Key phrase generation model training method and device and key phrase generation method and device

Abstract

The embodiment of the invention provides a key phrase generation model training method and device and a key phrase generation method and device. The training method comprises the following steps: acquiring a training document and a plurality of key phrase true values corresponding to the training document; distributing a plurality of key phrase true values to a plurality of control codes based on an optimal transmission theory; inputting the training document into a generator to enable the generator to output a plurality of candidate key phrases; inputting the plurality of candidate key phrases into a selector to enable the selector to output a decision label which is used for indicating that the corresponding candidate key phrases are reserved or discarded; and constructing a first loss function corresponding to the generator and a second loss function corresponding to the selector, and adjusting and optimizing the generator and the selector according to the first loss function and the second loss function to obtain a target key phrase generation model. According to the technical scheme, the accuracy and recall rate of key phrase generation can be considered, and the key phrase generation quality is guaranteed.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a training method for a key phrase generation model, a key phrase generation method and a device. Background Art

[0002] Keyphrases are words or phrases that represent or capture the core concepts and themes in documents, articles, reports, research papers or other text materials. Through keyphrases, users can know the main information of the document without having to read the entire document. In current technical solutions, the keyphrase generation (KPG) task is often processed using a single model, such as large language models (LLMs). However, keyphrase generation based on a single model cannot take into account both precision and recall. Therefore, how to balance the precision and recall of keyphrase generation and ensure the quality of keyphrase generation has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The embodiments of the present application provide a training method for a key phrase generation model, a key phrase generation method and a device, thereby at least to a certain extent taking into account both the accuracy and recall rate of key phrase generation, thereby ensuring the quality of key phrase generation.

[0004] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0005] According to one aspect of an embodiment of the present application, a training method for a key phrase generation model is provided, wherein the key phrase generation model includes a generator and a selector, wherein the generator is based on a Transformer, the decoder of the generator has a plurality of control codes, and the selector is constructed based on a large language model;

[0006] The method comprises:

[0007] Get the training documents and their corresponding truth values ​​of several key phrases;

[0008] Based on the optimal transmission theory, assigning a plurality of the key phrase truth values ​​to a plurality of the control codes as supervisory signals;

[0009] Inputting the training document into the generator so that the generator outputs a number of candidate key phrases corresponding to the training document;

[0010] Inputting a plurality of the candidate key phrases into the selector so that the selector outputs a decision tag corresponding to each of the candidate key phrases, wherein the decision tag is used to indicate whether to retain or discard the corresponding candidate key phrase;

[0011] A first loss function corresponding to the generator and a second loss function corresponding to the selector are constructed, and the generator and the selector are tuned according to the first loss function and the second loss function respectively to obtain a target key phrase generation model.

[0012] According to one aspect of an embodiment of the present application, a method for generating a key phrase is provided, comprising:

[0013] Get the documents to be processed;

[0014] The document to be processed is input into a pre-trained target key phrase generation model so that the target key phrase generation model outputs at least one target key phrase corresponding to the document to be processed, and the target key phrase generation model is trained by the training method described in the above embodiment.

[0015] According to one aspect of an embodiment of the present application, a training device for a key phrase generation model is provided, wherein the key phrase generation model includes a generator and a selector, wherein the generator is based on a Transformer, the decoder of the generator has a plurality of control codes, and the selector is constructed based on a large language model;

[0016] The device comprises:

[0017] The first acquisition module acquires the training document and the truth values ​​of several key phrases corresponding to it;

[0018] An allocation module, configured to allocate a plurality of key phrase truth values ​​to a plurality of control codes as supervisory signals based on an optimal transmission theory;

[0019] A first input module, configured to input the training document into the generator, so that the generator outputs a plurality of candidate key phrases corresponding to the training document;

[0020] A second input module, configured to input a plurality of the candidate key phrases into the selector, so that the selector outputs a decision tag corresponding to each of the candidate key phrases, wherein the decision tag is used to indicate whether to retain or discard the corresponding candidate key phrase;

[0021] The first processing module is used to construct a first loss function corresponding to the generator and a second loss function corresponding to the selector, and to tune the generator and the selector according to the first loss function and the second loss function respectively to obtain a target key phrase generation model.

[0022] According to one aspect of an embodiment of the present application, a key phrase generating device is provided, including:

[0023] The second acquisition module is used to acquire the document to be processed;

[0024] The second processing module is used to input the document to be processed into a pre-trained target key phrase generation model so that the target key phrase generation model outputs at least one target key phrase corresponding to the document to be processed, and the target key phrase generation model is trained by the training method described in the above embodiment.

[0025] According to one aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method described in the above embodiment is implemented.

[0026] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the methods described in the above embodiments.

[0027] According to one aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in the above embodiment.

[0028] In the technical solutions provided in some embodiments of the present application, the key phrase generation model includes a generator and a selector, wherein the generator is based on Transformer, the decoder of the generator has a number of control codes, and the selector is constructed based on a large language model; by obtaining the truth values ​​of training documents and their corresponding key phrases, based on the optimal transmission theory, the truth values ​​of the key phrases are assigned to the control codes as supervision signals, the training documents are input into the generator so that the generator outputs a number of candidate key phrases corresponding to the training documents, and then the number of candidate key phrases are input into the selector so that the selector outputs a decision label corresponding to each candidate key phrase, and the decision label is used to indicate whether to retain or discard the corresponding candidate key phrase; then, a first loss function corresponding to the generator and a second loss function corresponding to the selector are constructed, and the generator and the selector are tuned according to the first loss function and the second loss function respectively to obtain the target key phrase.

[0029] Therefore, the technical solution provided in the embodiment of the present application can correctly recall key phrases through the generator, and the selector can eliminate incorrect candidates, so that the key phrase generation model can take into account both the accuracy and recall rate of key phrase generation, thereby ensuring the quality of key phrase generation.

[0030] Furthermore, since the number of control codes usually exceeds the number of key phrase true values, many control codes are insufficiently trained. The technical solution provided in the embodiment of the present application is based on the optimal transmission theory to determine the distribution scheme between key phrase true values ​​and control codes, allowing one key phrase true value to be distributed to multiple control codes, thereby ensuring the rationality of the key phrase true value distribution and the training effect of each control code.

[0031] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0033] Figure 1 A schematic diagram showing a flow chart of a method for training a key phrase generation model according to an embodiment of the present application is shown;

[0034] Figure 2 A block diagram of a training device for a key phrase generation model according to an embodiment of the present application is shown;

[0035] Figure 3 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0036] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0037] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0038] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0039] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0040] Figure 1 A flow chart of a method for training a key phrase generation model according to an embodiment of the present application is shown. It should be noted that the method can be applied to a terminal device or a server, wherein the terminal device can include one or more of a smart phone, a tablet computer, a portable computer, and a desktop computer; the server can be a physical server or a cloud server.

[0041] In one embodiment, the key phrase generation model provided in the embodiment of the present application includes a generator and a selector, wherein the main goal of the generator is to generate a set of candidate key phrases that can represent the core concepts of a given document. Specifically, the generator is built based on the ONE2SET paradigm and uses the Transformer model as the backbone. The Transformer model adopts an encoder-decoder architecture, where the encoder is responsible for processing the input sequence and the decoder is responsible for generating the output sequence. In addition, the decoder is assigned multiple control codes, each of which is responsible for generating a candidate key phrase, thereby improving the recall rate.

[0042] The main function of the selector is to select the most relevant and accurate key phrases from the candidate key phrases generated by the generator. It is built on the basis of a large language model, thereby utilizing the powerful semantic understanding ability of the large language model to perform this task, ensuring that the final set of key phrases is both comprehensive and highly relevant. Specifically, the selector models the key phrase screening process as a sequence labeling task, in which each candidate key phrase is labeled as "retained" (T) or "discarded" (F). The selector generates the above decision labels autoregressively, thereby taking into account the choices that have been made before, reducing semantic duplication and improving the accuracy of screening.

[0043] Please continue to refer to Figure 1 The training method of the key phrase generation model at least includes steps S110 to S150, which are described in detail as follows (the method is described below by taking the application of the method to a terminal device as an example, hereinafter referred to as "terminal"):

[0044] In step S110, a training document and its corresponding truth values ​​of several key phrases are obtained.

[0045] In one embodiment, a person skilled in the art may pre-build a training database for model training, which may include several training documents and several key phrase truth values ​​corresponding to each training document, which may be obtained from manual annotations in the document or confirmed by an authoritative source, and which may accurately represent the core concepts and themes of the corresponding training document. Thus, before training, the terminal may obtain the training documents used for training and their corresponding key phrase truth values ​​from the pre-built training database in preparation for subsequent training.

[0046] In step S120, based on the optimal transmission theory, a number of the key phrase truth values ​​are assigned to a number of the control codes as supervisory signals.

[0047] In this embodiment, since the number of control codes usually exceeds the number of key phrase true values, it is easy to cause many control codes to be undertrained. To solve the above problem, the terminal can use the optimal transmission theory to train the key phrase true values. or φ(y M ) is dynamically assigned to the control code as a supervisory signal, thereby ensuring a reasonable distribution between the true value of the key phrase and the control code, and improving the training efficiency and quality. i represents the truth value of the i-th key phrase, and there are M-1 non-empty truth values ​​in total; φ represents the meaningless empty symbol used for filling, that is, the M-th truth value.

[0048] In one embodiment of the present application, based on the optimal transmission theory, a plurality of key phrase truth values ​​are assigned to a plurality of control codes as supervisory signals, including:

[0049] Generating a plurality of minimum text units for each of the control codes;

[0050] For each of the control codes, determining a matching score between the truth value of each key phrase and the minimum text unit corresponding to the control code;

[0051] Based on the optimal transmission theory, the target allocation scheme between the key phrase truth value and the control code is determined by calculating the matching score between the minimum text unit corresponding to each control code and the key phrase truth value.

[0052] In this embodiment, K tokens (i.e., at least one minimum text unit) may be generated for each control code as the prediction of the control code. The minimum text unit may be a word or a letter. Then, the terminal may use a pairwise matching function C match (*) Calculate the truth value y of each key phrase i With this prediction The matching score μ between ij :

[0053]

[0054]

[0055] Where τ is a normalization hyperparameter. K′=min(|y i |,K), K is the minimum number of text units to be predicted, Represents the smallest text unit In the probability distribution The probability value in .

[0056] After determining the matching score between each key phrase truth value and the prediction of each control code, the allocation between the key phrase truth value and the control code can be regarded as an optimal transmission problem based on the matching score, and the optimal allocation with Sinkhorn-Knopp iteration, i.e., the target allocation scheme, is searched.

[0057] In one embodiment, based on the optimal transmission theory, the target allocation scheme between the key phrase truth value and the control code is determined by calculating the matching score between the minimum text unit corresponding to each control code and the key phrase truth value, including:

[0058] Taking the true value of the key phrase as a supplier and constructing a corresponding supply vector, and taking the control code as a demander and constructing a corresponding demand vector;

[0059] Constructing an allocation cost matrix between the true value of the key phrase and the control code, wherein each element in the allocation cost matrix is ​​an allocation cost of allocating each true value of the key phrase to each control code, and the allocation cost is negatively correlated with the matching score between the two;

[0060] The optimal transmission theory is adopted to perform calculations based on the supply vector, the demand vector and the allocation cost matrix to determine a target allocation scheme between the true value of the key phrase and the control code.

[0061] In this embodiment, the control code can be regarded as the demand side, and its corresponding demand vector is constructed. Where N is the number of control codes, d j Represents the number of truth values ​​of key phrases assigned to the jth control code.

[0062] Consider the key phrase truth value as the supplier and construct its corresponding supply vector Among them, s i Represents the truth value y of the i-th key phrase i The number of control codes that can be assigned.

[0063] Next, we construct the allocation cost matrix between the key phrase truth value and the control code Among them, c ij Indicates that the truth value y of the i-th key phrase i The cost assigned to the jth control code.

[0064] It should be noted that, since assigning multiple key phrase truth values ​​to one control code at the same time may interfere with each other, it is possible to limit d j The value of is 1. Then, if y i If it matches more control codes, then the y i More control codes should be allocated, so s i Defined as Dynamic quantity of positive correlation:

[0065]

[0066] in, represents rounding up, and k is a predefined hyperparameter.

[0067] And, to simulate the truth value of the key phrase y i and prediction The higher the match score between i The lower the cost assigned to the jth control code, the lower the cost of c ij for:

[0068]

[0069] With the above vectors and matrices, we can find the optimal allocation π* according to the following objective function:

[0070]

[0071] In this way, each control code can be assigned the key phrase truth value or φ with the maximum matching score, which is the target assignment scheme.

[0072] Please continue to refer to Figure 1 In step S130, the training document is input into the generator so that the generator outputs a number of candidate key phrases corresponding to the training document.

[0073] In this embodiment, the terminal may input the training document into the generator, so that each control code in the generator generates a candidate key phrase for subsequent screening.

[0074] In step S140 , a plurality of candidate key phrases are input into the selector, so that the selector outputs a decision label corresponding to each candidate key phrase, wherein the decision label is used to indicate whether to retain or discard the corresponding candidate key phrase.

[0075] In this embodiment, after acquiring several key phrases, the terminal may input them into the selector, so as to use the selector to filter out the most relevant and accurate key phrases from the candidate key phrases generated by the generator.

[0076] Specifically, the key phrase selection process can be modeled as a sequence labeling task, and the selector generates a decision label for each candidate key phrase, and the decision label is used to indicate whether to retain or discard the corresponding candidate key phrase. In one example, the decision label may include "T" and "F", where "T" indicates that the candidate key phrase should be retained and "F" indicates that the candidate key phrase should be discarded. The selector can generate the decision label autoregressively, so that the selections made before can be considered, reducing semantic duplication and improving the accuracy of the selection.

[0077] In one example, when entering a selector, the instruction template used may be as follows:

[0078] Task definition: You need to perform a sequence labeling task to select multiple key phrases from numbered candidates based on a given document. Use the label "T" to indicate that a candidate is selected and the label "F" to indicate that it is rejected. For example, the label sequence "TFF" indicates that candidate [1] is selected and candidates [2] and [3] are rejected.

[0079] ###enter:

[0080] Document: {document}

[0081] Candidates:

[0082] [1]{candidate1}

[0083] …

[0084] [n]{candidaten}

[0085] ###response:

[0086] Label sequence: {label_sequence}

[0087] Please continue to refer to Figure 1 In step S150, a first loss function corresponding to the generator and a second loss function corresponding to the selector are constructed, and the generator and the selector are tuned according to the first loss function and the second loss function respectively to obtain a target key phrase generation model.

[0088] In this embodiment, a first loss function corresponding to the generator and a second loss function corresponding to the selector can be constructed respectively, so that the generator can be tuned according to the first loss function, and the selector can be tuned according to the second loss function, and then the target key phrase generation model can be obtained, so that key phrases can be generated for specified documents in the future according to the target key phrase generation model.

[0089] In one embodiment of the present application, some of the control code parts are used to generate existing key phrases, and the remaining parts are used to generate missing key phrases; then constructing the first loss function corresponding to the generator includes:

[0090] determining a first target allocation scheme corresponding to the existing key phrases and a second target allocation scheme corresponding to the missing key phrases;

[0091] According to the first target allocation scheme and the second target allocation scheme, a cross entropy loss between the two is determined as a first loss function.

[0092] In this embodiment, several control code parts are used to generate present key phrases, and the remaining parts are used to generate missing key phrases, wherein present key phrases refer to those phrases that can be directly found or obviously mentioned in the document, which directly reflect the core concepts and themes of the document; missing key phrases refer to those phrases that are not directly mentioned in a given document but are crucial to understanding the core concepts and themes of the document.

[0093] Therefore, the loss function of the generator can be composed of two parts, one for existing key phrases and the other for missing key phrases. Specifically, the optimal allocation plan can be found for existing key phrases and missing key phrases respectively. (i.e. the first target allocation scheme) and (i.e., the second target allocation scheme). Then, the corresponding cross entropy loss is calculated according to the two allocation schemes as the first loss function of the generator.

[0094] In one example, the cross entropy loss can be determined according to the following formula:

[0095]

[0096] Among them, L p and L a are the losses of existing keyphrases and missing keyphrases respectively, θ represents the model parameters, and are the predictions corresponding to existing keyphrases and missing keyphrases, respectively.

[0097] In one example, the parameters of the generator can be updated by a back-propagation algorithm using the calculated first loss function, and the model can be optimized using stochastic gradient descent or its variants. The above process is repeated until the performance of the model on the validation set cannot be improved or a predetermined number of iterations is reached.

[0098] In one embodiment of the present application, the second loss function corresponding to the selector is determined according to the following formula:

[0099]

[0100] Among them, φ is the parameter of the selector, N T and N F The number of decision labels for “retain” and “discard”, respectively, Y is composed of N T "Reserved" and N F “Discarded” label sequences, X is the input data without label sequences, i.e., candidate key phrases, p φ (Y t |X,Y<t ) is the probability that the selector predicts the current label given the previous choices.

[0101] In one embodiment of the present application, in each round of training, a number of candidate key phrases are randomly sorted and adjusted before being input into the selector.

[0102] In this embodiment, during the training process, the input order of the candidate key phrases can be randomly sorted and adjusted so that the selector will not rely on a specific input order, and is forced to learn how to evaluate the importance and relevance of each candidate key phrase independently of the input order, thereby enhancing the generalization ability of the model when processing inputs of different orders.

[0103] In one embodiment, during the reasoning phase of the selector, the candidate key phrases may be ranked based on the scores generated by the generator, so that the selector can more effectively process those candidates that are most likely to be selected as the final key phrases. In one example, the score may be calculated based on a variety of factors, such as the semantic relevance of the candidate key phrase to the document topic, the frequency of occurrence in the document, or the position in the document (e.g., phrases appearing in the title, abstract, or keyword field may be given a higher weight). In another example, the score may be the logarithmic probability corresponding to the candidate key phrase.

[0104] In one embodiment of the present application, a key phrase generation method is also provided, including:

[0105] Get the documents to be processed;

[0106] The document to be processed is input into a pre-trained target key phrase generation model so that the target key phrase generation model outputs at least one target key phrase corresponding to the document to be processed, and the target key phrase generation model is trained by the training method described in any of the above embodiments.

[0107] Thus, by using the target key phrase generation model obtained by the training method described in any of the above embodiments, the target key phrase generation model includes a generator and a selector, wherein the generator is based on Transformer, the decoder of the generator has a plurality of control codes, and the selector is constructed based on a large language model; during training, by obtaining the true values ​​of the training document and the corresponding key phrases, based on the optimal transmission theory, the key phrase true values ​​are assigned to the control codes as supervision signals, the training document is input into the generator so that the generator outputs a plurality of candidate key phrases corresponding to the training document, and then the plurality of candidate key phrases are input into the selector so that the selector outputs a decision label corresponding to each candidate key phrase, and the decision label is used to indicate whether to retain or discard the corresponding candidate key phrase; then, a first loss function corresponding to the generator and a second loss function corresponding to the selector are constructed, and the generator and the selector are tuned according to the first loss function and the second loss function respectively to obtain the target key phrase.

[0108] Therefore, the generator can correctly recall the key phrases, and the selector can eliminate the wrong candidates, so that the key phrase generation model can take into account both the accuracy and recall rate of key phrase generation, ensuring the quality of key phrase generation.

[0109] Furthermore, since the number of control codes usually exceeds the number of key phrase true values, many control codes are insufficiently trained. The technical solution provided in the embodiment of the present application is based on the optimal transmission theory to determine the distribution scheme between key phrase true values ​​and control codes, allowing one key phrase true value to be distributed to multiple control codes, thereby ensuring the rationality of the key phrase true value distribution and the training effect of each control code.

[0110] The following describes an embodiment of the device of the present application, which can be used to execute the method in the above embodiment of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the above embodiment of the method of the present application.

[0111] Figure 2 A block diagram of a training device for a key phrase generation model according to an embodiment of the present application is shown.

[0112] Reference Figure 2 As shown, according to a training device for a key phrase generation model of an embodiment of the present application, the key phrase generation model includes a generator and a selector, wherein the generator is based on a Transformer, the decoder of the generator has a plurality of control codes, and the selector is constructed based on a large language model;

[0113] The device includes:

[0114] The first acquisition module acquires the training document and the truth values ​​of several key phrases corresponding to it;

[0115] An allocation module, configured to allocate a plurality of key phrase truth values ​​to a plurality of control codes as supervisory signals based on an optimal transmission theory;

[0116] A first input module, configured to input the training document into the generator, so that the generator outputs a plurality of candidate key phrases corresponding to the training document;

[0117] A second input module, configured to input a plurality of the candidate key phrases into the selector, so that the selector outputs a decision tag corresponding to each of the candidate key phrases, wherein the decision tag is used to indicate whether to retain or discard the corresponding candidate key phrase;

[0118] The first processing module is used to construct a first loss function corresponding to the generator and a second loss function corresponding to the selector, and to tune the generator and the selector according to the first loss function and the second loss function respectively to obtain a target key phrase generation model.

[0119] The embodiment of the present application also provides a key phrase generating device, which includes:

[0120] The second acquisition module is used to acquire the document to be processed;

[0121] The second processing module is used to input the document to be processed into a pre-trained target key phrase generation model so that the target key phrase generation model outputs at least one target key phrase corresponding to the document to be processed, and the target key phrase generation model is trained by the training method described in any of the above embodiments.

[0122] Figure 3 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown.

[0123] It should be noted that Figure 3 The computer system of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0124] like Figure 3As shown, the computer system includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 302 or the program loaded from the storage part 308 to the random access memory (RAM) 303, such as executing the method described in the above embodiment. In RAM 303, various programs and data required for system operation are also stored. CPU 301, ROM 302 and RAM 303 are connected to each other through bus 304. Input / output (I / O) interface 305 is also connected to bus 304.

[0125] The following components are connected to the I / O interface 305: an input section 306 including a keyboard, a mouse, etc.; an output section 307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. A removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 310 as needed so that a computer program read therefrom is installed into the storage section 308 as needed.

[0126] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 309, and / or installed from a removable medium 311. When the computer program is executed by a central processing unit (CPU) 301, various functions defined in the system of the present application are executed.

[0127] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0128] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0129] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0130] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

[0131] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0132] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0133] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0134] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for training a key phrase generation model, characterized in that: The key phrase generation model includes a generator and a selector, wherein the generator is based on Transformer, the decoder of the generator has a number of control codes, and the selector is constructed based on a large language model; The method comprises: Get the training documents and their corresponding truth values ​​of several key phrases; Based on the optimal transmission theory, assigning a plurality of the key phrase truth values ​​to a plurality of the control codes as supervisory signals; Inputting the training document into the generator so that the generator outputs a number of candidate key phrases corresponding to the training document; Inputting a plurality of the candidate key phrases into the selector so that the selector outputs a decision tag corresponding to each of the candidate key phrases, wherein the decision tag is used to indicate whether to retain or discard the corresponding candidate key phrase; A first loss function corresponding to the generator and a second loss function corresponding to the selector are constructed, and the generator and the selector are tuned according to the first loss function and the second loss function respectively to obtain a target key phrase generation model.

2. The method according to claim 1, characterized in that: Based on the optimal transmission theory, the truth values ​​of the key phrases are assigned to the control codes as supervisory signals, including: Generating a plurality of minimum text units for each of the control codes; For each of the control codes, determining a matching score between the truth value of each key phrase and the minimum text unit corresponding to the control code; Based on the optimal transmission theory, the target allocation scheme between the key phrase truth value and the control code is determined by calculating the matching score between the minimum text unit corresponding to each control code and the key phrase truth value.

3. The method according to claim 2, characterized in that Based on the optimal transmission theory, the target allocation scheme between the key phrase truth value and the control code is determined by calculating the matching score between the minimum text unit corresponding to each control code and the key phrase truth value, including: Taking the true value of the key phrase as a supplier and constructing a corresponding supply vector, and taking the control code as a demander and constructing a corresponding demand vector; Constructing an allocation cost matrix between the true value of the key phrase and the control code, wherein each element in the allocation cost matrix is ​​an allocation cost of allocating each true value of the key phrase to each control code, and the allocation cost is negatively correlated with the matching score between the two; The optimal transmission theory is adopted to perform calculations based on the supply vector, the demand vector and the allocation cost matrix to determine a target allocation scheme between the true value of the key phrase and the control code.

4. The method according to claim 3, characterized in that Some of the control code portions are used to generate existing key phrases, and the remaining portions are used to generate missing key phrases; Then constructing the first loss function corresponding to the generator includes: determining a first target allocation scheme corresponding to the existing key phrases and a second target allocation scheme corresponding to the missing key phrases; According to the first target allocation scheme and the second target allocation scheme, a cross entropy loss between the two is determined as a first loss function.

5. The method according to claim 1, characterized in that According to the following formula, the second loss function corresponding to the selector is determined: Among them, φ is the parameter of the selector, N T and N F The number of decision labels to “keep” and “discard” respectively, Y is the sum of N T "Reserved" and N F The label sequence composed of "discarded", X is the input data without label sequence, that is, candidate key phrases, p φ (Y t |X,Y <t ) is the probability that the selector predicts the current label given the previous choices.

6. The method according to claim 1, characterized in that In each round of training, a number of candidate key phrases are randomly sorted and adjusted before being input into the selector.

7. A method for generating a key phrase, characterized in that: include: Get the documents to be processed; The document to be processed is input into a pre-trained target key phrase generation model so that the target key phrase generation model outputs at least one target key phrase corresponding to the document to be processed, and the target key phrase generation model is trained by the training method according to any one of claims 1 to 6.

8. A training device for a key phrase generation model, characterized in that: The key phrase generation model includes a generator and a selector, wherein the generator is based on Transformer, the decoder of the generator has a number of control codes, and the selector is constructed based on a large language model; The device comprises: The first acquisition module acquires the training document and the truth values ​​of several key phrases corresponding to it; An allocation module, configured to allocate a plurality of key phrase truth values ​​to a plurality of control codes as supervisory signals based on an optimal transmission theory; A first input module, configured to input the training document into the generator, so that the generator outputs a plurality of candidate key phrases corresponding to the training document; A second input module, configured to input a plurality of the candidate key phrases into the selector, so that the selector outputs a decision tag corresponding to each of the candidate key phrases, wherein the decision tag is used to indicate whether to retain or discard the corresponding candidate key phrase; The first processing module is used to construct a first loss function corresponding to the generator and a second loss function corresponding to the selector, and to tune the generator and the selector according to the first loss function and the second loss function respectively to obtain a target key phrase generation model.

9. A key phrase generating device, characterized in that: include: The second acquisition module is used to acquire the document to be processed; The second processing module is used to input the document to be processed into a pre-trained target key phrase generation model so that the target key phrase generation model outputs at least one target key phrase corresponding to the document to be processed, and the target key phrase generation model is trained by the training method according to any one of claims 1 to 6.

10. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the method according to any one of claims 1 to 7.

Images