Text generation method based on entity description
Through the text generation method based on entity description, the pre-trained model and beam search technology are used to solve the problem of lack of universality of existing information extraction methods, and the effective processing of multiple information extraction tasks and the utilization of rich label semantics is realized, thereby improving the improvement of the improvement of the ability to effectively process multiple information extraction tasks. extraction effect and model adaptability.
Patent Information
- Application Number
- CN202210520980.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-13
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2042-05-13
Smart Images

Figure CN115099235B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of natural language processing, and in particular to a text generation method and device based on entity description. Background Art
[0002] The task of information extraction is to extract specific events or entities from natural language text, and then automatically classify or extract a large amount of content. The extracted information includes entities, relations, events and their categories, etc. Subtasks include named entity recognition, relation extraction, entity linking, semantic role labeling, event extraction, sentiment analysis, etc.
[0003] Existing work on various information extraction tasks mainly deals with labeling and classification, and designs different prediction networks according to various specific tasks. Although these methods are effective, they ignore the rich label semantics and require a lot of task-specific designs, making it impossible to construct a general framework. Summary of the invention
[0004] The present application aims to solve one of the technical problems in the related art at least to some extent.
[0005] To this end, the first purpose of this application is to propose a text generation method based on entity description, which solves the technical problem that the existing methods are not universal and can well complete various information extraction tasks.
[0006] The second objective of this application is to propose a text generation device based on entity description.
[0007] To achieve the above-mentioned purpose, the first aspect of the present application proposes a text generation method based on entity description, including: obtaining a to-be-processed set of a target information extraction task; using a pre-trained text generation model to infer the to-be-processed set, applying beam search to decode the inference results, and obtaining all possible paths, wherein the text generation model is generated by training the pre-trained model according to the target information extraction task using the corresponding training set; all possible paths are screened according to probability to obtain candidate paths; and according to the type of the target information extraction task, the candidate paths are pruned to obtain the target information.
[0008] The text generation method based on entity description in the embodiment of the present application constructs a general text generation framework, designs different entity descriptions as output sequences for different information extraction tasks. When there are multiple tuples to be extracted in a text, the more effective "sequence to path" approach is used to avoid the order and independence problems between multiple tuples in the generated sequence, thereby completing various information extraction tasks well.
[0009] Optionally, in one embodiment of the present application, before using the pre-trained text generation model to perform reasoning on the processing set, the method further includes:
[0010] Constructing an enhanced data set, wherein the enhanced data set includes positive samples and negative samples;
[0011] Use the enhanced dataset to train the pre-trained model to obtain a text generation model.
[0012] Optionally, in one embodiment of the present application, constructing an enhanced data set includes:
[0013] Get the training set corresponding to the target task;
[0014] Preprocess the processing set and the training set to generate positive samples, where each text in the training set corresponds to an output tuple, and the number of positive samples is the same as the number of output tuples;
[0015] According to the type of target task, each output tuple is converted into a natural language sentence as an output sequence according to the entity description of the positive sample;
[0016] Use the training set to train the pre-trained model and generate a sample generation model;
[0017] Use the sample generation model to infer the training set, apply beam search to decode the inference results, and obtain all possible paths and prediction results, where the prediction result is a combination of possible paths;
[0018] The wrong part of the prediction result is used as the first negative sample, and the first negative sample is used to randomly replace the elements of different tuples in the positive sample to generate the second negative sample;
[0019] An enhanced dataset is constructed based on the positive samples, the first negative samples, and the second negative samples.
[0020] Optionally, in one embodiment of the present application, when the pre-trained model is trained using an enhanced data set, the loss function of an input sample in the enhanced data set is the average of the loss functions of each output sequence.
[0021] To achieve the above-mentioned purpose, the second embodiment of the present invention proposes a text generation device based on entity description, including: an acquisition module, a path generation module, a path screening module, and a pruning module, wherein:
[0022] An acquisition module is used to obtain a pending set of target information extraction tasks;
[0023] A path generation module is used to use a pre-trained text generation model to infer the processing set, and apply beam search to decode the inference results to obtain all possible paths. The text generation model is generated by training the pre-trained model using the corresponding training set according to the target information extraction task;
[0024] The path screening module is used to screen all possible paths according to probability to obtain candidate paths;
[0025] The pruning module is used to prune candidate paths to obtain target information according to the type of target information extraction task.
[0026] Optionally, in one embodiment of the present application, a text generation model training module is further included, which is specifically used to:
[0027] Constructing an enhanced data set, wherein the enhanced data set includes positive samples and negative samples;
[0028] Use the enhanced dataset to train the pre-trained model to obtain a text generation model.
[0029] Optionally, in one embodiment of the present application, constructing an enhanced data set includes:
[0030] Get the training set corresponding to the target task;
[0031] Preprocess the processing set and the training set to generate positive samples, where each text in the training set corresponds to an output tuple, and the number of positive samples is the same as the number of output tuples;
[0032] According to the type of target task, each output tuple is converted into a natural language sentence as an output sequence according to the entity description of the positive sample;
[0033] Use the training set to train the pre-trained model and generate a sample generation model;
[0034] Use the sample generation model to infer the training set, apply beam search to decode the inference results, and obtain all possible paths and prediction results, where the prediction result is a combination of possible paths;
[0035] The wrong part of the prediction result is used as the first negative sample, and the first negative sample is used to randomly replace the elements of different tuples in the positive sample to generate the second negative sample;
[0036] An enhanced dataset is constructed based on the positive samples, the first negative samples, and the second negative samples.
[0037] Optionally, in one embodiment of the present application, when the pre-trained model is trained using an enhanced data set, the loss function of an input sample in the enhanced data set is the average of the loss functions of each output sequence.
[0038] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0040] Figure 1 A flowchart of a text generation method based on entity description provided in Example 1 of the present application;
[0041] Figure 2 This is an example diagram of a text generation method based on entity description according to an embodiment of the present application;
[0042] Figure 3 A schematic diagram of the structure of a text generation device based on entity description provided in an embodiment of the present application. DETAILED DESCRIPTION
[0043] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.
[0044] In order to solve the problem that existing methods are not universal, we consider building a unified generative framework that can solve various information extraction problems. The generative model can make full use of natural language encoding, enrich label semantics, and insert language labels into the target output. In addition, this unified generative model can seamlessly adapt to multiple tasks without introducing additional task-specific model design.
[0045] Therefore, all kinds of information extraction tasks can be transformed into a sequence-to-sequence text translation problem. However, the design of the output sequence is crucial. Taking the joint extraction of entities and relations as an example, the output form is designed as (subject, relation, object). Compared with the output form (subject, object, relation), it is more in line with the semantic order of natural language and can obtain better extraction results. Therefore, for more common situations, if a complete sentence is directly used as the output sequence to describe the elements to be extracted, the effect will be further improved.
[0046] Although the sequence-to-sequence generation framework has achieved good results on many data sets, it still has limitations: (1) The order problem. Sentiment tuples are parallel and there is no concept of order, but the sequence-to-sequence method outputs them serially in order. (2) The independence problem. According to the concept of the sequence-to-sequence model, the subsequent tuple to be extracted depends on the previous tuple, but in fact the tuples are independent of each other and there is no such dependency. Therefore, the output text of the entity description in this application adopts the "sequence to path" method.
[0047] In addition, in order to expand the data set, it is necessary to use negative sample methods for data augmentation.
[0048] The following describes the text generation method and device based on entity description in an embodiment of the present application with reference to the accompanying drawings.
[0049] Figure 1 A flowchart of a text generation method based on entity description provided in Example 1 of the present application.
[0050] like Figure 1 As shown, the text generation method based on entity description includes the following steps:
[0051] Step 101, obtaining a to-be-processed set of target information extraction tasks;
[0052] Step 102, using a pre-trained text generation model to infer the processing set, applying beam search to decode the inference results, and obtaining all possible paths, wherein the text generation model is generated by training the pre-trained model using the corresponding training set according to the target information extraction task;
[0053] Step 103, screening all possible paths according to probability to obtain candidate paths;
[0054] Step 104: Prune the candidate paths according to the type of the target information extraction task to obtain the target information.
[0055] The text generation method based on entity description in the embodiment of the present application constructs a general text generation framework, designs different entity descriptions as output sequences for different information extraction tasks. When there are multiple tuples to be extracted in a text, the more effective "sequence to path" approach is used to avoid the order and independence problems between multiple tuples in the generated sequence, thereby completing various information extraction tasks well.
[0056] Optionally, in one embodiment of the present application, before using the pre-trained text generation model to perform reasoning on the processing set, the method further includes:
[0057] Constructing an enhanced data set, wherein the enhanced data set includes positive samples and negative samples;
[0058] Use the enhanced dataset to train the pre-trained model to obtain a text generation model.
[0059] Optionally, in one embodiment of the present application, constructing an enhanced data set includes:
[0060] Get the training set corresponding to the target task;
[0061] Preprocess the processing set and the training set to generate positive samples, where each text in the training set corresponds to an output tuple, and the number of positive samples is the same as the number of output tuples;
[0062] According to the type of target task, each output tuple is converted into a natural language sentence as an output sequence according to the entity description of the positive sample;
[0063] Use the training set to train the pre-trained model and generate a sample generation model;
[0064] Use the sample generation model to infer the prediction training set, apply beam search to decode the inference results, and obtain all possible paths and prediction results, where the prediction result is a combination of possible paths;
[0065] The wrong part of the prediction result is used as the first negative sample, and the first negative sample is used to randomly replace the elements of different tuples in the positive sample to generate the second negative sample;
[0066] An enhanced dataset is constructed based on the positive samples, the first negative samples, and the second negative samples.
[0067] Optionally, in one embodiment of the present application, when the pre-trained model is trained using an enhanced data set, the loss function of an input sample in the enhanced data set is the average of the loss functions of each output sequence.
[0068] This application proposes a text generation method based on entity description. The optimization design method of this application is used in multiple information extraction tasks such as entity recognition, relationship extraction, entity linking, event extraction, semantic role labeling, and sentiment analysis. In the experiment, Nvidia 1080Ti GPU single graphics card is used for training, and the pre-trained model uses the t5-base model in huggingface. Figure 2 The text generation process of this application is introduced by taking practical application as an example:
[0069] Step 1: Taking the extraction of sentiment element triplets in sentiment analysis as an example, the conventional sequence-to-sequence method of extracting tuples generates samples according to the number of input texts, which is:
[0070] Input text → "(a1,o1,s1),(a2,o2,s2)"
[0071] Based on the sequence-to-path method, samples are generated according to the number of input tuples, and each piece of input text corresponds to an output tuple:
[0072]
[0073] Among them, the entities to be extracted for different subtasks are shown in Table 1.
[0074] Table 1 Tuples to be extracted for each information extraction subtask
[0075]
[0076]
[0077] Step 2: For different subtasks, convert each output tuple as a positive sample into a natural language sentence as an output sequence according to the following method:
[0078] (1) Named Entity Recognition Task
[0079] The extraction target is the text ( <entity> , <category>) tuple, where <category>For Entity <entity>The linearized natural sentence is as follows:
[0080] Positive Sample <entiiy>one <category>entity.
[0081] Negative samples <entity>is nota <categoty>entity.
[0082] (2) Joint extraction of entities and relations
[0083] The extraction target is the text ( <entity1> , <relation> , <entity2>) triples, where <entity1>As the main body, <entity2>For the object, <relation>is the relationship between the two. The linearized natural sentence is as follows:
[0084] Positive Sample Therelationbetween <entity1>and <entity2>is <relation>.
[0085] Negative sample The relation between <entity1>and <entity2>is not <relation>.
[0086] (3) Semantic role analysis task
[0087] For a given predicate ( <predicate>), the extraction target is ( <argument> , <role>) tuples, the linearized natural sentences are as follows:
[0088] Positive Sample <argument>isthe <role>of <predicate>.
[0089] Negative samples <argument>is not the <role>of <predicate>.
[0090] (4) Event extraction task
[0091] Using the pipeline approach, the first step is to extract the trigger, and the second step is to extract the argument.
[0092] ①The goal of extracting trigger is (<event trigger> ,<event category> ) tuples, the linearized natural language sentences are as follows:
[0093] Positive Sample <eventtrigger>one <eventcategory>event.
[0094] Negative samples <eventtrigger>is not a <eventcategory>event.
[0095] ②The goal of extracting argument is (<event argument> ,<entity type> ,<entity role> ) triples, the linearized natural language sentence is as follows:
[0096] Positive Sample
[0097] <eventargument>one <entitytype>type,anditisthe <entityrole>.
[0098] Negative sample
[0099] <event argument> is not a <entity type> type, or it is not the <entityrole>.
[0100] (5) Entity Linking Task
[0101] For a given entity <entity>, extracted ( <argument>)and <entity>After completing the link, the linearized natural language sentence is as follows:
[0102] Positive Sample <argument>report <entity>.
[0103] Negative samples <argument>not refer to <entity>.
[0104] (6) Sentiment analysis task
[0105] ①AOPE mission
[0106] Positive sample F a (a)isF o (o).
[0107] Negative samples F a (a) is not F o (o).
[0108] ②ASTE Mission
[0109] Positive sample ItisF s (s)becauseF a (a)isF o (o).
[0110] Negative SampleIt is F s (s) not because F a (a)is F o (o).
[0111] ③TASD mission
[0112] Positive sample F a (a)isaF c (c)categoryandF a (a)isF o (o).
[0113] Negative samples F a (a)isnotaF c (c)categoryorF a (a)is not o (o).
[0114] ④UABSA Mission
[0115] Positive sample F a (a)isF s (s)
[0116] Negative samples F a (a)is not s (s)
[0117] ⑤ACOS tasks
[0118] Positive sample F c (c)isF s (s)becauseF a (a)isF o (o).
[0119] Negative samples F c (c)isF s (s) not because a (a)isF o (o).
[0120] in,
[0121]
[0122] Step 3: Based on the pre-trained model T5, train the text generation model M based on the training set aug , used for subsequent data enhancement;
[0123] Step 4: Use M aug Predict the training set, according to M aug The wrong prediction results are used to construct negative sample D′2, and then the elements of different tuples in the positive sample D are randomly replaced to construct negative sample D′1. The construction method of D′1: In order to improve the model's ability to match tuple elements, the elements in the tuple are randomly replaced; the construction method of D′2: In order to improve the model's ability to filter out wrong elements, we first use M aug Generate some wrong examples and add them to negative samples. Dataset D after data enhancement aug =D∪D′=D∪D′1∪D′2. Take sentiment analysis triple extraction as an example: the input text is "Those rolls are big, but not good and sashimi wasn't fresh.", and the tuples to be extracted are (rolls, big, positive), (rolls, notgood, negative), (rolls, sashimi, fresh), then the negative samples in D′1 can be (rolls, wasn'tfresh, positive), (sashimi, big, negative, false), and the negative samples in D′2 can be (sashimi, n'tfresh, negative, false).
[0124] Step 5: Similar to step 1, aug A T5-based model M is trained on the CNN. The loss function of an input sample is the average of the loss functions of each output sequence, expressed as:
[0125]
[0126] in
[0127] Step 6: Use the trained model M to infer the test set, apply beam search for decoding, sort the paths in descending order of probability, and the k paths with the highest probability are the candidate paths.
[0128] Step 7: Prune k paths for different subtasks to get the final result. During model inference, the decoded words should not belong to the full vocabulary, but should come from the original input text or some other classification tags. For example, in the ASTE task in sentiment analysis, the search range of sentiment polarity s is only in {positive, negative, neutral}. The candidate word set is denoted as T, which consists of the words T(x) of the input text and the classification tag T(task).
[0129] Right now
[0130] T(x,task)=T(x)∪T(task)
[0131] However, in the actual model reasoning process, the generated words do not necessarily come from the candidate set T, such as the difference between singular and plural or uppercase and lowercase. Therefore, for the case where the generated word e does not belong to the elements in T, the Levenshtein distance from e to all candidate elements can be measured, and the one with the smallest distance can be selected to replace e. A simpler way is to directly limit the generated words to the range of T. Furthermore, the elements to be generated can be divided into extraction-type elements and classification-type elements. The extraction-type elements come from T(x), and the classification-type elements come from T(task). Each type of element has its own limited range.
[0132] In summary, this application can effectively complete various information extraction tasks and has good versatility. Table 2 is the experimental results on the sentiment analysis (ACOS subtask) dataset, and Table 3 is the experimental results on the relationship extraction dataset, which proves that this method is effective.
[0133] Table 2 Sentiment analysis experimental results (taking ACOS task as an example)
[0134]
[0135]
[0136] Table 3. Relationship extraction experimental results
[0137]
[0138] In order to implement the above embodiment, the present application also proposes a text generation device based on entity description.
[0139] Figure 3 A schematic diagram of the structure of a text generation device based on entity description provided in an embodiment of the present application.
[0140] like Figure 3 As shown, the text generation device based on entity description includes: an acquisition module, a path generation module, a path screening module, and a pruning module, wherein:
[0141] An acquisition module is used to obtain a pending set of target information extraction tasks;
[0142] A path generation module is used to use a pre-trained text generation model to infer the processing set, and apply beam search to decode the inference results to obtain all possible paths. The text generation model is generated by training the pre-trained model using the corresponding training set according to the target information extraction task;
[0143] The path screening module is used to screen all possible paths according to probability to obtain candidate paths;
[0144] The pruning module is used to prune candidate paths to obtain target information according to the type of target information extraction task.
[0145] Optionally, in one embodiment of the present application, a text generation model training module is further included, which is specifically used to:
[0146] Constructing an enhanced data set, wherein the enhanced data set includes positive samples and negative samples;
[0147] Use the enhanced dataset to train the pre-trained model to obtain a text generation model.
[0148] Optionally, in one embodiment of the present application, constructing an enhanced data set includes:
[0149] Get the training set corresponding to the target task;
[0150] Preprocess the processing set and the training set to generate positive samples, where each text in the training set corresponds to an output tuple, and the number of positive samples is the same as the number of output tuples;
[0151] According to the type of target task, each output tuple is converted into a natural language sentence as an output sequence according to the entity description of the positive sample;
[0152] Use the training set to train the pre-trained model and generate a sample generation model;
[0153] Use the sample generation model to infer the training set, apply beam search to decode the inference results, and obtain all possible paths and prediction results, where the prediction result is a combination of possible paths;
[0154] The wrong part of the prediction result is used as the first negative sample, and the first negative sample is used to randomly replace the elements of different tuples in the positive sample to generate the second negative sample;
[0155] An enhanced dataset is constructed based on the positive samples, the first negative samples, and the second negative samples.
[0156] Optionally, in one embodiment of the present application, when the pre-trained model is trained using an enhanced data set, the loss function of an input sample in the enhanced data set is the average of the loss functions of each output sequence.
[0157] It should be noted that the aforementioned explanation of the embodiment of the text generation method based on entity description is also applicable to the text generation device based on entity description of this embodiment, and will not be repeated here.
[0158] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0159] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0160] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.
[0161] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.
[0162] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0163] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.
[0164] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
[0165] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.< / entity> < / argument> < / entity> < / argument> < / entity> < / argument> < / entity> < / entityrole> < / entityrole> < / entitytype> < / eventargument> < / eventcategory> < / eventtrigger> < / eventcategory> < / eventtrigger> < / predicate> < / role> < / argument> < / predicate> < / role> < / argument> < / role> < / argument> < / predicate> < / relation> < / relation> < / relation> < / relation> < / entity1> < / categoty> < / entity> < / category> < / entiiy> < / entity> < / category> < / category> < / entity>
Claims
1. A text generation method based on entity description, characterized in that: The following steps are involved: Obtain the pending set of target information extraction tasks; Use a pre-trained text generation model to infer the set to be processed, apply beam search to decode the inference results, and obtain all possible paths, wherein the text generation model is generated by training the pre-trained model using the corresponding training set according to the target information extraction task; Screening all possible paths according to probability to obtain candidate paths; According to the type of the target information extraction task, the candidate paths are pruned to obtain the target information.
2. The method according to claim 1, characterized in that Before using the pre-trained text generation model to reason about the set to be processed, the method further includes: Constructing an enhanced data set, wherein the enhanced data set includes positive samples and negative samples; The pre-trained model is trained using the enhanced data set to obtain the text generation model.
3. The method according to claim 2, characterized in that The constructing of the enhanced data set comprises: Get the training set corresponding to the target task; Preprocessing the set to be processed and the training set to generate positive samples, wherein each text segment corresponds to an output tuple during preprocessing, and the number of the positive samples is the same as the number of the output tuples; According to the type of the target task, each output tuple is converted into a natural language sentence as an output sequence according to the entity description of the positive sample; Use the training set to train the pre-trained model and generate a sample generation model; Using the sample generation model to infer the training set, applying beam search to decode the inference results, and obtaining all possible paths and prediction results, wherein the prediction result is a combination of possible paths; Taking the wrong part of the prediction result as a first negative sample, and using the first negative sample to randomly replace elements of different tuples in the positive sample to generate a second negative sample; The enhanced dataset is constructed according to the positive sample, the first negative sample and the second negative sample.
4. The method according to claim 3, characterized in that When the enhanced data set is used to train the pre-trained model, the loss function of an input sample in the enhanced data set is the average of the loss functions of each output sequence.
5. A text generation device based on entity description, characterized in that: It includes acquisition module, path generation module, path screening module and pruning module, among which, The acquisition module is used to acquire a to-be-processed set of target information extraction tasks; The path generation module is used to use a pre-trained text generation model to infer the set to be processed, and apply beam search to decode the inference results to obtain all possible paths, wherein the text generation model is generated by training the pre-trained model using the corresponding training set according to the target information extraction task; The path screening module is used to screen all possible paths according to probability to obtain candidate paths; The pruning module is used to prune the candidate paths to obtain target information according to the type of the target information extraction task.
6. The device according to claim 5, characterized in that It also includes a text generation model training module, which is specifically used for: Constructing an enhanced data set, wherein the enhanced data set includes positive samples and negative samples; The pre-trained model is trained using the enhanced data set to obtain the text generation model.
7. The device according to claim 6, characterized in that The constructing of the enhanced data set comprises: Get the training set corresponding to the target task; Preprocessing the set to be processed and the training set to generate positive samples, wherein each text segment in the training set corresponds to an output tuple, and the number of the positive samples is the same as the number of the output tuples; According to the type of the target task, each output tuple is converted into a natural language sentence as an output sequence according to the entity description of the positive sample; Use the training set to train the pre-trained model and generate a sample generation model; Using the sample generation model to infer the training set, applying beam search to decode the inference results, and obtaining all possible paths and prediction results, wherein the prediction result is a combination of possible paths; Taking the wrong part of the prediction result as a first negative sample, and using the first negative sample to randomly replace elements of different tuples in the positive sample to generate a second negative sample; The enhanced dataset is constructed according to the positive sample, the first negative sample and the second negative sample.
8. The device according to claim 7, characterized in that When the enhanced data set is used to train the pre-trained model, the loss function of an input sample in the enhanced data set is the average of the loss functions of each output sequence.
