A training data synthesis method and device for an event extraction task

Abstract

The present application relates to the technical field of natural language processing, in particular to a training data synthesis method and device for event extraction task. The training data synthesis method for event extraction task comprises the following steps: constructing an event list according to event extraction task information, and constructing a trigger word library; constructing a template according to the event list and the trigger word library to obtain positive data prompt templates and adversarial data prompt templates; obtaining first synthetic training data through a generated large language model after instruction fine-tuning; revising the first synthetic training data to obtain fourth synthetic training data; obtaining a weight set through a weight distribution model according to the fourth synthetic training data; sampling the fourth synthetic training data based on the weight set and a preset weight threshold, and combining sample data in the event list to obtain sixth synthetic training data. The present application is a training data synthesis method with good diversity and high data quality for event extraction task.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a method and apparatus for synthesizing training data for event extraction tasks. Background Technology

[0002] Event extraction is a technique in natural language processing (NLP) for extracting event information from text, widely used in fields such as healthcare, finance, and law. Event extraction extracts events of interest to users from unstructured natural language text and presents them in a structured manner. Currently, the industry primarily employs deep learning technology, training neural network models based on large-scale manually labeled data and extracting predefined event information from these models. Deep learning methods rely on sufficient and diverse training data, but labeling event data requires significant human and material resources, thus many methods face challenges in resource-constrained scenarios. To address the problem of insufficient training data, the current mainstream approach proposes using a small amount of data as samples and leveraging the generative capabilities of pre-trained models to synthesize training data.

[0003] The primary goal of data synthesis methods for event extraction is to efficiently synthesize accurate and diverse data containing event information. Currently, the main approaches are heuristic rule-based and generative model-based methods. Heuristic rule-based methods synthesize new data by replacing or modifying parts of the original samples using various rules, including entity modification and synonym replacement. However, these methods have significant drawbacks. The generated data lacks diversity, and synthesizing data based on only a few rules results in synthesized results that are semantically similar to the original data, hindering large-scale scalability. Furthermore, the synthesis results of this method have poor training performance, as the synthesized data can only provide a limited amount of new knowledge during training, thus offering limited improvement to the event extraction model.

[0004] Model-based methods utilize the generative capabilities of pre-trained languages ​​to synthesize new data. Language models trained on large-scale corpora contain rich linguistic knowledge and are adept at continuating and supplementing linguistic text. These methods primarily use models like BERT and T5, pre-trained via a masking-prediction task, to mask some non-trigger words and non-argument neighbors in the original sample data, and then synthesize new training data based on the new text fragments predicted by the model. Model-based data synthesis methods improve the quality of synthesized data compared to heuristic rule-based methods, but they still heavily rely on the original sample data and struggle to synthesize more structurally complex and semantically rich data, thus limiting their practical applications. Furthermore, the current study uses synthesized training data alongside standard data for training without more detailed sampling, resulting in limited performance improvements for the model.

[0005] In the existing technology, there is a lack of a training data synthesis method that offers good diversity and high data quality for event extraction tasks. Summary of the Invention

[0006] To address the problems of poor quality and low diversity of synthesized data in existing technologies, this invention provides a method and apparatus for synthesizing training data for event extraction tasks. The technical solution is as follows:

[0007] On the one hand, a method for synthesizing training data for an event extraction task is provided. This method is implemented by a training data synthesis device and includes:

[0008] Obtain event extraction task information; construct an event list based on the event extraction task information;

[0009] A template is constructed based on the event list to obtain a prompt template; the prompt template is then input into a generative large language model that has been fine-tuned by instructions to obtain a trigger word library;

[0010] Based on the event list and the trigger word library, templates are constructed to obtain positive data prompt templates and adversarial data prompt templates;

[0011] Based on the positive data prompt template and the adversarial data prompt template, the first synthetic training data is obtained through the generative large language model after fine-tuning the instructions;

[0012] Based on the trigger word library, the first synthetic training data is revised to obtain the fourth synthetic training data;

[0013] Based on the fourth synthetic training data, a weight set is obtained through a weight allocation model;

[0014] Based on the weight set and the preset weight threshold, the fourth synthetic training data is sampled to obtain the fifth synthetic training data; based on the fifth synthetic training data and the sample data in the event list, the sixth synthetic training data is obtained.

[0015] The event list includes the event type, event description, and sample data of the events to be extracted in the event extraction task.

[0016] Optionally, the step of obtaining the first synthetic training data based on the positive data cue template and the adversarial data cue template, through a generative large language model fine-tuned by instructions, includes:

[0017] The positive data prompt template is used to fine-tune the generative large language model after inputting the positive data prompt template to obtain positive training data.

[0018] The adversarial training data is obtained by fine-tuning the generative large language model using the adversarial data prompt template input command.

[0019] Based on the positive training data and the adversarial training data, the first synthetic training data is obtained.

[0020] Optionally, revising the first synthetic training data based on the trigger word library to obtain the fourth synthetic training data includes:

[0021] Based on the trigger word library, the first synthetic training data is labeled to obtain a candidate event list;

[0022] Based on the characteristics of error events and accompanying events, a template is constructed according to the candidate event list and the trigger word library to obtain the initial revision template;

[0023] Based on the initial revision template, the first synthetic training data is revised using a preset language judgment model to obtain the second synthetic training data;

[0024] Based on the characteristics of competing events, a template is constructed according to the candidate event list and the trigger word library to obtain a secondary revision template;

[0025] Based on the second revision template, the second synthetic training data is revised using a preset language judgment model to obtain the third synthetic training data;

[0026] Based on the characteristics of linguistic sentence structure, a template is constructed according to the candidate event list and the trigger word library to obtain the argument-argument role revision template;

[0027] Based on the argument-argument role revision template, the third synthetic training data is revised using a preset language judgment model to obtain the fourth synthetic training data.

[0028] Optionally, obtaining the weight set based on the fourth synthetic training data through a weight allocation model includes:

[0029] The fourth synthetic training data is vectorized into feature vectors to obtain event type vectors, trigger word vectors, argument vectors, and sentence vectors.

[0030] The event type vector is determined as the query vector;

[0031] The trigger word vector and the argument vector are concatenated to obtain the key vector;

[0032] Based on the query vector and the key vector, a first feature fusion vector is obtained by calculation using a bilinear model;

[0033] The first fusion vector and the sentence vector are concatenated to obtain the second feature fusion vector;

[0034] Based on the second feature fusion vector, weights are allocated using a weight allocation model to obtain a weight set.

[0035] The weight allocation model is constructed based on a multilayer perceptron structure.

[0036] On the other hand, a training data synthesis apparatus for an event extraction task is provided, which is applied to a training data synthesis method for an event extraction task. The apparatus includes:

[0037] The event list acquisition module is used to acquire event extraction task information and construct an event list based on the event extraction task information.

[0038] The trigger word acquisition module is used to construct a template based on the event list to obtain a prompt template; and input the prompt template into the generative large language model after fine-tuning the instructions to obtain the trigger word.

[0039] The template building module is used to build templates based on the event list and the trigger word library to obtain positive data prompt templates and adversarial data prompt templates.

[0040] The training data synthesis module is used to obtain the first synthesized training data based on the positive data prompt template and the adversarial data prompt template, through a generative large language model that has been fine-tuned by instructions.

[0041] The training data revision module is used to revise the first synthetic training data based on the trigger word library to obtain the fourth synthetic training data;

[0042] The weight set acquisition module is used to obtain a weight set based on the fourth synthetic training data and through a weight allocation model.

[0043] The training data sampling module is used to sample the fourth synthetic training data based on the weight set and a preset weight threshold to obtain the fifth synthetic training data; and to obtain the sixth synthetic training data based on the fifth synthetic training data and the sample data in the event list.

[0044] The event list includes the event type, event description, and sample data of the events to be extracted in the event extraction task.

[0045] Optionally, the training data synthesis module is further configured to:

[0046] The positive data prompt template is used to fine-tune the generative large language model after inputting the positive data prompt template to obtain positive training data.

[0047] The adversarial training data is obtained by fine-tuning the generative large language model using the adversarial data prompt template input command.

[0048] Based on the positive training data and the adversarial training data, the first synthetic training data is obtained.

[0049] Optionally, the training data revision module is further configured to:

[0050] Based on the trigger word library, the first synthetic training data is labeled to obtain a candidate event list;

[0051] Based on the characteristics of error events and accompanying events, a template is constructed according to the candidate event list and the trigger word library to obtain the initial revision template;

[0052] Based on the initial revision template, the first synthetic training data is revised using a preset language judgment model to obtain the second synthetic training data;

[0053] Based on the characteristics of competing events, a template is constructed according to the candidate event list and the trigger word library to obtain a secondary revision template;

[0054] Based on the second revision template, the second synthetic training data is revised using a preset language judgment model to obtain the third synthetic training data;

[0055] Based on the characteristics of linguistic sentence structure, a template is constructed according to the candidate event list and the trigger word library to obtain the argument-argument role revision template;

[0056] Based on the argument-argument role revision template, the third synthetic training data is revised using a preset language judgment model to obtain the fourth synthetic training data.

[0057] Optionally, the weight set acquisition module is further configured to:

[0058] The fourth synthetic training data is vectorized into feature vectors to obtain event type vectors, trigger word vectors, argument vectors, and sentence vectors.

[0059] The event type vector is determined as the query vector;

[0060] The trigger word vector and the argument vector are concatenated to obtain the key vector;

[0061] Based on the query vector and the key vector, a first feature fusion vector is obtained by calculation using a bilinear model;

[0062] The first fusion vector and the sentence vector are concatenated to obtain the second feature fusion vector;

[0063] Based on the second feature fusion vector, weights are allocated using a weight allocation model to obtain a weight set.

[0064] The weight allocation model is constructed based on a multilayer perceptron structure.

[0065] On the other hand, a training data synthesis device is provided, the training data synthesis device comprising: a processor; a memory storing computer-readable instructions, wherein when the computer-readable instructions are executed by the processor, any one of the methods described above for training data synthesis of event extraction tasks.

[0066] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement any of the above-described methods for training data synthesis for event extraction tasks.

[0067] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0068] This invention proposes a training data synthesis method for event extraction tasks. By synthesizing positive and adversarial training data, the robustness and accuracy of the trained model can be improved. Through multiple revisions, data quality is enhanced, addressing issues such as accompanying events, competing events, and erroneous events in the synthesized data. A dynamic sampling method for the synthesized training data is proposed, achieving high-quality data sampling based on multi-source fusion features and an instance-level weight allocation model, further improving the performance of the trained model. This invention can synthesize more diverse, higher-quality, and infinitely numerous event extraction training data, avoiding the problem of poor diversity caused by over-reliance on original samples, and improving event extraction performance under low resource conditions. This invention is a training data synthesis method with good diversity and high data quality for event extraction tasks. Attached Figure Description

[0069] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0070] Figure 1 This is a flowchart of a training data synthesis method for an event extraction task provided by an embodiment of the present invention;

[0071] Figure 2 This is a block diagram of a training data synthesis device for an event extraction task provided in an embodiment of the present invention;

[0072] Figure 3 This is a schematic diagram of the structure of a training data synthesis device provided in an embodiment of the present invention. Detailed Implementation

[0073] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0074] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0075] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, their intended meanings are consistent. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, their intended meanings are consistent.

[0076] In this embodiment of the invention, sometimes a subscript such as W1 may be mistakenly written as a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0077] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0078] This invention provides a method for synthesizing training data for an event extraction task. This method can be implemented by a training data synthesis device, which can be a terminal or a server. Figure 1 The flowchart shown is for a training data synthesis method for an event extraction task. The processing flow of this method may include the following steps:

[0079] S1. Obtain event extraction task information; construct an event list based on the event extraction task information.

[0080] The event list includes the event type, event description, and sample data of the events to be extracted in the event extraction task.

[0081] In one possible implementation, in the event list marked E in this invention, event e i For an element of this list, e iBy triplet<et,des,sam> Composition. To facilitate data parsing and synthesis in subsequent steps, the information in this step is stored in JSON text format.

[0082] Event e i The specific form is as follows:

[0083] {"et": "birth", "des": "the origin of life, usually a person or an animal", "sam": [{"text": "he was born in a beautiful village", "trigger word": "birth", "argument": {"subject": "he"}}]}.

[0084] In the example above, et represents the event type, which can be a specific event such as birth or marriage.

[0085] Des stands for Event Description, which is the specific definition of the event type. The event description for a birth event is: the emergence of life, usually involving a person or an animal. Event descriptions are generally derived from the documentation explaining the event extraction task or are set according to application requirements.

[0086] sam represents a small amount of raw sample data, which serves as a starting example for the subsequent synthesis process. The sample data is manually annotated, and each data point contains complete event information, namely the event type, event trigger word, argument, and argument role appearing in the text.

[0087] S2. Construct a template based on the event list to obtain a prompt template; input the prompt template into the generative large language model after fine-tuning the instructions to obtain the trigger vocabulary.

[0088] In one feasible implementation, the event list is traversed iteratively, retrieving event e in each iteration. i According to e i Construct prompt templates for a synthetic trigger word library. The prompt templates are built based on a preset original template, which is prompt text with slots, as shown below:

[0089] "Please generate event trigger words for <Slot 1> that match the event description below, with the event description as: <Slot 2>. Example: Trigger word: <Slot 3>"

[0090] The slots in the original template are using the retrieved e i Replace and update; the updated prompt template is shown below:

[0091] "Please generate 5 event trigger words that match the following event description and example. Event description: The birth of life, usually the subject is a person or an animal. Example: Trigger word: birth"

[0092] The updated template is input into the finely tuned generative large language model to trigger lexicon generation, and the event list is updated to quadruples.<et,des,sam,tr> `tr` represents the trigger word library for this event. The number of trigger words can be flexibly adjusted in this step according to the actual application needs.

[0093] A language model fine-tuned by instructions is a language model that has been fine-tuned on an instruction dataset, such as the Chat Generative Pre-trained Transformer (ChatGPT) or the General Language Model Dialogue System (ChatGLM). The instruction dataset used for instruction fine-tuning contains various task instructions and labeled results, so the above models can complete tasks based on prompts and generalize the prompt information to other uncommon tasks.

[0094] Taking ChatGPT as an example, it is an autoregressive language model based on the Transformer architecture. Unlike bidirectional encoder representations from transformers (BERT) or text-to-text transfer transformers (T5), it is not a single encoder or encoder-decoder structure model, but a unidirectional attention language model with a single decoder structure. It uses the next word as the pre-training target, is pre-trained on large-scale natural language text, has a much larger number of parameters than BERT and T5, and after fine-tuning, it can follow the preset instructions in the original template to synthesize the required trigger words.

[0095] S3. Based on the event list and trigger word library, construct templates to obtain positive data prompt templates and adversarial data prompt templates.

[0096] In one feasible implementation, to synthesize the two training data sets, a complete prompt template should first be constructed and then fed into the language model for generation. The updated E is iteratively traversed, retrieving event e in each iteration. i Then iterate through e. i tr in i Represents the current trigger word, based on e i and tr i Construct a prompt template corresponding to the training data.

[0097] The original template format for positive training data is as follows:

[0098] Positive training data: "Please refer to the example to generate <slot 2> sentences that match the event description below for <slot 1>, event description: <slot 3>. Example: Trigger words: <slot 3>, <slot 4>"

[0099] The original template format of adversarial training data is as follows:

[0100] Adversarial training data: "Please use <slot 1> to generate <slot 2> sentences that do not conform to or express the event description below, event description: <slot 3>."

[0101] e i and tr i Enter the information into the slots of the above template to obtain the following prompt template:

[0102] Positive training data: "Please refer to the example to generate 5 sentences that match the following event description: the birth of life, usually a person or an animal. Example: Trigger word: birth, he was born in a beautiful village."

[0103] Adversarial training data: "Please generate 3 sentences that do not conform to or express the following event description: the birth of life, the subject is usually a human or an animal."

[0104] S4. Based on the positive data prompt template and the adversarial data prompt template, the first synthetic training data is obtained through the generative large language model after fine-tuning the instructions.

[0105] Optionally, based on the positive data cue template and the adversarial data cue template, the first synthetic training data is obtained through a generative large language model fine-tuned by instructions, including:

[0106] The generative large language model, after being fine-tuned by inputting positive data prompt templates, obtains positive training data.

[0107] The adversarial training data is obtained by fine-tuning the generative large language model with adversarial data prompt template input instructions.

[0108] The first synthetic training data is obtained based on the frontal training data and the adversarial training data.

[0109] In one feasible implementation, positive training data in this invention refers to data containing a certain trigger word and conforming to a specific event type; adversarial training data refers to data containing a certain trigger word but not conforming to that event type.

[0110] Take the "birth" event as an example:

[0111] Positive data: {"Sentence": "The birth of the child brought endless joy to the family", "Trigger word": "birth"};

[0112] Adversarial data: {"Sentence": "A brilliant idea was born in his mind", "Trigger word": "born"};

[0113] Positive training data and adversarial training data can significantly improve the accuracy and robustness of event extraction models during training.

[0114] The prompt template is input into the language model to obtain the synthesized training data. This process is iterated until all trigger words in E are exhausted. The model's returned results do not include the start and end indices of the trigger words. Therefore, a regular expression matching tool is used to add the start and end positions of the first matched trigger words to the training data. The forms of the positive and adversarial training data are shown below:

[0115] Positive training data: {"Sentence": "The birth of the child brought endless joy to the family", "Trigger word": ["birth", 3, 4]};

[0116] Adversarial training data: {"Sentence": "A brilliant idea was born in his mind", "Trigger word": ["Born", 13, 14]}.

[0117] S5. Based on the trigger word library, revise the first synthetic training data to obtain the fourth synthetic training data.

[0118] Optionally, based on the trigger lexicon, the first synthetic training data is revised to obtain the fourth synthetic training data, including:

[0119] Based on the trigger word library, the first synthetic training data is labeled to obtain a list of candidate events;

[0120] Based on the characteristics of error events and accompanying events, a template is constructed according to the candidate event list and trigger word library to obtain the initial revision template;

[0121] Based on the initial revision template, the first synthetic training data is revised using a preset language judgment model to obtain the second synthetic training data;

[0122] Based on the characteristics of competing events, a template is constructed according to the candidate event list and the trigger word library to obtain a second-revision template;

[0123] Based on the second revision template, the second synthetic training data is revised using a preset language judgment model to obtain the third synthetic training data;

[0124] Based on the characteristics of linguistic sentence structure, templates are constructed according to the candidate event list and trigger word library to obtain argument-argument role revision templates;

[0125] Based on the argument-argument role revision template, the third synthetic training data is revised using a preset language judgment model to obtain the fourth synthetic training data.

[0126] In one feasible implementation, the training data obtained in the above steps is relatively coarse and has the following three problems: there are accompanying events in the synthesized training data, erroneous events in the synthesized training data, and competing events in the synthesized training data.

[0127] When constructing prompt templates, type synthesis is only performed on trigger words and events. However, in reality, the synthesized data often contains a large number of accompanying events. For example, in the synthesis of the "birth" event, using "birth" as the trigger word, the following sentence might be synthesized: "In the second year of their marriage, their eldest son was born."

[0128] The composite data above includes not only the "birth" event but also the accompanying event "marriage," triggered by the word "marriage." In real-world applications, accompanying events often occur alongside the main composite event.

[0129] There is a chance of errors when the model generates results, which may cause the trigger words and event types to not match, resulting in erroneous events in the synthesized data.

[0130] Regarding competition events, in fine-grained event types, some trigger words may appear in e i In, e can also appear. j In this context, when two events are in a race condition, it is necessary to determine the specific event type.

[0131] Iterate through each e in the first synthetic training data E i The trigger word library is used to determine whether a trigger word appears in the data using regular expressions. If it exists, the event to which the trigger word belongs is set (et∈e). i The candidate events are marked as such, and the candidate events are collected to obtain a list of candidate events.

[0132] To address the issues of error events and accompanying events in the synthetic data, the candidate event list is traversed, and the following template is constructed based on the event type and trigger word of the current event:

[0133] Does the word "<slot 1>" enclosed in $$ in the following sentence match the event description? Event description: <slot 2>. Sentence: <slot 3>.

[0134] The above is updated to obtain the template as follows:

[0135] Does the word "birth" enclosed in $$ in the following sentence match the event description? Event description: The birth of life, usually the subject is a person or an animal. Sentence: The birth of the child brought endless joy to the family.

[0136] In this step, positive training data and adversarial training data are not distinguished. The above template is fed into a preset language judgment model to obtain a yes or no result. If yes, the event is retained; otherwise, the event is deleted.

[0137] In the second revision, the problem of competing events is addressed by iterating through the candidate event list, retrieving all competing events from the list, and constructing the following template:

[0138] "In the sentence below, the word '<slot 1>' enclosed in $$ best matches which of the following event descriptions? Event description: A: <slot 2>; B: <slot 3>... Sentence: <slot 4>."

[0139] After the template is updated, it will look like this:

[0140] "In the sentence below, the word 'birth' enclosed in $$ best describes which of the following events? Event description: A: The emergence of life, usually a person or animal; B: The appearance of something new. Sentence: A brilliant idea was born in his mind."

[0141] The candidate event list is updated based on the results generated by the language model. If the answer is A, the "birth" event is retained and the "appearance" event is discarded, and vice versa.

[0142] After revising the training data based on trigger words for event extraction, high-quality synthetic data is obtained. Then, the arguments and argument roles in the training data are determined. This maximizes the accuracy of arguments and argument roles in the pipeline synthesis mode.

[0143] Based on the event type and trigger word of the current event, construct the following template:

[0144] "Please identify the corresponding slots 2, 3, ... in the sentences below where the word <slot 1> is enclosed in $$. If it does not exist, please indicate with 'none'. Sentence: <slot 4>. Example: <slot 5>, <slot 6>, <slot 7>."

[0145] The updated template is shown below:

[0146] "Please identify the subject corresponding to the word 'birth' enclosed in $$ in the following sentence, and indicate 'none' if it does not exist. Sentence: The birth of the child brought endless joy to the family. Example: He was born in a beautiful village. Subject: He."

[0147] Using a regular expression matching tool, the start and end positions of the words that first match the arguments are added to the training data:

[0148] {"Sentence": "The birth of the child brought endless joy to the family", "Trigger word": ["birth", 3, 4], Argument: [["subject", 0, 1]]}.

[0149] After this step is completed, the fourth synthetic training data, which has undergone a complete revision process, is obtained.

[0150] S6. Based on the fourth synthetic training data, obtain the weight set through the weight allocation model.

[0151] Optionally, based on the fourth synthetic training data, a weight set is obtained through a weight allocation model, including:

[0152] The fourth synthetic training data is vectorized into feature vectors to obtain event type vectors, trigger word vectors, argument vectors, and sentence vectors.

[0153] Use the event type vector as the query vector;

[0154] The trigger word vector and argument vector are concatenated to obtain the key vector;

[0155] Based on the query vector and key vector, the first feature fusion vector is obtained by calculating using a bilinear model;

[0156] The first fusion vector and the sentence vector are concatenated to obtain the second feature fusion vector;

[0157] Based on the second feature fusion vector, weights are allocated through a weight allocation model to obtain a weight set.

[0158] In one possible implementation, the revised fourth synthetic training data S syn While achieving high accuracy, the event extraction model inevitably introduces noise as the scale of synthetic training data increases, negatively impacting its performance. Therefore, this invention proposes a multi-source feature fusion module, Mix:S syn →P can be obtained from the fourth synthetic training data, which contains fused features P.

[0159] Based on the aforementioned feature fusion module, this invention constructs an instance-level weight allocation model g(x, θ), which can obtain the weight score β of each synthetic training data instance. Where β = g(Mix(S) syn The weight scores β of all training data instances together constitute the weight set.

[0160] In sampling synthetic data, it is necessary to remove noisy data as much as possible and retain high-quality data. In practice, the inventors found that the interrelationships between trigger words, arguments, and event types are closely related to the quality of synthetic data. The above features and sentence information are fused and then transmitted to the weight allocation model.

[0161] Instance s of obtaining the fourth synthetic training data i , for s i The sentences, trigger words, arguments, and argument roles in the text are assembled according to the following pattern. The format is as follows:

[0162] Subject: Child;

[0163] Will The input is encoded into a pre-trained language model such as BERT to obtain a vector representation, including the trigger word E. tri Argument On meta-role Where n is the number of arguments, and then... and Add them together to obtain the final vector of arguments.

[0164] Initialize the learnable type vector. Where t is the maximum number of event types. According to s i Event type, retrieve event type vector And treat it as the query vector q, and set E tri and E arg The concatenation is used as the key value k and transmitted to the bilinear model.

[0165] After obtaining q and k, the attention score α is calculated using a bilinear model, and the expected vector under this attention distribution is obtained based on the score. The calculation formulas for this process are as follows: (1), (2), (3), (4), (5):

[0166]

[0167] α=softmax(score(k,q)) (2)

[0168] Score(k, q) = k T Wq (3)

[0169]

[0170]

[0171] Where J is the length of the key value k, score is the scoring function, and W is a learnable parameter, ultimately yielding the first feature fusion vector p. i =attention(k, q).

[0172] The first feature fusion vector, which integrates trigger words, arguments, argument roles, and event types, and the sentence vector are concatenated to form the second feature fusion vector P. i The final multi-source feature vector is transmitted to the weight allocation model: P i =[p i [cls]]. Here, [cls] is the sentence vector obtained during the encoding stage.

[0173] The weight allocation model is based on a multilayer perceptron structure for model construction.

[0174] In one feasible implementation, we feed the multi-source feature vectors as input into the weight allocation model and calculate the weight score for each instance. The allocation model is expressed by the following formula (6):

[0175] g(x)=sigmoid(MLP(x)) (6)

[0176] In this context, MLP stands for Neural Network, and sigmoid is the activation function. Using multi-source features as input, the final weight g(Mix(x)) ranges from [0, 1].

[0177] In practical applications, the following two instances of birth events, "A brilliant idea was born in his mind" and "His arrival brought new hope to the family," are both noise samples in the synthetic training data, contradicting the type in terms of the argument: ["subject": "idea,"] and the trigger word: "arrival," respectively. After the multi-source feature fusion module, their multi-source feature vectors can capture these connections. Through weight allocation model calculations, it can be seen that the weights of the two instances are close to 0.

[0178] To optimize the parameters of the weight allocation model during multi-round synthetic training data sampling, a small standard dataset S with balanced event types is constructed based on the limited amount of original sample data mentioned earlier. gold , will S syn The dataset is randomly divided into y subsets, where the event types in each subset are kept balanced. syn =[s syn_1 , ..., s syn_y The above training data is used to optimize the allocation model parameters for y rounds.

[0179] In each round of parameter optimization, the weight allocation model is adjusted from a subset of the dataset. A classification network f(x, w) is built to learn trigger words and arguments. In the learning of this classification network, a weighted loss function is used to assign weight scores to the loss of each data instance, and its complete loss function is shown in equations (7) and (8):

[0180]

[0181]

[0182] Where l is the cross-entropy loss function, and in each batch of iterations, we fix the weights and assign the model parameters θ, and update the weights w at time t according to the gradient. t For w t+1 .

[0183] Then, based on the classification network f(x, w) at time t+1 t+1 ), in S gold Optimize θ, and maintain w during optimization. t+1 The loss function remains unchanged. At this point, the loss function is as follows: (9) and (10):

[0184]

[0185]

[0186] In each batch, w and θ are updated sequentially, and g(x) learns the optimal parameters, identifying which multi-source feature vectors represent synthetic training data with potential noise. The learning process of the assignment network is iteratively performed across multiple rounds, ensuring that g(x) avoids falling into biased local optima.

[0187] S7. Based on the weight set and the preset weight threshold, sample the fourth synthetic training data to obtain the fifth synthetic training data; based on the fifth synthetic training data and the sample data in the event list, obtain the sixth synthetic training data.

[0188] In one feasible implementation, after the weight allocation model has been learned, dynamic data sampling is achieved using g(x). Specifically, the fourth synthetic training data S... syn As input, via β=g(Mix(S) syn Obtain the weight set, and then determine whether to retain or discard the synthetic training data based on the weight score and threshold of each instance in the weight set. The preset weight threshold can be dynamically adjusted according to the experimental results.

[0189] Based on the aforementioned dynamic sampling system that integrates multi-source features, it is possible to perform sampling on S. syn Dynamic sampling is implemented to obtain the final fifth synthetic training data S. good Put S good and sample data S goldBy combining the best synthetic data, we obtain the sixth synthetic training data.

[0190] This invention proposes a training data synthesis method for event extraction tasks. By synthesizing positive and adversarial training data, the robustness and accuracy of the trained model can be improved. Through multiple revisions, data quality is enhanced, addressing issues such as accompanying events, competing events, and erroneous events in the synthesized data. A dynamic sampling method for the synthesized training data is proposed, achieving high-quality data sampling based on multi-source fusion features and an instance-level weight allocation model, further improving the performance of the trained model. This invention can synthesize more diverse, higher-quality, and infinitely numerous event extraction training data, avoiding the problem of poor diversity caused by over-reliance on original samples, and improving event extraction performance under low resource conditions. This invention is a training data synthesis method with good diversity and high data quality for event extraction tasks.

[0191] Figure 2 This is a block diagram illustrating a training data synthesis apparatus for an event extraction task, according to an exemplary embodiment. The apparatus is used in a method for synthesizing training data for an event extraction task. (Refer to...) Figure 2 The device includes an event list acquisition module 210, a trigger lexicon acquisition module 220, a template construction module 230, a training data synthesis module 240, a training data revision module 250, a weight set acquisition module 260, and a training data sampling module 270, wherein:

[0192] The event list acquisition module 210 is used to acquire event extraction task information and construct an event list based on the event extraction task information.

[0193] The trigger word acquisition module 220 is used to construct templates based on the event list to obtain prompt templates; the prompt templates are then input into the generative large language model after fine-tuning the instructions to obtain the trigger word library;

[0194] Template building module 230 is used to build templates based on the event list and trigger word library to obtain positive data prompt templates and adversarial data prompt templates.

[0195] The training data synthesis module 240 is used to obtain the first synthesized training data based on the positive data prompt template and the adversarial data prompt template, through the generative large language model after fine-tuning by instructions.

[0196] The training data revision module 250 is used to revise the first synthetic training data based on the trigger word library to obtain the fourth synthetic training data;

[0197] The weight set acquisition module 260 is used to obtain the weight set based on the fourth synthetic training data through the weight allocation model;

[0198] The training data sampling module 270 is used to sample the fourth synthetic training data based on the weight set and the preset weight threshold to obtain the fifth synthetic training data; and to obtain the sixth synthetic training data based on the fifth synthetic training data and the sample data in the event list.

[0199] The event list includes the event type, event description, and sample data of the events to be extracted in the event extraction task.

[0200] Optionally, the training data synthesis module 240 is further used for:

[0201] The generative large language model, after being fine-tuned by inputting positive data prompt templates, obtains positive training data.

[0202] The adversarial training data is obtained by fine-tuning the generative large language model with adversarial data prompt template input instructions.

[0203] The first synthetic training data is obtained based on the frontal training data and the adversarial training data.

[0204] Optionally, the training data revision module 250 is further used for:

[0205] Based on the trigger word library, the first synthetic training data is labeled to obtain a list of candidate events;

[0206] Based on the characteristics of error events and accompanying events, a template is constructed according to the candidate event list and trigger word library to obtain the initial revision template;

[0207] Based on the initial revision template, the first synthetic training data is revised using a preset language judgment model to obtain the second synthetic training data;

[0208] Based on the characteristics of competing events, a template is constructed according to the candidate event list and the trigger word library to obtain a second-revision template;

[0209] Based on the second revision template, the second synthetic training data is revised using a preset language judgment model to obtain the third synthetic training data;

[0210] Based on the characteristics of linguistic sentence structure, templates are constructed according to the candidate event list and trigger word library to obtain argument-argument role revision templates;

[0211] Based on the argument-argument role revision template, the third synthetic training data is revised using a preset language judgment model to obtain the fourth synthetic training data.

[0212] Optionally, the weight set acquisition module 260 is further used for:

[0213] The fourth synthetic training data is vectorized into feature vectors to obtain event type vectors, trigger word vectors, argument vectors, and sentence vectors.

[0214] Use the event type vector as the query vector;

[0215] The trigger word vector and argument vector are concatenated to obtain the key vector;

[0216] Based on the query vector and key vector, the first feature fusion vector is obtained by calculating using a bilinear model;

[0217] The first fusion vector and the sentence vector are concatenated to obtain the second feature fusion vector;

[0218] Based on the second feature fusion vector, weights are allocated through a weight allocation model to obtain a weight set.

[0219] The weight allocation model is based on a multilayer perceptron structure for model construction.

[0220] This invention proposes a training data synthesis method for event extraction tasks. By synthesizing positive and adversarial training data, the robustness and accuracy of the trained model can be improved. Through multiple revisions, data quality is enhanced, addressing issues such as accompanying events, competing events, and erroneous events in the synthesized data. A dynamic sampling method for the synthesized training data is proposed, achieving high-quality data sampling based on multi-source fusion features and an instance-level weight allocation model, further improving the performance of the trained model. This invention can synthesize more diverse, higher-quality, and infinitely numerous event extraction training data, avoiding the problem of poor diversity caused by over-reliance on original samples, and improving event extraction performance under low resource conditions. This invention is a training data synthesis method with good diversity and high data quality for event extraction tasks.

[0221] Figure 3 This is a schematic diagram of the structure of a training data synthesis device provided in an embodiment of the present invention, as shown below. Figure 3 As shown, the training data synthesis device may include the above-mentioned Figure 2 The illustrated training data synthesis apparatus for an event extraction task. Optionally, the training data synthesis apparatus 310 may include a first processor 2001.

[0222] Optionally, the training data synthesis device 310 may also include a memory 2002 and a transceiver 2003.

[0223] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0224] The following is combined with Figure 3A detailed description of each component of the training data synthesis device 310 is provided below:

[0225] The first processor 2001 is the control center of the training data synthesis device 310. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0226] Optionally, the first processor 2001 can perform various functions of the training data synthesis device 310 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0227] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 3 CPU0 and CPU1 are shown in the diagram.

[0228] In a specific implementation, as one example, the training data synthesis device 310 may also include multiple processors, for example... Figure 3 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, "processor" can refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0229] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0230] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be connected via the interface circuit of the training data synthesis device 310. Figure 3 (Not shown in the figure) is coupled to the first processor 2001, and the embodiments of the present invention do not specifically limit this.

[0231] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0232] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 3 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the sending function.

[0233] Optionally, the transceiver 2003 can be integrated with the first processor 2001, or it can exist independently and be connected to the interface circuit of the training data synthesis device 310. Figure 3 (Not shown in the figure) is coupled to the first processor 2001, and the embodiments of the present invention do not specifically limit this.

[0234] It should be noted that, Figure 3 The structure of the training data synthesis device 310 shown does not constitute a limitation on the router. Actual knowledge structure recognition devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0235] Furthermore, the technical effect of the training data synthesis device 310 can be referred to the technical effect of the training data synthesis method for event extraction tasks described in the above method embodiments, and will not be repeated here.

[0236] It should be understood that the first processor 2001 in this embodiment of the invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0237] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0238] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0239] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0240] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0241] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0242] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0243] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0244] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0245] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0246] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0247] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0248] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for synthesizing training data for an event extraction task, characterized in that, The method includes: Obtain event extraction task information; construct an event list based on the event extraction task information; A prompt template is constructed based on the event list to obtain a prompt template; the prompt template is then input into a generative large language model that has been fine-tuned by instructions to obtain a trigger word library; Based on the event list and the trigger word library, templates are constructed to obtain positive data prompt templates and adversarial data prompt templates; Based on the positive data prompt template and the adversarial data prompt template, the first synthetic training data is obtained through the generative large language model after instruction fine-tuning; Based on the trigger vocabulary, the first synthetic training data is revised to obtain the fourth synthetic training data, including: Based on the trigger word library, the first synthetic training data is labeled to obtain a candidate event list; Based on the characteristics of error events and accompanying events, a template is constructed according to the candidate event list and the trigger word library to obtain the initial revision template; Based on the initial revision template, the first synthetic training data is revised using a preset language judgment model to obtain the second synthetic training data; Based on the characteristics of competing events, a template is constructed according to the candidate event list and the trigger word library to obtain a secondary revision template; Based on the secondary revision template, the second synthetic training data is revised using a preset language judgment model to obtain the third synthetic training data; Based on the characteristics of linguistic sentence structure, a template is constructed according to the candidate event list and the trigger word library to obtain the argument-argument role revision template; Based on the argument-argument role revision template, the third synthetic training data is revised using a preset language judgment model to obtain the fourth synthetic training data; Based on the fourth synthetic training data, a weight set is obtained through a weight allocation model; Based on the weight set and the preset weight threshold, the fourth synthetic training data is sampled to obtain the fifth synthetic training data; based on the fifth synthetic training data and the sample data in the event list, the sixth synthetic training data is obtained.

2. The method for synthesizing training data for an event extraction task according to claim 1, characterized in that, The event list includes the event type, event description, and sample data of the events to be extracted in the event extraction task.

3. The method for synthesizing training data for an event extraction task according to claim 1, characterized in that, The first synthetic training data is obtained by using a generative large language model fine-tuned by instructions based on the positive data cue template and the adversarial data cue template, including: The positive data prompt template is input into the generative large language model after fine-tuning the instruction to obtain positive training data. The adversarial data prompt template is input into the fine-tuned generative large language model to obtain adversarial training data. Based on the positive training data and the adversarial training data, the first synthetic training data is obtained.

4. The method for synthesizing training data for an event extraction task according to claim 1, characterized in that, The step of obtaining a weight set based on the fourth synthetic training data through a weight allocation model includes: The fourth synthetic training data is vectorized into feature vectors to obtain event type vectors, trigger word vectors, argument vectors, and sentence vectors. The event type vector is determined as the query vector; The trigger word vector and the argument vector are concatenated to obtain the key vector; Based on the query vector and the key vector, a first feature fusion vector is obtained by calculating using a bilinear model; The first fusion vector and the sentence vector are concatenated to obtain the second feature fusion vector; Based on the second feature fusion vector, weights are allocated using a weight allocation model to obtain a weight set.

5. The method for synthesizing training data for an event extraction task according to claim 4, characterized in that, The weight allocation model is constructed based on a multilayer perceptron structure.

6. A training data synthesis apparatus for an event extraction task, wherein the training data synthesis apparatus for an event extraction task is used to implement any one of the training data synthesis methods for an event extraction task as described in claims 1-5, characterized in that, The device includes: The event list acquisition module is used to acquire event extraction task information and construct an event list based on the event extraction task information. The trigger word acquisition module is used to construct a template based on the event list to obtain a prompt template; and input the prompt template into the generative large language model after fine-tuning the instructions to obtain the trigger word. The template building module is used to build templates based on the event list and the trigger word library to obtain positive data prompt templates and adversarial data prompt templates. The training data synthesis module is used to obtain the first synthesized training data based on the positive data prompt template and the adversarial data prompt template, through a generative large language model that has been fine-tuned by instructions. The training data revision module is used to revise the first synthetic training data based on the trigger word library to obtain the fourth synthetic training data; The weight set acquisition module is used to obtain a weight set based on the fourth synthetic training data and through a weight allocation model. The training data sampling module is used to sample the fourth synthetic training data based on the weight set and a preset weight threshold to obtain the fifth synthetic training data; and to obtain the sixth synthetic training data based on the fifth synthetic training data and the sample data in the event list.

7. A training data synthesis device, characterized in that, The training data synthesis device includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 5.

Images