Method and device for performing data enhancement on event extraction training data set

Abstract

The embodiment of the invention relates to a method and a device for carrying out data enhancement on an event extraction training data set. The method comprises the following steps: marking a public data set as a first data set; forming a corresponding first model set by N pre-trained large language models; performing fine tuning training on each first large language model of the first model set based on the first data set; after fine tuning training of the first model set is finished, event information templates of various event types in the first data set are identified to obtain a corresponding first event information template set, and data set generation processing is carried out according to the first model set and the first event information template set to obtain a corresponding second data set; and performing sample synthesis processing according to the first data set and the second data set to obtain a corresponding third data set, and taking the third data set as an enhanced data set and storing the enhanced data set. According to the invention, the construction period of the data set can be shortened, and the construction cost of the data set is reduced.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method and device for performing data enhancement on an event extraction training data set. Background Art

[0002] Common large language models (LLMs) include BERT series models, BART series models, GPT series models, T5 series models, Wenxin series models, etc. These large language models are widely used in a variety of natural language processing (NLP) tasks, and event extraction (EE) tasks are one of the typical tasks. The training phase of large language models used to process EE tasks is roughly divided into two steps: 1) pre-training phase based on massive text datasets, which is to train the model's natural language understanding and text generation capabilities through text corpora; 2) fine-tuning training phase based on event extraction datasets, which uses massive event label corpora of event extraction datasets (datasets used for event extraction training) to improve the model's EE task processing performance. At present, the construction of such event extraction datasets is mostly done manually. Affected by human resource factors, this traditional manual processing mode will inevitably encounter challenges such as long processing cycles and high processing costs when preparing massive event label corpora. In view of this, how to shorten the construction cycle of event extraction datasets and reduce the construction cost has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The purpose of the present invention is to provide a method, device, electronic device and computer-readable storage medium for data enhancement of event extraction training data set in view of the defects of the prior art. The present invention records a public data set for event extraction training as the corresponding first data set, and a corresponding first model set is composed of multiple large language models that have completed pre-training, and each first large language model is fine-tuned based on the first data set; and after the fine-tuning training of the first model set is completed, the event information templates of various event types in the first data set are identified to obtain the corresponding first event information template set, and the data set generation processing is performed according to the first model set and the first event information template set to obtain the corresponding second data set, and the sample synthesis processing is performed according to the first and second data sets to obtain the corresponding third data set, and the third data set is used as an enhanced data set for event extraction training and saved. It can be seen that, on the one hand, the present invention uses multiple types of large language models with event extraction functions to automatically generate corpora to obtain corresponding automatically generated data sets (second data sets); on the other hand, it also constructs a synthetic data set (third data set) with larger data volume and better accuracy by mixing corpus labels of public data sets and automatically generated data sets; on the other hand, it also improves the data balance of the synthetic data set by supplementing the number of data records of a small number of event types; the present invention can ensure the data quality of the data set, shorten the construction cycle of the data set, and reduce the construction cost of the data set.

[0004] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present invention provides a method for data enhancement of an event extraction training data set, the method comprising:

[0005] A public data set for event extraction training is recorded as a corresponding first data set; N pre-trained large language models are combined into a corresponding first model set; and fine-tuning training is performed on each first large language model of the first model set based on the first data set; N is a preset first number, and the first number N is a positive integer; the first model set includes N first large language models;

[0006] After the fine-tuning training of the first model set is completed, the event information templates of each event type in the first data set are identified to obtain the corresponding first event information template set; and a data set generation process is performed based on the first model set and the first event information template set to obtain the corresponding second data set; and a sample synthesis process is performed based on the first and second data sets to obtain the corresponding third data set; and the third data set is used as an enhanced data set for event extraction training and saved.

[0007] Preferably, the public data set includes at least the ACE2005 data set released by the Language Data Alliance, the event extraction data set provided by the Kaggle / Google / GitHub / HDX / FiveThirtyEight platform, and the event extraction data set released by the Chinese Data Alliance;

[0008] The first language model includes at least BERT series models, BART series models, GPT series models, T5 series models, and Wenxin series models;

[0009] The first data set includes multiple first data records; the first data record includes a first event text and a first event label; the first event label includes a first event type label, a first trigger word label and multiple first element labels; the first element label includes a first element type label and a first element text label; the first event type label is the event type corresponding to the first event text; the first trigger word label is the trigger word text in the first event text that triggers the current type of event; each of the first element labels corresponds to an event element of the current event type; the first element type label is the event element type corresponding to the current event element; the first element text label is the element text in the first event text that corresponds to the current event element;

[0010] The first event information template set includes a plurality of first event information templates; each of the first event information templates corresponds to a class of event types; the first event information template includes a first event type and a plurality of first event element types;

[0011] The second data set includes a plurality of second data records; the second data records include a second event text and a second event tag; the second event tag includes a second event type tag, a second trigger word tag and a plurality of second element tags; the second element tag includes a second element type tag and a second element text tag;

[0012] The third data set includes multiple third data records; the third data record includes a third event text and a third event label; the third event label includes a third event type label, a third trigger word label and multiple third element labels; the third element label includes a third element type label and a third element text label.

[0013] Preferably, the fine-tuning training of each first large language model of the first model set based on the first data set specifically includes:

[0014] The first data set is divided into two data subsets according to a preset first segmentation ratio, which are recorded as a corresponding first training set and a first evaluation set; and a round of training is performed on the event extraction task processing function of each of the first large language models based on the first training set; and when each of the first large language models completes a round of training, the event extraction task processing performance of the current large language model is evaluated based on the first evaluation set to obtain a corresponding model evaluation result for that time, and when the model evaluation result for that time does not meet the standard, another round of training is performed on the current large language model based on the first training set until the latest model evaluation result for that time meets the standard; and after all the first large language models in the first model set meet the standard, it is confirmed that the fine-tuning training of the first model set is completed; the first training set and the first evaluation set are both composed of multiple first data records; the ratio of the total number of records in the first training set to the total number of records in the first evaluation set satisfies the first segmentation ratio.

[0015] Preferably, identifying the event information templates of various event types in the first data set to obtain the corresponding first event information template set specifically includes:

[0016] Counting the event type types of all the first event type labels in the first data set to obtain a corresponding first event type set; the first event type set includes multiple types of the first event types;

[0017] and grouping all the first data records in the first data set whose first event type labels match the same type of the first event type into one category to form a corresponding first category record set;

[0018] and obtaining a corresponding first event element type set by counting the event element type types of all the first element type tags in each of the first type record sets; the first event element type set includes multiple types of the first event element types;

[0019] Each first event type and all the first event element types of its corresponding first event element type set form a corresponding first event information template;

[0020] And all the obtained first event information templates form a corresponding first event information template set.

[0021] Preferably, the step of performing data set generation processing according to the first model set and the first event information template set to obtain the corresponding second data set specifically includes:

[0022] Step 51, taking the first of the first event information templates in the first event information template set as the corresponding current event information template;

[0023] Step 52, initializing the count value of the first counter to 1;

[0024] Step 53, taking the first of the first large language models in the first model set as the corresponding current large language model;

[0025] Step 54, bringing the current event information template into the first event text generation instruction template corresponding to the current large language model to perform corresponding event text generation instruction assembly to obtain the corresponding first instruction text; and inputting the first instruction text into the current large language model to perform corresponding creative event text generation task processing and using the generated text output by the model as the corresponding first event text;

[0026] Among them, the first event text generation instruction template corresponding to each of the first large language models is a formatting template; each of the first event text generation instruction templates is used to record the first event type of the input current event information template as the corresponding current event type, and form a corresponding current event element type set from all the first event element types of the current event information template, and inform the corresponding first large language model through a narrative text to generate a creative event text based on the current event type and its corresponding current event element type set, and require that the creative event text generated at this time should contain a trigger word corresponding to the current event type, and require that the minimum set of event element types contained in the creative text generated at this time should satisfy the current event element type set;

[0027] Step 55, bringing the first event text into the event extraction instruction template corresponding to the current large language model to perform corresponding event extraction instruction assembly to obtain the corresponding second instruction text; and inputting the second instruction text into the current large language model to perform corresponding event extraction task processing and using the generated text output by the model as the corresponding first event information text;

[0028] Among them, the event extraction instruction template corresponding to each of the first large language models is a formatting template; each of the event extraction instruction templates is used to record the input first event text as the corresponding current event text, and inform the corresponding first large language model through a narrative text to perform event information extraction processing on the current event text, and perform corresponding event information text generation processing according to the extracted event type, event trigger words, and element types and element texts corresponding to all event elements in a specified event information text output format;

[0029] Step 56, extracting the corresponding event type text from the first event information text as the corresponding first event identification type; and extracting the corresponding event trigger word text from the first event information text as the corresponding first trigger word identification text; and extracting the element type and element text corresponding to each event element in the first event information text as the corresponding first element identification type and first element identification text to form a corresponding first event identification element, and forming a corresponding first element identification type set from all the obtained first element identification types;

[0030] Step 57, identifying the first event identification type and the first element identification type set; if the first event identification type does not match the first event type of the current event information template or at least one type of the first event element type in the current event information template is not included in the first element identification type set, setting the corresponding enhanced data record to empty; if the first event identification type matches the first event type of the current event information template and the first element identification type set includes all the first event element types of the current event information template, taking the first event text as a corresponding second event text, and taking the first event identification type as a corresponding second event type label, and taking the first trigger word identification text as a corresponding second trigger word label, and the first element identification type and the first element identification text of each first event identification element as the corresponding second element type label and the second element text label form a corresponding second element label, and the obtained second event type label, the second trigger word label and all the second element labels form a corresponding second event label, and the second event text and the second event label form a corresponding enhanced data record;

[0031] Step 58: if the enhanced data record obtained this time is not empty, then add 1 to the count value of the first counter, and use the enhanced data record obtained this time as a corresponding second data record;

[0032] Step 59, whether the count value of the first counter is greater than the preset first total record threshold is identified; if the count value of the first counter is less than or equal to the first total record threshold, whether the current large language model is the last first large language model of the first model set is identified, if so, return to step 53, if not, use the next first large language model of the first model set as the new current large language model and return to step 54; if the count value of the first counter is greater than the first total record threshold, whether the current event information template is the last first event information template of the first event information template set is identified, if not, extract the next first event information template of the first event information template set as the new current event information template and return to step 52, if so, all the second data records obtained constitute the corresponding second data set.

[0033] Preferably, performing sample synthesis processing on the first and second data sets to obtain a corresponding third data set specifically includes:

[0034] Step 61, counting the event type types of all the first event type labels in the first data set to obtain a corresponding second event type set;

[0035] Wherein, the second event type set includes multiple types of second event types;

[0036] Step 62, grouping all the first data records in the first data set whose first event type tags match the same type of the second event type into one class to form a corresponding second class record set; and grouping all the second data records in the second data set whose second event type tags match the same type of the second event type into one class to form a corresponding third class record set;

[0037] Step 63, initialize an empty record set and record it as the corresponding first record set;

[0038] Step 64, perform a round of traversal on all the second-category record sets; and in this round of traversal, record the second-category record set currently traversed as the corresponding current second-category record set; and use the third-category record set corresponding to the current second-category record set as the corresponding current third-category record set; and perform a round of sample mixing processing on all the first data records of the current second-category record set and all the second data records of the current third-category record set, and update the first record set based on the mixing processing result;

[0039] Step 65, group all the third data records in the first record set whose third event type tags match the same type of the second event type into one category to form a corresponding fourth category record set; and count the total number of records in each of the fourth category record sets to obtain the corresponding first total number of records; and record the largest and smallest total number of the first records as the corresponding maximum total number of records and minimum total number of records; and calculate the ratio of the minimum total number of records to the maximum total number of records to obtain the corresponding first ratio = minimum total number of records / maximum total number of records;

[0040] Step 66, identifying whether the first ratio is less than a preset first ratio threshold; if the first ratio is greater than or equal to the first ratio threshold, go to step 67; if the first ratio is less than the first ratio threshold, record the second event type corresponding to the minimum total number of records as the corresponding current event type, and randomly select a corresponding first data record from the second type record set corresponding to the current event type as the corresponding current record, and perform a round of sample mixing processing on the current record and all the second data records in the third type record set corresponding to the current event type, and update the first record set based on the mixing processing result, and return to step 65 at the end of this round of record set update;

[0041] Step 67: Use the first record set as the corresponding third data set.

[0042] Further, performing a round of sample mixing processing on all the first data records of the current second-category record set and all the second data records of the current third-category record set and updating the first record set based on the mixing processing result specifically includes:

[0043] Step 71, extracting the first of the first data records in the current second-category record set as the corresponding current master record;

[0044] Step 72, extracting the first of the second data records in the current three-category record set as the corresponding current auxiliary record;

[0045] Step 73, randomly select a tag from the second trigger word tag and all the second element tags of the current auxiliary record as the corresponding currently selected tag; and identify the currently selected tag; if the currently selected tag is the second trigger word tag, go to step 74; if the currently selected tag is a second element tag, go to step 75;

[0046] Step 74, taking the selected label as the corresponding first corpus text, and taking the first trigger word label of the current main record as the corresponding second corpus text, and performing corpus mixing processing on the first and second corpus texts according to the preset corpus mixing rule to obtain the corresponding third corpus text; and taking the third corpus text as the corresponding third trigger word label; and taking each of the first element labels of the current main record as the corresponding third element label; and taking the first event type label of the current main record as the corresponding third event type label; and going to step 76;

[0047] Step 75, taking the first element tag in the current main record that matches the second element type tag of the tag selected this time as the corresponding current element tag; taking the second element text tag of the tag selected this time as the corresponding fourth corpus text; taking the first element text tag of the current element tag as the corresponding fifth corpus text; and performing corpus mixing processing on the fourth and fifth corpus texts according to the corpus mixing rule to obtain the corresponding sixth corpus text; and taking the second element type tag corresponding to the tag selected this time and the sixth corpus text as a group of corresponding third element type tags and third element text tags to form a corresponding third element tag; and taking the first event type tag and the first trigger word tag of the current main record as the corresponding third event type tag and the third trigger word tag; and taking the remaining first element tags in the current main record except the current element tag as a corresponding third element tag;

[0048] Step 76, the third event type label, the third trigger word label and all the third element labels obtained this time are used to form a corresponding third event label; and one of the first large language models is selected from the first model set as the corresponding current large language model; and the third event label obtained this time is brought into the first event text generation instruction template corresponding to the current large language model to perform the corresponding event text generation instruction assembly to obtain the corresponding third instruction text; and the third instruction text is input into the current large language model to perform the corresponding creative event text generation task processing and the generated text output by the model is used as the corresponding third event text; and the third event text obtained this time and the third event label form a corresponding third data record;

[0049] The second event text generation instruction template corresponding to each of the first language models is a formatting template; each of the second event text generation instruction templates is used to record the input third event label as the corresponding current event label, and inform the corresponding first language model through a narrative text to generate a creative event text based on the event type label, trigger word label and all element labels of the current event label;

[0050] Step 77, identifying whether the current auxiliary record is the last second data record of the current three-category record set; if not, extracting the next second data record of the current three-category record set as the new current auxiliary record, and returning to step 73; if yes, going to step 78;

[0051] Step 78, identify whether the current main record is the last of the first data record in the current second-category record set; if not, extract the next first data record of the current second-category record set as the new current main record, and return to step 72; if so, add all the third data records obtained in this round of sample mixing processing to the first record set.

[0052] Further, performing a round of sample mixing processing on all the second data records in the third type of record set corresponding to the current record and the current event type and updating the first record set based on the mixing processing result specifically includes:

[0053] Step 81, taking the current record as the corresponding current main record; and taking the third type of record set corresponding to the current event type as the corresponding current record set;

[0054] Step 82, extracting the first second data record of the current record set as the corresponding current auxiliary record;

[0055] Step 83, randomly selecting a tag from the second trigger word tag and all the second element tags of the current auxiliary record as the corresponding currently selected tag; and identifying the currently selected tag; if the currently selected tag is the second trigger word tag, go to step 84; if the currently selected tag is a second element tag, go to step 85;

[0056] Step 84, taking the selected label as the corresponding seventh corpus text, taking the first trigger word label of the current main record as the corresponding eighth corpus text, and performing corpus mixing processing on the seventh and eighth corpus texts according to the corpus mixing rule to obtain the corresponding ninth corpus text; taking the ninth corpus text as the corresponding third trigger word label; taking each of the first element labels of the current main record as the corresponding third element label; taking the first event type label of the current main record as the corresponding third event type label; and going to step 86;

[0057] Step 85, taking the first element tag in the current main record that matches the second element type tag of the currently selected tag as the corresponding current element tag; taking the second element text tag of the currently selected tag as the corresponding tenth corpus text; taking the first element text tag of the current element tag as the corresponding eleventh corpus text; and performing corpus mixing processing on the tenth and eleventh corpus texts according to the corpus mixing rule to obtain the corresponding twelfth corpus text; and taking the second element type tag corresponding to the currently selected tag and the twelfth corpus text as a group of corresponding third element type tags and third element text tags to form a corresponding third element tag; and taking the first event type tag and the first trigger word tag of the current main record as the corresponding third event type tag and the third trigger word tag; and taking the remaining first element tags in the current main record except the current element tag as a corresponding third element tag;

[0058] Step 86, the third event type label, the third trigger word label and all the third element labels obtained this time are used to form a corresponding third event label; and one of the first large language models is selected from the first model set as the corresponding current large language model; and the third event label obtained this time is brought into the first event text generation instruction template corresponding to the current large language model to perform the corresponding event text generation instruction assembly to obtain the corresponding fourth instruction text; and the fourth instruction text is input into the current large language model to perform the corresponding creative event text generation task processing and the generated text output by the model is used as the corresponding third event text; and the third event text obtained this time and the third event label form a corresponding third data record;

[0059] Step 87, identify whether the current auxiliary record is the last second data record of the current record set; if not, extract the next second data record of the current record set as the new current auxiliary record, and return to step 83; if so, add all the third data records obtained in this round of sample mixing processing to the first record set.

[0060] A second aspect of an embodiment of the present invention provides a device for implementing the method for data enhancement of an event extraction training data set described in the first aspect, the device comprising: a data generation model preparation module, an enhanced data set generation module;

[0061] The data generation model preparation module is used to record a public data set for event extraction training as a corresponding first data set; and form N pre-trained large language models into a corresponding first model set; and fine-tune the first large language models of the first model set based on the first data set; N is a preset first number, and the first number N is a positive integer; the first model set includes N first large language models;

[0062] The enhanced data set generation module is used to identify the event information templates of various event types in the first data set after the fine-tuning training of the first model set is completed to obtain the corresponding first event information template set; and perform data set generation processing based on the first model set and the first event information template set to obtain the corresponding second data set; and perform sample synthesis processing based on the first and second data sets to obtain the corresponding third data set; and save the third data set as an enhanced data set for event extraction training.

[0063] A third aspect of an embodiment of the present invention provides an electronic device, including: a memory, a processor, and a transceiver;

[0064] The processor is used to be coupled to the memory, read and execute instructions in the memory, so as to implement the method steps described in the first aspect above;

[0065] The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

[0066] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions. When the computer instructions are executed by a computer, the computer executes the instructions of the method described in the first aspect above.

[0067] The embodiment of the present invention provides a method, device, electronic device and computer-readable storage medium for data enhancement of an event extraction training data set. As can be seen from the above content, the embodiment of the present invention records a public data set for event extraction training as the corresponding first data set, and a corresponding first model set is composed of multiple large language models that have completed pre-training, and each first large language model is fine-tuned based on the first data set; and after the fine-tuning training of the first model set is completed, the event information templates of various event types in the first data set are identified to obtain the corresponding first event information template set, and the data set generation processing is performed according to the first model set and the first event information template set to obtain the corresponding second data set, and the sample synthesis processing is performed according to the first and second data sets to obtain the corresponding third data set, and the third data set is used as an enhanced data set for event extraction training and saved. It can be seen that, on the one hand, the embodiment of the present invention uses multiple types of large language models with event extraction functions to automatically generate corpus to obtain the corresponding automatically generated data set (second data set); on the other hand, it also constructs a synthetic data set (third data set) with larger data volume and better accuracy by mixing corpus labels of the public data set and the automatically generated data set; on the other hand, it improves the data balance of the synthetic data set by supplementing the number of data records of a small number of event types; through the embodiment of the present invention, the data quality of the data set is guaranteed, the construction cycle of the data set is shortened, and the construction cost of the data set is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 A schematic diagram of a method for performing data enhancement on an event extraction training data set provided in Embodiment 1 of the present invention;

[0069] Figure 2 A module structure diagram of a device for performing data enhancement on an event extraction training data set provided in Embodiment 2 of the present invention;

[0070] Figure 3 A schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] Embodiment 1 of the present invention provides a method for data enhancement of an event extraction training data set, such as Figure 1As shown in a schematic diagram of a method for data enhancement of an event extraction training data set provided in the first embodiment of the present invention, the method mainly includes the following steps:

[0073] Step 1: record a public data set for event extraction training as the corresponding first data set; and form a corresponding first model set with N pre-trained large language models; and fine-tune each first large language model of the first model set based on the first data set;

[0074] Specifically comprising: step 11, recording a public data set for event extraction training as a corresponding first data set;

[0075] Here, the public data set of the embodiment of the present invention, i.e., the first data set, includes at least the ACE2005 data set released by the Linguistic Data Consortium (LDC), the event extraction data set provided by the Kaggle / Google / GitHub / HDX / FiveThirtyEight platform, and the event extraction data set released by the Chinese Data Consortium; these data sets are all high-quality data sets that have undergone data quality inspection, also known as golden data sets; fine-tuning the large language model based on these data sets can enable the model to have the corresponding event extraction task processing capabilities;

[0076] The first data set includes multiple first data records; the first data record includes a first event text and a first event label; the first event label includes a first event type label, a first trigger word label and multiple first element labels; the first element label includes a first element type label and a first element text label; the first event type label is the event type corresponding to the first event text; the first trigger word label is the trigger word text in the first event text that triggers the current type of event; each first element label corresponds to an event element of the current event type; the first element type label is the event element type corresponding to the current event element; the first element text label is the element text in the first event text that corresponds to the current event element;

[0077] Step 12, and forming the N pre-trained large language models into a corresponding first model set;

[0078] Here, the preset first number N is a positive integer; the first model set of the embodiment of the present invention is composed of N first largest language models; the first largest language models mentioned here include at least BERT series models, BART series models, GPT series models, T5 series models, and Wenxin series models;

[0079] Step 13, fine-tuning each first language model of the first model set based on the first data set;

[0080] Specifically, the method includes: dividing the first data set into two data subsets according to a preset first division ratio and recording them as corresponding first training sets and first evaluation sets; and performing a round of training on the event extraction task processing function of each first large language model based on the first training set; and performing an evaluation on the event extraction task processing performance of the current large language model based on the first evaluation set when each first large language model completes a round of training to obtain a corresponding current model evaluation result; and when the current model evaluation result is not up to standard, performing another round of training on the current large language model based on the first training set until the latest current model evaluation result is up to standard; and after all the first large language models of the first model set have met the standard, confirming that the fine-tuning training of the first model set is completed;

[0081] Among them, the first segmentation ratio is a preset ratio parameter, such as 8:2; the first training set and the first evaluation set are both composed of multiple first data records; the ratio of the total number of records in the first training set to the total number of records in the first evaluation set satisfies the first segmentation ratio.

[0082] Step 2: After the fine-tuning training of the first model set is completed, the event information templates of various event types in the first data set are identified to obtain the corresponding first event information template set; and the data set generation processing is performed based on the first model set and the first event information template set to obtain the corresponding second data set; and the sample synthesis processing is performed based on the first and second data sets to obtain the corresponding third data set; and the third data set is used as an enhanced data set for event extraction training and saved;

[0083] Specifically, it includes: step 21, after the fine-tuning training of the first model set is completed, the event information templates of various event types in the first data set are identified to obtain the corresponding first event information template set;

[0084] The first event information template set includes a plurality of first event information templates; each first event information template corresponds to a type of event; the first event information template includes a first event type and a plurality of first event element types;

[0085] Specifically, it includes: step 211, counting the event type types of all first event type labels in the first data set to obtain a corresponding first event type set;

[0086] The first event type set includes multiple first event types;

[0087] Step 212, all first data records in the first data set whose first event type tags match the same type of first event type are grouped into one category to form a corresponding first category record set;

[0088] Step 213, counting the event element type types of all first element type labels in each first type record set to obtain a corresponding first event element type set;

[0089] The first event element type set includes multiple types of first event element types;

[0090] Step 214, each first event type and all first event element types of its corresponding first event element type set form a corresponding first event information template;

[0091] Step 215, all the obtained first event information templates form a corresponding first event information template set;

[0092] Step 22, performing data set generation processing according to the first model set and the first event information template set to obtain a corresponding second data set;

[0093] Among them, the second data set includes multiple second data records; the second data record includes a second event text and a second event label; the second event label includes a second event type label, a second trigger word label and multiple second element labels; the second element label includes a second element type label and a second element text label; Here, the embodiment of the present invention uses multiple types of large language models with event extraction functions to automatically generate corpora to obtain the corresponding automatically generated data set, i.e., the second data set, thereby achieving the purpose of quickly increasing the total amount of data and shortening the data construction cycle;

[0094] The current step 22 specifically includes:

[0095] Step 221, taking the first first event information template of the first event information template set as the corresponding current event information template;

[0096] Step 222, initializing the count value of the first counter to 1;

[0097] Step 223, taking the first first large language model of the first model set as the corresponding current large language model;

[0098] Step 224, bringing the current event information template into the first event text generation instruction template corresponding to the current large language model to perform corresponding event text generation instruction assembly to obtain the corresponding first instruction text; and inputting the first instruction text into the current large language model to perform corresponding creative event text generation task processing and using the generated text output by the model as the corresponding first event text;

[0099] Here, the first event text generation instruction template corresponding to each first language model in the embodiment of the present invention is a formatting template; each first event text generation instruction template is used to record the first event type of the input current event information template as the corresponding current event type, and form a corresponding current event element type set from all first event element types of the current event information template, and inform the corresponding first language model through a narrative text to generate a creative event text based on the current event type and its corresponding current event element type set, and require that the creative event text generated at this time should contain a trigger word corresponding to the current event type, and require that the minimum set of event element types contained in the creative text generated at this time should satisfy the current event element type set;

[0100] Step 225, bringing the first event text into the event extraction instruction template corresponding to the current large language model to perform corresponding event extraction instruction assembly to obtain the corresponding second instruction text; and inputting the second instruction text into the current large language model to perform corresponding event extraction task processing and using the generated text output by the model as the corresponding first event information text;

[0101] Here, the event extraction instruction template corresponding to each first language model in the embodiment of the present invention is a formatting template; each event extraction instruction template is used to record the input first event text as the corresponding current event text, and inform the corresponding first language model through a narrative text to perform event information extraction processing on the current event text, and perform corresponding event information text generation processing according to the extracted event type, event trigger words, and element types and element texts corresponding to all event elements in a specified event information text output format;

[0102] Step 226, extract the corresponding event type text from the first event information text as the corresponding first event identification type; extract the corresponding event trigger word text from the first event information text as the corresponding first trigger word identification text; and extract the element type and element text corresponding to each event element in the first event information text as the corresponding first element identification type and first element identification text to form a corresponding first event identification element, and form a corresponding first element identification type set from all the obtained first element identification types;

[0103] Step 227, identify the first event identification type and the first element identification type set; if the first event identification type does not match the first event type of the current event information template or at least one type of first event element type in the current event information template is not included in the first element identification type set, set the corresponding enhanced data record to empty; if the first event identification type matches the first event type of the current event information template and the first element identification type set includes all first event element types of the current event information template, then take the first event text as a corresponding second event text, and take the first event identification type as a corresponding second event type label, and take the first trigger word identification text as a corresponding second trigger word label, and use the first element identification type and the first element identification text of each first event identification element as the corresponding second element type label and the second element text label to form a corresponding second element label, and use the obtained second event type label, the second trigger word label and all second element labels to form a corresponding second event label, and use the second event text and the second event label to form a corresponding enhanced data record;

[0104] Step 228: if the enhanced data record obtained this time is not empty, then the count value of the first counter is increased by 1, and the enhanced data record obtained this time is used as a corresponding second data record;

[0105] Step 229, whether the count value of the first counter is greater than the preset first total record threshold is identified; if the count value of the first counter is less than or equal to the first total record threshold, whether the current large language model is the last first large language model of the first model set is identified, if so, return to step 223, if not, the next first large language model of the first model set is used as the new current large language model and return to step 224; if the count value of the first counter is greater than the first total record threshold, whether the current event information template is the last first event information template of the first event information template set is identified, if not, the next first event information template of the first event information template set is extracted as the new current event information template and return to step 222, if so, all the second data records obtained constitute the corresponding second data set;

[0106] Here, the first total record threshold is a preset threshold parameter; the total number of second data records corresponding to each event type in the second data set generated by the above steps 221-229 is consistent with the first total record threshold;

[0107] Step 23, performing sample synthesis processing according to the first and second data sets to obtain a corresponding third data set;

[0108] Among them, the third data set includes multiple third data records; the third data record includes a third event text and a third event label; the third event label includes a third event type label, a third trigger word label and multiple third element labels; the third element label includes a third element type label and a third element text label; here, although the total amount of data can be quickly expanded by step 22, its data quality is still relatively weak compared to the data quality of the first data set, so the embodiment of the present invention will construct a synthetic data set with larger data volume and better accuracy, namely the third data set, by mixing the corpus labels of the first and second data sets after the second data set;

[0109] The current step 23 specifically includes:

[0110] Step 231, counting the event type types of all first event type labels in the first data set to obtain a corresponding second event type set;

[0111] The second event type set includes multiple types of second event types;

[0112] Step 232, grouping all first data records in the first data set whose first event type tags match the same second event type into one class to form a corresponding second class record set; and grouping all second data records in the second data set whose second event type tags match the same second event type into one class to form a corresponding third class record set;

[0113] Step 233, initialize an empty record set and record it as the corresponding first record set;

[0114] Step 234, perform a round of traversal on all second-category record sets; and in this round of traversal, record the currently traversed second-category record set as the corresponding current second-category record set; and record the third-category record set corresponding to the current second-category record set as the corresponding current third-category record set; and perform a round of sample mixing processing on all first data records of the current second-category record set and all second data records of the current third-category record set, and update the first record set based on the mixing processing result;

[0115] Among them, performing a round of sample mixing processing on all first data records of the current second-category record set and all second data records of the current third-category record set and updating the first record set based on the mixing processing result specifically includes:

[0116] Step A1, extracting the first first data record of the current second-category record set as the corresponding current main record;

[0117] Step A2, extracting the first second data record of the current three-category record set as the corresponding current auxiliary record;

[0118] Step A3, randomly select a tag from the second trigger word tag and all second element tags of the current auxiliary record as the corresponding currently selected tag; and identify the currently selected tag; if the currently selected tag is the second trigger word tag, go to step A4; if the currently selected tag is a second element tag, go to step A5;

[0119] Step A4, taking the selected label as the corresponding first corpus text, and taking the first trigger word label of the current main record as the corresponding second corpus text, and performing corpus mixing processing on the first and second corpus texts according to the preset corpus mixing rule to obtain the corresponding third corpus text; and taking the third corpus text as a corresponding third trigger word label; and taking each first element label of the current main record as a corresponding third element label; and taking the first event type label of the current main record as a corresponding third event type label; and going to step A6;

[0120] Here, the corpus mixing rules of the embodiment of the present invention can be customized based on specific application requirements; one processing method of the customized rule is a replacement method, that is, the second corpus text is directly used as the corresponding third corpus text; another processing method of the customized rule is a fusion method, that is, the first and second corpus texts are spliced ​​to obtain the corresponding third corpus text; another processing method of the customized rule is to perform a fusion two-feature prediction based on the text features of the first and second corpus texts based on a data augmentation strategy such as Mixup, and then perform text decoding based on the predicted fusion features, and then use the decoded text as the corresponding third corpus text;

[0121] Step A5, take the first element tag in the current main record that matches the second element type tag of the tag selected this time as the corresponding current element tag; take the second element text tag of the tag selected this time as the corresponding fourth corpus text; take the first element text tag of the current element tag as the corresponding fifth corpus text; and perform corpus mixing processing on the fourth and fifth corpus texts according to the corpus mixing rules to obtain the corresponding sixth corpus text; and take the second element type tag and the sixth corpus text corresponding to the tag selected this time as a group of corresponding third element type tags and third element text tags to form a corresponding third element tag; and take the first event type tag and the first trigger word tag of the current main record as the corresponding third event type tag and the third trigger word tag; and take the remaining first element tags in the current main record except the current element tag as a corresponding third element tag;

[0122] Step A6, the third event type label, the third trigger word label and all the third element labels obtained this time are used to form a corresponding third event label; and a first large language model is selected from the first model set as the corresponding current large language model; and the third event label obtained this time is brought into the first event text generation instruction template corresponding to the current large language model to perform the corresponding event text generation instruction assembly to obtain the corresponding third instruction text; and the third instruction text is input into the current large language model to perform the corresponding creative event text generation task processing and the generated text output by the model is used as the corresponding third event text; and the third event text and the third event label obtained this time form a corresponding third data record;

[0123] Here, the second event text generation instruction template corresponding to each first language model in the embodiment of the present invention is a formatting template; each second event text generation instruction template is used to record the input third event label as the corresponding current event label, and inform the corresponding first language model through a narrative text to generate a creative event text based on the event type label, trigger word label and all element labels of the current event label;

[0124] Step A7, identifying whether the current auxiliary record is the last second data record of the current three-category record set; if not, extracting the next second data record of the current three-category record set as the new current auxiliary record, and returning to step A3; if yes, going to step A8;

[0125] Step A8, identifying whether the current master record is the last first data record of the current second-category record set; if not, extracting the next first data record of the current second-category record set as the new current master record, and returning to step A2; if so, adding all third data records obtained by this round of sample mixing processing to the first record set;

[0126] Step 235, all third data records in the first record set whose third event type tags match the same type of second event type are grouped into one category to form a corresponding fourth category record set; and the total number of records in each fourth category record set is counted to obtain the corresponding first total number of records; and the largest and smallest first total number of records are recorded as the corresponding maximum total number of records and minimum total number of records; and the ratio of the minimum total number of records to the maximum total number of records is calculated to obtain the corresponding first ratio = minimum total number of records / maximum total number of records;

[0127] Step 236, identifying whether the first ratio is less than a preset first ratio threshold; if the first ratio is greater than or equal to the first ratio threshold, go to step 237; if the first ratio is less than the first ratio threshold, record the second event type corresponding to the minimum total number of records as the corresponding current event type, and randomly select a corresponding first data record from the second type record set corresponding to the current event type as the corresponding current record, and perform a round of sample mixing processing on the current record and all the second data records in the third type record set corresponding to the current event type, and update the first record set based on the mixing processing result, and return to step 235 at the end of this round of record set update;

[0128] Here, the first ratio threshold is a preset threshold parameter; when the first ratio is less than the first ratio threshold, it indicates that the current event type is a small number event type. In order to improve the sample data balance of the data set for all event types, the embodiment of the present invention will perform a round of quantity supplement for the data records corresponding to this small number event type;

[0129] The specific supplementary operation is: performing a round of sample mixing processing on all second data records in the third type of record set corresponding to the current record and the current event type and updating the first record set based on the mixing processing result, specifically including:

[0130] Step B1, taking the current record as the corresponding current main record; and taking the third type of record set corresponding to the current event type as the corresponding current record set;

[0131] Step B2, extracting the first second data record of the current record set as the corresponding current auxiliary record;

[0132] Step B3, randomly select a tag from the second trigger word tag and all second element tags of the current auxiliary record as the corresponding tag for this selection; and identify the tag for this selection; if the tag for this selection is the second trigger word tag, go to step B4; if the tag for this selection is a second element tag, go to step B5;

[0133] Step B4, taking the selected label as the corresponding seventh corpus text, taking the first trigger word label of the current main record as the corresponding eighth corpus text, and performing corpus mixing processing on the seventh and eighth corpus texts according to the corpus mixing rule to obtain the corresponding ninth corpus text; and taking the ninth corpus text as a corresponding third trigger word label; and taking each first element label of the current main record as a corresponding third element label; and taking the first event type label of the current main record as a corresponding third event type label; and going to step B6;

[0134] Step B5, take the first element tag in the current main record that matches the second element type tag of the tag selected this time as the corresponding current element tag; take the second element text tag of the tag selected this time as the corresponding tenth corpus text; take the first element text tag of the current element tag as the corresponding eleventh corpus text; perform corpus mixing processing on the tenth and eleventh corpus texts according to the corpus mixing rule to obtain the corresponding twelfth corpus text; take the second element type tag corresponding to the tag selected this time and the twelfth corpus text as a group of corresponding third element type tags and third element text tags to form a corresponding third element tag; take the first event type tag and the first trigger word tag of the current main record as the corresponding third event type tag and the third trigger word tag; take the remaining first element tags in the current main record except the current element tag as a corresponding third element tag;

[0135] Step B6, the third event type label, the third trigger word label and all the third element labels obtained this time are used to form a corresponding third event label; and a first large language model is selected from the first model set as the corresponding current large language model; and the third event label obtained this time is brought into the first event text generation instruction template corresponding to the current large language model to perform the corresponding event text generation instruction assembly to obtain the corresponding fourth instruction text; and the fourth instruction text is input into the current large language model to perform the corresponding creative event text generation task processing and the generated text output by the model is used as the corresponding third event text; and the third event text and the third event label obtained this time form a corresponding third data record;

[0136] Step B7, identifying whether the current auxiliary record is the last second data record of the current record set; if not, extracting the next second data record of the current record set as the new current auxiliary record, and returning to step B3; if so, adding all third data records obtained by this round of sample mixing processing to the first record set;

[0137] Step 237, using the first record set as the corresponding third data set;

[0138] Step 24, and save the third data set as an enhanced data set for event extraction training.

[0139] Figure 2 The module structure diagram of a device for data enhancement of event extraction training data set provided in the second embodiment of the present invention, the device is a terminal device or server implementing the aforementioned method embodiment, and can also be a device that enables the aforementioned terminal device or server to implement the aforementioned method embodiment, for example, the device can be a device or chip system of the aforementioned terminal device or server. Figure 2As shown, the device includes: a data generation model preparation module 201 and an enhanced data set generation module 202.

[0140] The data generation model preparation module 201 is used to record a public data set for event extraction training as the corresponding first data set; and to form a corresponding first model set with N large language models that have completed pre-training; and to fine-tune the various first large language models of the first model set based on the first data set; N is a preset first number, and the first number N is a positive integer; the first model set includes N first large language models.

[0141] The enhanced data set generation module 202 is used to identify the event information templates of various event types in the first data set after the fine-tuning training of the first model set is completed, to obtain the corresponding first event information template set; and to perform data set generation processing based on the first model set and the first event information template set to obtain the corresponding second data set; and to perform sample synthesis processing based on the first and second data sets to obtain the corresponding third data set; and to save the third data set as an enhanced data set for event extraction training.

[0142] An embodiment of the present invention provides a device for data enhancement of an event extraction training data set, which can execute the method steps in the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here.

[0143] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the data generation model preparation module can be a separately established processing element, or it can be integrated in a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The function of the above-mentioned module is determined. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0144] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more digital signal processors (DSP), or one or more field programmable gate arrays (FPGA). For another example, when a module above is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0145] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the above method embodiment is generated in whole or in part. The above computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The above computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the above computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, Bluetooth, microwave, etc.) mode to another website site, computer, server or data center. The above computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, data center, etc. that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0146] Figure 3 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. The electronic device may be a terminal device or a server for implementing the method of the aforementioned embodiment, or may be a terminal device or a server for implementing the method of the aforementioned embodiment connected to the aforementioned terminal device or server. Figure 3As shown, the electronic device may include: a processor 301 (such as a CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transceiver 303. Various instructions may be stored in the memory 302 to complete various processing functions and implement the processing steps described in the aforementioned embodiment method. Preferably, the electronic device involved in the embodiment of the present invention also includes: a power supply 304, a system bus 305 and a communication port 306. The system bus 305 is used to realize the communication connection between components. The above-mentioned communication port 306 is used for connecting and communicating between the electronic device and other peripherals.

[0147] exist Figure 3 The system bus 305 mentioned in the figure can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface is used to realize the communication between the database access device and other devices (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM), and may also include non-volatile memory (Non-Volatile Memory), such as at least one disk storage.

[0148] The above-mentioned processor can be a general-purpose processor, including a central processing unit CPU, a network processor (Network Processor, NP), a graphics processing unit (Graphics Processing Unit, GPU), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0149] It should be noted that an embodiment of the present invention further provides a computer-readable storage medium, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the method and processing process provided in the above embodiments.

[0150] The embodiment of the present invention provides a method, device, electronic device and computer-readable storage medium for data enhancement of an event extraction training data set. As can be seen from the above content, the embodiment of the present invention records a public data set for event extraction training as the corresponding first data set, and a corresponding first model set is composed of multiple large language models that have completed pre-training, and each first large language model is fine-tuned based on the first data set; and after the fine-tuning training of the first model set is completed, the event information templates of various event types in the first data set are identified to obtain the corresponding first event information template set, and the data set generation processing is performed according to the first model set and the first event information template set to obtain the corresponding second data set, and the sample synthesis processing is performed according to the first and second data sets to obtain the corresponding third data set, and the third data set is used as an enhanced data set for event extraction training and saved. It can be seen that, on the one hand, the embodiment of the present invention uses multiple types of large language models with event extraction functions to automatically generate corpus to obtain the corresponding automatically generated data set (second data set); on the other hand, it also constructs a synthetic data set (third data set) with larger data volume and better accuracy by mixing corpus labels of the public data set and the automatically generated data set; on the other hand, it improves the data balance of the synthetic data set by supplementing the number of data records of a small number of event types; through the embodiment of the present invention, the data quality of the data set is guaranteed, the construction cycle of the data set is shortened, and the construction cost of the data set is reduced.

[0151] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0152] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for data enhancement of an event extraction training data set, characterized in that: The method comprises: A public data set for event extraction training is recorded as a corresponding first data set; N pre-trained large language models are combined into a corresponding first model set; and fine-tuning training is performed on each first large language model of the first model set based on the first data set; N is a preset first number, and the first number N is a positive integer; the first model set includes N first large language models; After the fine-tuning training of the first model set is completed, the event information templates of each event type in the first data set are identified to obtain the corresponding first event information template set; and a data set generation process is performed based on the first model set and the first event information template set to obtain the corresponding second data set; and a sample synthesis process is performed based on the first and second data sets to obtain the corresponding third data set; and the third data set is used as an enhanced data set for event extraction training and saved.

2. The method for data enhancement of an event extraction training data set according to claim 1, characterized in that: The public data sets include at least the ACE2005 data set released by the Language Data Alliance, the event extraction data set provided by the Kaggle / Google / GitHub / HDX / FiveThirtyEi ght platform, and the event extraction data set released by the Chinese Data Alliance; The first language model includes at least BERT series models, BART series models, GPT series models, T5 series models, and Wenxin series models; The first data set includes multiple first data records; the first data record includes a first event text and a first event label; the first event label includes a first event type label, a first trigger word label and multiple first element labels; the first element label includes a first element type label and a first element text label; the first event type label is the event type corresponding to the first event text; the first trigger word label is the trigger word text in the first event text that triggers the current type of event; each of the first element labels corresponds to an event element of the current event type; the first element type label is the event element type corresponding to the current event element; The first element text label is the element text corresponding to the current event element in the first event text; The first event information template set includes a plurality of first event information templates; each of the first event information templates corresponds to a class of event types; the first event information template includes a first event type and a plurality of first event element types; The second data set includes a plurality of second data records; the second data records include a second event text and a second event tag; the second event tag includes a second event type tag, a second trigger word tag and a plurality of second element tags; the second element tag includes a second element type tag and a second element text tag; The third data set includes multiple third data records; the third data record includes a third event text and a third event label; the third event label includes a third event type label, a third trigger word label and multiple third element labels; the third element label includes a third element type label and a third element text label.

3. The method for data enhancement of an event extraction training data set according to claim 2, characterized in that: The fine-tuning training of each first large language model of the first model set based on the first data set specifically includes: The first data set is divided into two data subsets according to a preset first segmentation ratio, which are recorded as a corresponding first training set and a first evaluation set; and a round of training is performed on the event extraction task processing function of each of the first large language models based on the first training set; and when each of the first large language models completes a round of training, the event extraction task processing performance of the current large language model is evaluated based on the first evaluation set to obtain a corresponding model evaluation result for that time, and when the model evaluation result for that time does not meet the standard, another round of training is performed on the current large language model based on the first training set until the latest model evaluation result for that time meets the standard; and after all the first large language models in the first model set meet the standard, it is confirmed that the fine-tuning training of the first model set is completed; the first training set and the first evaluation set are both composed of multiple first data records; the ratio of the total number of records in the first training set to the total number of records in the first evaluation set satisfies the first segmentation ratio.

4. The method for data enhancement of an event extraction training data set according to claim 2, characterized in that: The identifying of the event information templates of each event type in the first data set to obtain the corresponding first event information template set specifically includes: Counting the event type types of all the first event type labels in the first data set to obtain a corresponding first event type set; the first event type set includes multiple types of the first event types; and grouping all the first data records in the first data set whose first event type labels match the same type of the first event type into one category to form a corresponding first category record set; and obtaining a corresponding first event element type set by counting the event element type types of all the first element type tags in each of the first type record sets; the first event element type set includes multiple types of the first event element types; Each first event type and all the first event element types of its corresponding first event element type set form a corresponding first event information template; And all the obtained first event information templates form a corresponding first event information template set.

5. The method for data enhancement of an event extraction training data set according to claim 2, characterized in that: The step of performing data set generation processing according to the first model set and the first event information template set to obtain a corresponding second data set specifically includes: Step 51, taking the first of the first event information templates in the first event information template set as the corresponding current event information template; Step 52, initializing the count value of the first counter to 1; Step 53, taking the first of the first large language models in the first model set as the corresponding current large language model; Step 54, bringing the current event information template into the first event text generation instruction template corresponding to the current large language model to perform corresponding event text generation instruction assembly to obtain the corresponding first instruction text; and inputting the first instruction text into the current large language model to perform corresponding creative event text generation task processing and using the generated text output by the model as the corresponding first event text; Among them, the first event text generation instruction template corresponding to each of the first large language models is a formatting template; each of the first event text generation instruction templates is used to record the first event type of the input current event information template as the corresponding current event type, and form a corresponding current event element type set from all the first event element types of the current event information template, and inform the corresponding first large language model through a narrative text to generate a creative event text based on the current event type and its corresponding current event element type set, and require that the creative event text generated at this time should contain a trigger word corresponding to the current event type, and require that the minimum set of event element types contained in the creative text generated at this time should satisfy the current event element type set; Step 55, bringing the first event text into the event extraction instruction template corresponding to the current large language model to perform corresponding event extraction instruction assembly to obtain the corresponding second instruction text; and inputting the second instruction text into the current large language model to perform corresponding event extraction task processing and using the generated text output by the model as the corresponding first event information text; Among them, the event extraction instruction template corresponding to each of the first large language models is a formatting template; each of the event extraction instruction templates is used to record the input first event text as the corresponding current event text, and inform the corresponding first large language model through a narrative text to perform event information extraction processing on the current event text, and perform corresponding event information text generation processing according to the extracted event type, event trigger words, and element types and element texts corresponding to all event elements in a specified event information text output format; Step 56, extracting the corresponding event type text from the first event information text as the corresponding first event identification type; and extracting the corresponding event trigger word text from the first event information text as the corresponding first trigger word identification text; and extracting the element type and element text corresponding to each event element in the first event information text as the corresponding first element identification type and first element identification text to form a corresponding first event identification element, and forming a corresponding first element identification type set from all the obtained first element identification types; Step 57, identifying the first event identification type and the first element identification type set; if the first event identification type does not match the first event type of the current event information template or at least one type of the first event element type in the current event information template is not included in the first element identification type set, setting the corresponding enhanced data record to empty; if the first event identification type matches the first event type of the current event information template and the first element identification type set includes all the first event element types of the current event information template, taking the first event text as a corresponding second event text, and taking the first event identification type as a corresponding second event type label, and taking the first trigger word identification text as a corresponding second trigger word label, and the first element identification type and the first element identification text of each first event identification element as the corresponding second element type label and the second element text label form a corresponding second element label, and the obtained second event type label, the second trigger word label and all the second element labels form a corresponding second event label, and the second event text and the second event label form a corresponding enhanced data record; Step 58: if the enhanced data record obtained this time is not empty, then add 1 to the count value of the first counter, and use the enhanced data record obtained this time as a corresponding second data record; Step 59, whether the count value of the first counter is greater than the preset first total record threshold is identified; if the count value of the first counter is less than or equal to the first total record threshold, whether the current large language model is the last first large language model of the first model set is identified, if so, return to step 53, if not, use the next first large language model of the first model set as the new current large language model and return to step 54; if the count value of the first counter is greater than the first total record threshold, whether the current event information template is the last first event information template of the first event information template set is identified, if not, extract the next first event information template of the first event information template set as the new current event information template and return to step 52, if so, all the second data records obtained constitute the corresponding second data set.

6. The method for data enhancement of an event extraction training data set according to claim 2, characterized in that: The performing sample synthesis processing according to the first and second data sets to obtain a corresponding third data set specifically includes: Step 61, counting the event type types of all the first event type labels in the first data set to obtain a corresponding second event type set; Wherein, the second event type set includes multiple types of second event types; Step 62, grouping all the first data records in the first data set whose first event type tags match the same type of the second event type into one class to form a corresponding second class record set; and grouping all the second data records in the second data set whose second event type tags match the same type of the second event type into one class to form a corresponding third class record set; Step 63, initialize an empty record set and record it as the corresponding first record set; Step 64, perform a round of traversal on all the second-category record sets; and in this round of traversal, record the second-category record set currently traversed as the corresponding current second-category record set; and use the third-category record set corresponding to the current second-category record set as the corresponding current third-category record set; and perform a round of sample mixing processing on all the first data records of the current second-category record set and all the second data records of the current third-category record set, and update the first record set based on the mixing processing result; Step 65, group all the third data records in the first record set whose third event type tags match the same type of the second event type into one category to form a corresponding fourth category record set; and count the total number of records in each of the fourth category record sets to obtain the corresponding first total number of records; and record the largest and smallest total number of the first records as the corresponding maximum total number of records and minimum total number of records; and calculate the ratio of the minimum total number of records to the maximum total number of records to obtain the corresponding first ratio = minimum total number of records / maximum total number of records; Step 66, identifying whether the first ratio is less than a preset first ratio threshold; if the first ratio is greater than or equal to the first ratio threshold, go to step 67; if the first ratio is less than the first ratio threshold, record the second event type corresponding to the minimum total number of records as the corresponding current event type, and randomly select a corresponding first data record from the second type record set corresponding to the current event type as the corresponding current record, and perform a round of sample mixing processing on the current record and all the second data records in the third type record set corresponding to the current event type, and update the first record set based on the mixing processing result, and return to step 65 at the end of this round of record set update; Step 67: Use the first record set as the corresponding third data set.

7. The method for data enhancement of an event extraction training data set according to claim 6, characterized in that: The performing a round of sample mixing processing on all the first data records of the current second-category record set and all the second data records of the current third-category record set and updating the first record set based on the mixing processing result specifically includes: Step 71, extracting the first of the first data records in the current second-category record set as the corresponding current master record; Step 72, extracting the first of the second data records in the current three-category record set as the corresponding current auxiliary record; Step 73, randomly select a tag from the second trigger word tag and all the second element tags of the current auxiliary record as the corresponding currently selected tag; and identify the currently selected tag; if the currently selected tag is the second trigger word tag, go to step 74; if the currently selected tag is a second element tag, go to step 75; Step 74, taking the selected label as the corresponding first corpus text, and taking the first trigger word label of the current main record as the corresponding second corpus text, and performing corpus mixing processing on the first and second corpus texts according to the preset corpus mixing rule to obtain the corresponding third corpus text; and taking the third corpus text as the corresponding third trigger word label; and taking each of the first element labels of the current main record as the corresponding third element label; and taking the first event type label of the current main record as the corresponding third event type label; and going to step 76; Step 75, taking the first element tag in the current main record that matches the second element type tag of the tag selected this time as the corresponding current element tag; taking the second element text tag of the tag selected this time as the corresponding fourth corpus text; taking the first element text tag of the current element tag as the corresponding fifth corpus text; and performing corpus mixing processing on the fourth and fifth corpus texts according to the corpus mixing rule to obtain the corresponding sixth corpus text; and taking the second element type tag corresponding to the tag selected this time and the sixth corpus text as a group of corresponding third element type tags and third element text tags to form a corresponding third element tag; and taking the first event type tag and the first trigger word tag of the current main record as the corresponding third event type tag and the third trigger word tag; and taking the remaining first element tags in the current main record except the current element tag as a corresponding third element tag; Step 76, the third event type label, the third trigger word label and all the third element labels obtained this time are used to form a corresponding third event label; and one of the first large language models is selected from the first model set as the corresponding current large language model; and the third event label obtained this time is brought into the first event text generation instruction template corresponding to the current large language model to perform the corresponding event text generation instruction assembly to obtain the corresponding third instruction text; and the third instruction text is input into the current large language model to perform the corresponding creative event text generation task processing and the generated text output by the model is used as the corresponding third event text; and the third event text obtained this time and the third event label form a corresponding third data record; The second event text generation instruction template corresponding to each of the first language models is a formatting template; each of the second event text generation instruction templates is used to record the input third event label as the corresponding current event label, and inform the corresponding first language model through a narrative text to generate a creative event text based on the event type label, trigger word label and all element labels of the current event label; Step 77, identifying whether the current auxiliary record is the last second data record of the current three-category record set; if not, extracting the next second data record of the current three-category record set as the new current auxiliary record, and returning to step 73; if yes, going to step 78; Step 78, identify whether the current main record is the last of the first data record in the current second-category record set; if not, extract the next first data record of the current second-category record set as the new current main record, and return to step 72; if so, add all the third data records obtained in this round of sample mixing processing to the first record set.

8. The method for data enhancement of event extraction training data set according to claim 7, characterized in that: The step of performing a round of sample mixing processing on all the second data records in the third type of record set corresponding to the current record and the current event type and updating the first record set based on the mixing processing result specifically includes: Step 81, taking the current record as the corresponding current main record; and taking the third type of record set corresponding to the current event type as the corresponding current record set; Step 82, extracting the first second data record of the current record set as the corresponding current auxiliary record; Step 83, randomly selecting a tag from the second trigger word tag and all the second element tags of the current auxiliary record as the corresponding currently selected tag; and identifying the currently selected tag; if the currently selected tag is the second trigger word tag, go to step 84; if the currently selected tag is a second element tag, go to step 85; Step 84, taking the selected label as the corresponding seventh corpus text, taking the first trigger word label of the current main record as the corresponding eighth corpus text, and performing corpus mixing processing on the seventh and eighth corpus texts according to the corpus mixing rule to obtain the corresponding ninth corpus text; taking the ninth corpus text as the corresponding third trigger word label; taking each of the first element labels of the current main record as the corresponding third element label; taking the first event type label of the current main record as the corresponding third event type label; and going to step 86; Step 85, taking the first element tag in the current main record that matches the second element type tag of the currently selected tag as the corresponding current element tag; taking the second element text tag of the currently selected tag as the corresponding tenth corpus text; taking the first element text tag of the current element tag as the corresponding eleventh corpus text; and performing corpus mixing processing on the tenth and eleventh corpus texts according to the corpus mixing rule to obtain the corresponding twelfth corpus text; and taking the second element type tag corresponding to the currently selected tag and the twelfth corpus text as a group of corresponding third element type tags and third element text tags to form a corresponding third element tag; and taking the first event type tag and the first trigger word tag of the current main record as the corresponding third event type tag and the third trigger word tag; and taking the remaining first element tags in the current main record except the current element tag as a corresponding third element tag; Step 86, the third event type label, the third trigger word label and all the third element labels obtained this time are used to form a corresponding third event label; and one of the first large language models is selected from the first model set as the corresponding current large language model; and the third event label obtained this time is brought into the first event text generation instruction template corresponding to the current large language model to perform the corresponding event text generation instruction assembly to obtain the corresponding fourth instruction text; and the fourth instruction text is input into the current large language model to perform the corresponding creative event text generation task processing and the generated text output by the model is used as the corresponding third event text; and the third event text obtained this time and the third event label form a corresponding third data record; Step 87, identify whether the current auxiliary record is the last second data record of the current record set; if not, extract the next second data record of the current record set as the new current auxiliary record, and return to step 83; if so, add all the third data records obtained in this round of sample mixing processing to the first record set.

9. A device for executing the method for data enhancement of an event extraction training data set according to any one of claims 1 to 8, characterized in that: The device comprises: a data generation model preparation module and an enhanced data set generation module; The data generation model preparation module is used to record a public data set for event extraction training as a corresponding first data set; and form N pre-trained large language models into a corresponding first model set; and fine-tune the first large language models of the first model set based on the first data set; N is a preset first number, and the first number N is a positive integer; the first model set includes N first large language models; The enhanced data set generation module is used to identify the event information templates of various event types in the first data set after the fine-tuning training of the first model set is completed to obtain the corresponding first event information template set; and perform data set generation processing based on the first model set and the first event information template set to obtain the corresponding second data set; and perform sample synthesis processing based on the first and second data sets to obtain the corresponding third data set; and save the third data set as an enhanced data set for event extraction training.

10. An electronic device, characterized in that: include: memory, processors, and transceivers; The processor is used to couple with the memory, read and execute instructions in the memory, so as to implement the method according to any one of claims 1 to 8; The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a computer, the computer is enabled to execute the method according to any one of claims 1 to 8.

Images