Event extraction large model construction method based on data enhancement

Abstract

The application provides an event extraction large model construction method based on data enhancement, relates to the technical field of data processing, and comprises the following steps: determining an event annotation system corresponding to an event data set and a first prompt word, wherein the first prompt word comprises event extraction task definition, event extraction task instruction and sample annotation data; calling a large language model to perform event extraction on the event data set according to the event annotation system and the first prompt word, and obtaining first annotation data; calling the large language model to perform multiple iterations on the first annotation data based on a second prompt word, and obtaining target annotation data, wherein the second prompt word comprises a feedback prompt word for event extraction error feedback and a perfecting prompt word for data perfection based on the generated event extraction error feedback; obtaining a data-enhanced training data set based on the target annotation data and the sample annotation data, applying the data-enhanced training data set to the large language model, and obtaining a trained event extraction large model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to an event extraction large model construction method based on data enhancement. BACKGROUND

[0002] Event extraction is an important and challenging subtask in the field of information extraction, aiming to extract structured event information from unstructured text, mainly including event trigger words, event categories, event arguments, etc. Event extraction has a wide range of applications in automatic summarization, automatic question answering, information retrieval, knowledge graph construction, etc.

[0003] Currently, event extraction mainly relies on manually annotated data, resulting in high costs for related applications. Event extraction is a complex task that requires careful annotation of text. Traditional event extraction annotation usually involves multiple levels of structure, including event range identification, event trigger word identification, event type classification, and event argument identification. These annotation tasks all require human involvement, which is time-consuming and labor-intensive. Due to the complexity of the annotated data, inconsistencies and subjectivity can easily occur during the annotation process, making it difficult to establish a high-quality annotated dataset. In addition, the cost of annotated data is also an important issue. Since event extraction annotation requires specialized domain knowledge and language understanding capabilities, the cost of obtaining manually annotated data is very high. However, due to the inherent complexity of the event extraction task, related supervised event extraction large models require a large amount of annotated training data to reach a usable level, and currently mainly rely on manually annotated data, which also results in high costs for developing event extraction-related applications in the real world. SUMMARY

[0004] The present application provides an event extraction large model construction method and device based on data enhancement, to solve the problem of relying on manually annotated data in the prior art.

[0005] The present application provides an event extraction large model construction method based on data enhancement, comprising:

[0006] determining an event annotation system corresponding to an event dataset and a first prompt word, the event annotation system including events, trigger words, arguments, all event type names, event types, argument role sets corresponding to each event type, and at least one of each argument role, and the first prompt word including event extraction task definitions, event extraction task instructions, and sample annotation data of the event dataset;

[0007] According to the event annotation system and the first prompt word, a large language model is called to perform event extraction on the event dataset to obtain first annotation data;

[0008] Based on the second prompt word, the large language model is called to update the first annotation data multiple times to obtain target annotation data;

[0009] Based on the second prompt word, the large language model is called to update the first annotation data multiple times to obtain target annotation data, and the second prompt word includes a feedback prompt word for event extraction error feedback and a perfect prompt word for data perfection based on the generated event extraction error feedback.

[0010] Based on the target annotation data and the sample annotation data, data-enhanced training data sets are obtained, which are applied to the large language model to obtain a trained event extraction large model.

[0011] According to the event extraction large model construction method based on data enhancement provided by the application, the event extraction task definition includes event definition, trigger word definition, argument definition and event type definition related to the event extraction task.

[0012] According to the event extraction large model construction method based on data enhancement provided by the application, the first annotation data includes first event data and first event structure, and the event extraction task instruction is used to instruct the large language model to mark the position of the trigger word and argument in the first event data when extracting the first event data, and return the first event structure in the form of a code block, wherein the position marking of the argument is determined based on the position marking of the trigger word.

[0013] According to the event extraction large model construction method based on data enhancement provided by the application, before calling the large language model to extract events from the event data set each time, a plurality of sample annotation data belonging to the same event type are randomly sampled from the event data set.

[0014] According to the event extraction large model construction method based on data enhancement provided by the application, based on the second prompt word, the large language model is called to update the first annotation data multiple times to obtain target annotation data, comprising:

[0015] Determine the second annotation data corresponding to the first annotation data in the last iteration update;

[0016] In the current iteration update, the feedback prompt word is used to identify errors in the second annotation data from a preset checking dimension to determine an error identification result;

[0017] Based on the error identification result and the second annotation data, the perfect prompt word in the current iteration update is generated;

[0018] Based on the perfect prompt word, the large language model is called to perform data perfection on the second labeled data, to obtain third labeled data corresponding to the first labeled data under current iterative updating, until a preset iteration requirement is reached, and the third labeled data is determined as target labeled data.

[0019] According to the event extraction large model construction method based on data enhancement provided by the application, the preset checking dimension includes at least one of the following:

[0020] Whether the trigger word and the argument in the event structure are sub-sequences of the event data extracted by the event extraction;

[0021] Whether the trigger word in the event structure represents the occurrence of the event;

[0022] Whether the argument in the event structure participates in the event;

[0023] Whether the argument in the event structure plays a given role in the event structure;

[0024] Whether the argument in the event data extracted by the event extraction does not exist in the event structure.

[0025] The application embodiment provides an event extraction large model construction device based on data enhancement, comprising:

[0026] The first event extraction data enhancement module is configured to determine an event annotation system corresponding to an event data set and a first prompt word, the event annotation system includes an event, a trigger word, an argument, all event type names, an event type, an argument role set corresponding to each event type, and at least one of each argument role, and the first prompt word includes an event extraction task definition, an event extraction task instruction, and sample labeled data of the event data set;

[0027] The second event extraction data enhancement module is configured to call a large language model to perform event extraction on the event data set according to the event annotation system and the first prompt word, to obtain first labeled data;

[0028] The third event extraction data enhancement module is configured to call the large language model to perform multiple iterative updates on the first labeled data based on a second prompt word, to obtain target labeled data, and the second prompt word includes a feedback prompt word for event extraction error feedback and a perfect prompt word for data perfection based on the generated event extraction error feedback;

[0029] The event extraction large model construction module is configured to obtain a data-enhanced training data set based on the target labeled data and the sample labeled data, apply the data-enhanced training data set to the large language model, and obtain a trained event extraction large model.

[0030] The application further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the method for constructing a large event extraction model based on data enhancement according to any one of the above methods when executing the program.

[0031] The application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and the computer program is executable by a processor to implement the method for constructing a large event extraction model based on data enhancement according to any one of the above methods.

[0032] The application further provides a computer program product, including a computer program, and the computer program is executable by a processor to implement the method for constructing a large event extraction model based on data enhancement according to any one of the above methods.

[0033] The application provides a method and device for constructing a large event extraction model based on data enhancement, which determines an event annotation system corresponding to an event data set and a first prompt word, the event annotation system includes an event, a trigger word, an argument, all event type names, an event type, an argument role set corresponding to each event type, and at least one of each argument role, and the first prompt word includes an event extraction task definition, an event extraction task instruction, and sample annotation data of the event data set; according to the event annotation system and the first prompt word, a large language model is called to perform event extraction on the event data set to obtain first annotation data; based on a second prompt word, the large language model is called to perform multiple iterations on the first annotation data to obtain target annotation data, and the second prompt word includes a feedback prompt word for event extraction error feedback and a perfecting prompt word for data perfecting based on the generated event extraction error feedback. Finally, based on the target annotation data and the sample annotation data, a training data set with data enhancement is obtained, which is applied to the large language model to obtain a trained event extraction large model. Thus, the application realizes extraction of annotation data in the event data set by defining a detailed event annotation system, clear task definition, task instruction, and sample annotation data prompt, and the powerful capability of the large language model, and realizes efficient and accurate event extraction by combining the algorithm of alternating iterations between the two generation steps of event extraction error feedback and data perfecting to self-correct the extracted annotation data. Finally, the training data set with data enhancement is obtained in combination with the original sample annotation data, the large language model is supervised and fine-tuned by the training data set with data enhancement, and the trained event extraction large model is obtained. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to make the technical solutions in the present application or the prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those of ordinary skill in the art without any creative effort.

[0035] Figure 1 The flowchart of the method for constructing the event extraction large model based on data enhancement provided by the embodiment of the present application is shown in the figure.

[0036] Figure 2 The structural schematic diagram of the device for constructing the event extraction large model based on data enhancement provided by the embodiment of the present application is shown in the figure.

[0037] Figure 3 The structural schematic diagram of the electronic device provided by the present application is shown in the figure. DETAILED DESCRIPTION

[0038] In order to make the technical solutions in the present application or the prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those of ordinary skill in the art without any creative effort.

[0039] Figure 1 The flowchart of the method for constructing the event extraction large model based on data enhancement provided by the embodiment of the present application is shown in the figure. Figure 1 As shown in the figure, the method comprises steps 110, 120 and 130.

[0040] Step 110: determining an event annotation system corresponding to an event data set and a first prompt word, the event annotation system comprising an event, a trigger word, an argument, all event type names, an event type, an argument role set corresponding to each event type, and at least one of each argument role, and the first prompt word comprising an event extraction task definition, an event extraction task instruction and sample annotation data of the event data set.

[0041] Here, the event data set is a text collection containing multiple event instances. These event instances can come from news reports, social media, academic papers and other sources. In this embodiment, the event data set includes some sample event data that has been annotated with event structure.

[0042] Here, the event annotation system is a structured framework for describing and annotating events recorded in the event data set, mainly including the following parts: event, trigger word, argument, all event type names, event type, argument role set corresponding to each event type, and each argument role.

[0043] Here, the event refers to a specific occurrence or action described, such as "earthquake", "meeting held" and the like. The trigger word is the most critical word or phrase in the event, which directly indicates the occurrence of the event. For example, in "earthquake caused great loss", "earthquake" is the trigger word. The argument is the participating entity or attribute in the event, which describes the detailed situation of the event. They usually have a clear semantic relationship with the event, such as the arguments of the "earthquake" event may include "time", "place", "magnitude" and the like. The event type refers to the category to which a specific event belongs, such as "disaster", "meeting" and the like. In addition, for each event type, there will be one or more argument role sets, which define the arguments that the event type may contain and their roles. And each argument role needs to be assigned at least one entity or attribute value to fully describe the event.

[0044] In this embodiment, when performing the event extraction task, in order to clarify the task target and guide the annotation work, the first prompt word is used to guide the subsequent large language model to generate new annotation data according to the given prompt information.

[0045] Here, the first prompt word includes event extraction task definition, event extraction task instruction, and sample annotation data of the event data set.

[0046] Specifically, the event extraction task definition includes event definition, trigger word definition, argument definition, and event type definition related to the event extraction task.

[0047] Here, the event extraction task definition needs to give a clear definition of the terms related to the event extraction task. Since event annotation is complex, the annotator needs to accurately understand the definitions of the above event extraction related terms and the rules of event annotation. The large model also needs to understand the definitions of "event", "trigger word", "argument", "argument role" and the event type to which the generated data sample belongs before generating data. Therefore, the definitions of the above terms are first given in the prompt word.

[0048] In addition, in this embodiment, since the names of argument roles themselves have sufficient semantic information (such as "borrower", "lender", etc.), the definition of each argument role can not be added to the first prompt word.

[0049] Specifically, the first annotation data includes first event data and a first event structure, the event extraction task instruction is used to instruct the large language model to mark the positions of trigger words and arguments in the first event data when extracting the first event data, and return the first event structure in the form of a code block, wherein the position marking of the arguments is determined based on the position marking of the trigger words.

[0050] Here, the event extraction task instruction is used to instruct the large language model to generate event data from the event data set according to the given requirements and mark relevant information such as trigger words, arguments, and their position information (such as start position and end position) in the generated event data.

[0051] Therefore, in this embodiment, in order to output the position information of the trigger words in the generated first event data after processing using the large language model, the event extraction task is used to instruct the large language model to insert special symbols around the trigger words in the generated first event data, so as to realize the position positioning of the trigger words.

[0052] Further, in this embodiment, considering that the number of arguments in the generated first event data is large, the position marking of the arguments is determined based on the position marking of the trigger words. Specifically, the event extraction task instruction is used to instruct that after the position information of the trigger words is parsed, the relevant text of the arguments in the generated first event structure is matched in the generated first event data. If multiple results are matched, the result closest to the trigger word is selected as the position information of the argument.

[0053] In addition, the event extraction task instruction is also used to instruct that after processing using the large language model, the first event structure corresponding to the generated first event data is returned in the form of a code block, so that a complete event structure can be more conveniently and intuitively represented. For example, an event structure is defined using a dictionary in Python, the keys of the dictionary are trigger words and argument roles, and the values of the dictionary are trigger-related text and arguments.

[0054] Step 120, according to the event annotation system and the first prompt word, calling a large language model to perform event extraction on the event data set to obtain first annotation data;

[0055] Here, the first annotation data includes first event data extracted by the event extraction and a first event structure.

[0056] In this embodiment, a large language model is used to perform event extraction from an event dataset to extract events that are not annotated with event structures (i.e., first event data) and corresponding event structure annotations (i.e., first event structures). It should be understood that the event structures herein include events, trigger words, arguments, event types, event type names, argument roles, and the like defined in an event annotation system.

[0057] At step 130, based on a second prompt word, the large language model is called to perform multiple iterations of updating the first annotation data to obtain target annotation data. The second prompt word includes a feedback prompt word for event extraction error feedback and a perfecting prompt word for data perfection based on the generated event extraction error feedback.

[0058] In this embodiment, the second prompt word is used to guide the large language model to perform alternating iterations between the "feedback" and "perfecting" steps.

[0059] Here, the feedback prompt word can indicate which event extraction results are incorrect and the specific types of errors (e.g., trigger word recognition errors, argument annotation errors, etc.). After identifying the errors, the perfecting prompt word is used to guide the large language model to perform data perfection based on the generated event extraction error feedback.

[0060] In this embodiment, the feedback prompt word is used to identify errors in the first annotation data. Based on the perfecting prompt word, the large language model is guided to correct the identified errors. The corrected annotation results are then updated to the current annotation data to form new annotation data. The feedback prompt word is used to evaluate the new annotation version to determine whether there are still errors or further improvements are needed. Based on the evaluation results, new feedback prompt words and perfecting prompt words are generated for the next iteration. The iteration continues: repeat the above steps until a preset number of iterations is reached, the annotation data quality meets the requirements, or no further significant improvements can be made. The annotation data obtained after multiple iterations is referred to as target annotation data. Compared with the first annotation data, the target annotation data has higher accuracy and completeness.

[0061] At step 140, based on the target annotation data and the sample annotation data, data-enhanced training data sets are obtained for the large language model, and a trained event extraction large model is obtained.

[0062] Here, the large language model is a pre-trained or constructed base model. In other words, the large language model generally has the basic ability to perform event extraction tasks, but is not optimal.

[0063] Finally, in this embodiment, the synthesized new target annotation data is mixed into the original sample annotation data, that is, the data enhanced training data set is obtained, and the large language model is supervised fine-tuning training using the data enhanced training data set, so that the trained event extraction large model is obtained.

[0064] The event extraction large model construction method based on data enhancement proposed in the embodiment of the application realizes the extraction of the annotation data in the event data set by defining a detailed event annotation system, clear task definition, task instruction and sample annotation data prompt and the powerful capability of the large language model, and the extracted annotation data is self-corrected by combining the alternating iteration algorithm between the two generation steps of event extraction error feedback and data perfection, so that efficient and accurate event extraction is realized. Finally, the original sample annotation data is combined to obtain the data enhanced training data set, and the large language model is supervised fine-tuning training using the data enhanced training data set, so that the trained event extraction large model is obtained.

[0065] In some embodiments, before each call of the large language model for event extraction on the event data set, a plurality of sample annotation data belonging to the same event type are randomly sampled from the event data set.

[0066] It should be noted that although the related definitions of the event extraction terms are given in the first prompt word, the definitions of the terms such as "event", "trigger word", "argument", "argument role" cannot be accurately understood only according to this, and the generated annotation data does not meet the requirements and the format is not uniform, which is not conducive to parsing the event structure from the generated event data. Therefore, in this embodiment, k sample annotation data of annotated event structures are also provided in the first prompt word to realize subsequent utilization of the context learning capability of the large language model, enhancement of the understanding of the large language model on the event extraction task, and guidance of the large language model to generate new annotation data according to the required format.

[0067] Here, in order to increase the diversity of the generated annotation data, a dynamic prompt word strategy is used, that is, each time the large language model is called to generate new annotation data, k sample annotation data belonging to the same event type are randomly sampled as examples, and these sample annotation data are connected in the form of an ordered list and added to the first prompt word.

[0068] In some embodiments, the calling of the large language model on the first annotation data for multiple iteration updates based on the second prompt word to obtain the target annotation data comprises:

[0069] determining the second annotation data corresponding to the first annotation data after the last iteration update;

[0070] In the current iteration update, the feedback prompt word is used to identify errors in the second annotation data from the preset check dimension, and an error identification result is determined;

[0071] Based on the error identification result and the second annotation data, the perfect prompt word under the current iteration update is generated;

[0072] Based on the perfect prompt word, the large language model is called to perform data perfection on the second annotation data, and the third annotation data corresponding to the first annotation data under the current iteration update is obtained, until the preset iteration requirement is reached, and the third annotation data is determined as the target annotation data.

[0073] In the current iteration update, a preset feedback prompt word is used to prompt the large language model to detect whether there is an error in a certain dimension in the generated annotation data from multiple preset check dimensions, and an error identification result is generated. Based on the error identification result and the second annotation data, a perfect prompt word under the current iteration update is generated. This perfect prompt word aims to guide the large language model to correct the identified errors. Using the generated perfect prompt word, the large language model is called to continue data perfection on the second annotation data. After the above steps, the third annotation data under the current iteration update is obtained. Then, the above steps are repeated until the preset iteration requirement is reached, such as the iteration number, the threshold of the annotation data quality, and other iteration requirements, and the third annotation data that meets the preset iteration requirement is determined as the target annotation data.

[0074] For example, P_fb is used to represent the feedback prompt word for event extraction error feedback, and P_refine is used to represent the perfect prompt word for data perfection based on the generated event extraction error feedback. In the t-1 step of iteration update, the feedback prompt word P_fb is first used to identify possible errors in the second annotation data (X_(t-1), Y_(t-1)) under the current t-1 step of iteration from multiple preset check dimensions, and then these identified errors and the second annotation data (X_(t-1), Y_(t-1)) are filled into a pre-designed perfect prompt word P_refine to feed back to the large language model, so that the large language model can continue to perfect the second annotation data based on the detected errors to generate the third annotation data (X_t, Y_t).

[0075] Here, the preset check dimension includes at least one of the following:

[0076] Whether the trigger word and the argument in the event structure are a subsequence of the event data extracted by event extraction; for example, it is checked whether the trigger word and the argument actually exist in the annotated text and are part of the text (i.e., a subsequence).

[0077] whether the trigger word in the event structure represents the occurrence of the event; in other words, the trigger word should have clear action or state change, which can trigger the identification and understanding of a specific event.

[0078] whether the argument in the event structure participates in the event; here, it is required that each argument must have a direct relationship with the event, that is, they must be part of the event or affected by the event. If the argument is not related to the event, it should not be included in the event structure.

[0079] whether the argument in the event structure plays the role given in the event structure in an event; here, it is required that the argument must accurately play the role it is given in the event structure.

[0080] whether the event data extracted from the event contains arguments that do not exist in the event structure; it is noted here that during the process of event extraction or annotation, some non-existent arguments, i.e. illusory arguments, may be introduced. These arguments are not part of the original event structure, but they are mistakenly added to the extracted or annotated event data.

[0081] The event extraction large model construction method based on data enhancement provided in the embodiments of the present application combines the algorithm of alternating iteration between the two generation steps of event extraction error feedback and data improvement to self-correct the extracted and annotated data, realizing efficient and accurate event extraction.

[0082] Figure 2 The event extraction large model construction device based on data enhancement provided for the embodiments of the present application, as shown in Figure 2 includes a first event extraction data enhancement module 210, a second event extraction data enhancement module 220, a third event extraction data enhancement module 230, and an event extraction large model construction module 240.

[0083] The first event extraction data enhancement module 210 is configured to determine an event annotation system corresponding to an event data set and a first prompt word, the event annotation system including an event, a trigger word, an argument, all event type names, an event type, a set of argument roles corresponding to each event type, and at least one of each argument role, and the first prompt word including an event extraction task definition, an event extraction task instruction, and sample annotation data of the event data set.

[0084] The second event extraction data enhancement module 220 is configured to call a large language model to perform event extraction on the event data set according to the event annotation system and the first prompt word, and obtain first annotation data.

[0085] The third event extraction data enhancement module 230 is configured to call the large language model to update the first annotation data multiple times based on a second prompt word to obtain target annotation data, the second prompt word including a feedback prompt word for event extraction error feedback and a perfecting prompt word for data perfecting based on the generated event extraction error feedback.

[0086] The event extraction large model construction module 240 is configured to obtain data-enhanced training data set based on the target annotation data and the sample annotation data, and apply the data-enhanced training data set to the large language model to obtain a trained event extraction large model.

[0087] In the embodiments of the present application, by defining a detailed event annotation system, clear task definition, task instruction, prompt of sample annotation data and powerful large language model, the annotation data in the event data set is extracted, and the extracted annotation data is self-corrected by combining the alternating iterative algorithm between the two generation steps of event extraction error feedback and data perfecting, realizing efficient and accurate event extraction. Finally, combined with the original sample annotation data, a data-enhanced training data set is obtained, and the large language model is supervised fine-tuning trained through the data-enhanced training data set, so as to obtain a trained event extraction large model.

[0088] Figure 3 is a structural schematic diagram of an electronic device provided by the present application, as Figure 3 shown, the electronic device can include a processor 310, a communications interface 320, a memory 330 and a communications bus 340, wherein the processor 310, the communications interface 320, the memory 330 complete mutual communication through the communications bus 340. The processor 310 can call the logic instructions in the memory 330 to execute the event extraction large model construction method based on data enhancement, the method comprising:

[0089] determining an event annotation system corresponding to an event data set and a first prompt word, the event annotation system including events, trigger words, arguments, all event type names, event types, argument role sets corresponding to each event type, and at least one of each argument role, the first prompt word including event extraction task definition, event extraction task instruction and sample annotation data of the event data set;

[0090] According to the event annotation system and the first prompt word, the large language model is called to perform event extraction on the event data set to obtain first annotation data;

[0091] Based on the second prompt word, the large language model is called to update the first annotation data multiple times to obtain target annotation data, and the second prompt word includes a feedback prompt word for event extraction error feedback and a perfect prompt word for data perfection based on the generated event extraction error feedback.

[0092] Based on the target annotation data and the sample annotation data, data-enhanced training data sets are obtained, which are applied to the large language model to obtain a trained event extraction large model.

[0093] In addition, the logical instructions in the memory 330 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to make a computer device (which can be a personal computer, server, or network device, etc.) execute all or part of the steps of the method described in various embodiments of the present application. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, and various media that can store program codes.

[0094] On the other hand, the present application also provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor, so that the computer can execute the event extraction large model construction method based on data enhancement provided by the above-mentioned method, which comprises:

[0095] determining an event annotation system corresponding to an event data set and a first prompt word, the event annotation system comprising events, trigger words, arguments, all event type names, event types, argument role sets corresponding to each event type, and at least one of each argument role, and the first prompt word comprising event extraction task definition, event extraction task instruction and sample annotation data of the event data set;

[0096] According to the event annotation system and the first prompt word, the large language model is called to perform event extraction on the event data set to obtain first annotation data;

[0097] Based on the second prompt word, the large language model is called to update the first annotation data for multiple iterations to obtain target annotation data, and the second prompt word includes a feedback prompt word for event extraction error feedback and a perfect prompt word for data perfection based on the generated event extraction error feedback.

[0098] Based on the target annotation data and the sample annotation data, a data-enhanced training data set is obtained, which is applied to the large language model to obtain a trained event extraction large model.

[0099] In another aspect, the present application also provides a non-transitory computer readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data-enhanced event extraction large model construction method provided by the above methods, and the method comprises:

[0100] determining an event annotation system corresponding to an event data set and a first prompt word, the event annotation system including events, trigger words, arguments, all event type names, event types, argument role sets corresponding to each event type, and at least one of each argument role, and the first prompt word including event extraction task definition, event extraction task instruction and sample annotation data of the event data set;

[0101] According to the event annotation system and the first prompt word, a large language model is called to perform event extraction on the event data set to obtain first annotation data;

[0102] Based on a second prompt word, the large language model is called to update the first annotation data for multiple iterations to obtain target annotation data, and the second prompt word includes a feedback prompt word for event extraction error feedback and a perfect prompt word for data perfection based on the generated event extraction error feedback;

[0103] Based on the target annotation data and the sample annotation data, a data-enhanced training data set is obtained, which is applied to the large language model to obtain a trained event extraction large model.

[0104] The device embodiments described above are only schematic, wherein the units shown as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement it without creative labor.

[0105] Those skilled in the art can clearly understand the technical solutions of the various embodiments from the above description of the embodiments, and the various embodiments can be implemented by means of software with the necessary general hardware platforms, and of course, can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that makes a contribution, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in the various embodiments or some parts of the embodiments.

[0106] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data enhancement-based event extraction large model construction method, characterized in that, The method comprises the following steps: determining an event annotation system corresponding to an event data set and a first prompt word, the event annotation system comprising at least one of an event, a trigger word, an argument, all event type names, an event type, an argument role set corresponding to each event type, and the first prompt word comprising an event extraction task definition, an event extraction task instruction, and sample annotation data of the event data set; calling a large language model to perform event extraction on the event data set according to the event annotation system and the first prompt word, and obtaining first annotation data; based on a second prompt word, calling the large language model to perform multiple iterations of updating on the first annotation data to obtain target annotation data, the second prompt word comprising a feedback prompt word for event extraction error feedback and a perfecting prompt word for data perfecting based on the generated event extraction error feedback; based on the target annotation data and the sample annotation data, obtaining a data-enhanced training data set, applying it to the large language model, and obtaining a trained event extraction large model; the event extraction task definition comprises event definition, trigger word definition, argument definition and event type definition in the event annotation system; the first annotation data comprises first event data and a first event structure, and the event extraction task instruction is used to instruct the large language model to mark the positions of trigger words and arguments in the first event data when extracting the first event data, and return the first event structure in the form of a code block, wherein the position marking of the argument is determined based on the position marking of the trigger word; wherein, before calling the large language model to perform event extraction on the event data set each time, a plurality of sample annotation data belonging to the same event type are randomly sampled from the event data set.

2. The data augmentation-based event extraction large model construction method according to claim 1, characterized in that, the second prompt word, calling the large language model to perform multiple iterations of updating on the first annotation data to obtain target annotation data, comprising: determining second annotation data corresponding to the first annotation data after the last iteration update; in the current iteration update, using the feedback prompt word to identify errors in the second annotation data from a preset checking dimension to determine an error identification result; based on the error identification result and the second annotation data, generating the perfecting prompt word under the current iteration update; based on the perfecting prompt word, calling the large language model to perfect the data of the second annotation data to obtain third annotation data corresponding to the first annotation data under the current iteration update, until a preset iteration requirement is reached, and determining the third annotation data as the target annotation data.

3. The data augmentation-based event extraction large model construction method according to claim 2, characterized in that, the preset checking dimension comprises at least one of the following: whether the trigger word and the argument in the event structure are subsequences of the event data extracted by event extraction; whether the trigger word in the event structure represents the occurrence of an event; whether the argument in the event structure participates in the event; whether the argument in the event structure plays a given role in the event structure in an event; whether the argument in the event structure is contained in the event data extracted by event extraction.

4. A data augmentation-based event extraction large model construction device, characterized in that, ​ The first event extraction data enhancement module is configured to determine an event annotation system corresponding to an event data set and a first prompt word, the event annotation system including at least one of an event, a trigger word, an argument, all event type names, an event type, an argument role set corresponding to each event type, and the first prompt word including an event extraction task definition, an event extraction task instruction, and sample annotation data of the event data set. The second event extraction data enhancement module is configured to call a large language model to perform event extraction on the event data set according to the event annotation system and the first prompt word, to obtain first annotation data. The third event extraction data enhancement module is configured to call the large language model to iteratively update the first annotation data based on a second prompt word to obtain target annotation data, the second prompt word including a feedback prompt word for event extraction error feedback and a perfecting prompt word for data perfecting based on the generated event extraction error feedback. The event extraction large model construction module is configured to obtain a data-enhanced training data set based on the target annotation data and the sample annotation data, apply the data-enhanced training data set to the large language model, and obtain a trained event extraction large model. The event extraction task definition includes event definition, trigger word definition, argument definition, and event type definition in the event annotation system. The first annotation data includes first event data and a first event structure, the event extraction task instruction is used to instruct the large language model to mark the positions of trigger words and arguments in the first event data when extracting the first event data, and return the first event structure in the form of a code block, wherein the position marking of the arguments is determined based on the position marking of the trigger words. Before calling the large language model to perform event extraction on the event data set each time, a plurality of sample annotation data belonging to the same event type are randomly sampled from the event data set.

5. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the data-enhanced event extraction large model construction method of any one of claims 1 to 3.

6. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the data-enhanced event extraction large model construction method of any one of claims 1 to 3.

7. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the data-enhanced event extraction large model construction method of any one of claims 1 to 3.

Images