Event extraction large model construction method based on data enhancement

Abstract

The invention provides an event extraction large model construction method based on data enhancement, and relates to the technical field of data processing.The method comprises the steps that an event labeling system and a first cue word corresponding to an event data set are determined, and the first cue word comprises an event extraction task definition, an event extraction task instruction and sample labeling data; calling a large language model to perform event extraction on the event data set according to the event tagging system and the first cue word to obtain first tagging data; based on a second cue word, calling a large language model to carry out multiple times of iterative updating on the first annotation data to obtain target annotation data, the second cue word comprising a feedback cue word used for carrying out event extraction error feedback and a perfection cue word used for carrying out data perfection based on the generated event extraction error feedback; and obtaining a data enhanced training data set based on the target annotation data and the sample annotation data, and applying the data enhanced training data set to a large language model to obtain a trained event extraction large model.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method for constructing a large event extraction model based on data enhancement. Background Art

[0002] Event extraction is an important and challenging subtask in the field of information extraction, which aims to extract structured event information from unstructured text, mainly including event trigger words, event categories, event arguments, etc. Event extraction has a wide range of applications in automatic summarization, automatic question answering, information retrieval, knowledge graph construction and other fields.

[0003] At present, event extraction mainly relies on manually annotated data, resulting in high costs for related applications. Event extraction is a complex task that requires detailed annotation of text. Traditional event extraction annotation usually involves multiple levels of structure, including event scope identification, event trigger word identification, event type classification, and event argument identification. These annotation tasks all require manual participation, which is time-consuming and laborious. Due to the complexity of the annotated data, the annotation process is prone to inconsistency and subjectivity, making it difficult to establish a high-quality annotated dataset. In addition, the cost of annotated data is also an important issue. Since event extraction annotation requires professional domain knowledge and language understanding ability, the cost of obtaining manually annotated data is very high. However, due to the inherent complexity of the event extraction task, the related supervised event extraction large model requires a large amount of annotated training data to reach a usable level. At present, it mainly relies on manually annotated data, which also leads to high costs for developing event extraction related applications in the real world. Summary of the invention

[0004] The present invention provides a method and device for constructing a large event extraction model based on data enhancement, so as to solve the problems caused by reliance on manually labeled data in the prior art.

[0005] The present invention provides a method for constructing a large event extraction model based on data enhancement, comprising: Determine an event annotation system and a first prompt word corresponding to the event data set, wherein the event annotation system includes events, trigger words, arguments, all event type names, event types, a set of argument roles corresponding to each event type, and at least one item of each argument role, and the first prompt word includes an event extraction task definition, an event extraction task instruction, and sample annotation data of the event data set; According to the event annotation system and the first prompt word, calling a large language model to extract events from the event data set to obtain first annotated data; Based on the second prompt word, calling the large language model to iteratively update the first labeled data multiple times to obtain target labeled data; Based on the second prompt word, calling the large language model to iteratively update the first labeled data multiple times to obtain target labeled data, wherein the second prompt word includes a feedback prompt word for event extraction error feedback and a completion prompt word for data completion based on the generated event extraction error feedback; Based on the target labeled data and the sample labeled data, a data-enhanced training data set is obtained, which is applied to the large language model to obtain a trained event extraction large model.

[0006] According to a method for constructing a large event extraction model based on data enhancement provided by the present invention, the event extraction task definition includes event definition, trigger word definition, argument definition and event type definition related to the event extraction task.

[0007] According to a method for constructing a large event extraction model based on data enhancement provided by the present invention, the first annotated data includes first event data and a first event structure, and the event extraction task instruction is used to instruct the large language model to mark the positions of trigger words and arguments in the first event data when extracting the first event data, and to return the first event structure in the form of a code block, wherein the position mark of the argument is determined based on the position mark of the trigger word.

[0008] According to a method for constructing a large event extraction model based on data enhancement provided by the present invention, before each call of the large language model to extract events from the event dataset, a plurality of sample annotation data belonging to the same event type are randomly sampled from the event dataset.

[0009] According to a method for constructing a large event extraction model based on data enhancement provided by the present invention, based on the second prompt word, calling the large language model to perform multiple iterations to update the first annotated data to obtain target annotated data includes: Determine the second annotated data corresponding to the first annotated data in the last iterative update; During the current iteration update, the feedback prompt word is used to perform error recognition on the second labeled data from a preset inspection dimension to determine an error recognition result; Based on the error recognition result and the second annotation data, generating the improvement prompt word under the current iterative update; Based on the improvement prompt word, the large language model is called to perform data improvement on the second annotated data to obtain third annotated data corresponding to the first annotated data under the current iterative update, until the preset iteration requirement is met, and the third annotated data is determined as the target annotated data.

[0010] According to a method for constructing a large event extraction model based on data enhancement provided by the present invention, the preset inspection dimension includes at least one of the following: Whether the trigger words and arguments in the event structure are subsequences of the event data extracted from the event; Whether the trigger words in the event structure represent the occurrence of the event; Whether the arguments in the event structure participate in the event; Whether the arguments in an event structure play a role in an event given in the event structure; Whether the event data extracted by the event contains arguments that do not exist in the event structure.

[0011] The present application embodiment provides a device for constructing a large event extraction model based on data enhancement, including: A first event extraction data enhancement module, used to determine an event annotation system and a first prompt word corresponding to an event data set, wherein the event annotation system includes events, trigger words, arguments, all event type names, event types, a set of argument roles corresponding to each event type, and at least one item of each argument role, and the first prompt word includes an event extraction task definition, an event extraction task instruction, and sample annotation data of the event data set; A second event extraction data enhancement module is used to call a large language model to extract events from the event data set according to the event annotation system and the first prompt word to obtain first annotation data; a third event extraction data enhancement module, configured to call the large language model to perform multiple iterations on the first annotated data to obtain target annotated data based on a second prompt word, wherein the second prompt word includes a feedback prompt word for event extraction error feedback and a completion prompt word for data completion based on the generated event extraction error feedback; The event extraction large model construction module is used to obtain a data-enhanced training data set based on the target annotation data and the sample annotation data, and apply it to the large language model to obtain a trained event extraction large model.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for constructing a large event extraction model based on data enhancement as described above is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for constructing a large event extraction model based on data enhancement.

[0014] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned methods for constructing a large event extraction model based on data enhancement.

[0015] The present invention provides a method and device for constructing a large event extraction model based on data enhancement. The method and device determine an event annotation system and a first prompt word corresponding to an event data set. The event annotation system includes events, trigger words, arguments, all event type names, event types, a set of argument roles corresponding to each event type, and at least one item of each argument role. The first prompt word includes an event extraction task definition, an event extraction task instruction, and sample annotation data of the event data set. According to the event annotation system and the first prompt word, a large language model is called to perform event extraction on the event data set to obtain first annotation data. Based on a second prompt word, the large language model is called to perform multiple iterations on the first annotation data to obtain target annotation data. The second prompt word includes a feedback prompt word for performing event extraction error feedback and a completion prompt word for performing data completion based on the generated event extraction error feedback. Finally, based on the target labeled data and the sample labeled data, a data-enhanced training data set is obtained, which is applied to the large language model to obtain a trained event extraction large model. Thus, the present invention realizes the extraction of labeled data in the event data set by defining a detailed event labeling system, clear task definitions, task instructions, and prompts for sample labeled data, as well as the powerful capabilities of the large language model, and combines the event extraction error feedback and data improvement The algorithm of alternating iterations between the two generation steps performs self-correction on the extracted labeled data to achieve efficient and accurate event extraction. Finally, combined with the original sample labeled data, a data-enhanced training data set is obtained, and the large language model is fine-tuned in a supervised manner through the data-enhanced training data set to obtain a trained event extraction large model. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A schematic diagram of a method for constructing a large event extraction model based on data enhancement provided in an embodiment of the present application; Figure 2 A schematic diagram of the structure of a device for constructing a large event extraction model based on data enhancement provided in an embodiment of the present application; Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] Figure 1 A flow chart of a method for constructing a large event extraction model based on data enhancement provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, it includes: step 110, step 120 and step 130.

[0020] Step 110, determining an event annotation system and a first prompt word corresponding to the event data set, wherein the event annotation system includes events, trigger words, arguments, all event type names, event types, a set of argument roles corresponding to each event type, and at least one item of each argument role, and the first prompt word includes an event extraction task definition, an event extraction task instruction, and sample annotation data of the event data set; Here, the event data set is a text collection containing multiple event instances. These event instances may come from a variety of sources such as news reports, social media, academic papers, etc. In this embodiment, the event data set includes some sample event data that have been annotated with event structures.

[0021] Here, the event annotation system is a structured framework used to describe and annotate events that occur in the event data recorded in the event dataset. It mainly includes the following parts: events, trigger words, arguments, all event type names, event types, the set of argument roles corresponding to each event type, and each argument role.

[0022] Here, an event refers to a specific situation or action that occurs, such as "earthquake", "conference held", etc. The trigger word is the most critical word or phrase in the event, which directly indicates the occurrence of the event. For example, in "the earthquake caused huge losses", "earthquake" is the trigger word. Arguments are participating entities or attributes in the event, which are used to describe the details of the event. They usually have a clear semantic relationship with the event. For example, the arguments of the "earthquake" event may include "time", "place", "magnitude", etc. The event type refers to the category to which a specific event belongs, such as the above-mentioned "disaster", "conference", etc. In addition, for each event type, there will be one or more argument role sets, which define the arguments and their roles that may be contained in this type of event. Each argument role needs to be assigned at least one entity or attribute value to fully describe the event.

[0023] In this embodiment, when performing an event extraction task, in order to clarify the task goal and guide the annotation work, the designed first prompt word is used to guide the subsequent large language model to generate new annotation data according to the given prompt information.

[0024] Here, the first prompt word includes the event extraction task definition, the event extraction task instruction and the sample annotation data of the event data set.

[0025] Specifically, the event extraction task definition includes event definition, trigger word definition, argument definition and event type definition related to the event extraction task.

[0026] Here, the event extraction task definition needs to give a clear definition of the terms related to the event extraction task. Since event annotation is complex, the annotator needs to accurately understand the above event extraction related term definitions and event annotation rules. Requiring a large model to generate data also requires it to first understand the definitions of "events", "trigger words", "arguments", "argument roles" and the event types to which the data samples need to be generated. Therefore, the definitions of the above terms are first given in the prompt words.

[0027] In addition, in this embodiment, since the name of the argument role itself has sufficient semantic information (such as "borrower", "lender", etc.), the definition of each argument role does not need to be added to the first prompt word.

[0028] Specifically, the first annotated data includes first event data and a first event structure, and the event extraction task instruction is used to instruct the large language model to mark the positions of trigger words and arguments in the first event data when extracting the first event data, and to return the first event structure in the form of a code block, wherein the position mark of the argument is determined based on the position mark of the trigger word.

[0029] Here, the event extraction task instruction is used to instruct the large language model to generate event data from the event dataset according to given requirements and mark relevant information, such as trigger words, arguments, and their position information in the generated event data (such as the starting position and the ending position).

[0030] Therefore, in this embodiment, in order to output the position information of the trigger word in the generated first event data after processing using the large language model, the pre-built event extraction task instructs the large language model to insert special symbols around the trigger word in the generated first event data, so as to achieve the position positioning of the trigger word.

[0031] Furthermore, in this embodiment, considering that there are a large number of arguments in the generated first event data, the position mark of the argument is determined by the position mark of the trigger word. Specifically, the event extraction task instruction is used to indicate that after parsing to obtain the position information of the trigger word, the relevant text of the argument in the generated first event structure is matched in the generated first event data. If multiple results are matched, the result closest to the trigger word is selected as the position information of the argument.

[0032] In addition, the event extraction task instruction is also used to indicate that after processing using the large language model, the first event structure corresponding to the generated first event data is returned in the form of a code block, so that a complete event structure can be more conveniently and intuitively represented. For example, a dictionary in Python is used to define an event structure, the dictionary keys are trigger words and argument roles, and the dictionary values ​​are trigger word related text and arguments.

[0033] Step 120, based on the event annotation system and the first prompt word, calling a large language model to extract events from the event dataset to obtain first annotated data; Here, the first annotation data includes first event data and a first event structure extracted from the event.

[0034] In this embodiment, a large language model is used to extract events from an event data set to extract event data (i.e., first event data) that have not been annotated with event structures, and corresponding event structure annotations (i.e., first event structures). It should be understood that the event structure here includes events, trigger words, arguments, event types, event type names, argument roles, etc. defined in the event annotation system.

[0035] Step 130, based on the second prompt word, calling the large language model to perform multiple iterative updates on the first annotation data to obtain target annotation data, wherein the second prompt word includes a feedback prompt word for event extraction error feedback and a completion prompt word for data completion based on the generated event extraction error feedback.

[0036] In this embodiment, a predefined second prompt word is used to guide the large language model to perform alternating iterations between the two steps of "feedback" and "perfection" on the first annotated data.

[0037] Here, the feedback prompt words can indicate which event extraction results are wrong and the specific types of errors (such as trigger word recognition errors, argument labeling errors, etc.). After the errors are identified, the improvement prompt words are used to guide the large language model to perform data improvement based on the generated event extraction error feedback.

[0038] In this embodiment, feedback prompt words are used to identify errors in the first annotation data. According to the improvement prompt words, the large language model is guided to correct the identified errors. The corrected annotation results are then updated to the current annotation data to form new annotation data. The new annotation version is continued to be evaluated using feedback prompt words to determine whether there are still errors or areas that need further improvement. Based on the evaluation results, new feedback prompt words and improvement prompt words are generated for the next round of iteration. Iteration continues: Repeat the above steps until the preset number of iterations is reached, the quality of the annotation data meets the requirements or cannot be significantly improved. The annotation data obtained after multiple iterative updates is called target annotation data. Compared with the first annotation data, the target annotation data has higher accuracy and completeness.

[0039] Step 140, based on the target labeled data and the sample labeled data, a data-enhanced training data set is obtained, and applied to the large language model to obtain a trained event extraction large model.

[0040] Here, the large language model is a pre-trained or pre-built basic model. In other words, this large language model usually has the basic ability to perform event extraction tasks, but it is not optimal.

[0041] Finally, in this embodiment, the synthesized new target annotation data is mixed into the original sample annotation data, that is, a data-enhanced training data set is obtained. The data-enhanced training data set is used to perform supervised fine-tuning training on the large language model to obtain a trained event extraction large model.

[0042] The method for constructing a large event extraction model based on data enhancement proposed in the embodiment of the present invention realizes the extraction of labeled data in the event data set by defining a detailed event annotation system, clear task definition, task instructions and prompts for sample annotation data and the powerful capabilities of the large language model, and combines the event extraction error feedback and data improvement two generation steps to perform self-correction on the extracted labeled data, thereby achieving efficient and accurate event extraction. Finally, the original sample annotation data is combined to obtain a data-enhanced training data set, and the large language model is fine-tuned in a supervised manner through the data-enhanced training data set to obtain a trained event extraction large model.

[0043] In some embodiments, before calling the large language model to extract events from the event dataset each time, a plurality of sample annotation data belonging to the same event type are randomly sampled from the event dataset.

[0044] It should be noted that although the relevant definitions of event extraction terms are given in the first prompt word, the definitions of terms such as "event", "trigger word", "argument", and "argument role" cannot be accurately understood based on this alone. The generated annotation data does not meet the requirements and the format is different, which is not conducive to parsing the event structure from the generated event data. Therefore, in this embodiment, k sample annotation data with annotated event structures are also provided in the first prompt word to realize the subsequent use of the context learning ability of the large language model, enhance the large language model's understanding of the event extraction task, and guide the large language model to generate new annotation data in the required format.

[0045] Here, in order to increase the diversity of the generated annotation data, a dynamic prompt word strategy is used, that is, each time the large language model is called to generate new annotation data, k sample annotation data belonging to the same event type are randomly sampled as examples, and these sample annotation data are connected in an ordered list and added to the first prompt word.

[0046] In some embodiments, based on the second prompt word, calling the large language model to perform multiple iterations to update the first labeled data to obtain target labeled data includes: Determine the second annotated data corresponding to the first annotated data in the last iterative update; During the current iteration update, the feedback prompt word is used to perform error recognition on the second labeled data from a preset inspection dimension to determine an error recognition result; Based on the error recognition result and the second annotation data, generating the improvement prompt word under the current iterative update; Based on the improvement prompt word, the large language model is called to perform data improvement on the second annotated data to obtain third annotated data corresponding to the first annotated data under the current iterative update, until the preset iteration requirement is met, and the third annotated data is determined as the target annotated data.

[0047] During the current iterative update, the preset feedback prompt words are used to prompt the large language model to detect whether there is an error in a certain dimension in the generated annotation data from multiple preset inspection dimensions, and generate error recognition results. Based on the error recognition results and the second annotation data, the improvement prompt words under the current iterative update are generated. This improvement prompt word is intended to guide the large language model on how to correct the identified errors. Using the generated improvement prompt words, the large language model is called to continue to improve the second annotation data. After the above steps, the third annotation data under the current iterative update is obtained. Then, the above steps are repeated until the preset iteration requirements are met, such as the number of iterations, the threshold of the annotation data quality, and other iteration requirements, and the third annotation data that meets the preset iteration requirements is determined as the target annotation data.

[0048] For example, P_fb is used to represent the feedback prompt word for event extraction error feedback, and P_refine is used to represent the improvement prompt word for data improvement based on the generated event extraction error feedback. In the t-1th iteration of the iterative update, the feedback prompt word P_fb is first used to identify the possible errors in the second labeled data (X_(t-1), Y_(t-1)) under the current t-1th iteration from multiple preset inspection dimensions, and then these identified errors and the second labeled data (X_(t-1), Y_(t-1)) are filled into a pre-designed improvement prompt word P_refine and fed back to the large language model, so that the large language model can continue to improve the second labeled data according to the detected errors and generate the third labeled data (X_t, Y_t).

[0049] Here, the preset inspection dimension includes at least one of the following: Whether the trigger words and arguments in the event structure are subsequences of the event data extracted by the event; for example, check whether the trigger words and arguments actually exist in the annotated text and are part of the text (i.e., subsequence).

[0050] Whether the trigger words in the event structure represent the occurrence of the event; in other words, the trigger words should have clear action or state change characteristics, which can trigger the recognition and understanding of specific events.

[0051] Whether the arguments in the event structure participate in the event; here, each argument must be directly related to the event, that is, they must be part of the event or affected by the event. If the argument is irrelevant to the event, then it should not be included in the event structure.

[0052] Whether the arguments in an event structure play the role given in the event structure in an event; here, the argument must accurately play the role assigned to it in the event structure.

[0053] Does the event data extracted from the event contain arguments that do not exist in the event structure? It should be noted here that some non-existent arguments, namely hallucination arguments, may be introduced during the event extraction or annotation process. These arguments are not part of the original event structure, but they are mistakenly added to the extracted or annotated event data.

[0054] The method for constructing a large event extraction model based on data enhancement proposed in an embodiment of the present invention combines an algorithm that alternates and iterates between the two generation steps of event extraction error feedback and data improvement to self-correct the extracted labeled data, thereby achieving efficient and accurate event extraction.

[0055] Figure 2 The event extraction large model construction device based on data enhancement provided in the embodiment of the present application is as follows: Figure 2 As shown, it includes: a first event extraction data enhancement module 210, a second event extraction data enhancement module 220, a third event extraction data enhancement module 230 and an event extraction large model construction module 240.

[0056] A first event extraction data enhancement module 210 is used to determine an event annotation system and a first prompt word corresponding to an event data set, wherein the event annotation system includes an event, a trigger word, an argument, all event type names, an event type, an argument role set corresponding to each event type, and at least one item of each argument role, and the first prompt word includes an event extraction task definition, an event extraction task instruction, and sample annotation data of the event data set; A second event extraction data enhancement module 220 is used to extract events from the event dataset by calling a large language model according to the event annotation system and the first prompt word to obtain first annotation data; The third event extraction data enhancement module 230 is used to call the large language model to perform multiple iterative updates on the first annotation data based on the second prompt word to obtain the target annotation data, wherein the second prompt word includes a feedback prompt word for event extraction error feedback and a completion prompt word for data completion based on the generated event extraction error feedback.

[0057] The event extraction large model construction module 240 is used to obtain a data-enhanced training data set based on the target annotation data and the sample annotation data, and apply it to the large language model to obtain a trained event extraction large model.

[0058] In the embodiment of the present application, by defining a detailed event annotation system, clear task definitions, task instructions, and prompts for sample annotation data, as well as the powerful capabilities of a large language model, the extraction of annotation data from an event data set is achieved, and the algorithm of alternating iterations between the two generation steps of event extraction error feedback and data improvement is combined to self-correct the extracted annotation data, thereby achieving efficient and accurate event extraction. Finally, the original sample annotation data is combined to obtain a data-enhanced training data set, and the large language model is fine-tuned in a supervised manner through the data-enhanced training data set to obtain a trained event extraction large model.

[0059] Figure 3 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 3As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the event extraction large model construction method based on data enhancement, and the method includes: Determine an event annotation system and a first prompt word corresponding to the event data set, wherein the event annotation system includes events, trigger words, arguments, all event type names, event types, a set of argument roles corresponding to each event type, and at least one item of each argument role, and the first prompt word includes an event extraction task definition, an event extraction task instruction, and sample annotation data of the event data set; According to the event annotation system and the first prompt word, calling a large language model to extract events from the event data set to obtain first annotated data; Based on the second prompt word, calling the large language model to iteratively update the first labeled data multiple times to obtain target labeled data, wherein the second prompt word includes a feedback prompt word for event extraction error feedback and a completion prompt word for data completion based on the generated event extraction error feedback; Based on the target labeled data and the sample labeled data, a data-enhanced training data set is obtained, which is applied to the large language model to obtain a trained event extraction large model.

[0060] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0061] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the event extraction large model construction method based on data enhancement provided by the above methods, the method includes: Determine an event annotation system and a first prompt word corresponding to the event data set, wherein the event annotation system includes events, trigger words, arguments, all event type names, event types, a set of argument roles corresponding to each event type, and at least one item of each argument role, and the first prompt word includes an event extraction task definition, an event extraction task instruction, and sample annotation data of the event data set; According to the event annotation system and the first prompt word, calling a large language model to extract events from the event data set to obtain first annotated data; Based on the second prompt word, calling the large language model to iteratively update the first labeled data multiple times to obtain target labeled data, wherein the second prompt word includes a feedback prompt word for event extraction error feedback and a completion prompt word for data completion based on the generated event extraction error feedback; Based on the target labeled data and the sample labeled data, a data-enhanced training data set is obtained, which is applied to the large language model to obtain a trained event extraction large model.

[0062] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the method for building a large event extraction model based on data enhancement provided by the above methods, the method comprising: Determine an event annotation system and a first prompt word corresponding to the event data set, wherein the event annotation system includes events, trigger words, arguments, all event type names, event types, a set of argument roles corresponding to each event type, and at least one item of each argument role, and the first prompt word includes an event extraction task definition, an event extraction task instruction, and sample annotation data of the event data set; According to the event annotation system and the first prompt word, calling a large language model to extract events from the event data set to obtain first annotated data; Based on the second prompt word, calling the large language model to iteratively update the first labeled data multiple times to obtain target labeled data, wherein the second prompt word includes a feedback prompt word for event extraction error feedback and a completion prompt word for data completion based on the generated event extraction error feedback; Based on the target labeled data and the sample labeled data, a data-enhanced training data set is obtained, which is applied to the large language model to obtain a trained event extraction large model.

[0063] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0064] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a large event extraction model based on data enhancement, characterized in that: include: Determine an event annotation system and a first prompt word corresponding to the event data set, wherein the event annotation system includes events, trigger words, arguments, all event type names, event types, a set of argument roles corresponding to each event type, and at least one item of each argument role, and the first prompt word includes an event extraction task definition, an event extraction task instruction, and sample annotation data of the event data set; According to the event annotation system and the first prompt word, calling a large language model to extract events from the event data set to obtain first annotated data; Based on the second prompt word, calling the large language model to iteratively update the first labeled data multiple times to obtain target labeled data, wherein the second prompt word includes a feedback prompt word for event extraction error feedback and a completion prompt word for data completion based on the generated event extraction error feedback; Based on the target labeled data and the sample labeled data, a data-enhanced training data set is obtained, which is applied to the large language model to obtain a trained event extraction large model.

2. The method for constructing a large event extraction model based on data enhancement according to claim 1, characterized in that: The event extraction task definition includes event definition, trigger word definition, argument definition and event type definition in the event annotation system.

3. The method for constructing a large event extraction model based on data enhancement according to claim 1, characterized in that: The first annotated data includes first event data and a first event structure. The event extraction task instruction is used to instruct the large language model to mark the positions of trigger words and arguments in the first event data when extracting the first event data, and to return the first event structure in the form of a code block, wherein the position mark of the argument is determined based on the position mark of the trigger word.

4. The method for constructing a large event extraction model based on data enhancement according to claim 1, characterized in that: Before calling the large language model to extract events from the event dataset each time, a plurality of sample annotation data belonging to the same event type are randomly sampled from the event dataset.

5. The method for constructing a large event extraction model based on data enhancement according to claim 1, characterized in that: The method of calling the large language model to iteratively update the first labeled data multiple times based on the second prompt word to obtain target labeled data includes: Determine the second annotated data corresponding to the first annotated data in the last iterative update; During the current iteration update, the feedback prompt word is used to perform error recognition on the second labeled data from a preset inspection dimension to determine an error recognition result; Based on the error recognition result and the second annotation data, generating the improvement prompt word under the current iterative update; Based on the improvement prompt word, the large language model is called to perform data improvement on the second annotated data to obtain third annotated data corresponding to the first annotated data under the current iterative update, until the preset iteration requirement is met, and the third annotated data is determined as the target annotated data.

6. The method for constructing a large event extraction model based on data enhancement according to claim 5, characterized in that: The preset inspection dimension includes at least one of the following: Whether the trigger words and arguments in the event structure are subsequences of the event data extracted from the event; Whether the trigger words in the event structure represent the occurrence of the event; Whether the arguments in the event structure participate in the event; Whether the arguments in an event structure play a role in an event given in the event structure; Whether the event data extracted by the event contains arguments that do not exist in the event structure.

7. A device for constructing a large event extraction model based on data enhancement, characterized in that: include: A first event extraction data enhancement module, used to determine an event annotation system and a first prompt word corresponding to an event data set, wherein the event annotation system includes events, trigger words, arguments, all event type names, event types, a set of argument roles corresponding to each event type, and at least one item of each argument role, and the first prompt word includes an event extraction task definition, an event extraction task instruction, and sample annotation data of the event data set; A second event extraction data enhancement module is used to call a large language model to extract events from the event data set according to the event annotation system and the first prompt word to obtain first annotation data; a third event extraction data enhancement module, configured to call the large language model to perform multiple iterations on the first annotated data to obtain target annotated data based on a second prompt word, wherein the second prompt word includes a feedback prompt word for event extraction error feedback and a completion prompt word for data completion based on the generated event extraction error feedback; The event extraction large model construction module is used to obtain a data-enhanced training data set based on the target annotation data and the sample annotation data, and apply it to the large language model to obtain a trained event extraction large model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for constructing a large event extraction model based on data enhancement as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for constructing a large event extraction model based on data enhancement as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for constructing a large event extraction model based on data enhancement as described in any one of claims 1 to 6 is implemented.

Images