Long text event extraction method and system based on abstract generation

Abstract

The invention belongs to the technical field of event extraction, and discloses a long text event extraction method and system based on abstract generation. The method comprises the following steps: defining the type of an event to be extracted and an event argument role; preparing and preprocessing annotation data, training an event extraction model in a low-resource scene through sample data, and evaluating and storing the event extraction model; and carrying out fine tuning on the large model in a Prompt instruction fine tuning mode, and training an abstract generation model of the long text. And integrating the abstract generation model and the event extraction model of the long text to complete event extraction of the long text. By generating the abstract content of the long text and performing event extraction based on the abstract content, event information in the text can be extracted more accurately, and the long text event extraction precision is greatly improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of event extraction, and in particular relates to a long text event extraction method and system based on summary generation. Background Art

[0002] Most existing event extraction methods rely on a large amount of labeled data to train the model. However, in low-resource scenarios, due to the scarcity of labeled data, traditional event extraction models are difficult to achieve ideal performance. In addition, event information in long texts is often relatively scattered, and directly extracting events from long texts has the problems of large computational complexity and low efficiency. Directly labeling data for long texts is difficult, time-consuming, and labor-intensive, with high labor costs. Traditional summary generation methods can generate relatively accurate summaries, but they often focus on retaining the overall information and key points of the text when generating summaries, while ignoring the extraction and retention of specific event information, and cannot fully meet the needs of event extraction.

[0003] Through the above analysis, the problems and defects of the existing technology are: low efficiency, large amount of calculation, low accuracy of long text event extraction in low-resource scenarios, and difficulty in long text event extraction and annotation in zero-sample situations. In addition, traditional summary generation methods focus more on summarizing text content, and do not necessarily directly optimize for event extraction needs. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present invention provides a long text event extraction method and system based on summary generation.

[0005] The technical solution is as follows: A long text event extraction method based on summary generation comprises the following steps: S1, define the type of event to be extracted and the role of event arguments; S2, labeling data preparation and preprocessing, training the event extraction model in low-resource scenarios through sample data, and evaluating and saving the event extraction model; S3, fine-tuning the large model through the Prompt instruction fine-tuning method, training the summary generation model of long text.

[0006] S4 integrates the summary generation model and event extraction model of long text to complete event extraction of long text.

[0007] In step S1, the types of events to be extracted and the role of event arguments are defined, and a schema dictionary for event extraction is generated based on the defined event types and event argument roles.

[0008] In step S2, the labeled data is prepared, including: collecting Chinese open source labeled data, integrating data and selecting a small amount of high-quality data, and manually labeling some event type data; Preprocessing includes: aligning the annotations of labeled data from different sources; constructing negative data, and the relationship between negative data and positive data is expressed as: the amount of negative data = 5 × the amount of positive data; converting the data set into the input data format of the model, and dividing the data into training set, validation set and test set in a ratio of 8:1:1.

[0009] In step S2, the event extraction model in the low-resource scenario is trained by sample data, including the following steps: (2.1) Basic model selection: The universal information extraction model uie-base is selected as the basic model of the event extraction model; (2.2) Configure training hyperparameters: Configure the training hyperparameters of the event extraction model according to the task characteristics, dataset characteristics, and training resources. The configured training hyperparameters include: learning rate of 1e-5, maximum sequence length of 512, batch size of 16, training rounds of 10, and AdamW optimizer; (2.3) Event extraction model training: Load the training set and validation set data for training. During the training process, the event extraction model is trained on the training set and evaluated on the validation set to monitor the performance changes of the event extraction model during the training process. (2.4) Event extraction model effect evaluation: The event extraction model is evaluated using the macro average precision Macro_P, macro average recall Macro_R and macro average F1 value Macro_F1. The event extraction model evaluation process includes event type recognition evaluation and event argument recognition evaluation. (2.5) Model iterative optimization: Based on the evaluation results of the above event extraction model, the event extraction model is iteratively optimized and trained by adjusting the training hyperparameters and data enhancement strategies; (2.6) Save the event extraction model: After training is completed, save the model file. The saved content includes the weight and bias parameter information of the event extraction model and the structural information of the event extraction model.

[0010] In step S3, a summary generation model for a long text is trained, including the following steps: (3.1) Selection of base large language model: The large language model Qwen2-72B-Instruct is used as the base model of the long text summary extraction model; (3.2) Large model deployment and interface encapsulation: Based on the vLLM framework, deploy the large language model Qwen2-72B-Instruct to achieve offline reasoning, and encapsulate the large language model reasoning service into an OpenAI format HTTP service; (3.3) Prompt instruction design: Prompt instructions are designed to guide the large model to complete the task of generating summaries of long texts. The instruction design adopts the ideas of establishing roles, setting goals and requirements, supplementing information, and controlling output format requirements. (3.4) Interact with the large model to generate summary results: Through the HTTP interface service, input the prompt command and configure the model name & path, maximum model length, and multi-GPU operation parameter path to interact with the large model, so that the large model can perform summary generation tasks according to the command requirements and return results; (3.5) Optimization and testing: Fine-tune and optimize the large model by optimizing the prompt so that the model output meets the expected results; (3.6) Post-processing of results: The output results of the large language model are processed by manual verification and summary.

[0011] In step (3.3), establishing roles includes: defining the role information played by the large model, including role identity, role skills, and role task objectives; Setting goals and requirements includes: clarifying the task objectives of the large model, as well as the content, format, and word count requirements when completing the task; Supplementary information: By introducing few-shot context learning, explanatory information, and world knowledge, the model is assisted in completing the task. By adding one-shot, the large model is assisted in completing the final task. Output format requirement: The large model output format is output in the form of one line of text.

[0012] In step (3.5), the large model is fine-tuned and optimized by optimizing prompt so that the model output meets the expected results, including the following steps: (1) Adjust the details and format of the prompt: including adjusting the use of certain details and changing the word order of the instruction; (2) Adding suggestive enumeration: During the test, the time information that is easy to be lost in the large model is added in the instructions in the form of examples; (3) Reasoning emphasis: Improve the instruction-following ability of large models through reasoning emphasis.

[0013] Further, in step S4, the summary generation model and the event extraction model of the long text are integrated to complete the event extraction of the long text, including the steps of: S4.1, data input: input the long text data that needs to be extracted for events one by one; S4.2, data processing is performed through three processes: data cleaning, data filtering, and data language conversion; S4.3, summary generation: input the processed long text into the summary generation model to generate the summary content of the long text; S4.4, judgment of summary generation results: for the cases where the summary generation results are discarded due to poor generation results or the large model itself does not generate a summary, judge. If there is no summary content, use the original text for the next step of event extraction; S4.5, event extraction: input the generated summary text or original text into the event extraction model, extract events from the text, and return the extraction results, which include: event type, event trigger words, event argument information, and non-existent argument information is replaced with null values. If there are multiple events, the event information of multiple events is returned; S4.6, post-processing: format and organize the output results of the event extraction model to facilitate user understanding and use; S4.7, result return: return the final sorted extraction results to the user.

[0014] In step S4.6, formatting is performed for the event extraction model extraction results that contain probability information and text start and end information, and the result presentation method that is inconvenient to view. Formatting includes: removing probability information and text start and end information, retaining only text descriptions; improving the readability of the results, and changing the final output format of the results to a dictionary list format.

[0015] Another object of the present invention is to provide a long text event extraction system based on summary generation, which is implemented by the long text event extraction method based on summary generation, and the system includes: The event type and argument definition module is used to define the type of events to be extracted and the role of event arguments; The event extraction model training module is used for labeling data preparation and preprocessing, training event extraction models in low-resource scenarios through sample data, and evaluating and saving event extraction models; The summary generation model acquisition module is used to fine-tune the large model through the prompt instruction fine-tuning method to train the summary generation model of long text; The event extraction module for long texts is used to integrate the summary generation model and event extraction model for long texts to complete event extraction for long texts.

[0016] Combining all the above technical solutions, the beneficial effects of the present invention are as follows: the present invention uses a small amount of labeled data and transfer learning and other technical means to enable the event extraction model to maintain good performance based on a small amount of sample data training in low-resource scenarios; the present invention fine-tunes the large language model through instructions, applies the powerful text generation ability of the large language model to event extraction, so that it automatically generates summary content of long texts, and fine-tunes the prompt through reasoning emphasis, context learning and other methods, so that the key argument information of the event is retained in the generated summary information.

[0017] The present invention combines text summary generation technology and event extraction technology, firstly extracts the summary content of the long text, and then extracts events based on the summary information, thereby effectively reducing the computational complexity and complexity of directly extracting events from the long text. At the same time, since the summary information contains the main information of the text and the key information points of the event, event extraction based on the summary information can more accurately extract the event information in the text, greatly improving the accuracy of long text event extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings herein are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description, serve to explain the principles of the present disclosure; Figure 1 is a flow chart of a long text event extraction method based on summary generation provided by an embodiment of the present invention; Figure 2 is a flow chart of a process of obtaining an event extraction model provided by an embodiment of the present invention; Figure 3 This is a diagram of a schema dictionary interface provided by an embodiment of the present invention; Figure 4 is a schematic diagram of a prompt instruction provided by an embodiment of the present invention; Figure 5 It is a flow chart of event extraction of the entire long text provided by an embodiment of the present invention; Figure 6 Schematic diagram of a long text event extraction system based on summary generation provided by an embodiment of the present invention; In the figure: 1. Event type and argument definition module; 2. Event extraction model training module; 3. Summary generation model acquisition module; 4. Event extraction module for long texts. DETAILED DESCRIPTION

[0019] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention, so the present invention is not limited by the specific implementation disclosed below.

[0020] The innovation of the present invention lies in: Event extraction model training in low-resource scenarios: The present invention uses a small amount of high-quality annotated data and reasonable model parameter configuration to train an event extraction model that still maintains good performance in low-resource scenarios, reducing the dependence on a large amount of annotated data; Based on the powerful text generation capability of the large model, a reasonable prompt instruction is designed through reasoning emphasis, context learning, etc. to generate a summary generation model suitable for long text time extraction, which shortens the length of long text events while retaining key event information.

[0021] The text summary generation technology and event extraction technology are innovatively combined. By first summarizing the content of the long text and then extracting events based on the summary information, the computational complexity and complexity of directly extracting events from long texts are effectively reduced, while the accuracy of long text event extraction is greatly improved, providing new ideas and methods for long text event extraction.

[0022] Embodiment 1, as Figure 1 As shown, the long text event extraction method based on summary generation provided by the embodiment of the present invention includes: S1, define the type of event to be extracted and the role of event arguments; S2, labeling data preparation and preprocessing, training the event extraction model in low-resource scenarios through sample data, and evaluating and saving the event extraction model; S3, fine-tuning the large model Qwen2-72B-Instruct through the Prompt instruction fine-tuning method to train the summary generation model of long text; S4 integrates the summary generation model and event extraction model of long text to complete event extraction of long text.

[0023] Exemplarily, in step S1, the type of event to be extracted and the role of event arguments are defined, and a schema dictionary for event extraction is generated based on the defined event type and event argument roles.

[0024] like Figure 2As shown in the figure, in the process of obtaining the event extraction model, event extraction refers to extracting structured event information from unstructured text. Event extraction tasks usually include two tasks: the first task: event detection, that is, judging the event type by identifying event trigger words; the second task: event element identification, defining the argument role in the event according to the event type, and extracting the element information of each argument.

[0025] Specifically, it includes: defining the types of events to be extracted and the role of event arguments. The event types and argument roles extracted by this method are shown in the following table, which include 6 event categories, 48 ​​event subcategories, and 171 arguments. At the same time, based on the defined event types and event argument roles, a schema dictionary for event extraction is generated. For specific categories and argument information, see Appendix 1; for schema dictionary, see Figure 2 shown.

[0026] The present invention defines 48 event types and 171 arguments, because for each event type, the role of its argument is different.

[0027] Table 1 Event categories and argument information

[0028] Exemplarily, in step S2, the preparation of labeled data includes: collecting Chinese open source labeled data, integrating data and selecting a small amount of high-quality data, and manually labeling a small amount of specific event type data; Preprocessing includes: aligning the annotations of labeled data from different sources; constructing negative data, and the relationship between negative data and positive data is expressed as: the amount of negative data = 5 × the amount of positive data; converting the data set into the input data format of the model, and dividing the data into training set, validation set and test set in a ratio of 8:1:1.

[0029] Specifically, the data preparation and processing stage requires the preparation of a small amount of annotated data of predefined event types. Each piece of annotated data needs to contain annotation information such as event type, event trigger words, and event arguments, and be preprocessed and organized into a model input format to facilitate subsequent model training.

[0030] Preferably, the entire data preparation process includes obtaining labeled data: on the one hand, by sorting out event labeled data on Chinese event extraction open source data sets such as ACE2005, CMNEE, and DUEE1.0, and selecting some labeled data therefrom (selecting 30-50 data for each event type), a part of labeled data is obtained; Among them, the selection principles for some annotated data are: good text quality; one text should contain multiple event types as much as possible; and moreover, ambiguous data texts should be removed.

[0031] On the other hand, for some event types, there is currently no open source annotated data set, so they are manually annotated and supplemented, with about 30 annotated data for each event type. In summary, the annotated data of all 46 event subtypes is obtained, with a total of 1,500 annotated data. Since this method extracts summaries from long texts before extracting events, when preparing the annotated data for the event extraction model, there is no need to deliberately consider the length of the data. Sentence-level, paragraph-level, and text-level annotated data are all acceptable, further reducing the difficulty of data collection and annotation.

[0032] Preferably, the annotation alignment process is as follows: Integrate all annotated data of the same event type; Event type label alignment: The same event type in different data sets is aligned to the same label; Argument role label alignment: According to the defined argument role information, verify the argument role labels of the annotated data, align different descriptions to the same description, and remove redundant argument role labels; for example, in dataset A, the label of the marriage event is: wedding event (time, wedding location, subject, object), with the argument role information in brackets; while in dataset B, it is: marriage event (time, location, first person, second person); according to the defined marriage event information, align the labels to: marriage (time, location, first person, second person).

[0033] The function of the constructed negative example quantity technology is to construct negative example data with empty labels for event types and time argument roles that are not in the labeled data; the number of negative examples uses the model's default 5*number of positive sample samples.

[0034] In another exemplary embodiment, in step S2, the event extraction model in the low-resource scenario is trained by sample data, including the steps of: (2.1) Basic model selection: This paper selects Baidu's open source universal information extraction model uie-base as the basic model of the event extraction model. The base of the universal information extraction model uie-base is the pre-trained model ERNIE 3.0 series, which has been pre-trained on a large scale through advanced pre-training technology, so that the universal information extraction model uie-base has rich semantic knowledge and strong language understanding ability. Its excellent small sample fine-tuning ability is very helpful for this method to train an event extraction model in a low-resource scenario.

[0035] (2.2) Configure training hyperparameters: Configure the model's training hyperparameters, including learning rate, batch size, training rounds, etc., based on the task characteristics, dataset characteristics, and training resources. The event extraction model of this method is trained on an A100 GPU with the following parameter configurations: learning rate 1e-5, maximum sequence length 512, batch size 16, training rounds 10, and AdamW optimizer; (2.3) Event extraction model training: Load the training set and validation set data for training. During the training process, the event extraction model is trained on the training set and evaluated on the validation set (for the evaluation method, refer to (2.4)) to monitor the performance changes of the model during the training process.

[0036] (2.4) Event extraction model effect evaluation: The event extraction model is evaluated using macro average precision (Macro_P), macro average recall (Macro_R) and macro average F1 value (Macro_F1). The model evaluation process includes event type recognition evaluation and event argument recognition evaluation. The relevant formulas are as follows: .

[0037] .

[0038] .

[0039] For the evaluation of event type recognition, Indicates The accuracy of event type recognition , Indicates Recall rate of event type recognition , Indicates The F1 value of event type recognition, Indicates the total number of event types, according to the defined event types, here ; Similarly, for event argument recognition, They represent the The accuracy of event argument recognition , recall rate and value, Represents the total number of event arguments for all event types, according to the predefined event arguments, here .

[0040] (2.5) Model iterative optimization: Based on the evaluation results of the above model, the model is iteratively optimized and trained by adjusting training hyperparameters, data enhancement and other strategies. It can be understood that the present invention changes the input parameters for optimization, that is, optimization is performed by continuously adjusting hyperparameters and training data.

[0041] (2.6) Save the event extraction model: After training is completed, save the model file, including the model parameter information (weights and biases, etc.), model structure information, etc. In the subsequent use process, the trained event extraction model can be directly loaded for use.

[0042] In another exemplary embodiment, in step S3, training a summary generation model for a long text includes the following steps: (3.1) Selection of the base large language model: The base large language model used in this method is the large language model Qwen2-72B-Instruct with a parameter scale of 72 billion developed by Alibaba Cloud. It has significant effects on multiple evaluation tasks, surpassing existing open source models, and has powerful language processing capabilities; it also supports contextual tasks up to 128K, breaking through the limitation of training sequence length, and is very suitable for processing long texts; it uses the GQA (Grouped Query Attention) mechanism, which brings the advantages of accelerated reasoning and reduced video memory usage. Therefore, considering the comprehensive language understanding and processing capabilities, long text support capabilities, reasoning speed, and actual evaluation, this method selects Qwen2-72B-Instruct as the base model of the long text summary extraction model of the present invention.

[0043] (3.2) Large model deployment and interface encapsulation: Based on the vLLM framework, deploy the large language model Qwen2-72B-Instruct to achieve offline reasoning, and encapsulate the large language model reasoning service into an OpenAI format HTTP service for use in subsequent steps; It can be understood that vLLM is a Python-based LLM reasoning and service framework, and its main advantages are ease of use and high performance. Through PagedAttention technology, continuous batch processing, CUDA core optimization, and distributed reasoning support, vLLM can significantly improve the reasoning speed of LLM, reduce video memory usage, and better meet the needs of actual applications. This method uses the vLLM framework to deploy Qwen2-72B-Instruct on a local server and provides an OpenAI access interface to facilitate subsequent calls to the summary generation model.

[0044] Preferably, deploying Qwen2-72B-Instruct includes: Create a virtual conda environment and install dependent environments such as Vllm and pytorch; Download the Qwen2-72B-Instruct model; Verify that the Qwen2-72B-Instruct model is available; Configure parameters and start the service: After verifying that the Qwen2-72B-Instruct model is available, use the module provided by vLLM to package the entire service into an OpenAI format http service and provide it to upper-layer applications.

[0045] Use the curl command to verify that the summary generation model OpenAI interface service is available.

[0046] (3.3) Prompt instruction design: A prompt is a paragraph of text, a sentence, or a question, which is used to guide the large model to generate text or perform a specific task. In this method, the large model is guided to complete the task of generating a summary of a long text by designing a suitable prompt instruction. In this method, the overall idea of ​​prompt instruction design includes establishing roles, setting goals and requirements, supplementing information, and designing output format requirements; Among them, role establishment: defines the role information played by the big model, usually including role identity, role skills, role task objectives, etc. In this method, in order to control the length of the input token, we simply define the big model role, defining the big model as a summary generation expert (role identity) that needs to generate long text summary content (role task objectives) as required. This method does not describe the big model role skills in detail.

[0047] Set goals and requirements: clarify the task objectives of the large model, as well as the content, format, word count and other task requirements when completing the task. This method defines the goal of the large model to receive the input long text and generate the summary content of the long text, and requires the summary content to contain key elements such as time, place, subject and object, cause, result, etc., and also sets requirements for the word count of the summary.

[0048] Supplementary information: Help the model complete the task by introducing few-shot (including one-shot) context learning, explanatory information, world knowledge, etc. In this method, we add one-shot to assist and guide the large model to complete the final task.

[0049] Output format requirements: The final output format of the large model is required to facilitate user viewing or further processing. In this method, the large model output format is required to be output in the format of one line of text.

[0050] The final prompt instruction principle of this method is as follows Figure 4 shown.

[0051] (3.4) Interact with the big model to generate summary results: Through the HTTP interface service, pass in the prompt command and configure parameters such as model name & path, maximum model length, multi-GPU operation parameter path, etc. to interact with the big model, so that the big model can perform summary generation tasks according to the command requirements and return results.

[0052] (3.5) Optimization and testing: When interacting with a large model, the model output may not meet expectations, so we need to continuously test and fine-tune the output of the large model. This method fine-tunes the large model by optimizing the prompt so that the model output meets the expected results, including the following steps: (1) Adjust the details and format of the prompt: such as adjusting the use of certain details and changing the word order of the command; (2) Adding suggestive enumeration: For example, during the testing process, it was found that the large model was prone to losing information describing time in the form of "day of the week" or "year before the current year", while time information is extremely important for events. Therefore, the present invention adds it in the instruction in the form of examples; (3) Reasoning emphasis: Since the current large model has not yet achieved complete consistency in following instructions, sometimes the instructions are not fully executed. Therefore, we use reasoning emphasis to improve the large model's ability to follow instructions. Figure 4 In the present invention, the large model is required to extract key information time and characters again through a "special reminder".

[0053] (3.6) Post-processing of results: Although the task execution of the large model is optimized by continuously adjusting prompts, the large language model does not follow the prompt instructions completely consistently, and its output results do not necessarily fully meet the requirements of subsequent tasks. Therefore, the present invention uses manual verification and summary to process the output results of the large language model, such as removing redundant information such as "The summary of the above text is..." at the beginning of the result generated by the large model, redundant key information dictionaries in the model output, and the model's reasoning process output, etc., and deleting or discarding the generated results for these situations.

[0054] It is understandable that for event extraction from long texts, the trigger words and argument information of the event are very important, and it is necessary to ensure that these important information are not lost during the summary generation process. At the same time, it is necessary to ensure that the events in the generated summary are the key events described in the long text. Therefore, it is very important to fine-tune the prompt instruction process, which requires repeated iterative optimization. Finally, this method fine-tunes the model by adding few-shots to provide a small number of summary generation task training samples to the summary generation model, helping the summary generation model to better understand the task and generate a summary that meets the application needs.

[0055] In the process of obtaining a long text summary generation model, it is necessary to continuously iterate and fine-tune the prompt instruction to improve the quality of long text summary generation. The prompt instruction fine-tuning methods include: enabling context learning by adding few-shot to provide a small number of summary generation task labeled samples to the large language model, giving the large language model a role identity definition, and repeating and emphasizing key information in the prompt, and finally generating a long text summary that meets application needs.

[0056] In step S4, the summary generation model and event extraction model of the long text are integrated to complete the event extraction of the long text, as shown in the attached figure. Figure 5 , including the steps of: S4.1, data input: input the long text data that needs to be extracted for events one by one; S4.2, data processing is performed through three processes: data cleaning, data filtering, and data language conversion; Specifically, data cleaning: remove HTML tags, non-encoded characters, special characters, noise symbols and some irrelevant content in the input text; data filtering: since this method only extracts events from Chinese data, the data filtering process mainly filters non-Chinese text; data language conversion: convert traditional Chinese into simplified Chinese; Among them, the methods of data cleaning and data filtering: mainly adopt a rule-based method, using regular expressions and other means to remove; data language conversion is implemented using the zhconv library specifically used for traditional and simplified Chinese conversion.

[0057] S4.3, summary generation: input the processed long text into the summary generation model to generate the summary content of the long text; S4.4, judgment of summary generation result: In the post-processing process of the summary generation model, the summary generation result may be discarded due to poor generation results or the large model itself may not generate a summary. This method will judge this situation. If there is no summary content, the original text will be used for the next step of event extraction; S4.5, event extraction: input the generated summary text or the original text (if there is no summary generation result) into the event extraction model, extract events from the text, and return the extraction results, which include: event type, event trigger words, event argument information, and non-existent argument information is replaced with null values. If there are multiple events, the event information of multiple events is returned; S4.6, post-processing: format and organize the output results of the event extraction model to facilitate user understanding and use; S4.7, result return: return the final sorted extraction results to the user.

[0058] In step S4.6, formatting is performed. Since the model extraction results contain probability information and text start and end information, and the result presentation is not convenient for viewing, formatting includes: removing probability information and text start and end information, retaining only text descriptions; improving the readability of the results, and changing the final output format of the results to a dictionary list format.

[0059] Such as: {Event type: praise event; Event trigger words: commendation; Praise: Principal; Time: August 10, 2023; Praise recipient: Teacher Zhang}.

[0060] It can be seen from the above embodiments that in terms of improving the efficiency of event extraction: the present invention significantly improves the efficiency of event extraction by first generating a summary of the long text and then extracting events from the summary. The traditional long text event extraction method needs to process a large amount of redundant information, while summary generation can remove non-critical parts of the text and retain core information, so that the event extraction process focuses more on key events, thereby reducing the amount of calculation and improving the processing speed.

[0061] In terms of improving the accuracy of event extraction: Based on the large language model summary generation, key sentences or paragraphs in the article are extracted. These key contents often contain the core information of the event. Therefore, event extraction based on the summary can more accurately capture the key elements of the event, such as time, place, people, event type, etc., improving the accuracy and recall rate of event extraction.

[0062] It can be seen from the above embodiments that the present invention applies the powerful text generation capability of the large model to event extraction, and provides new ideas and methods for event extraction of long texts by first generating a summary and then extracting the event.

[0063] The present invention solves the problem that in the process of extracting long text events, it is difficult to accurately capture event information due to the lengthy text, and the event extraction accuracy is low.

[0064] The present invention is not the only or optimal method for directly extracting events from long texts. By generating a large model summary, the text length can be shortened and key event information can be retained, which can provide new possibilities for research and application in related fields.

[0065] Embodiment 2, as Figure 6 As shown, the long text event extraction system based on summary generation provided by the embodiment of the present invention includes: Event type and argument definition module 1, used to define the type of event to be extracted and the role of event argument; Event extraction model training module 2 is used to prepare and preprocess the annotation data, train the event extraction model in low-resource scenarios, and evaluate and save the event extraction model; Summary generation model acquisition module 3 is used to fine-tune the large model Qwen2-72B-Instruct through the Prompt instruction fine-tuning method to train the summary generation model of long text; The event extraction module 4 of the long text is used to integrate the summary generation model and the event extraction model of the long text to complete the event extraction of the long text.

[0066] A comparative experiment was conducted on the test set to compare the method of the present invention with the universal information extraction model uie-base, and the evaluation indicators used were macro average precision (Macro_P), macro average recall (Macro_R) and macro average F1 value (Macro_F1).

[0067] It can be seen from the experimental results that compared with the general information extraction model uie-base for event extraction, the method of the present invention has greatly improved various evaluation indicators by integrating the large language model summary generation technology into the event extraction task, as shown in Table 2.

[0068] Table 2 Comparison between the present invention and the general information extraction model uie-base

[0069] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered within the protection scope of the present invention.

Claims

1. A long text event extraction method based on summary generation, characterized in that: The method comprises the steps of: S1, define the type of event to be extracted and the role of event arguments; S2, annotated data preparation and preprocessing, training the event extraction model in low-resource scenarios through sample data, and evaluating the saved event extraction model; S3, fine-tuning the large model through the Prompt instruction fine-tuning method to train the summary generation model of long text; S4 integrates the summary generation model and event extraction model of long text to complete event extraction of long text.

2. The long text event extraction method based on summary generation according to claim 1 is characterized in that: In step S1, the type of event to be extracted and the role of event arguments are defined, including: generating a schema dictionary for event extraction based on the defined event type and event argument roles.

3. The long text event extraction method based on summary generation according to claim 1 is characterized in that: In step S2, the annotation data is prepared, including: collecting Chinese open source annotation data, integrating and selecting data, and annotating some event type data; Preprocessing includes: aligning the annotation data from different sources; constructing negative data, and the relationship between negative data and positive data is expressed as: the amount of negative data = 5 × the amount of positive data; Convert the dataset into the model's input data format and divide the data into training set, validation set, and test set in a ratio of 8:1:

1.

4. The long text event extraction method based on summary generation according to claim 1 is characterized in that: In step S2, the event extraction model in the low-resource scenario is trained by sample data, including the following steps: (2.1) Basic model selection: The universal information extraction model uie-base is selected as the basic model of the event extraction model; (2.2) Configure training hyperparameters: Configure the training hyperparameters of the event extraction model based on the task characteristics, dataset characteristics, and training resources; (2.3) Event extraction model training: Load the training set and validation set data for training. During the training process, the event extraction model is trained on the training set and evaluated on the validation set to monitor the performance changes of the event extraction model during the training process. (2.4) Event extraction model effect evaluation: The event extraction model is evaluated using the macro average precision Macro_P, macro average recall Macro_R and macro average F1 value Macro_F1; (2.5) Model iterative optimization: Based on the evaluation results of the event extraction model, the event extraction model is iteratively optimized and trained by adjusting the training hyperparameters and data enhancement strategies; (2.6) Save the event extraction model: After training is completed, save the model file. The saved content includes the weight and bias parameter information of the event extraction model and the structural information of the event extraction model.

5. The long text event extraction method based on summary generation according to claim 1 is characterized in that: In step S3, a summary generation model for a long text is trained, including the following steps: (3.1) Selection of base large language model: The large language model Qwen2-72B-Instruct is used as the base model of the long text summary extraction model; (3.2) Large model deployment and interface encapsulation: Deploy the large language model Qwen2-72B-Instruct based on the vLLM framework to implement offline reasoning, and encapsulate the large language model reasoning service into an HTTP service in OpenAI format; (3.3) Prompt instruction design: Prompt instructions are designed to guide the large model to complete the task of generating summaries of long texts. Instruction design includes establishing roles, setting goals and requirements, supplementing information, and controlling output format requirements. (3.4) Interact with the large model to generate summary results: Through the HTTP interface service, input the prompt command and configure the model name and path, maximum model length, and multi-GPU operation parameter path to achieve interaction with the large model, so that the large model performs the summary generation task according to the command requirements and returns the result; (3.5) Optimization and testing: Fine-tune and optimize the large model by optimizing the prompt so that the model output meets the expected results; (3.6) Post-processing of results: The output results of the large language model are processed by verification and summary.

6. The long text event extraction method based on summary generation according to claim 5 is characterized in that: In step (3.3), establishing a role includes: defining the role information played by the large model, including role identity, role skills, and role task objectives; Setting goals and requirements includes: clarifying the task objectives of the large model, and the content, format, and word count requirements when completing the task; Supplementary information: By introducing few-shot context learning, explanatory information, and world knowledge, the model is assisted in completing the task. By adding one-shot, the large model is assisted in completing the final task. Output format requirement: The large model output format is output in the form of one line of text.

7. The long text event extraction method based on summary generation according to claim 5 is characterized in that: In step (3.5), the large model is fine-tuned and optimized by optimizing prompt so that the model output meets the expected results, including the following steps: (1) Adjust the details and format of the prompt: including adjusting the use of certain details and changing the word order of the instruction; (2) Adding suggestive enumeration: During the test, the time information that is easy to be lost in the large model is added in the instructions in the form of examples; (3) Reasoning emphasis: Improve the instruction-following ability of large models through reasoning emphasis.

8. The long text event extraction method based on summary generation according to claim 1 is characterized in that: In step S4, the summary generation model and event extraction model of the long text are integrated to complete the event extraction of the long text, including the following steps: S4.1, data input: input the long text data that needs to be extracted for events one by one; S4.2, data processing is performed through three processes: data cleaning, data filtering, and data language conversion; S4.3, summary generation: input the processed long text into the summary generation model to generate the summary content of the long text; S4.4, judgment of summary generation results: for the cases where the summary generation results are discarded due to poor generation results or the large model itself does not generate a summary, judge. If there is no summary content, use the original text for the next step of event extraction; S4.5, event extraction: input the generated summary text or original text into the event extraction model, extract events from the text, and return the extraction results, which include: event type, event trigger words, event argument information, and non-existent argument information is replaced with null values. If there are multiple events, the event information of multiple events is returned; S4.6, post-processing: format and organize the output results of the event extraction model to facilitate user understanding and use; S4.7, result return: return the final sorted extraction results to the user.

9. The long text event extraction method based on summary generation according to claim 8 is characterized in that: In step S4.6, formatting is performed for the event extraction model extraction results that contain probability information and text start and end information, and the result presentation method that is inconvenient to view; The formatting includes: removing probability information and text start and end information, retaining only text description; improving the readability of the results, and changing the final output format of the results into a dictionary list format.

10. A long text event extraction system based on summary generation, characterized in that: The system is implemented by the long text event extraction method based on summary generation according to any one of claims 1 to 9, and the system includes: The event type and argument definition module (1) is used to define the type of event to be extracted and the role of event arguments; The event extraction model training module (2) is used for labeling data preparation and preprocessing, training the event extraction model in low-resource scenarios through sample data, and evaluating and saving the event extraction model; The summary generation model acquisition module (3) is used to fine-tune the large model through the prompt instruction fine-tuning method to train the summary generation model of the long text; The long text event extraction module (4) is used to integrate the long text summary generation model and the event extraction model to complete the event extraction of the long text.

Images