A multi-template selection system for event extraction field based on prompt learning

Abstract

The application belongs to the field of information extraction, and provides a multi-template selection system for event extraction based on prompt learning, which comprises a context encoding module, an extended event type mining module, a context decoding module, a training module and an inference module. The application widens the field of view of the pre-training language model through the context decoding module, and improves the accuracy of the event extraction task by using the extended event type mining module to assist the model in extracting correct arguments.

Description

Technical Field

[0001] The present invention relates to the field of information extraction, and in particular to a multi-template selection system for event extraction based on prompt learning. Background Art

[0002] Event extraction is a crucial task in natural language processing, involving the identification, classification, and extraction of structured event information from text. It has significant applications in a wide range of fields, including information retrieval, intelligence analysis, text mining, automatic summarization, semantic understanding, cross-lingual analysis, and legal review. It is a key technology for intelligent information processing and knowledge management. Previous research typically divides event extraction into four sub-problems: trigger identification, trigger classification, argument identification, and argument role classification. Various sub-modules are typically combined into classification or joint models to address these four issues. Alternatively, reading comprehension approaches have been used to address these issues by leveraging the question and answer history of multiple questions across pre-trained language models. However, classification-based approaches often require the manual design of multiple sub-modules and suffer from the problem of error inheritance between sub-modules. Furthermore, reading comprehension-based approaches require the design of multiple questions and models for multiple rounds of question answering. Recently, prompt-based generation methods have emerged in the field of event extraction. These methods avoid the problem of error propagation and are more flexible than previous classification- and reading comprehension-based approaches. They can also be extended to new events and change their types by modifying prompt templates and decoding strategies. Furthermore, prompt-based generative models perform well in resource-poor environments and require only a small amount of data to achieve acceptable model performance. In the prompt-based generation method, prompts are first manually created for different event types in the event ontology. It is worth noting that the construction of prompts in previous methods varies. Previous construction methods can be mainly divided into two categories. One is that the prompt consists of a template. In the other construction method, the prompt consists of a template, an event type description, and an event keyword. The generated prompt is then connected to the mentioned event and input into the model. The prompt instructs the pre-trained language model to generate a result for the corresponding event type in the input sentence. Finally, the corresponding decoding method is used to extract the trigger words and arguments corresponding to the role from the generated sentence and obtain the event record.

[0003] However, existing methods only consider the template corresponding to the current event type for extraction, failing to fully utilize the potential of the pre-trained language model. Summary of the Invention

[0004] In order to overcome the shortcomings of the existing technology, the purpose of the present invention is to provide a multi-template selection system for the field of event extraction based on prompt learning, broaden the field of vision of the pre-trained language model, and adopt an extended event type mining module to assist the model in extracting correct arguments to improve the accuracy of the event extraction task.

[0005] To achieve the above object, the present application provides the following scheme:

[0006] A multi-template selection system for event extraction based on prompt learning, comprising:

[0007] A context encoding module is configured to obtain a system input by connecting a mention event, a keyword and a target event type by a word piece, encode the system input in an encoder of a preset BART model to obtain an encoded representation, and send the encoded representation to the BART model.

[0008] An extended event type mining module is configured to measure a correlation between an event type mining matrix and the encoded representation by using an information fusion function, multiply the correlation and an output of the BART model to obtain an adaptive representation of all event types, perform fusion and normalization operations on the adaptive representation and the event type mining matrix to obtain a normalized matching score, sort the normalized matching scores in descending order, and determine an extended template corresponding to a second normalized matching score as an event type template if the second normalized matching score is greater than a predetermined threshold.

[0009] A context decoding module is configured to add a prefix to the extended template to obtain a final template, input the final template into a decoder of the BART model to obtain generated text, and the prefix is related to a possible event.

[0010] A training module is configured to extract a span of the mention event, replace a token placeholder in the encoded representation with the span, and train the BART model.

[0011] An inference module is configured to list all event types for each piece of predicted data, connect a candidate list of trigger words corresponding to the event types, feed the connection result to the BART model for text generation, extract a trigger and a parameter of an output result of the BART model, and select the generated text corresponding to the trigger and the parameter as a final event extraction result if the trigger and the parameter have corresponding matches in the mention event.

[0012] Preferably, the system further comprises:

[0013] A prompt template is configured to extract role semantic information of the mention event and use the role semantic information as a role placeholder of the encoded representation.

[0014] The present application has the following technical effects:

[0015] The present invention provides a multi-template selection system for event extraction based on prompt learning. Through the context decoding module, the defect of the conventional language model with a small field of view is solved, and a matching model with a prefix is ​​realized. By adopting an extended event type mining module to assist the model in extracting correct arguments, the problem of the conventional language model's fixed event type template is solved, and the optimal event type template is determined. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 A schematic diagram of a multi-template selection system provided by an embodiment of the present invention;

[0018] Figure 2 This is a flow chart of the multi-template selection system provided by an embodiment of the present invention.

[0019] Description of reference numerals:

[0020] 1-Context encoding module, 2-Extended event type mining module, 3-Context decoding module, 4-Training module, 5-Inference module. DETAILED DESCRIPTION

[0021] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0022] The purpose of the present invention is to provide a multi-template selection system for event extraction based on prompt learning, broaden the field of vision of pre-trained language models, and adopt an extended event type mining module to assist the model in extracting correct arguments to improve the accuracy of event extraction tasks.

[0023] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] Figure 1 A schematic diagram of a multi-template selection system provided by an embodiment of the present invention, Figure 2 The flowchart of the multi-template selection system provided by the embodiment of the present invention is as follows: Figure 1 and Figure 2 As shown, the present invention provides a multi-template selection system for event extraction based on prompt learning, comprising:

[0025] Context encoding module 1 is used to connect the mentioned events, keywords and target event types through word units to obtain system input, and then send the system input to the encoder of the preset BART model for encoding to obtain the encoded representation;

[0026] An extended event type mining module 2 is configured to measure the correlation between the event type mining matrix and the encoded representation using an information fusion function, multiply the correlation by the output of the BART model to obtain an adapted representation of all event types, fuse and normalize the adapted representation and the event type mining matrix to obtain a normalized matching score, sort the normalized matching scores in descending order, and if a second normalized matching score is greater than a predetermined threshold, determine that the extended template corresponding to the second normalized matching score is an event type template;

[0027] Context decoding module 3 is used to add a prefix to the extended template to obtain the final template. The final template is then input into the decoder of the BART model for decoding to obtain the generated text; the prefix is ​​related to the possible events;

[0028] Training module 4 is used to extract spans of mentioned events, replace token placeholders in the encoded representation with spans, and train the BART model;

[0029] Reasoning module 5 is used to list all event types for each predicted data, connect the candidate list of trigger words corresponding to the event type and feed the connection result into the BART model for text generation, extract the trigger and parameters of the output result of the BART model, and if the trigger and parameter have corresponding matches in the mentioned event, select the generated text corresponding to the trigger and parameter as the final event extraction result.

[0030] Preferably, it also includes:

[0031] The prompt template is used to extract the role semantic information of the mentioned event and use the role semantic information as the role placeholder of the encoded representation.

[0032] Preferably, the extended event type mining module 2 is first used to automatically determine the potential extended event types in the input. Then, the extended template corresponding to the extended event type is used to interact with the template corresponding to the event summary of the event type currently to be extracted to obtain multi-template information, thereby providing more information for pre-training generation results. The prompt-based event extraction method applied in resource-scarce scenarios can be divided into: context encoding module 1, extended event type mining module 2, context decoding module 3, training module 4, reasoning module 5, and prompt template design.

[0033] Specifically, context encoding module 1: given an event mentioned First combine it with the event keyword and target event type Connected, where and as well as is achieved through special lemmas ( <sep>) are distinguished and then fed together into the encoder part of the BART model for encoding to obtain the encoded representation.

[0034] Furthermore, the extended event type mining module 2: in order to mine the extended templates mentioned by the current event that are helpful for event extraction, an extended event type mining matrix is ​​used to calculate the matching scores between the current input and all event types, and then the corresponding extended templates are obtained according to the matching scores. The matching score is obtained by defining the information fusion function To measure the correlation between the extended event type mining matrix and each vector representation, the information fusion function is obtained through a learnable function. This part can be specifically divided into: 1) Information fusion of the extended event type mining matrix and the BART encoder output vector. 2) The adaptive matrix W m ∈R 4d×1 Multiply by the information fusion function The obtained value fully captures the similarity information of different aspects, and obtains the similarity between event types and the vector representation of each input word. 3) Multiply the obtained similarity by the output of the pre-trained language model to obtain the adaptation representation between the mentioned event and each event type 4) Adaptive Representation The expanded event type mining matrix is ​​then merged and the fused information is normalized. 5) The fused normalized scores are sorted and the second largest normalized match score is selected. If this value is greater than a predetermined threshold, the original event type template is expanded to the corresponding event type template. Otherwise, no other expanded event type template exists.

[0035] Specifically, context decoding module 3: after obtaining the extended template Finally, in order to obtain the final template, Add a prefix that is related to the possible events that the expanded template may become related to Related events, and then compare them with connected, called the final template This template, along with the output of the pre-trained language model, is fed into the BART decoder for guidance in model generation to produce the target sentence.

[0036] Preferably, training module 4: The training goal of the designed model is to extract spans from event mentions to replace triggers and role placeholders in the template. The model does not directly generate event records containing triggers and arguments, but instead generates sentences containing correct triggers and arguments. Specifically, the target generated sentence for trigger word extraction is expressed as a sentence: "The event trigger word is <trigger>" and the real trigger span in the event mention is replaced with a tagged placeholder <trigger>The target generated sentence for extracting arguments is a sentence generated after replacing the placeholders in the template with the argument span extracted from the event mention.

[0037] Further, reasoning module 5: First, all event types are listed for each prediction data. Then, the candidate list of trigger words corresponding to the event type is connected and fed into the model to generate the result. Triggers and parameters are extracted from the generated results, and if these trigger words and parameters have corresponding matches in the event mention, the result is selected as the final event extraction result of the event mention. The prefix is ​​related to the possible event, that is, the extended template may become Related events.

[0038] Refer to Table 1 for event types and their corresponding prompt templates. Prompt Template Design: In previous prompt-based generative models, templates are considered a fundamental factor influencing model extraction. To fully leverage the semantic information in roles, this example designs a new set of templates for event extraction on the ACE2005-EN dataset. These templates use a placeholder for the role in the form of "Some + role name."

[0039] Table 1

[0040]

[0041]

[0042] The beneficial effects of the present invention are as follows:

[0043] The present invention broadens the field of view of the pre-trained language model through the context decoding module; and improves the accuracy of the event extraction task by adopting the extended event type mining module to assist the model in extracting correct arguments.

[0044] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0045] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.< / trigger> < / trigger> < / sep>

Claims

1. A multi-template selection system for event extraction based on prompt learning, characterized in that: include: A context encoding module is used to obtain system input by connecting mentioned events, keywords, and target event types through word units, and to feed the system input into the encoder of the preset BART model for encoding to obtain an encoded representation; An extended event type mining module is configured to measure the correlation between an event type mining matrix and the encoded representation using an information fusion function, multiply the correlation by the output of the BART model to obtain an adapted representation of all event types, fuse and normalize the adapted representation and the event type mining matrix to obtain a normalized matching score, sort the normalized matching scores in descending order, and if a second normalized matching score is greater than a predetermined threshold, determine that the extended template corresponding to the second normalized matching score is an event type template; A context decoding module, configured to add a prefix to the extended template to obtain a final template, and input the final template into the decoder of the BART model for decoding to obtain generated text; Said prefix is ​​related to the event that may occur; a training module for extracting spans of the mentioned events, replacing token placeholders in the encoded representation with the spans, and training the BART model; An inference module is used to list all event types for each piece of predicted data, connect a candidate list of trigger words corresponding to the event type and feed the connection result into the BART model for text generation, extract the trigger and parameters of the output result of the BART model, and if the trigger and the parameter have a corresponding match in the mentioned event, select the generated text corresponding to the trigger and the parameter as the final event extraction result.

2. A multi-template selection system for event extraction based on prompt learning according to claim 1, characterized in that: Also includes: A prompt template is used to extract the role semantic information of the mentioned event and use the role semantic information as the role placeholder of the encoded representation.

Images