Prompt learning-based multi-template selection system for event extraction field

Abstract

The invention belongs to the field of information extraction, and provides a prompt learning-based multi-template selection system for the field of event extraction, which comprises a context coding module, an extended event type mining module, a context decoding module, a training module and a reasoning module. According to the invention, through the context decoding module, the view of the pre-training language model is broadened; by adopting the expansion event type mining module to assist the model in extracting the correct argument, the accuracy of the event extraction task is improved.

Description

Technical Field

[0001] The present invention relates to the field of information extraction, and in particular to a multi-template selection system for event extraction based on prompt learning. Background Art

[0002] Event extraction is an important task in the field of natural language processing. It involves identifying, classifying and extracting structured event information from text. It has important application value in many fields such as information retrieval, intelligence analysis, text mining, automatic summarization, semantic understanding, cross-language analysis and legal review. It is one of the key technologies for realizing intelligent information processing and knowledge management. Previous studies usually divide event extraction into four sub-problems: trigger identification, trigger classification, argument identification and argument role classification. Various sub-modules are usually designed to be combined into classification or joint models to solve the above four problems. In addition, there are also reading comprehension methods that use the question and answer history of multiple questions between pre-trained language models to solve the above problems. However, classification-based methods usually require manual design of multiple sub-modules, and there is an error inheritance problem between sub-modules. At the same time, reading comprehension-based methods need to design multiple questions and models for multiple rounds of question answering. Recently, prompt-based generation methods have emerged in the field of event extraction. It does not have the problem of error propagation, is more flexible than previous classification-based and reading comprehension-based methods, and can extend the model to new events and change the type by modifying the prompt template and decoding strategy. In addition, prompt-based generative models perform well in resource-poor environments and only require a small amount of data to achieve acceptable model performance. In the prompt-based generation method, prompts are first manually created for different event types in the event ontology. It is worth noting that the construction of prompts in previous methods is different. Previous construction methods can be mainly divided into two categories. One is that the prompt consists of a template. In the other construction method, the prompt consists of a template, an event type description, and an event keyword. The generated prompt is then connected to the mentioned event and input into the model. The prompt instructs the pre-trained language model to generate the result of the corresponding event type in the input sentence. Finally, the corresponding decoding method is used to extract the trigger words and arguments corresponding to the role from the generated sentence and obtain the event record.

[0003] However, existing methods only consider the template corresponding to the current event type for extraction, and fail to fully utilize the potential of the pre-trained language model. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide a multi-template selection system for the field of event extraction based on prompt learning, broaden the field of vision of the pre-trained language model, and adopt an extended event type mining module to assist the model in extracting correct arguments to improve the accuracy of the event extraction task.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A multi-template selection system for event extraction based on prompt learning, comprising:

[0007] A context encoding module, used to obtain system input by connecting mentioned events, keywords and target event types through word units, and send the system input to the encoder of the preset BART model for encoding to obtain an encoded representation;

[0008] An extended event type mining module is used to measure the correlation between the event type mining matrix and the coding representation using an information fusion function, multiply the correlation by the output of the BART model to obtain an adaptation representation of all event types, fuse and normalize the adaptation representation and the event type mining matrix to obtain a normalized matching score, sort the normalized matching score in descending order, and if the second normalized matching score is greater than a predetermined threshold, determine that the extended template corresponding to the second normalized matching score is an event type template;

[0009] A context decoding module is used to add a prefix to the extended template to obtain a final template. The final template is input into the decoder of the BART model for decoding to obtain a generated text; the prefix is ​​related to the event that may occur;

[0010] A training module, configured to extract the span of the mentioned event, replace the marker placeholder in the encoded representation with the span, and train the BART model;

[0011] An inference module is used to list all event types for each piece of predicted data, connect a candidate list of trigger words corresponding to the event type and feed the connection result into the BART model for text generation, extract the trigger and parameters of the output result of the BART model, and if the trigger and the parameter have corresponding matches in the mentioned event, select the generated text corresponding to the trigger and the parameter as the final event extraction result.

[0012] Preferably, it also includes:

[0013] A prompt template is used to extract the role semantic information of the mentioned event and use the role semantic information as the role placeholder of the encoded representation.

[0014] The present invention discloses the following technical effects:

[0015] The present invention provides a multi-template selection system for event extraction based on prompt learning. Through a context decoding module, the defect of a conventional language model with a small field of view is solved, and a matching model with a prefix is ​​realized. By adopting an extended event type mining module to assist the model in extracting correct arguments, the problem of fixed event type templates of conventional language models is solved, and the determination of the optimal event type template is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0017] Figure 1 A schematic diagram of a multi-template selection system provided by an embodiment of the present invention;

[0018] Figure 2 A flow chart of a multi-template selection system provided in an embodiment of the present invention.

[0019] Description of reference numerals:

[0020] 1-Context encoding module, 2-Extended event type mining module, 3-Context decoding module, 4-Training module, 5-Inference module. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] The purpose of the present invention is to provide a multi-template selection system for event extraction based on prompt learning, broaden the field of vision of pre-trained language models, and use an extended event type mining module to assist the model in extracting correct arguments to improve the accuracy of event extraction tasks.

[0023] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] Figure 1 A schematic diagram of a multi-template selection system provided by an embodiment of the present invention, Figure 2 A flow chart of a multi-template selection system provided by an embodiment of the present invention, such as Figure 1 and Figure 2 As shown, the present invention provides a multi-template selection system for event extraction based on prompt learning, comprising:

[0025] Context encoding module 1, used to connect the mentioned events, keywords and target event types through word units to obtain system input, and send the system input to the encoder of the preset BART model for encoding to obtain the encoded representation;

[0026] An extended event type mining module 2 is used to measure the correlation between the event type mining matrix and the coding representation using an information fusion function, multiply the correlation by the output of the BART model to obtain an adaptation representation of all event types, fuse and normalize the adaptation representation and the event type mining matrix to obtain a normalized matching score, sort the normalized matching score in descending order, and if the second normalized matching score is greater than a predetermined threshold, determine that the extended template corresponding to the second normalized matching score is an event type template;

[0027] Context decoding module 3 is used to add a prefix to the extended template to obtain the final template. The final template is input into the decoder of the BART model for decoding to obtain the generated text; the prefix is ​​related to the possible events;

[0028] Training module 4 is used to extract the spans of mentioned events, replace the token placeholders in the encoded representation with the spans, and train the BART model;

[0029] Reasoning module 5 is used to list all event types for each predicted data, connect the candidate list of trigger words corresponding to the event type and feed the connection result into the BART model for text generation, extract the trigger and parameters of the output result of the BART model, and if the trigger and parameter have corresponding matches in the mentioned event, select the generated text corresponding to the trigger and parameter as the final event extraction result.

[0030] Preferably, it also includes:

[0031] The prompt template is used to extract the role semantic information of the mentioned event and use the role semantic information as the role placeholder of the encoded representation.

[0032] Preferably, the extended event type mining module 2 is first used to automatically determine the potential extended event types in the input, and then the extended template corresponding to the extended event type is used to interact with the template corresponding to the event summary of the event type currently to be extracted to obtain multi-template information, thereby providing more information for the pre-training generation result. The prompt-based event extraction method applied in the resource-scarce scenario can be divided into: context encoding module 1, extended event type mining module 2, context decoding module 3, training module 4, reasoning module 5 and prompt template design.

[0033] Specifically, context encoding module 1: given an event mentioned First, combine it with the event keyword and the target event type Connected, where and as well as is through special lemmas ( <sep>) are distinguished and then fed together into the encoder part of the BART model for encoding to obtain the encoded representation.

[0034] Further, the extended event type mining module 2: in order to mine the extended templates mentioned by the current event that are helpful for event extraction, an extended event type mining matrix is ​​used to calculate the matching scores between the current input and all event types, and then the corresponding extended templates are obtained according to the matching scores. The matching score is defined by the information fusion function To measure the correlation between the extended event type mining matrix and each vector representation, the information fusion function is obtained through a learnable function. This part can be specifically divided into: 1) Information fusion of the extended event type mining matrix and the BART encoder output vector. 2) The adaptive matrix W m ∈R 4d×1 Multiply by the information fusion function The obtained value fully captures the similarity information of different aspects, and obtains the similarity between event types and the vector representation of each input word. 3) Multiply the obtained similarity by the output of the pre-trained language model to obtain the adaptation representation between the mentioned event and each event type 4) Adaptive Representation The expanded event type mining matrix is ​​merged with each other, and the fused information is normalized. 5) The fused normalized scores are sorted, and then the second largest normalized matching score is selected. If the value is greater than the preset threshold, the original event type template is expanded to the corresponding event type template. Otherwise, there is no other template for the expanded event type.

[0035] Specifically, context decoding module 3: after obtaining the extended template Finally, in order to obtain the final template, Add a prefix that is related to the possible events that the expanded template may become associated with Related events, and then compare them with connected, called the final template This template, along with the output of the pre-trained language model, is fed into the BART decoder for guidance in model generation to produce the target sentence.

[0036] Preferably, training module 4: The training goal of the designed model is to extract spans from event mentions to replace triggers and role placeholders in the template. The model does not directly generate event records containing triggers and arguments, but generates sentences containing correct triggers and arguments. Specifically, the target generated sentence for trigger word extraction is expressed as a sentence: "The event trigger word is <trigger>", and the actual trigger span in the event mention is replaced with the marked placeholder <trigger>The target generated sentence for extracting arguments is the sentence generated after replacing the placeholders in the template with the argument span extracted from the event mention.

[0037] Further, reasoning module 5: First, all event types are listed for each prediction data. Subsequently, the candidate list of trigger words corresponding to the event type is connected and fed into the model to generate results. Triggers and parameters are extracted from the generated results, and if these trigger words and parameters have corresponding matches in the event mention, the result is selected as the final event extraction result of the event mention. The prefix is ​​related to the possible event, that is, the extended template may become Related events.

[0038] Referring to Table 1, event types and their corresponding prompt templates are shown in the table; Prompt template design: In previous prompt-based generation models, templates are considered to be the basic factors affecting model extraction. In order to make full use of the semantic information in the role, this embodiment designs a set of new templates for the event extraction task on the ACE2005-EN dataset, which uses the form of "Some+role name" as the placeholder for the role in the template.

[0039] Table 1

[0040]

[0041]

[0042] The beneficial effects of the present invention are as follows:

[0043] The present invention broadens the field of view of the pre-trained language model through a context decoding module; and improves the accuracy of the event extraction task by using an extended event type mining module to assist the model in extracting correct arguments.

[0044] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0045] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.< / trigger> < / trigger> < / sep>

Claims

1. A multi-template selection system for event extraction based on prompt learning, characterized in that: include: A context encoding module, used to obtain system input by connecting mentioned events, keywords and target event types through word units, and send the system input to the encoder of the preset BART model for encoding to obtain an encoded representation; An extended event type mining module is used to measure the correlation between the event type mining matrix and the coding representation using an information fusion function, multiply the correlation by the output of the BART model to obtain an adaptation representation of all event types, fuse and normalize the adaptation representation and the event type mining matrix to obtain a normalized matching score, sort the normalized matching score in descending order, and if the second normalized matching score is greater than a predetermined threshold, determine that the extended template corresponding to the second normalized matching score is an event type template; A context decoding module is used to add a prefix to the extended template to obtain a final template, and input the final template into the decoder of the BART model for decoding to obtain a generated text; The prefix is ​​associated with an event that may occur; A training module, configured to extract the span of the mentioned event, replace the marker placeholder in the encoded representation with the span, and train the BART model; An inference module is used to list all event types for each piece of predicted data, connect a candidate list of trigger words corresponding to the event type and feed the connection result into the BART model for text generation, extract the trigger and parameters of the output result of the BART model, and if the trigger and the parameter have corresponding matches in the mentioned event, select the generated text corresponding to the trigger and the parameter as the final event extraction result.

2. According to claim 1, a multi-template selection system for event extraction based on prompt learning is characterized in that: Also includes: A prompt template is used to extract the role semantic information of the mentioned event and use the role semantic information as the role placeholder of the encoded representation.

Images