Method and device for extracting events at a chapter level based on machine reading comprehension, equipment and storage medium

Abstract

The application discloses a kind of based on machine reading comprehension's chapter-level event extraction method, device and equipment and storage medium.It is given input chapter and corresponding fill-in-the-blank template, model generates fill-in-the-blank according to the corresponding argument slot in template on the basis of chapter content, realizes the extraction of event argument.The application is aimed at chapter-level event extraction and few-sample event extraction problem, realizes using machine reading comprehension mechanism to traditional event extraction task, provides a kind of fill-in-the-blank template level event extraction mode information fusion method and machine reading comprehension method based on pre-training language model.The application further discloses an end-to-end encoder-decoder model, by using the natural language understanding ability of pre-training language model, excavates the potential knowledge of pre-training language model, can improve the generalization performance of model in few-sample situation, improve the adaptive ability of model on extraction task in different fields.

Description

Technical Field

[0001] This invention relates to the field of natural language processing in the field of information technology, and particularly to the technical fields of deep learning and information extraction. Specifically, this invention provides a method, apparatus, electronic device, and storage medium for text-level event extraction based on machine reading comprehension. Background Technology

[0002] Event extraction is defined as the identification and extraction of important information from unstructured text in a structured form. In event extraction, words representing the occurrence of an event action and the main theme of the event information are called trigger words. Entities related to the event, such as people, time, and place, centered around the trigger words, are called event arguments. The role of an event argument in the event is called an argument role. Event extraction is an important subtask of information extraction and natural language understanding, providing support for applications in various fields such as information retrieval, emergency detection, and public opinion analysis. Traditional event extraction methods are mostly limited to the sentence dimension, assuming that the extracted events are located within a single sentence in an attempt to simplify the problem. However, in the real world, event information often exists in passages composed of multiple sentences, and passages often contain multiple events. Traditional event extraction methods are mostly data-driven, defining event extraction as a classification or sequence labeling task. They require a large amount of labeled training data to achieve ideal performance during model training, thus performing poorly in low-resource domains and lacking the ability to extract unknown events. The chapter-level event extraction task can be defined as a machine reading comprehension task based on chapter content understanding and text fragment matching.

[0003] This invention provides a method, apparatus, device, and storage medium for text-level event extraction based on machine reading comprehension. Given an input text and a corresponding fill-in-the-blank template, the model generates fill-in-the-blanks for the corresponding argument slots in the template based on the text content, thereby extracting event arguments. This invention proposes an end-to-end encoder-decoder model, which, by maintaining consistency between upstream and downstream tasks of the pre-trained language model, better utilizes the natural language understanding capabilities of the pre-trained language model, improves the model's generalization performance in few-shot scenarios, and enhances the model's adaptability to extraction tasks in different domains. Summary of the Invention

[0004] The purpose of this invention is to provide a method, apparatus, electronic device, and storage medium for text-level event extraction based on machine reading comprehension. This method provides an input text and a corresponding fill-in-the-blank template. The model generates fill-in-the-blanks for the corresponding argument slots in the template based on the text content, thereby extracting event arguments. This invention addresses the problems of text-level event extraction and few-shot event extraction by reconstructing the traditional event extraction task format, providing a fill-in-the-blank template-level event information fusion method and a machine reading comprehension method based on a pre-trained language model. This invention further discloses an end-to-end encoder-decoder model. By keeping the upstream and downstream tasks of the pre-trained language model consistent, it can better utilize the natural language understanding capabilities of the pre-trained language model, utilize cue learning to mine potential knowledge from the pre-trained language model, improve the model's generalization performance in few-shot scenarios, and enhance the model's adaptability in extraction tasks across different domains. This invention achieves event extraction on Chinese and English text datasets, improving the performance of event extraction and its performance in low-resource scenarios.

[0005] The document-level event extraction method based on machine reading comprehension described in this invention is performed according to the following steps:

[0006] a. Obtain the document-level event extraction dataset, perform text preprocessing on the dataset, including data cleaning, document length filtering, Chinese word segmentation, and special character filtering, and divide the dataset into training set, validation set and test set;

[0007] b. Construct an event extraction task mode definition, and construct a structured extraction definition based on event type, event trigger words, and event argument roles. Label the event type, event trigger words, and event argument roles of the corresponding text segments in the dataset texts involved in step a.

[0008] c. Define tag categories based on event type and event trigger words, construct event extraction fill-in-the-blank templates based on concise natural language descriptions for events, perform Chinese and English word segmentation, length filtering and first and last identifier marking preprocessing on the fill-in-the-blank templates, define the types of event extraction arguments in step b, construct corresponding fill-in-the-blank templates with trigger words and event types as template category identifiers, and construct an event extraction fill-in-the-blank template database.

[0009] d. Utilize the event extraction structured pattern information integration mechanism to integrate the event trigger words and event argument roles in the event extraction task pattern definition as external information into the construction of the fill-in template in step c. Constrain the event argument prediction slots in the fill-in template with argument roles and constrain the event description trigger words in the fill-in template with trigger words to construct a fill-in template based on event extraction pattern information.

[0010] e. Using the event extraction fill-in-the-blank template database from step c and the text of the dataset from step a, fill-in-the-blank templates are matched and spliced ​​based on event type and event trigger words.

[0011] f. Concatenate the passage text and fill-in-the-blank template from step e, add a special separator [SEP] and an argument slot identifier [ARG] to form the input of the encoding layer, use a bidirectional attention mechanism to calculate the attention relationship between the passage text and the fill-in-the-blank template, and output the encoded representation vector and the last hidden vector.

[0012] g. Using the encoded representation vector obtained in step f and the last hidden vector of the encoding layer as input, the model performs fill-in-the-blank prediction on the fill-in-the-blank template in the form of machine reading comprehension through the autoregressive decoding layer, generating the argument filling prediction sequence of the fill-in-the-blank template.

[0013] h. Perform event argument generation and mapping operations on the fill-in template argument prediction sequence described in step g. According to the event extraction mode definition described in step b, perform mapping and matching between the fill-in template argument slot values ​​and the event extraction task mode definition to obtain the final event argument extraction result.

[0014] The document-level event extraction dataset mentioned in step a is specifically in two languages, English and Chinese, and is a document-level event extraction dataset in the fields of news and finance. In the document-level event extraction dataset, the document contains multiple event descriptions and multiple event types, corresponding to multiple event trigger words and event argument roles.

[0015] The event extraction task paradigm described in step b is defined as a structured description of event information, and the set of key attributes in the key-value pair set is defined as the event extraction task pattern.

[0016] The fill-in-the-blank template in step c is set as a natural language short sentence with slot values ​​for event types. The argument slots to be predicted are represented by special identifiers [ARG]. The fill-in-the-blank template database covers the event types and trigger words involved in the dataset. The event type and trigger word are used as unique primary keys to match the corresponding fill-in-the-blank template. The event extraction task mode information integration mechanism specifically uses the event type, event trigger word, argument role and other information defined in the event extraction task mode to integrate into the argument slots corresponding to the fill-in-the-blank template, and uses the machine reading comprehension mechanism of the model to provide slot type hints.

[0017] The method for combining the passage and fill-in-the-blank template described in step f:

[0018] Given a text D = {d1, d2, ..., d...} n} and fill-in-the-blank template T={t1,t2,...,t m} is formed by splicing and inserting delimiters.<s>{d1,d2,...,d n } <sep>{t1,t2,...,t m } <s>As a new input sequence, it is input into the coding layer, where multiple encoder attention heads are concatenated and linearly transformed to obtain the final encoded vector.

[0019] In step g, the autoregressive decoding layer receives the encoded vector from the encoding layer and generates a unidirectional autoregressive sequence by right shifting. Each decoder in the decoding layer receives the generated vector from the previous time step and the last hidden vector from step f, and calculates the generation probability of the next time step through a cross-attention mechanism.

[0020] The event argument generation and mapping described in step h involves filling the predicted sequence with argument-filling templates as described in step g. It matches and transforms the event extraction task paradigm with the fill-in-the-blank template, using regular expressions to locate the coordinates of slots in the fill-in-the-blank template, and achieving a structured mapping of event arguments and argument roles for the predicted slot values ​​in the fill-in-the-blank template. A large-scale machine reading comprehension dataset is used as the training dataset for the model pre-training stage. Cross-task pre-training enables the model to acquire prior knowledge of machine reading comprehension, improving its generalization ability in low-sample scenarios for downstream target tasks.

[0021] A text-level event extraction device based on machine reading comprehension, comprising a data preprocessing module, a bidirectional word embedding encoding module, an autoregressive decoding module, an argument generation and mapping module, and a pre-training and fine-tuning module, wherein:

[0022] The data preprocessing module acquires news and financial report event extraction datasets and preprocesses the Chinese and English datasets, including: article length filtering, Chinese word segmentation, special character filtering, and encoding format standardization. The datasets are then divided into training, validation, and test sets. The module also acquires event extraction fill-in-the-blank templates and preprocesses them in both Chinese and English, including event type and trigger word annotation, fill-in-the-blank template argument slot annotation, and fill-in-the-blank template database construction.

[0023] Bidirectional word embedding encoding module: Given a text passage and a corresponding fill-in-the-blank template, the passage and the fill-in-the-blank template are concatenated, and special separators and argument slot identifiers are added; the input is then embedded and represented by the bidirectional word embedding encoding module. Each layer in the encoding module calculates the word encoding vector and the last hidden layer vector through a multi-head self-attention mechanism, thus obtaining the representation relationship between the passage and the fill-in-the-blank template;

[0024] The autoregressive decoding module receives the output of the bidirectional word embedding encoding module. The input and output of the autoregressive decoding module are generated by right shifting. The prediction result of the previous time step is used as the input vector of the current time step. Each decoder layer additionally receives the hidden vector of the encoding layer. The predicted word vector is obtained by using the cross-attention mechanism. After reaching the maximum generation length or generating the sentence end identifier, the sequence generation process is completed. In the form of machine reading comprehension, the module predicts the filling of the argument slots in the fill-in-the-blank template based on the content of the passage text and generates the fill-in-the-blank template after filling in the blanks.

[0025] The argument generation and mapping module extracts event arguments and corresponding argument roles from the fill-in-the-blank template prediction results. It replaces the fill-in-the-blank template argument slots with a unified argument identifier [arg] using regular expression matching and adds them to the vocabulary. In the autoregressive decoding layer's prediction of fill-in-the-blank template argument slots, if an argument role has multiple corresponding event arguments, it merges the arguments using connectives. The pre-training and fine-tuning module trains the model for the target task. It uses domain-similar tasks for cross-task pre-training, enabling the model to acquire machine reading comprehension capabilities. Through knowledge transfer, it transfers external prior knowledge to the target task to assist training and learning, improving the model's performance and generalization ability on few-shot target tasks.

[0026] An electronic device includes at least one processor, at least one GPU computing card, and a memory communicatively connected to the processor; wherein: the memory stores instructions executed by at least one processor, the instructions being executed by at least one processor or at least one GPU computing card to enable at least one processor or at least one GPU computing card to perform the above-described method;

[0027] A non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the methods described above.

[0028] This invention provides a method for extracting passage-level events in machine reading comprehension, the method comprising:

[0029] Obtain the chapter-level event extraction dataset, perform text preprocessing on the dataset, including data cleaning, chapter length filtering, Chinese word segmentation, and special character filtering, and divide the dataset into training set, validation set and test set;

[0030] The event extraction task mode definition is constructed, and a structured extraction mode based on event type, event trigger words, and event argument roles is constructed. The event type, event trigger words, and event argument roles of the corresponding text segments in the relevant texts of the dataset are labeled.

[0031] Label categories are defined based on event type and event trigger words. Event extraction fill-in-the-blank templates are constructed based on slotted natural language short sentences oriented towards event types. The fill-in-the-blank templates are preprocessed by Chinese and English word segmentation, word embedding, length filtering, and first-order identifier marking. Corresponding fill-in-the-blank templates are constructed using trigger words and event types as template category identifiers, and an event extraction fill-in-the-blank template database is built. By utilizing the event extraction task pattern information integration mechanism, event trigger words and event argument roles in the event extraction task paradigm are integrated as structured information into the fill-in-the-blank templates. The machine reading comprehension ability of the pre-trained language model is used to provide argument role hints for the event argument prediction slots in the fill-in-the-blank templates and type hints for the event description trigger words in the fill-in-the-blank templates. Fill-in-the-blank templates based on event information integration are constructed. The fill-in-the-blank template database and the dataset text are used to perform fill-in-the-blank template matching based on event type and event trigger words.

[0032] The text and fill-in-the-blank template are concatenated, and special separators [SEP] and argument slot identifiers [ARG] are added to form the input to the encoding layer. A bidirectional attention mechanism is used to calculate the attention relationship between the text and the fill-in-the-blank template, outputting an encoded representation vector and the last hidden vector. The encoded representation vector and the last hidden vector of the encoding layer are used as input to the autoregressive decoding layer, which enables the model to predict the fill-in-the-blank template in a machine reading comprehension manner, generating a fill-in-the-blank template argument filling prediction sequence. The fill-in-the-blank template argument filling prediction sequence is input to the event argument generation and mapping layer. According to the event extraction task mode definition, the fill-in-the-blank template argument slot values ​​are mapped and matched with the event extraction task mode structure to obtain the final event argument extraction result.

[0033] This invention provides a passage-level event extraction device based on machine reading comprehension, the device comprising:

[0034] Data preprocessing module: Acquires news and financial report event datasets, preprocesses Chinese and English datasets, including: article length filtering, Chinese word segmentation, special character filtering and encoding format unification, and divides the dataset into training set, validation set and test set;

[0035] Obtain event extraction fill-in-the-blank templates and preprocess the Chinese and English event extraction fill-in-the-blank templates, including event type and trigger word annotation, fill-in-the-blank template argument slot annotation, and fill-in-the-blank template database construction.

[0036] Bidirectional word embedding encoding module: Given a text passage and a corresponding fill-in-the-blank template, the passage and the fill-in-the-blank template are concatenated, and special delimiters and argument slot identifiers are added. The input is then embedded and represented by the bidirectional word embedding encoding module. Each layer in the encoding module calculates the word encoding vector and the last hidden layer vector through a multi-head self-attention mechanism, thus obtaining the representation relationship between the passage and the fill-in-the-blank template;

[0037] The autoregressive decoding module receives the output of the bidirectional word embedding encoding module. The input and output of the autoregressive decoding module are generated by right shifting. The prediction result of the previous time step is used as the input vector of the current time step. Each decoder layer additionally receives the hidden vector of the encoding layer. The predicted word vector is obtained by using the cross-attention mechanism. After reaching the maximum generation length or generating the sentence end identifier, the sequence generation process is completed. In the form of machine reading comprehension, the module predicts the filling of the argument slots in the fill-in-the-blank template based on the content of the passage text and generates the fill-in-the-blank template after filling in the blanks.

[0038] Argument generation and mapping module: This module extracts event arguments and corresponding argument roles from the fill-in-the-blank template prediction results, and replaces the fill-in-the-blank template argument slots with a unified argument identifier [arg] using regular expression matching and adds them to the vocabulary. In the autoregressive decoding layer's prediction of fill-in-the-blank template argument slots, if an argument role has multiple corresponding event arguments, the arguments are merged using a connector word.

[0039] Pre-training and fine-tuning module: Used to train the target task. It uses domain-similar tasks to perform cross-task pre-training on the model, enabling the model to acquire machine reading comprehension capabilities. Through knowledge transfer, it transfers external prior knowledge to the target task to assist training and learning, thereby improving the model's performance and generalization ability on target tasks with few samples.

[0040] This invention provides an electronic device, the device comprising:

[0041] At least one processor; at least one GPU computing card; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor or the at least one GPU computing card to enable the at least one processor or the at least one GPU computing card to perform the method described in any one of the embodiments of this application; a non-transitory computer-readable storage medium storing computer instructions, characterized in that the computer instructions are used to cause the computer to perform the method described in any one of the embodiments of this application. The technology according to this application can complete the task of extracting chapter-level events in low-resource scenarios (Chinese and English), improving the accuracy of chapter-level event extraction in low-resource scenarios.

[0042] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings. The technical solutions of the invention will now be described in further detail with reference to the accompanying drawings and embodiments. Attached Figure Description

[0043] Figure 1 A schematic diagram of the structure of the passage-level event extraction method based on machine reading comprehension provided by the present invention;

[0044] Figure 2 A flowchart of a passage-level event extraction method based on machine reading comprehension provided by the present invention;

[0045] Figure 3 Example diagram of the passage-level event extraction method based on machine reading comprehension provided by the present invention;

[0046] Figure 4 The flowchart of the event extraction and fill-in-the-blank template construction and document matching provided by the present invention;

[0047] Figure 5 The structural diagram of the bidirectional encoding module of the passage-level event extraction method based on machine reading comprehension provided by the present invention;

[0048] Figure 6 The structure diagram of the autoregressive decoding module of a passage-level event extraction method based on machine reading comprehension provided by this invention;

[0049] Figure 7 The structural diagram of the event information integration module in the fill-in-the-blank template in the passage-level event extraction method based on machine reading comprehension provided by the present invention;

[0050] Figure 8 A flowchart of the transfer learning module in a passage-level event extraction method based on machine reading comprehension provided by this invention;

[0051] Figure 9 A module structure diagram of a passage-level event extraction method based on machine reading comprehension provided by the present invention;

[0052] Figure 10 This is a block diagram of the electronic device of the present invention. Detailed Implementation

[0053] To enable those skilled in the art to better understand the present invention, a further detailed description of the invention is provided below with reference to the accompanying drawings. This includes various details of the embodiments of this application to aid understanding, and these should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0054] Example

[0055] This invention provides a method for extracting passage-level events in machine reading comprehension, the method comprising:

[0056] Obtain the chapter-level event extraction dataset, perform text preprocessing on the dataset, including data cleaning, chapter length filtering, Chinese word segmentation, and special character filtering, and divide the dataset into training set, validation set and test set;

[0057] The event extraction task mode definition is constructed, and a structured extraction mode based on event type, event trigger words, and event argument roles is constructed. The event type, event trigger words, and event argument roles of the corresponding text segments in the relevant texts of the dataset are labeled.

[0058] Label categories are defined based on event type and event trigger words. Event extraction fill-in-the-blank templates are constructed based on slotted natural language short sentences oriented towards event types. The fill-in-the-blank templates are preprocessed by Chinese and English word segmentation, word embedding, length filtering, and first-order identifier marking. Corresponding fill-in-the-blank templates are constructed using trigger words and event types as template category identifiers, and an event extraction fill-in-the-blank template database is built. By utilizing the event extraction task pattern information integration mechanism, event trigger words and event argument roles in the event extraction task paradigm are integrated as structured information into the fill-in-the-blank templates. The machine reading comprehension ability of the pre-trained language model is used to provide argument role hints for the event argument prediction slots in the fill-in-the-blank templates and type hints for the event description trigger words in the fill-in-the-blank templates. Fill-in-the-blank templates based on event information integration are constructed. The fill-in-the-blank template database and the dataset text are used to perform fill-in-the-blank template matching based on event type and event trigger words.

[0059] The text and fill-in-the-blank template are concatenated, and special separators [SEP] and argument slot identifiers [ARG] are added to form the input to the encoding layer. A bidirectional attention mechanism is used to calculate the attention relationship between the text and the fill-in-the-blank template, outputting an encoded representation vector and the last hidden vector. The encoded representation vector and the last hidden vector of the encoding layer are used as input to the autoregressive decoding layer, which enables the model to predict the fill-in-the-blank template in a machine reading comprehension manner, generating a fill-in-the-blank template argument filling prediction sequence. The fill-in-the-blank template argument filling prediction sequence is input to the event argument generation and mapping layer. According to the event extraction task mode definition, the fill-in-the-blank template argument slot values ​​are mapped and matched with the event extraction task mode structure to obtain the final event argument extraction result.

[0060] This invention provides a passage-level event extraction device based on machine reading comprehension, the device comprising:

[0061] The data preprocessing module acquires news and financial report event extraction datasets and preprocesses the Chinese and English datasets, including: article length filtering, Chinese word segmentation, special character filtering, and encoding format standardization. The datasets are then divided into training, validation, and test sets. The module also acquires event extraction fill-in-the-blank templates and preprocesses them in both Chinese and English, including event type and trigger word annotation, fill-in-the-blank template argument slot annotation, and fill-in-the-blank template database construction.

[0062] Bidirectional word embedding encoding module: Given a text passage and a corresponding fill-in-the-blank template, the passage and the fill-in-the-blank template are concatenated, and special delimiters and argument slot identifiers are added. The input is then embedded and represented by the bidirectional word embedding encoding module. Each layer in the encoding module calculates the word encoding vector and the last hidden layer vector through a multi-head self-attention mechanism, thus obtaining the representation relationship between the passage and the fill-in-the-blank template;

[0063] The autoregressive decoding module receives the output of the bidirectional word embedding encoding module. The input and output of the autoregressive decoding module are generated by right shifting. The prediction result of the previous time step is used as the input vector of the current time step. Each decoder layer additionally receives the hidden vector of the encoding layer. The predicted word vector is obtained by using the cross-attention mechanism. After reaching the maximum generation length or generating the sentence end identifier, the sequence generation process is completed. In the form of machine reading comprehension, the module predicts the filling of the argument slots in the fill-in-the-blank template based on the content of the passage text and generates the fill-in-the-blank template after filling in the blanks.

[0064] The argument generation and mapping module extracts event arguments and corresponding argument roles from the fill-in-the-blank template prediction results. It replaces the fill-in-the-blank template argument slots with a unified argument identifier [arg] using regular expression matching and adds them to the vocabulary. In the autoregressive decoding layer's prediction of fill-in-the-blank template argument slots, if an argument role has multiple corresponding event arguments, it merges the arguments using connectives. The pre-training and fine-tuning module trains the model for the target task. It uses domain-similar tasks for cross-task pre-training, enabling the model to acquire machine reading comprehension capabilities. Through knowledge transfer, it transfers external prior knowledge to the target task to assist training and learning, improving the model's performance and generalization ability on few-shot target tasks.

[0065] This invention provides an electronic device, the device comprising:

[0066] At least one processor; at least one GPU computing card; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor or the at least one GPU computing card to enable the at least one processor or the at least one GPU computing card to perform the method described in any one of the embodiments of this application;

[0067] A non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the method described in any one of the examples of this application, the method being able to complete the task of extracting chapter-level events in low-resource scenarios in Chinese and English, and improve the accuracy of extracting chapter-level events in low-resource scenarios;

[0068] Figure 1 This is a structural diagram of the present invention; the method described in this invention is applicable to Chinese and English text-level event extraction datasets, enabling event extraction under low-resource conditions, extracting event arguments and corresponding argument roles from texts. This method is executed by a text-level event extraction device based on machine reading comprehension, which is implemented in software and / or hardware. See [link to relevant documentation]. Figure 2 ;

[0069] The event extraction method provided in this embodiment is performed according to the following steps:

[0070] Obtain the chapter-level event extraction dataset, perform text preprocessing on the dataset, including data cleaning, chapter length filtering, Chinese word segmentation, and special character filtering, and divide the dataset into training set, validation set and test set;

[0071] The specific steps included in the text preprocessing method are as follows:

[0072] Obtain the event extraction dataset at the chapter level in both Chinese and English;

[0073] For example, news articles are crawled from the Currents Events section of English Wikipedia using web crawling methods; and Chinese financial reports are obtained by collecting publicly available corporate financial reports and reports on financial companies.

[0074] The text is subjected to data cleaning, length filtering, Chinese word segmentation, and special character filtering. In the data cleaning stage, the original text containing web page tags and redundant encoded characters is filtered out using regular expression matching. The text is then subjected to encoding format unification, special character filtering, and word segmentation using the Spacy library.

[0075] The chapter-level event extraction dataset was divided into training, testing, and validation sets in a random 8:1:1 ratio.

[0076] For example, after data preprocessing, the dataset is stored in JSON / JSONLine format, which is represented as {doc_id:00001,title:Changfu Ruihua Company Shares Frozen,sentences:[[sent1],[sent2],…[sent n ]]}, where sent i This refers to the text of sentences separated by periods within a passage;

[0077] We construct a paradigm definition for event extraction tasks, and build a structured extraction paradigm based on event type, event trigger words, and event argument roles. We then label the event types, event trigger words, and event argument roles of corresponding text segments in the dataset.

[0078] The event extraction paradigm described is a structured method for extracting event information; for example, such as... Figure 3 As shown in the fill-in-the-blank template event information integration module, the event extraction paradigm includes event types such as shareholding increase, shareholding decrease, bankruptcy reorganization, and transportation actions; including but not limited to event trigger words based on gerunds such as bankruptcy, shareholding increase, shareholding decrease, transportation, and attack; and event argument roles including time, location, action subject, and action recipient.

[0079] The specific method for annotating the corresponding text segments in the dataset chapters is as follows: annotate the event information of the dataset chapters in json / jsonline format, and construct a structured key-value pair dictionary annotation form;

[0080] For example, the input event chapter D = {d1, d2, ..., d...} n }correspond Figure 3 The instance input text in the text has the known event type of "freeze" and the trigger word "freeze". The event argument roles corresponding to the event include: time, executor, company, and frozen shares. The event arguments in the text are marked in order.

[0081] After the event text is annotated, the attribute values ​​in the json / jsonline format data include the chapter identifier, chapter title, chapter sentence text, and event type set. The chapter sentence text includes a list of chapter sentences separated by periods, and the event type set includes the event type, event trigger word, event argument, and corresponding argument role.

[0082] For example, the annotated text is {doc_id:00001,title:Changfu Ruihua Company Shares Frozen,sentences:[[Announcement states: Changfu Ruihua Company originally held 520,000 shares. ],[On May 5, 2017 local time, the Dalian Municipal People's Court froze 520,000 shares of Changfu Ruihua;]],events:[[event_type:Shares Frozen,event_trigger:Frozen,event_argument:{Time:May 5, 2017,Executing Agency:Dalian Municipal People's Court,Company:Changfu Ruihua,Frozen Shares:520,000 shares}]]}.

[0083] Tag categories are defined based on event type and event trigger words. Event extraction fill-in-the-blank templates are constructed based on event-oriented slotted natural language short sentences. The fill-in-the-blank templates are preprocessed by Chinese and English word segmentation, word embedding, length filtering, and first-order identifier marking. Corresponding fill-in-the-blank templates are constructed using trigger words and event types as template category identifiers, and an event extraction fill-in-the-blank template database is built. In one embodiment, the specific steps of the method for constructing the fill-in-the-blank templates and the database are as follows:

[0084] Based on event type and event trigger words, and referring to the structured paradigm of event extraction tasks, the event argument roles and quantities are obtained, and a concise natural language description fill-in-the-blank template containing all event argument roles under the event type is constructed.

[0085] For example, a fill-in template is constructed for event type share freezing. The fill-in template is classified according to event type share freezing and event trigger word freezing. Based on the time, executor, company, and frozen share argument role defined in the event extraction paradigm, the fill-in template is constructed such that [arg1] holds [arg2] and [arg3] is frozen in [arg4], so that the arguments are included in the fill-in template in the form of slots.

[0086] The entire event extraction task paradigm defined in the dataset is converted into a fill-in-the-blank template. The fill-in-the-blank templates are categorized by event type and trigger word. The argument roles and quantities in the event extraction task paradigm are used as references for the fill-in-the-blank template slot types and quantities. Specifically, the fill-in-the-blank template [arg1] holding [arg2] being frozen by [arg3] and [arg4] belongs to the share freeze event. It is driven by the trigger word freeze and contains four event arguments that correspond to the argument roles of the company's share freeze execution structure time.

[0087] Type labeling and argument role number labeling are performed on the fill-in-the-blank templates. A fill-in-the-blank template database is built in JSON file format. The event types and trigger words involved in the dataset are covered to realize a fill-in-the-blank template matching and generation mechanism for any event type and trigger word.

[0088] By leveraging the information integration mechanism of the event extraction paradigm, event trigger words and event argument roles in the event extraction task paradigm are incorporated as structured information into the fill-in-the-blank template. The machine reading comprehension mechanism of the model is used to provide argument role hints for the event argument prediction slots in the fill-in-the-blank template and to provide constraint hints for the event description trigger words in the fill-in-the-blank template. A fill-in-the-blank template based on event information integration is constructed, and fill-in-the-blank template matching based on event type and event trigger words is performed using the fill-in-the-blank template database and the text of the dataset.

[0089] The event extraction paradigm information integration mechanism is specifically manifested in integrating event information into the fill-in-the-blank template. For details, please refer to [link / details]. Figure 7 ,include:

[0090] Event trigger words and argument roles are treated as external knowledge and information is incorporated into fill-in-the-blank templates in the form of constraint declarations, generating corresponding fill-in-the-blank templates for different chapters;

[0091] At the model output level, a passage-wide vocabulary constraint is implemented, shrinking the original generated vocabulary size from 32,000 in the pre-trained language model to the size of the input passage vocabulary. This achieves a constraint mechanism that confines the generated answer to the passage scope. Specifically, if the predicted word w is within the passage vocabulary V... c Inside, the autoregressive decoding layer outputs the decoding layer vector. The dot product of the predicted word embedding vector and the softmax result is used; otherwise, the output is 0.

[0092] Based on the regular expression matching method, the corresponding argument slots and trigger word positions are located. Based on the trigger words in the chapter and the corresponding argument role set, the argument role information is integrated into the prototype fill-in template. Finally, it is concatenated with the input chapter and sent to the model encoding layer.

[0093] For example, based on the event extraction task mode information, the argument slots of the fill-in-the-blank template are declared and constrained. Specifically, the event type of the fill-in-the-blank template [arg1] holding [arg2] is frozen by [arg3] in [arg4], and the event argument roles are time, executor, company, and frozen shares, respectively. The corresponding argument slots of the fill-in-the-blank template are matched and positioned to obtain the more specific fill-in-the-blank template after incorporating event information, which states that the shares [arg2] held by the company [arg1] are frozen by the executor [arg3] at time [arg4].

[0094] The text and the fill-in-the-blank template are concatenated, and special separators and argument slot identifiers are added to form the input to the encoding layer. A bidirectional attention mechanism is used to calculate the attention relationship between the text and the fill-in-the-blank template, and the output is an encoded representation vector and the last hidden vector. The encoded representation vector and the last hidden vector of the encoding layer are used as input, and an autoregressive decoding layer is passed to enable the model to predict the fill-in-the-blank template in the form of machine reading comprehension, generating a fill-in-the-blank template argument filling prediction sequence.

[0095] For specific methods on obtaining and combining passage and fill-in-the-blank templates, please refer to [link / reference]. Figure 4 The specific steps include:

[0096] Obtain the event types and trigger words from the event extraction, collect the types of event extraction paradigms and argument role definitions involved in the event extraction task, build a fill-in-the-blank template database, and classify and label the fill-in-the-blank templates using event types and trigger words as a joint primary key;

[0097] The dataset contains a passage D with a set of event types E. The input passage is matched with a corresponding fill-in-the-blank template t using a fill-in-the-blank template database B. in ∈B,t in It is a slotted natural language short phrase for event type e;

[0098] For the input passage, fill-in-the-blank templates are matched from the fill-in-the-blank template database. Text preprocessing is then performed on the fill-in-the-blank templates, and the template construction formula is t. in ={w i }∪{arg j },(i≤n,j≤|R e |). arg j Represents the set of argument characters R e The slot in the fill-in-the-blank template for the j-th event argument;

[0099] The encoding method of the input text is described in [reference needed]. Figure 5 ;

[0100] Given a passage D = {w1, w2, ..., w n } and corresponding fill-in-the-blank template t in , where w i To indicate words in a passage, use D and t. in The input is concatenated and processed through a word embedding layer (D,t). in Embedded representation;

[0101] The model's encoding and decoding layers share the parameters of the word embedding layer. Each layer of the encoding layer calculates an encoding vector through a multi-head self-attention mechanism, learning the representational relationship between the passage and the fill-in-the-blank template. Q, K, V are attention weight matrices. The attention heads of multiple encoders are concatenated and linearly transformed to obtain (D, t in The encoded vector representation is obtained by passing the input text through the encoding layer and providing the output word encoding representation and the final hidden vector to the autoregressive decoding layer.

[0102] The model aims to address the given passage and fill-in-the-blank template (D,t) in ), in the form of machine reading comprehension, for t in Predict the argument slots and output the filled template t. out ={w i }∪{α j },(i≤|t out |,j≤|R e |), α j Representative model for slot arg j The predicted arguments to be filled in;

[0103] The decoding method for the input text encoding vector is described in [reference needed]. Figure 5 It receives the hidden vector from the encoding layer, and generates an autoregressive sequence between the input and output by right shifting, thus generating a blank-filling template t. out Each decoder layer receives the previous generated vector and the latent vector of the encoding layer as input, and uses the cross-attention mechanism to calculate and obtain the predicted word vector. The model performs autoregressive sequence generation.

[0104] The fill-in-the-blank template argument filling prediction sequence is input into the event argument generation and mapping layer. Based on the event extraction task mode definition, the fill-in-the-blank template argument slot values ​​are mapped and matched with the event extraction task paradigm structure to obtain the final event argument extraction result. For example, the fill-in-the-blank template argument slots are replaced with a unified argument identifier [arg] and added to the vocabulary using regular expression matching. For the fill-in-the-blank template company [arg1]'s shares [arg2] held by the enforcement agency [arg3] frozen at time [arg4], [arg] is removed. i The tag number feature is replaced with a consistent slot tag [arg]; for the annotation of event trigger words, the special tag [arg] is added to the vocabulary of the pre-trained language model;

[0105] Furthermore, the processed fill-in-the-blank template format is: The company [arg] holds [arg] shares, which were frozen by the enforcement agency [arg] at time [arg].

[0106] In the prediction of fill template argument slots in the autoregressive decoding layer, if there are multiple event arguments and their corresponding argument roles, the arguments are merged using a connector.

[0107] For example, if the slot value of the company is an argument role in the fill-in-the-blank template and there are multiple event arguments in a single event record, the extraction result for the fill-in-the-blank template is the concatenation of arguments;

[0108] For text-based articles, please refer to the cross-task knowledge transfer method. Figure 8 ,include:

[0109] By learning from machine reading comprehension source tasks with similar task formats, effective external prior knowledge is acquired and transferred to the target task for training. Through knowledge transfer, the model learns stronger representation and natural language understanding capabilities on tasks with larger-scale corpora, and achieves stronger generalization ability and performance on the target task.

[0110] The model is pre-trained across tasks using machine reading comprehension tasks in similar domains, with the training objective being... Where T represents the pre-training task. The training samples represent the task. Indicates the given input Under the given conditions, the prediction results Maximum likelihood estimation;

[0111] Fine-tune the model that has acquired prior knowledge for the target task, and train the target. D represents the target task input passage, e represents a single event in D, and (r, α) represents the event argument and argument role pair in event e.

[0112] For example, in the cross-task pre-training stage, a large-scale Chinese text-level event extraction dataset is used for cross-task pre-training. A text-level event extraction device based on machine reading comprehension is used as input, along with the text and its corresponding fill-in-the-blank template. Predictions are obtained by filling in the argument slots in the fill-in-the-blank template based on the text content, in a machine reading comprehension format, to obtain the complete template. The generated predictions are then input into an event argument mapping layer to obtain event arguments and argument roles, resulting in the final event extraction result. Figure 9 This is a schematic diagram of the modular structure of the device of the present invention, including: a data preprocessing module, a bidirectional word embedding encoding module, an autoregressive decoding module, an argument generation and mapping module, and a pre-training and fine-tuning module, wherein:

[0113] The data preprocessing module acquires news and financial report event extraction datasets and preprocesses the Chinese and English datasets, including: article length filtering, Chinese word segmentation, special character filtering, and encoding format standardization. The datasets are then divided into training, validation, and test sets. The module also acquires event extraction fill-in-the-blank templates and preprocesses them in both Chinese and English, including event type and trigger word annotation, fill-in-the-blank template argument slot annotation, and fill-in-the-blank template database construction.

[0114] Bidirectional word embedding encoding module: Given a text passage and a corresponding fill-in-the-blank template, the passage and the fill-in-the-blank template are concatenated, and special delimiters and argument slot identifiers are added. The input is then embedded and represented by the bidirectional word embedding encoding module. Each layer in the encoding module calculates the word encoding vector and the last hidden layer vector through a multi-head self-attention mechanism, thus obtaining the representation relationship between the passage and the fill-in-the-blank template;

[0115] The autoregressive decoding module receives the output of the bidirectional word embedding encoding module. The input and output of the autoregressive decoding module are generated by right shifting. The prediction result of the previous time step is used as the input vector of the current time step. Each decoder layer additionally receives the hidden vector of the encoding layer. The predicted word vector is obtained by using the cross-attention mechanism. After reaching the maximum generation length or generating the sentence end identifier, the sequence generation process is completed. In the form of machine reading comprehension, the module predicts the filling of the argument slots in the fill-in-the-blank template based on the content of the passage text and generates the fill-in-the-blank template after filling in the blanks.

[0116] The argument generation and mapping module extracts event arguments and corresponding argument roles from the fill-in-the-blank template prediction results. It replaces the fill-in-the-blank template argument slots with a unified argument identifier [arg] using regular expression matching and adds them to the vocabulary. In the autoregressive decoding layer's prediction of fill-in-the-blank template argument slots, if an argument role has multiple corresponding event arguments, it merges the arguments using connectives. The pre-training and fine-tuning module trains the model for the target task. It uses domain-similar tasks for cross-task pre-training, enabling the model to acquire machine reading comprehension capabilities. Through knowledge transfer, it transfers external prior knowledge to the target task to assist training and learning, improving the model's performance and generalization ability on few-shot target tasks.

[0117] This invention provides an electronic device and a readable storage medium; such as Figure 10 The diagram shown is a block diagram of the electronic device of the present invention. The electronic device refers to various modern electronic digital computers, including, for example, personal computers, portable computers, and various server devices. The components, their interconnections, and functions shown in this invention are merely examples.

[0118] like Figure 10 As shown, the electronic device includes: one or more multi-core processors, one or more GPU computing cards, and memory. To enable interaction between the electronic device, it should also include: input devices and output devices. The various devices are interconnected and communicate via a bus.

[0119] The memory is the non-transitory computer-readable storage medium provided in this application. The memory stores instructions executable by at least one multi-core processor or at least one GPU computing card to cause the entity recognition and linking method provided in this application to be executed. The non-transitory computer-readable storage medium of this application stores computer instructions used to cause a computer to execute the entity recognition and linking method provided in this application;

[0120] Input devices provide and accept control signals from the user into the electronic device, including keyboards that generate numeric or character information and mice used to control the device to generate other key signals. Output devices provide feedback information to the user's electronic device, including displays that print out execution results or processes.

[0121] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practical disclosure. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and it should be understood that the invention is not limited to the precise structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope.< / s> < / sep> < / s>

Claims

1. A method for extracting passage-level events based on machine reading comprehension, characterized in that, Follow these steps: a. Obtain the document-level event extraction dataset, perform text preprocessing on the dataset, including data cleaning, document length filtering, Chinese word segmentation, and special character filtering, and divide the dataset into training set, validation set and test set; b. Construct an event extraction task mode definition, and construct a structured extraction definition based on event type, event trigger words, and event argument roles. Label the event type, event trigger words, and event argument roles of the corresponding text segments in the dataset texts involved in step a. c. Define label categories based on event type and event trigger words, construct event extraction fill-in-the-blank templates based on concise natural language descriptions for events, and preprocess the fill-in-the-blank templates by Chinese and English word segmentation, length filtering, and first and last identifier marking. For the event extraction argument definition categories in step b, construct corresponding fill-in-the-blank templates using trigger words and event types as template category identifiers, and build an event extraction fill-in-the-blank template database. The fill-in-the-blank template is set as a natural language short sentence with slot values ​​for event types, where the argument slots to be predicted are represented by special identifiers [ARG]. The fill-in-the-blank template database covers the event types and trigger words involved in the dataset, and uses the event type and trigger words as unique primary keys to match the corresponding fill-in-the-blank templates. The event extraction task mode information integration mechanism specifically uses the event type, event trigger words, and argument role information defined in the event extraction task mode to integrate into the argument slots corresponding to the fill-in-the-blank templates, and uses the machine reading comprehension mechanism of the model to provide slot type hints. d. Utilize the event extraction structured pattern information integration mechanism to integrate the event trigger words and event argument roles in the event extraction task pattern definition as external information into the construction of the fill-in template in step c. Constrain the event argument prediction slots in the fill-in template with argument roles and constrain the event description trigger words in the fill-in template with trigger words to construct a fill-in template based on event extraction pattern information. e. Using the event extraction fill-in-the-blank template database from step c and the text of the dataset from step a, fill-in-the-blank templates are matched and spliced ​​based on event type and event trigger words. f. Concatenate the passage text and fill-in-the-blank template from step e, add a special separator [SEP] and an argument slot identifier [ARG] to form the input of the encoding layer, use a bidirectional attention mechanism to calculate the attention relationship between the passage text and the fill-in-the-blank template, and output the encoded representation vector and the last hidden vector. g. Using the encoded representation vector obtained in step f and the last hidden vector of the encoding layer as input, the model performs fill-in-the-blank prediction on the fill-in-the-blank template in the form of machine reading comprehension through the autoregressive decoding layer, generating the argument filling prediction sequence of the fill-in-the-blank template. h. Perform event argument generation and mapping operations on the fill-in template argument prediction sequence described in step g. According to the event extraction mode definition described in step b, perform mapping and matching between the fill-in template argument slot values ​​and the event extraction task mode definition to obtain the final event argument extraction result.

2. The method for extracting passage-level events based on machine reading comprehension according to claim 1, characterized in that, The document-level event extraction dataset mentioned in step a is specifically in two languages, English and Chinese, and is a document-level event extraction dataset in the fields of news and finance. In the document-level event extraction dataset, the document contains multiple event descriptions and multiple event types, corresponding to multiple event trigger words and event argument roles.

3. The method for extracting passage-level events based on machine reading comprehension according to claim 1, characterized in that, The event extraction task argument definition described in step b is a structured description of event information, and the key attribute set in the key-value pair set is defined as the argument role of the event extraction task.

4. The method for extracting passage-level events based on machine reading comprehension according to claim 1, characterized in that, The method for splicing the passage and fill-in-the-blank template described in step f is characterized by: Given text With fill-in-the-blank template [s] is formed by concatenation and the insertion of delimiters. [sep] [s] is used as a new input sequence and input into the coding layer. Multiple sets of encoder attention heads are concatenated and linearly transformed to obtain the final coding vector.

5. The method for extracting passage-level events based on machine reading comprehension according to claim 1, characterized in that, In step g, the autoregressive decoding layer receives the encoded vector from the encoding layer and generates a unidirectional autoregressive sequence by right shifting. Each decoder in the decoding layer receives the generated vector from the previous time step and the last hidden vector from step f, and calculates the generation probability of the next time step through a cross-attention mechanism.

6. The method for extracting passage-level events based on machine reading comprehension according to claim 1, characterized in that, The event argument generation and mapping described in step h involves filling the prediction sequence with the fill-in template arguments described in step g, matching and transforming the event extraction task paradigm and the fill-in template according to step b, locating the coordinates of the slots in the fill-in template using regular expressions, and realizing the structured mapping of event arguments and argument roles for the predicted slot values ​​in the fill-in template.

7. A document-level event extraction device based on machine reading comprehension, wherein the device uses the document-level event extraction method of claims 1-6, characterized in that the device comprises a data preprocessing module, a bidirectional word embedding encoding module, an autoregressive decoding module, an argument generation and mapping module, and a pre-training and fine-tuning module. Composition, in which: The data preprocessing module acquires news and financial report event extraction datasets and preprocesses the Chinese and English datasets, including: article length filtering, Chinese word segmentation, special character filtering, and encoding format standardization; it also divides the datasets into training, validation, and test sets. The module acquires event extraction fill-in-the-blank templates and preprocesses them, including event type and trigger word annotation, fill-in-the-blank template argument slot annotation, and fill-in-the-blank template database construction. Bidirectional word embedding encoding module: Given a passage text and a corresponding fill-in-the-blank template, the passage and the corresponding fill-in-the-blank template are concatenated, and special separators and argument slot identifiers are added; the input is embedded and represented by the bidirectional word embedding encoding module; each layer in the encoding module calculates the word encoding vector and the last hidden layer vector through a multi-head self-attention mechanism to obtain the representation relationship between the passage and the fill-in-the-blank template; The autoregressive decoding module receives the output of the bidirectional word embedding encoding module. The input and output of the autoregressive decoding module are generated by right shifting. The prediction result of the previous time step is used as the input vector of the current time step. Each decoder layer additionally receives the hidden vector of the encoding layer. The predicted word vector is obtained by using the cross-attention mechanism. After reaching the maximum generation length or generating the sentence end identifier, the sequence generation process is completed. In the form of machine reading comprehension, the module predicts the filling of the argument slots in the fill-in-the-blank template based on the content of the passage text and generates the fill-in-the-blank template after filling in the blanks. Argument generation and mapping module: used to extract event arguments and corresponding argument roles from the fill-in-the-blank template prediction results, and replace the fill-in-the-blank template argument slots with unified argument identifiers through regular expression matching. Add to vocabulary; in the autoregressive decoding layer's prediction of fill-in template argument slots, if an argument role has multiple event arguments corresponding to it, use connecting words to merge the arguments; Pre-training and fine-tuning module: Used to train the target task. It uses domain-similar tasks to perform cross-task pre-training on the model, enabling the model to acquire machine reading comprehension capabilities. Through knowledge transfer, it transfers external prior knowledge to the target task to assist training and learning, thereby improving the model's performance and generalization ability on target tasks with few samples.

8. An electronic device, characterized in that, The device includes at least one processor, at least one GPU computing card, and a memory communicatively connected to the processor; wherein: the memory stores instructions executed by at least one processor, the instructions being executed by at least one processor or at least one GPU computing card, such that at least one processor or at least one GPU computing card can perform the method described in claims 1-6; 9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method described in claims 1-6.

Images