A method, system, storage medium and electronic device for chapter-level event extraction

Abstract

The application provides a kind of facing chapter-level event extraction method, system, storage medium and electronic equipment, it is related to natural language processing technical field.The application includes, obtaining to be analyzed document;According to to be analyzed document, event type is acquired based on machine learning understanding model;According to the document after labeling event type, the relationship pair of the corresponding event parameter type-entity candidate object is acquired using learning search model or deep neural network model;According to event type, parameter type and entity candidate object, fill in macro event template framework, as event extraction result return user.Each event type definition parameter is filled in set constraint to a pre-defined, but highly interdependent category set containing key event information, so that the event is expressed to the user in a relatively simple but information-rich way, and further processed by downstream analysis methods.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and particularly relates to a method and system for event extraction at a document level, a storage medium and an electronic device. BACKGROUND

[0002] The goal of event extraction is to identify instances of a class of events in a document and any arguments corresponding to the roles in the events. Existing event extraction frameworks are divided into sentence-level event extraction and document-level event extraction. Sentence-level event extraction refers to identifying and extracting individual events from each sentence in a document, as well as any entities playing argument roles in these events, but it cannot summarize the content of the document. Document-level event extraction is a challenging information extraction task that requires inference from the entire article. It has obvious applicability in the real world, as it allows users to quickly identify the characters, content, location and time of events in a document without having to read the entire document.

[0003] So far, event extraction methods have not been able to provide a satisfactory solution. Most current event extraction systems rely on local information at the phrase or sentence level and do not consider the article as a whole. Since an event often requires multiple sentences to be fully described, this approach cannot capture the relationships between events mentioned at the document level, as well as the event arguments in different sentences compared to event triggers, thereby limiting the extraction performance. At the same time, the event extraction work on the document is mainly focused on highly specific domains, and usually relies heavily on hand-crafted rules, which cannot be well generalized to new domains. SUMMARY

[0004] (I) Technical problems solved

[0005] In view of the deficiencies of the prior art, the present application provides a method and system for event extraction at a document level, a storage medium and an electronic device, which solves the technical problem of being unable to capture the relationships between events mentioned at the document level.

[0006] (II) Technical solutions

[0007] To achieve the above object, the present application is implemented by the following technical solutions:

[0008] A method for event extraction at a document level, a machine learning understanding model, a learning search model or a deep neural network model are pre-trained, and the extraction method comprises:

[0009] S1, obtaining a document to be analyzed;

[0010] S2, obtaining an event type based on the machine learning understanding model according to the document to be analyzed;

[0011] S3. Based on the document after annotating the event type, use the learning search model or deep neural network model to obtain the corresponding event parameter type-entity candidate object relationship pair;

[0012] S4. Based on the event type, parameter type, and entity candidate object, fill in the macro event template framework and return it to the user as the event extraction result.

[0013] Preferably, the machine learning understanding model is a gated attention (GA) reader, and its training process includes:

[0014] The event parameter type query and word embeddings of documents in the training set are obtained through a lookup table and processed in the subsequent K-layer network. The k-th layer takes the document embeddings from the (k-1)-th layer as input; the document word embeddings d of each layer are... i The parameter type query term embedding Q is obtained by transformation using a bidirectional gated recurrent unit (GRU), and then combined using a gated attention module to obtain the input x for the next layer. i Each GA box represents a gating-attention module that applies attention to the query parameters to the document representation.

[0015] After repeating this process more than K layers, scores are calculated for words in the document and converted into a probability distribution over the words using the Softmax() function; the resulting probability distribution is then used to select the query answer.

[0016] Preferably, the learning search model is trained using a learning search algorithm, including:

[0017] S10. Randomly select a document d from the labeled document D and generate a sequence of candidate relation pairs of entity and parameter types;

[0018] S20. Within a random sampling step t, execute the currently learned policy π on document d. i ;

[0019] S30. Execute strategy π within step size t based on the candidate relation pairs corresponding to document d. i Afterwards, all possible strategies of the remaining candidate relation pairs are relative to the optimal strategy π. * The strategy corresponding to the minimum loss generates a new training example and adds the new training sample to the training set.

[0020] S40. After generating k new examples, retrain the new policy using the new training set. Then jump back to step 1;

[0021] S50. After N iterations, the final training strategy is obtained.

[0022] wherein N is a predefined number of training times, m is a predefined number of sampling times, policy is an action mapping of filling a specific parameter slot with a specific candidate entity, is a manually confirmed optimal policy, and * is a manually confirmed optimal policy, and i is a manually confirmed optimal policy, and

[0023] Preferably, the deep neural network model comprises an input layer, two hidden layers and an output layer.

[0024] Preferably, the deep neural network model comprises an input layer, two hidden layers and an output layer.

[0025] Preferably, the training process of the deep neural network model comprises:

[0026] S100, using the current parameters W to predict each training sample x i , and comparing the prediction result with the real sample label y i to calculate the loss;

[0027] S200, updating the model parameters W through the back propagation process. After the parameter update is completed, the training of the next sample is performed;

[0028] S300, after the execution of all training samples is completed, the current model is evaluated using the validation sample set (V, Z) to calculate the F1 score; if the current model F1 score is lower than the historical highest score F1 best , then these parameters are stored as the new optimal model parameters W best , and the F1 best is updated to the current F1 score;

[0029] S400, after the k periods are ended, the historical optimal model parameters W best are returned.

[0030] A document-level event extraction system, which pre-trains a machine learning understanding model, a learning search model or a deep neural network model, comprises:

[0031] An acquisition module, configured to acquire a document to be analyzed;

[0032] A learning module, configured to acquire an event type based on the machine learning understanding model according to the document to be analyzed;

[0033] The search module is configured to obtain a corresponding event parameter type-entity candidate object relation pair by using the learning search model or the deep neural network model according to the document after the event type is labeled.

[0034] The filling module is configured to fill a macro event template framework according to the event type, the parameter type and the entity candidate object, and return the event extraction result to the user.

[0035] A storage medium storing a computer program for event extraction at a chapter level, wherein the computer program causes a computer to execute the event extraction method at a chapter level as described above.

[0036] An electronic device, comprising:

[0037] One or more processors;

[0038] Memory; and

[0039] One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the programs comprise a program for executing the event extraction method at a chapter level as described above.

[0040] (III) Beneficial Effects

[0041] The present application provides an event extraction method, system, storage medium and electronic device at a chapter level. Compared with the prior art, the present application has the following beneficial effects:

[0042] The present application comprises obtaining a document to be analyzed; obtaining an event type based on the machine learning understanding model according to the document to be analyzed; obtaining a corresponding event parameter type-entity candidate object relation pair by using the learning search model or the deep neural network model according to the document after the event type is labeled; filling a macro event template framework according to the event type, the parameter type and the entity candidate object, and returning the event extraction result to the user. By filling the definition parameter of each event type into a set of constraints to a pre-defined, but highly interdependent category set containing key event information, the event is represented to the user in a relatively simple but information-rich manner, and further processed by downstream analysis methods. It can be generalized in a wide range of event types, and the event structure has wide applicability. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0044] Figure 1 A block diagram of a chapter-oriented event extraction method provided by an embodiment of the present application;

[0045] Figure 2 A schematic diagram of a GA reader architecture provided by an embodiment of the present application;

[0046] Figure 3 A schematic diagram of a deep neural network architecture for event extraction provided by an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are some embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the present application.

[0048] The embodiments of the present application provide a chapter-oriented event extraction method, system, storage medium and electronic device, which solve the technical problem that the relationship between events mentioned at the document level cannot be captured.

[0049] The technical solutions in the embodiments of the present application are as follows to solve the above technical problem:

[0050] The embodiments of the present application include obtaining a document to be analyzed; obtaining an event type based on a machine learning understanding model according to the document to be analyzed; obtaining a corresponding event parameter type-entity candidate object relationship pair by using a learning search model or a deep neural network model according to the document after the event type is labeled; filling a macro event template framework according to the event type, parameter type and entity candidate object, and returning an event extraction result to a user. By filling the definition parameter of each event type into a set constraint to a pre-defined, but highly interdependent category set containing key event information, the event is represented to the user in a relatively simple but information-rich manner, and further processed by a downstream analysis method. It can be generalized in a wide range of event types, and the event structure has wide applicability.

[0051] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0052] Example:

[0053] like Figure 1 As shown, this embodiment of the invention provides a method for extracting events at the document level, which involves pre-training a machine learning understanding model, a learning search model, or a deep neural network model. The extraction method includes:

[0054] S1. Obtain the document to be analyzed;

[0055] S2. Based on the document to be analyzed, obtain the event type using the machine learning understanding model;

[0056] S3. Based on the document after annotating the event type, use the learning search model or deep neural network model to obtain the corresponding event parameter type-entity candidate object relationship pair;

[0057] S4. Based on the event type, parameter type, and entity candidate object, fill in the macro event template framework and return it to the user as the event extraction result.

[0058] This invention presents events to users in a concise yet informative manner by constraining the set of definition parameters for each event type to a predefined, highly interdependent set of categories containing key event information. Further processing is then performed using downstream analysis methods. This approach can be generalized across a wide range of event types, and the event structure has broad applicability.

[0059] The following will detail each step of the above solution:

[0060] In step S1, the document to be analyzed is obtained.

[0061] The technical means of obtaining the text can be any of the existing technologies, such as web scraping, which will not be elaborated here.

[0062] In step S2, the event type is obtained based on the machine learning understanding model according to the document to be analyzed.

[0063] This paper presents a method for event extraction based on machine reading comprehension, which trains a general question-answering system using existing large-scale machine reading comprehension corpora and directly applies the trained model to the macro-event template filling task. The core of this process is a gated attention (GA) reader. For example... Figure 2As shown, the GA Reader is a deep learning model that employs a multi-hop architecture combined with an attention mechanism. The multi-hop architecture simulates a person reading a text several times, each time refining their understanding of the current text. The attention mechanism serves to focus the reader's attention on a given question while reading the text, causing the model to pay more attention to information relevant to the event when processing the text.

[0064] The training process of the GA Reader described above includes obtaining the event parameter type query and the document word embeddings through lookup tables, and processing them in the following K-layer network, where the k-th layer network obtains the previous document embedding from the (k-1)-th layer network as input. Using a multi-layer network structure can build a more complex input document word embedding representation for the model, with each layer focusing on different aspects of the parameter type query. The document word embedding d i and the parameter type query word embedding Q are converted using bidirectional gated recurrent units (GRUs), respectively, and then combined using a gated attention module to obtain the input x i of the next layer. Each "GA" box represents a gated-attention module that applies attention to the parameter type query to the document representation. After repeating this process for more than K layers, scores are calculated for the words in the document and converted to a probability distribution over the words using the Softmax() function. The resulting probability distribution can be used to select the answer to the query.

[0065] The gated-attention module is applied to the document word embedding d i in the following way:

[0066] α i = softmax(Q T d i )

[0067]

[0068]

[0069] where Q is the parameter type query embedding representation and is the Hadamard product.

[0070] In the last layer, the document and query representations are used to calculate the score of each word using the inner product, and then obtain the probability distribution of the words in the document through the softmax layer. When a set of words appears multiple times in the document, the probability of the word is aggregated and re-normalized, and the final result is obtained by selecting the candidate word with the maximum probability:

[0071]

[0072]

[0073] where C is the candidate result set, s is the softmax() probability vector corresponding to the candidate word c, specifying the index of the document d corresponding to the candidate word c.

[0074] In step S3, according to the document after labeling the event type, the learning search model or the deep neural network model is used to obtain the corresponding event parameter type-entity candidate object relationship pair;

[0075] For the labeled text data, a learning search-based method or a deep neural network-based method is used to train on the existing data set, build a model, and complete the filling of the macro event template.

[0076] First, the learning search-based macro event template parameter filling:

[0077] For each document, the named entity recognition method is used to collect all entity candidates, and each entity is paired with all possible parameter types in the macro event template structure. For the formed parameter type-entity relationship pair, a decision needs to be made on whether to include it in the final macro event template. For each decision, not only the local features of the current relationship pair need to be considered, but also all decisions made by the model in the past, so that the past decisions can be included in the context when filling the template

[0078] The learning search model is trained using a learning search algorithm, including:

[0079] S10, randomly selecting a document d from the labeled document D and generating a sequence of entity and parameter type candidate relationship pairs;

[0080] S20, execute the currently learned strategy π on the document d within the randomly sampled step t i ;

[0081] S30, after the strategy π is executed on the candidate relationship pairs corresponding to the document d within the step t i , the strategy corresponding to the minimum loss of all possible strategies of the remaining candidate relationship pairs relative to the optimal strategy π * generates a new training example, and the new training sample is added to the training set;

[0082] S40, after generating k new examples, retrain the new strategy using the new training set, and then jump back to step 1;

[0083] S50, after N times of circulation, the final training strategy is obtained

[0084] Where N is the number of custom training iterations, m is the number of custom sampling iterations, and strategy π is the action mapping that fills specific parameter slots with specific candidate entities. * It is a manually confirmed optimal strategy, β i i = 1, ..., N represents the weights updated using a custom strategy.

[0085] Second, deep neural networks for event extraction:

[0086] In order to extract the main events in the text, given the input text and a set of entity candidate objects, this embodiment of the invention fills the macro event template by performing binary prediction on whether each (entity candidate object, parameter type) relation pair is included or excluded in the final template.

[0087] Model architecture such as Figure 3 As shown, the deep neural network model includes an input layer, two hidden layers, and an output layer; wherein, the input layer consists of text data features required by the model, containing information about the current candidate entity relationship pairs, and the output layer is the probability value of whether the current candidate relationship pair is included in the final template; a multi-layer network structure is used to learn more complex interaction information from the input features.

[0088] The model first applies a linear transformation W to convert the input layer text data features into low-dimensional hidden vectors. Then, it processes these low-dimensional hidden vectors through a nonlinear transformation, enabling the network to model more complex decisions. Commonly used nonlinear transformation functions include sigmoid(), tanh(), and ReLU(), and the transformation process is as follows:

[0089]

[0090]

[0091] ReLU(x) = max(0,x)

[0092] The nonlinear transformation process can be repeated multiple times. In the final layer, the data is transformed using the softmax() function, converting the output decision scores into a set of probabilities:

[0093]

[0094] Specifically, the training process of the deep neural network model includes:

[0095] S100, Apply the current parameter W to each training sample x i Make a prediction and compare the prediction results with the true sample labels y. i Compare and calculate the loss;

[0096] S200. Update the model parameters W through backpropagation. After the parameters are updated, train the next sample.

[0097] S300. After all training samples have been executed, evaluate the current model using the validation sample set (V,Z) and calculate the F1 score; if the current model's F1 score is lower than the historical highest F1 score... best Then these parameters are stored as the new optimal model parameters W. best and update F1 best This is the current F1 score;

[0098] S400. After k cycles, return the historically optimal model parameters W. best .

[0099] In step S4, the macro-event template framework is filled according to the event type, parameter type, and entity candidate object, and returned to the user as the event extraction result.

[0100] For the task of extracting text events at the chapter level, this invention proposes a method for template filling based on a macro-level event template framework to create a realistic and useful chapter-level event extraction model. The event template format is shown in Table 1, consisting of event type, event parameter type, and event parameter filling entity. Each parameter field can be filled with zero or one or more text characters.

[0101] Table 1

[0102]

[0103] This invention provides a document-level event extraction system, which pre-trains a machine learning understanding model, a learning search model, or a deep neural network model. The extraction system includes:

[0104] The acquisition module is used to acquire the document to be analyzed.

[0105] The learning module is used to obtain the event type based on the machine learning understanding model according to the document to be analyzed;

[0106] The search module is used to obtain the corresponding event parameter type-entity candidate object relationship pairs based on the documents after annotating event types, using the learning search model or deep neural network model.

[0107] The population module is used to populate the macro event template framework according to the event type, parameter type, and entity candidate object, and return it to the user as the event extraction result.

[0108] This invention provides a storage medium storing a computer program for document-level event extraction, wherein the computer program causes a computer to execute the document-level event extraction method described above.

[0109] This invention provides an electronic device, comprising:

[0110] One or more processors;

[0111] Memory; and

[0112] One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including methods for performing document-level event extraction as described above.

[0113] It is understood that the document-level event extraction system, storage medium, and electronic device provided in the embodiments of the present invention correspond to the document-level event extraction method provided in the embodiments of the present invention. The explanations, examples, and beneficial effects of the relevant content can be referred to the corresponding parts of the document-level event extraction method, and will not be repeated here.

[0114] In summary, compared with existing technologies, it has the following beneficial effects:

[0115] This invention includes: acquiring a document to be analyzed; obtaining event types based on the document using a machine learning understanding model; obtaining corresponding event parameter type-entity candidate object relationship pairs using a learning search model or a deep neural network model based on the document labeled with event types; and filling a macro-event template framework based on the event types, parameter types, and entity candidate objects, returning this as the event extraction result to the user. By constraining the set of defined parameters for each event type to a predefined, but highly interdependent set of categories containing key event information, events are represented to the user in a concise yet information-rich manner, and further processed by downstream analysis methods. This approach can generalize to a wide range of event types, and the event structure has broad applicability.

[0116] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0117] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for event extraction at a document level, characterized in that, The pre-trained machine learning understanding model, learning search model or deep neural network model, the extraction method comprises: S1, obtaining a document to be analyzed; S2, obtaining an event type based on the machine learning understanding model according to the document to be analyzed; S3, obtaining a corresponding event parameter type-entity candidate object relationship pair by using the learning search model or deep neural network model according to the document after labeling the event type; S4, filling a macro event template framework according to the event type, parameter type and entity candidate object, and returning an event extraction result to a user, The machine learning understanding model is a gated attention GA reader, and the training process comprises: The event parameter type query and the word embedding of the document word in the training set are obtained through a lookup table, and are processed in a K-layer network. The kth layer network obtains the previous document embedding from the (k-1)th layer network as input. The document word embedding d i and the parameter type query word embedding Q are respectively converted using a bidirectional gated recurrent unit (GRU), and then combined using a gated attention module to obtain the input x i of the next layer; wherein each GA box represents a gated-attention module that applies attention focusing on the parameter type query to the document representation; after repeating this process for more than K layers, a score is calculated for a word in the document, and is converted into a probability distribution over the word using a Softmax() function; and the final probability distribution is used to select the answer to the query. The learning search model is trained by using a learning search algorithm, comprising: S10, randomly extracting a document d from the labeled document D and generating a sequence of entity, parameter type and candidate relationship pair; S20, execute the currently learned policy p on the document d within the randomly sampled step size t i ; S30, performing the strategy p in the step size t from the candidate relation pair corresponding to the document d i After that, the strategy corresponding to the minimum loss of all possible strategies of the remaining candidate relation pairs relative to the best strategy p * generates a new training example, and adds the new training sample to the training set; S40, after generating k new examples, retrain the new policy using the new training set Then jump back to step 1; S50, obtaining a final training strategy after N times of circulation Wherein, N is the self-defined training number, m is the self-defined sampling number, the strategy π is an action mapping of filling a specific parameter slot with a specific candidate entity, π * is an optimal strategy confirmed by a person, β i , i = 1, …, N is the self-defined strategy update weight. 2.The method for a document-level event extraction according to claim 1, wherein, The deep neural network model comprises an input layer, two hidden layers and an output layer; The input layer is composed of text data features required by the model, and contains information of the current candidate entity relationship pair; the output layer is a probability value of whether the current candidate relationship pair is included in the final template; and a multi-layer network structure is used to learn more complex interaction information from the input features. 3.The method for a document-level event extraction according to claim 2, wherein, The training process of the deep neural network model comprises: S100, using the current parameter W on each training sample x i make a prediction, and compare the prediction with the true sample label y i compute the loss; S200, updating the model parameters W through a back propagation process, and after the parameter updating is completed, training of the next sample is performed; S300, after all training samples are executed, the current model is evaluated using the validation sample set (V, Z), and the F1 score is calculated; if the current model F1 score is higher than the historical highest score F1 best , then store these parameters as the new optimal model parameters W best , and update the F1 best as the current F1 score; S400, after the end of the k cycles, return the model parameter W providing the historical optimum best .

4. A document-level event extraction system, characterized by, The pre-trained machine learning understanding model, learning search model or deep neural network model, the extraction system comprises: An acquisition module configured to acquire a document to be analyzed; A learning module configured to obtain an event type based on the machine learning understanding model according to the document to be analyzed; A search module configured to obtain a corresponding event parameter type-entity candidate object relationship pair by using the learning search model or deep neural network model according to the document after labeling the event type; A filling module configured to fill a macro event template framework according to the event type, parameter type and entity candidate object, and return an event extraction result to a user, The machine learning understanding model is a gated attention GA reader, and the training process comprises: The event parameter type query and the word embedding of the document word in the training set are obtained through a lookup table, and are processed in a K-layer network. The kth layer network obtains the previous document embedding from the (k-1)th layer network as input. The document word embedding d i and the parameter type query word embedding Q are respectively converted using a bidirectional gated recurrent unit (GRU), and then combined using a gated attention module to obtain the input x of the next layer. i Each GA box represents a gated-attention module that applies attention focusing on the parameter type query to the document representation. After repeating this process for more than K layers, scores are calculated for the words in the document, and a Softmax() function is used to convert the scores into a probability distribution over the words. The final probability distribution is used to select the answer to the query. The learning search model is trained by using a learning search algorithm, comprising: S10, randomly extracting a document d from the labeled document D and generating a sequence of entity, parameter type and candidate relationship pair; S20, execute the currently learned policy p on the document d within the randomly sampled step size t i ; S30, performing the policy π in the step size t from the candidate relation pair corresponding to the document d i After that, the strategy corresponding to the minimum loss of all possible strategies of the remaining candidate relation pairs relative to the best strategy π * A new training example is generated, and the new training sample is added to the training set; S40, after generating k new examples, retrain the new policy using the new training set Then jump back to step 1; S50, obtaining a final training strategy after N times of circulation where N is a self-defined training number, m is a self-defined sampling number, the strategy π is an action mapping of filling a specific parameter slot with a specific candidate entity, π * is an optimal strategy confirmed by a human, β i , i = 1,..., N is a self-defined strategy update weight.

5. A storage medium, characterized by It stores a computer program for chapter-level event extraction, wherein the computer program enables a computer to execute the chapter-level event extraction method of any one of claims 1-3.

6. An electronic device, comprising: Comprise: One or more processors; Memory; And One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the programs comprise a program for executing the chapter-level event extraction method of any one of claims 1-3.

Images