Document-level event joint extraction method based on role-to-event backtracking graph

Abstract

The present invention proposes a document-level event joint extraction method based on a role-pair event backtracking graph, the method comprising: obtaining a role pair set of event types based on event information; obtaining role pairs corresponding to word pairs based on the role pair set of event types; constructing a target word pair adjacency matrix, and obtaining a new target word pair adjacency matrix based on the role pairs corresponding to the word pairs and the target word pair adjacency matrix; obtaining a gold target word pair adjacency matrix based on the document information of the corpus gold annotation as event information; training the constructed graph-enhanced document-level event joint extraction model using the gold target word pair adjacency matrix to obtain a trained graph-enhanced document-level event joint extraction model; and obtaining events and arguments corresponding to the events using the trained graph-enhanced document-level event joint extraction model. The present invention constructs a graph-enhanced document-level event joint extraction framework, avoids error propagation in the pipeline mode, and reduces the number of model parameters.

Description

Technical Field

[0001] The present invention relates to the technical field of information extraction, and in particular to a document-level event joint extraction method based on a role-to-event backtracking graph. Background Art

[0002] Sentence-level event extraction aims to extract various types of specified time information from sentences. Since it is impossible to handle event arguments across sentences, document-level event extraction has attracted attention, which requires completing subtasks such as entity extraction, event type judgment, and multi-event extraction.

[0003] Most existing studies use the strategy of independently executing the above subtasks in steps to achieve document-level event extraction. This not only causes error propagation, which reduces the extraction effect, but also causes the extraction effect to depend on the specified role order, which is time-consuming to train; and the independent classification of a single event breaks the associated semantics of the entity acting as an argument in multiple events. Summary of the invention

[0004] In view of the above situation, the main purpose of the present invention is to propose a document-level event joint extraction method based on role-to-event backtracking graph to solve the above technical problems.

[0005] The present invention proposes a document-level event joint extraction method based on a role-to-event backtracking graph, the method comprising the following steps:

[0006] Step 1: Based on the event information, all event types and the role sets contained in the event types are obtained, and the role sets are sorted based on the event types to obtain a role pair set of the event types;

[0007] Add label elements and traceback label values ​​to the role collection of the event type;

[0008] Step 2: Combine the words in any two adjacent role arguments in the role pair set of the event type to form a word pair set of the role arguments;

[0009] For each word pair in the word pair set of role arguments, a role pair corresponding to the word pair is formed;

[0010] Construct a target word pair adjacency matrix, and fill the label values ​​and traceback values ​​of the role pairs corresponding to the word pairs in the target word pair adjacency matrix to obtain a target traceback graph, and convert the target traceback graph into a new target word pair adjacency matrix;

[0011] Step 3: Take the document information of the golden annotation of the corpus as event information, execute steps 1 and 2 in sequence to obtain the golden backtracking graph, and convert the golden backtracking graph into the golden target word pair adjacency matrix;

[0012] Step 4: construct a graph-enhanced document-level event joint extraction model. The graph-enhanced document-level event joint extraction model is trained with the golden target word pair adjacency matrix as the approximation target to obtain the trained graph-enhanced document-level event joint extraction model.

[0013] Through the trained graph-enhanced document-level event joint extraction model, a new predicted word pair adjacency matrix is ​​obtained;

[0014] Step 5: Decode the adjacency matrix of the new predicted words to obtain the events and the arguments corresponding to the events.

[0015] Compared with the prior art, the present invention has the following beneficial effects:

[0016] 1. The present invention converts the entity-role correspondence for each event into a word-word correspondence for the entire document by designing a role-role relationship structure, aiming to reveal the correspondence between the roles of two words as arguments in a certain event;

[0017] 2. The present invention proposes a role-pair event backtracking graph with role-role pairs as the forward edge type and backtracking identifiers as the reverse edge type, so as to directly decode the arguments of what role a word plays in which events under which event type;

[0018] 3. The present invention constructs a graph-enhanced document-level event joint extraction framework based on the role-to-event backtracking graph, thereby avoiding error propagation in the pipeline mode and reducing the number of model parameters.

[0019] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description or learned through embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a flowchart of the document-level event joint extraction method based on role-to-event backtracking graph proposed by the present invention. DETAILED DESCRIPTION

[0021] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0022] These and other aspects of the embodiments of the present invention will be apparent with reference to the following description and accompanying drawings. In these descriptions and accompanying drawings, some specific implementations of the embodiments of the present invention are specifically disclosed to represent some ways of implementing the principles of the embodiments of the present invention, but it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0023] See also Figure 1 The embodiment of the present invention proposes a document-level event joint extraction method based on a role-to-event backtracking graph, the method comprising the following steps:

[0024] Step 1: Based on the event information, all event types and the role sets contained in the event types are obtained, and the role sets are sorted based on the event types to obtain a role pair set of the event types;

[0025] Add label elements and traceback label values ​​to the role collection of the event type;

[0026] In step 1, all event types and role sets contained in the event types are obtained based on the event information, and the role sets are sorted based on the event types to obtain a role pair set of the event type. The specific steps are as follows:

[0027] Get all event types, and for each event type, get the role set contained in the event type;

[0028] For each event type, the roles are randomly sorted to obtain a randomly sorted role set. The relationship between the corresponding process is:

[0029] ;

[0030] in, represents a randomly sorted set of roles, All represent roles after event types are randomly sorted. Indicates the number of roles included in the event type;

[0031] In ascending order, two adjacent roles in the randomly sorted role set are combined into role pairs to obtain a role pair set of event types. The corresponding process relationship is:

[0032] ;

[0033] in, A set of role pairs representing event types, — represents a connector, Indicates the roles after the event types are randomly sorted. Indicates a role pair.

[0034] Furthermore, after obtaining the set of role pairs of the event type, a continuous label value is assigned to each element in the set of role pairs of the event type starting from 2, and the label values ​​of all role-role pairs under the same event type are required to be adjacent;

[0035] Add a NULL label element and a backtracking label element to the role pair set of the event type, with label values ​​of 0 and 1 respectively.

[0036] Step 2: Combine the words in any two adjacent role arguments in the role pair set of the event type to form a word pair set of the role arguments;

[0037] For each word pair in the word pair set of role arguments, a role pair corresponding to the word pair is formed;

[0038] Construct a target word pair adjacency matrix, and fill the label values ​​and traceback values ​​of the role pairs corresponding to the word pairs in the target word pair adjacency matrix to obtain a target traceback graph, and convert the target traceback graph into a new target word pair adjacency matrix;

[0039] In step 2, the words in any two adjacent role arguments in the role pair set of the event type are combined to form a word pair set of the role argument. The corresponding process has the following relation:

[0040] ;

[0041] in, The set of word pairs representing role arguments, and Respectively represent the first and words, Represents a document, and Respectively represent words in the current event and words The two adjacent roles played;

[0042] For each word pair in the word pair set of role arguments, a role pair corresponding to the word pair is formed, and the relationship between the corresponding process is:

[0043] ;

[0044] in, Indicates the role pair corresponding to the word pair;

[0045] Construct the target word pair adjacency matrix, and fill in the label value and backtracking value of the role pair corresponding to the word pair in the target word pair adjacency matrix. The relationship between the corresponding process is:

[0046] ;

[0047] in, Represents the words in the adjacency matrix of the target word Corresponding lines and words The value corresponding to the corresponding column, Represents the words in the adjacency matrix of the target word pair Corresponding lines and words The value corresponding to the corresponding column, Indicates the processing of the label value function of the role pair.

[0048] Step 3: Take the document information of the golden annotation of the corpus as event information, execute steps 1 and 2 in sequence to obtain the golden backtracking graph, and convert the golden backtracking graph into the golden target word pair adjacency matrix.

[0049] Step 4: construct a graph-enhanced document-level event joint extraction model. The graph-enhanced document-level event joint extraction model is trained with the golden target word pair adjacency matrix as the approximation target to obtain the trained graph-enhanced document-level event joint extraction model.

[0050] The new predicted word pair adjacency matrix is ​​obtained through the trained graph-enhanced document-level event joint extraction model;

[0051] In step 4, a graph-enhanced document-level event joint extraction model is constructed, wherein the graph-enhanced document-level event joint extraction model consists of an encoding layer and a classification layer;

[0052] In the encoding layer of the graph-enhanced document-level event joint extraction model, the updated embedding representation of the word pairs is obtained based on the words in the document. The specific steps for obtaining the updated embedding representation of the word pairs are as follows:

[0053] The initial embedding representation of the words in the document is concatenated with the initial embedding representation of the word type to obtain the concatenated word embedding representation. The relationship between the corresponding process is:

[0054] ;

[0055] in, represents the concatenated word embedding representation, represents the initial embedding representation of the word, Initialized embedding representation representing the word type, Represents a splicing operation;

[0056] Based on the concatenated word embedding representation, the Bi-LSTM network is used to capture the position information of the words, and the concatenated word embedding representation is updated. The corresponding relationship in the process is:

[0057] ;

[0058] in, represents the concatenated word embedding representation updated by the forward computation, represents the concatenated word embedding representation updated by backward computation, It means that after forward LSTM processing, It means that after backward LSTM processing, Both represent the concatenated word embedding representations. represents the updated concatenated word embedding representation;

[0059] Based on the updated concatenated word embedding representation, the updated embedding representation of the word pair is obtained. The relationship between the corresponding process is:

[0060] ;

[0061] in, represents the updated embedding representation of the word pair, and Both represent the updated concatenated word embedding representation;

[0062] In the classification layer of the graph-enhanced document-level event joint extraction model, the word pairs are classified by calculating the probability distribution of the embedding representation of the word pairs on each label in the role pair set, and then the predicted word pair adjacency matrix is ​​obtained. The corresponding process has the following relationship:

[0063] ;

[0064] in, represents the probability distribution, represents the weight matrix, Indicates The embedding representation of word pairs is Indicates the parameters, represents the bias term, Indicates Parameters The final label output is as follows.

[0065] The graph-enhanced document-level event joint extraction model is trained with the golden target word pair adjacency matrix as the approximation target. During the training process, the weighted cross entropy loss function is used as the objective function, taking into account the possible difference in the number of labels in the training data. The objective function formula is:

[0066] ;

[0067] in, represents the cross entropy loss function, Indicates the number of words contained in the document. represents the size of the adjacency matrix of the gold target word pair, express The weight of the category.

[0068] Step 5: Decode the adjacency matrix of the new predicted word pair to obtain the event and the argument corresponding to the event;

[0069] In step 5, the adjacency matrix of the new predicted word pair is decoded to obtain the event and the argument corresponding to the event. The specific steps are as follows:

[0070] Create a node for each word corresponding to a value greater than 1 in the new predicted word pair adjacency matrix, and establish a node-to-node edge. At the same time, establish a reverse backtracking edge for all nodes with a value of 1 in the new predicted word pair adjacency matrix to obtain a backtracking graph of the role to the event.

[0071] Based on the role-to-event backtracking graph, in the current path, if the subsequent nodes all have reverse backtracking edges pointing to the previous nodes, then the current path corresponds to an event;

[0072] For each edge in the event path, the event type and the event type label value range are determined according to the edge type label value, and the role pair corresponding to the edge is determined according to the edge type label value. The relationship between the corresponding process is:

[0073] ;

[0074] in, Indicates the role pair, Indicates the function processing of the relationship structure after the id is converted into a role. Represents the word pair in the new predicted word pair adjacency matrix Corresponding row words The value of the corresponding column, represents the new predicted word pair adjacency matrix, Represents the value in the adjacency matrix of the new predicted word pair.

[0075] Furthermore, precision, recall, and F1 values ​​are used as evaluation indicators to evaluate the argument extraction results. The calculation formula is:

[0076] ;

[0077] in, Indicates the accuracy, represents the recall rate, represents the number of samples predicted to be positive and whose true value is positive, Represents the number of samples that are predicted to be positive but the true value is negative. Represents the number of samples that are predicted to be negative but the true value is positive. Represents the harmonic mean.

[0078] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0079] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0080] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A document-level event joint extraction method based on role-to-event backtracking graph, characterized in that: The method comprises the following steps: Step 1: Based on the event information, all event types and the role sets contained in the event types are obtained, and the role sets are sorted based on the event types to obtain a role pair set of the event types; Add label elements and traceback label values ​​to the role collection of the event type; Step 2: Combine the words in any two adjacent role arguments in the role pair set of the event type to form a word pair set of the role arguments; For each word pair in the word pair set of role arguments, a role pair corresponding to the word pair is formed; Construct a target word pair adjacency matrix, and fill the label values ​​and traceback values ​​of the role pairs corresponding to the word pairs in the target word pair adjacency matrix to obtain a target traceback graph, and convert the target traceback graph into a new target word pair adjacency matrix; Step 3: Take the document information of the golden annotation of the corpus as event information, execute steps 1 and 2 in sequence to obtain the golden backtracking graph, and convert the golden backtracking graph into the golden target word pair adjacency matrix; Step 4: construct a graph-enhanced document-level event joint extraction model. The graph-enhanced document-level event joint extraction model is trained with the golden target word pair adjacency matrix as the approximation target to obtain the trained graph-enhanced document-level event joint extraction model. Through the trained graph-enhanced document-level event joint extraction model, a new predicted word pair adjacency matrix is ​​obtained; Step 5: Decode the adjacency matrix of the new predicted word pair to obtain the event and the argument corresponding to the event; Among them, in the step 2, a target word pair adjacency matrix is ​​constructed, and the label value and the backtracking value of the role pair corresponding to the word pair are filled in the target word pair adjacency matrix. The relationship between the corresponding process is: ； in, Represents the words in the adjacency matrix of the target word Corresponding lines and words The value corresponding to the corresponding column, Represents the words in the adjacency matrix of the target word pair Corresponding lines and words The value corresponding to the corresponding column, It means that the label value function of the role pair has been processed. Indicates the role pair corresponding to the word pair; In step 5, the adjacency matrix of the new predicted word pair is decoded to obtain the event and the argument corresponding to the event. The specific steps are as follows: Create a node for each word corresponding to a value greater than 1 in the new predicted word pair adjacency matrix, and establish a node-to-node edge. At the same time, establish a reverse backtracking edge for all nodes with a value of 1 in the new predicted word pair adjacency matrix to obtain a backtracking graph of the role to the event. Based on the backtracking graph of the role to the event, in the current path, if the subsequent nodes all have reverse backtracking edges pointing to the previous nodes, then the current path corresponds to an event; For each edge in the event path, the event type and the event type label value range are determined according to the edge type label value, and the role pair corresponding to the edge is determined according to the edge type label value. The relationship between the corresponding process is: ； in, Indicates the role pair, Indicates the function processing of the relationship structure after the id is converted into a role. Represents the word pair in the new predicted word pair adjacency matrix Corresponding row words The value of the corresponding column, represents the new predicted word pair adjacency matrix, Represents the value in the adjacency matrix of the new predicted word pair.

2. The document-level event joint extraction method based on role-to-event backtracking graph according to claim 1 is characterized in that: In step 1, all event types and role sets contained in the event types are obtained based on the event information, and the role sets are sorted based on the event types to obtain a role pair set of the event type. The specific steps are as follows: Get all event types, and for each event type, get the role set contained in the event type; For each event type, the roles are randomly sorted to obtain a randomly sorted role set. The relationship between the corresponding process is: ； in, represents a randomly sorted set of roles, All represent roles after event types are randomly sorted. Indicates the number of roles included in the event type; In ascending order, two adjacent roles in the randomly sorted role set are combined into role pairs to obtain a role pair set of event types. The corresponding process relationship is: ； in, A set of role pairs representing event types, — represents a connector, Indicates the roles after the event types are randomly sorted. Indicates a role pair.

3. The document-level event joint extraction method based on role-to-event backtracking graph according to claim 2 is characterized in that: In step 2, the words in any two adjacent role arguments in the role pair set of the event type are combined to form a word pair set of the role argument. The corresponding process has the following relational expression: ； in, The set of word pairs representing role arguments, and Respectively represent the first and words, Represents a document, and Respectively represent words in the current event and words Two adjacent roles played.

4. The document-level event joint extraction method based on role-to-event backtracking graph according to claim 3 is characterized in that: In step 2, for each word pair in the word pair set of role arguments, a role pair corresponding to the word pair is formed, and the relationship between the corresponding process is: 。 5. The document-level event joint extraction method based on role-to-event backtracking graph according to claim 1 is characterized in that: In the step 4, a graph-enhanced document-level event joint extraction model is constructed, wherein the graph-enhanced document-level event joint extraction model is composed of an encoding layer and a classification layer; In the encoding layer of the graph-enhanced document-level event joint extraction model, the updated embedding representation of the word pairs is obtained based on the words in the document. The specific steps for obtaining the updated embedding representation of the word pairs are as follows: The initial embedding representation of the words in the document is concatenated with the initial embedding representation of the word type to obtain the concatenated word embedding representation. The relationship between the corresponding process is: ； in, represents the concatenated word embedding representation, represents the initial embedding representation of the word, The initial embedding representation representing the word type, Represents a splicing operation; Based on the concatenated word embedding representation, the Bi-LSTM network is used to capture the position information of the words, and the concatenated word embedding representation is updated. The corresponding relationship in the process is: ； in, represents the concatenated word embedding representation updated by the forward computation, represents the concatenated word embedding representation updated by backward computation, It means that after forward LSTM processing, It means that after backward LSTM processing, Both represent the concatenated word embedding representations. represents the updated concatenated word embedding representation; Based on the updated concatenated word embedding representation, the updated embedding representation of the word pair is obtained. The relationship between the corresponding process is: ； in, represents the updated embedding representation of the word pair, and Both represent the updated concatenated word embedding representation.

6. The document-level event joint extraction method based on role-to-event backtracking graph according to claim 5 is characterized in that: In the step 4, a graph-enhanced document-level event joint extraction model is constructed, wherein the graph-enhanced document-level event joint extraction model is composed of an encoding layer and a classification layer, and in the classification layer of the graph-enhanced document-level event joint extraction model; By calculating the probability distribution of the embedding representation of the word pair on each label in the role pair set, the word pairs are classified, and then the predicted word pair adjacency matrix is ​​obtained. The corresponding relationship in the process is: ； in, represents the probability distribution, represents the weight matrix, Indicates The embedding representation of word pairs is Indicates the parameters, represents the bias term, Indicates Parameters The final label output is as follows.

7. The document-level event joint extraction method based on role-to-event backtracking graph according to claim 6 is characterized in that: In step 4, the graph-enhanced document-level event joint extraction model is trained with the golden target word pair adjacency matrix as the approximation target; Among them, the weighted cross entropy loss function is used as the objective function during the training process, and the objective function formula is: ； in, represents the cross entropy loss function, Indicates the number of words contained in the document. represents the size of the adjacency matrix of the gold target word pair, express The weight of the category.

Images