A joint event extraction method based on matter logic

Abstract

The application discloses a joint event extraction method based on a matter logic, which comprises the following steps: inputting a sentence into a matter logic relation extraction model to obtain event relation pairs in the sentence; inputting each event in the event relation pairs in the sentence into a joint event extraction model based on a graph attention network to obtain an extraction result corresponding to the sentence; and the extraction result comprises trigger word classification and argument classification. The application improves the accuracy of multiple event extraction in a sentence.

Description

Technical Field

[0001] This invention relates to the field of event extraction technology, and in particular to a joint event extraction method based on event logic. Background Technology

[0002] With the rapid development of the internet and text mining technologies, research on event tasks has received increasing attention from researchers. A text often contains multiple events, which may all revolve around the same theme. Various logical relationships exist between these events, such as chronology, causality, conditionality, and transition. Analyzing these logical relationships allows for a deeper understanding of the evolution and development of events within the text and helps infer the relationships between them. Event extraction is a crucial task that extracts structured event information from unstructured data. It typically includes four sub-tasks: trigger word identification, event type detection, event argument identification, and argument role detection. Research methods for sentence-level event extraction can be categorized into pipeline-based and joint-based approaches. The pipeline approach first identifies the event type and then extracts event arguments; the joint approach learns trigger words and arguments jointly to avoid the impact of trigger word extraction errors on argument extraction. Event extraction is highly useful in many fields. For example, storing the extracted event information in a knowledge base can provide useful information for information retrieval and further facilitate knowledge reasoning.

[0003] Existing technical solutions:

[0004] The patent application "Event Extraction Method, Apparatus, Electronic Device, and Storage Medium" (Patent No.: 202110827424.5) from the Institute of Automation, Chinese Academy of Sciences, provides an event extraction method comprising the following steps: inputting the document to be extracted into an event extraction model, the model including a sentence-level feature extraction layer, a document-level feature extraction layer, a feature decoding layer, and an event prediction layer; the sentence-level feature extraction layer uses a Transformer model to encode each sentence in the document to be extracted, obtaining the corresponding context feature vector and event element representation vector; the document-level feature extraction layer then extracts features, obtaining a document encoding vector and a document event element representation vector; the feature decoding layer analyzes and obtains a role relationship representation vector, an event relationship representation vector, and an event-to-role relationship representation vector; finally, the event prediction layer extracts multiple events and assigns event elements, outputting the prediction result. The drawback of this method is that, in extracting events, it only considers the features of the sentence sequence, but ignores the syntactic features of the sentences, making it difficult for the model to obtain the correlation between multiple events in a sentence, and failing to assign different weights to different features.

[0005] Beijing Minglue Zhaohui Technology Co., Ltd.'s patent application, "Causal Relationship Extraction Method, Apparatus, Electronic Device, and Readable Storage Medium" (Patent No.: 202210308591.3), proposes a method for extracting causal relationships from events. The method includes the following steps: segmenting the text to be extracted to obtain multiple unit words, and tagging each unit word with its corresponding part-of-speech tag; obtaining a preset event rule set, combining the part-of-speech tag with the unit words matching the event sub-rules in the preset event rule set to obtain multiple unit events; obtaining a trained rule model, inputting the unit events into the trained rule model, and obtaining the causal relationship extraction result of the text to be extracted through the output of the trained rule model. The drawback of this method is that it does not consider the dependency relationships between words, nor does it utilize external lexical information, thus failing to fully utilize the semantics of characters. Furthermore, while manually constructed rules may have high accuracy for specific domains, their portability and generalization are weak, making them unsuitable for a wide range of domain data.

[0006] The patent application from Shanxi University, "A Method and Apparatus for Document-Level Event Extraction Based on Multi-Granularity Entity Heterogeneous Graph" (Patent No.: 202210348614.3), proposes a document-level event extraction method, including the following steps: Entity extraction is performed separately using contextual information based on sentences and paragraphs, and the entity sets of the two granularities are fused based on a multi-granularity entity selection strategy to improve the accuracy of entity extraction; a heterogeneous graph incorporating multi-granularity entities is constructed by combining sentences with selected candidate entities, and a graph convolutional network is used to obtain vectorized representations of entities and sentences with document-level context awareness, improving the ability of sentences and entities to perceive events; finally, multi-label classification of event types and event arguments is performed to achieve event detection and argument recognition. The drawback of this method is that it does not construct dependencies between words and does not calculate attention weight information for different features, resulting in important text features not playing a significant role in the output results.

[0007] The disadvantages of existing technologies include:

[0008] 1. In the extraction of events, only the features of the sentence sequence are considered, but the syntactic features of the sentences are ignored, making it difficult for the model to obtain the correlation of multiple events in a sentence, and different weight information is not assigned to different features.

[0009] 2. The method fails to consider the dependencies between words and does not utilize external lexical information, resulting in blurred and difficult-to-determine event boundaries. Furthermore, while the method of manually constructing rules may have high accuracy for specific domains, it lacks portability and generalization, making it unsuitable for a wide range of data across various domains.

[0010] 3. The dependencies between words were not constructed, and attention weight information was not calculated for different features, so that important text features did not play a greater role in the output results. Summary of the Invention

[0011] In view of this, the present invention provides a joint event extraction method based on event logic to solve the above-mentioned technical problems.

[0012] This invention discloses a joint event extraction method based on event logic, which includes:

[0013] Input the sentence into the logical relationship extraction model to obtain the event relationship pairs in the sentence;

[0014] Each event in the event relationship pair in the sentence is input into a joint event extraction model based on a graph attention network to obtain the extraction result corresponding to the sentence; wherein, the extraction result includes trigger word classification and argument classification;

[0015] The logic relationship extraction model includes an encoding layer, a feature extraction layer, and an event relationship recognition layer;

[0016] The process of inputting sentences into a logical relationship extraction model to obtain extraction results corresponding to the sentences includes:

[0017] The sentence is input into the encoding layer to obtain the text feature matrix corresponding to the sentence output by the encoding layer;

[0018] The text feature matrix is ​​input into the feature extraction layer to obtain the global and local feature representation matrices output by the feature extraction layer;

[0019] The global and local feature representation matrices are input into the event relationship recognition layer to identify event relationship pairs in the sentence.

[0020] Further, the step of inputting the sentence into the encoding layer to obtain the text feature matrix corresponding to the sentence output by the encoding layer includes:

[0021] The sentence is input into the embedding layer in the encoding layer to convert each word in the sentence into a word vector, which is then encoded by the BERT model to generate a word vector representation matrix.

[0022] The SoftLexicon method is used to import an external dictionary, which matches the characters in the sentence with the dictionary to obtain the words corresponding to the characters. Based on the position of the character in the word, the word is placed into four word sets: B, M, E, and S. These word sets represent the position of the character at the beginning, middle, end of the word, and when it forms a word on its own.

[0023] After obtaining the four word sets for each character in the sentence, each word set is represented as a fixed-length vector. The word frequency is used as the weight coefficient for each word. The word vector embeddings of all words in each set are weighted and calculated to obtain the vector of the word set for each character.

[0024] By concatenating the vectors of the four word sets corresponding to a character into the BERT word vector corresponding to that character, a new word vector representation matrix X1 is obtained.

[0025] Different weights are assigned to the event trigger word features, event sequence features, and relational connect word features, and then fused to obtain a multi-dimensional feature matrix X2;

[0026] Concatenating X2 and X1 yields the final text feature matrix.

[0027] Further, the step of inputting the text feature matrix into the feature extraction layer to obtain the global and local feature representation matrices output by the feature extraction layer includes:

[0028] The text feature matrix is ​​input into the convolutional layer of the feature extraction layer to obtain the final feature representation D of the multiple convolutional layers. CNN ;D CNN ∈R n×m Each row in the table represents the vocabulary-level features extracted from each word through multiple convolutional layers; m is the number of convolutional kernels, and n is the number of words in the sentence;

[0029] After performing max pooling on all words, we obtain matrix P, where P = [p1, p2, ..., p...]. n ], p i This is the vector obtained after performing max pooling on the i-th word;

[0030] D CNN The self-attention layer of the feature extraction layer is input to obtain the vocabulary-level features D'. CNN D' CNN ∈R n×m ;

[0031] The text feature matrix is ​​input into the bidirectional gated recurrent unit of the feature extraction layer to obtain the output matrix H. GRU The bidirectional gated loop unit consists of a forward GRU and a reverse GRU. If the number of hidden units is set to s, then H... GRU ∈R n ×(2×s) H GRU Each row in the text represents the sentence-level features extracted from each word through a bidirectional gated recurrent unit;

[0032] matrix HGRU Inputting the feature extraction layer into another self-attention layer yields sentence-level features H′. GRU ;

[0033] H′ GRU D' CNN The global attention mechanism layer of the feature extraction layer is input to obtain the output feature matrix G;

[0034] Put matrix P and matrix H l The features are concatenated into the output matrix of the global attention layer to output global and local feature representation matrices. Among them, H l This is the output matrix of the last one-dimensional hidden layer of the bidirectional gated recurrent unit layer.

[0035] Furthermore, the event relationship identification layer adopts a Conditional Random Field (CRF) model;

[0036] Suppose that a label sequence output by the CRF is L = [l1, l2, ..., l n If ], then the total score of a label sequence L is:

[0037]

[0038] Where A is the transition score matrix, Indicates from label l i To tag l i+1 The transition probability, This indicates that the i-th character is in label l i The score below;

[0039] Maximize the correct label sequence L * The log-likelihood estimation function and the objective function of the logical relation extraction model are calculated as follows:

[0040]

[0041] The model's loss function is defined as loss = -log(P(L)). * |Z)), optimize parameters through backpropagation.

[0042] Furthermore, the step of placing all events in the event relationship pairs in the sentence into a set to form a text set, and inputting this text set into a joint event extraction model based on a graph attention network to obtain the extraction result corresponding to the sentence, includes:

[0043] By concatenating the word vector representation matrix X1, the part-of-speech embedding matrix X2, and the entity category embedding matrix X3 together, we can obtain the text feature matrix X.

[0044] Input the text feature matrix X into the Bi-Short Memory (Bi-LSTM) network model to obtain the output matrix H. LSTM ;

[0045] DDParser is used to perform dependency parsing on sentences to obtain a syntactic dependency graph, which is then expanded.

[0046] Using the feature nodes and relation edges of the syntactic dependency graph as input to the m-th layer of an N-order graph attention neural network, the graph attention network processes the features v of each node in the graph. i Aggregate calculations are performed to obtain aggregated features v′ i Finally, the output set V' of the graph attention network layer is obtained, and the number of nodes in set V' is n+k+m.

[0047] The trigger word and argument recognition layer in the joint event extraction model based on graph attention network jointly extracts trigger words and arguments. The BIO annotation method is used for multi-classification tasks. The output matrix O of the previous layer is first input into a fully connected layer, and after passing through the activation function, matrix O' is obtained. Then, a softmax layer is connected to normalize all types of vectors, thereby realizing the classification of event trigger words.

[0048] After obtaining the candidate trigger words, the output matrix O' is used to classify the arguments of the entity list in the sentence; average pooling is then performed on the word vectors contained in the trigger words to obtain the vector representation T of the candidate trigger words. i Then put T i The vector E of each other word j The data is concatenated and fed into a fully connected network, followed by a softmax layer to achieve argument classification.

[0049] Furthermore, before concatenating the word vector representation matrix X1, the part-of-speech embedding matrix X2, and the entity category embedding matrix X3 to obtain the text feature matrix X, the method further includes:

[0050] The word vector representation matrix X1 is generated by encoding the ERNIE model in the joint event extraction model based on the graph attention network.

[0051] The joint event extraction model based on graph attention network performs word segmentation and part-of-speech tagging on the event text in the input sentence, and finally obtains the part-of-speech embedding matrix X2 corresponding to sentence S;

[0052] The text is labeled with entity categories according to the BIO tagging rules. Then, through random initialization and backpropagation optimization, a trained entity category vector is obtained, and the entity category embedding representation corresponding to each word is obtained. Finally, the entity category embedding matrix X3 corresponding to sentence S is obtained.

[0053] Furthermore, the joint event extraction model based on graph attention network performs word segmentation and part-of-speech tagging on the event text in the input sentence, ultimately obtaining the part-of-speech embedding matrix X2 corresponding to sentence S, including:

[0054] The joint event extraction model based on graph attention network performs word segmentation and part-of-speech tagging on the event text in the input sentence. Then, it tags the part of speech of each word according to the BIO tagging rule. The tags include B-pos, I-pos, E-pos, and words composed of a single character are represented by S-pos. pos refers to the part of speech of each word. Then, through random initialization and backpropagation optimization, the trained part-of-speech vector is obtained, and the part-of-speech embedding representation corresponding to each part of speech is obtained. Finally, the part-of-speech embedding matrix X2 corresponding to sentence S is obtained.

[0055] Furthermore, the process of expanding the syntactic dependency graph is as follows:

[0056] Define any two word vector nodes v that have a path between them. i ,v j The shortest path is p ij The edge between any two adjacent word vector nodes is defined as (w m ,w m+1 );w i This refers to the i-th word vector node;

[0057] A BiGRU network is used to fuse the features of all nodes on the shortest path between two word vector nodes. The outputs of the forward and backward GRUs are respectively and Will and By concatenating the features together, we obtain the fused feature vector h, which is the output of BiGRU at time t. Consider these as the surrounding nodes of each of these two nodes;

[0058] Finally, we obtain the expanded syntactic dependency graph G = (V, E), where V is a set of nodes containing three subsets Vi. c V w and V b V c It is a set of n character vector nodes, where n is the sentence length, V w It is the set of k word vector nodes after word segmentation, V b It is the set of surrounding nodes of each word vector node calculated by the shortest path algorithm, and its size is m.

[0059] Furthermore, the graph attention network focuses on the feature v of each node in the syntactic dependency graph. iAggregate calculations are performed to obtain aggregated features v′ i The calculation method is shown in the following formula:

[0060]

[0061] Where K is the number of attention heads, W k It is the weight matrix of the k-th attention head relative to the node. It calculates the weight coefficients for the k-th attention, N. i It is node v i In the syntactic dependency graph, all neighbor nodes v j The set of σ, where σ is a nonlinear activation function;

[0062] After the above calculations, the output set V' of the graph attention network layer is obtained. The number of nodes in set V' is n+k+m. However, in the subsequent classification process, it is not necessary to classify the k word vector nodes and the m surrounding nodes, so they are discarded, leaving only the first n character nodes, which are then converted into a matrix representation O.

[0063] Furthermore, the following formula is used to classify event trigger words:

[0064] O'=tanh(W O O+b O )

[0065]

[0066] in, W is the probability distribution of the trigger word type for the i-th entity. T It is a parameter matrix for classifying event-triggered words. Where n T n is the number of event types. c Indicates the size of the vector dimension;

[0067] Argument classification is achieved using the following formula:

[0068]

[0069] in, W is the probability distribution of the role played by the j-th entity in the event triggered by the i-th candidate trigger word. A It is the parameter matrix for event argument classification. n A It is the number of argument types.

[0070] By employing the above technical solutions, this invention has the following advantages: To address the problem of missing syntactic information in the extracted features, this invention inputs the dependency parsing results into a graph attention network, enabling the learning of syntactic structural features; to address the problem of ambiguous and difficult-to-determine event boundaries, this invention introduces external lexical information and multiple feature vector representations, simultaneously considering two different levels of features: lexical-level features and sentence-level features, thus solving the problems of ambiguous event boundaries and incomplete feature selection; to address the problem of not utilizing important textual features, this invention uses a graph attention network to aggregate features from the syntactic dependency graph during event extraction, and employs multiple attention mechanisms to construct dependencies between words during the extraction of logical relationships, calculating different attention weights for different features, thereby improving the accuracy of extracting multiple events from a sentence. Attached Figure Description

[0071] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in the embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0072] Figure 1 This is a framework diagram of the logic relationship extraction model based on multi-feature fusion according to an embodiment of the present invention;

[0073] Figure 2 This is a structural diagram of the BiGRU according to an embodiment of the present invention;

[0074] Figure 3 This is a framework diagram of the joint event extraction model based on graph attention network according to an embodiment of the present invention;

[0075] Figure 4 This is an example diagram of the syntactic dependency graph in an embodiment of the present invention. Detailed Implementation

[0076] The present invention will be further described in conjunction with the accompanying drawings and embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art should fall within the protection scope of the present invention.

[0077] See Figure 1 This invention provides an embodiment of a joint event extraction method based on event logic, which mainly includes two modules: an event logic relationship extraction model and a joint event extraction model. Firstly, the event logic relationship extraction model based on multi-feature fusion is divided into three layers: an encoding layer, a feature extraction layer, and a relationship recognition layer, as shown in the framework diagram below. Figure 1 As shown.

[0078] 1. Construction of text feature vectors.

[0079] This invention uses BERT as the encoder of the model. To further extract semantic information and abstract features from the text, this invention improves the overall model performance through fine-tuning. Given a sentence S, the embedding layer converts each word into a d-dimensional word vector, which is then encoded by the BERT model to generate a word vector representation matrix X1.

[0080] Next, this invention uses the SoftLexicon method to introduce external word information, matching characters in the sentence with a dictionary to obtain words corresponding to the characters. Based on the position of the characters in these words, these words are placed into four word sets: B, M, E, and S. These word sets represent the position of the character at the beginning, middle, end, and as a word on its own, respectively. If no word in the corresponding word set is found after matching the dictionary, None is used to fill the corresponding word set. The contents of the four word sets are shown in formula (1).

[0081]

[0082] Where L represents the external dictionary, c i This represents the i-th character in the sentence.

[0083] After obtaining the four word sets for each character in the sentence, each word set is represented as a vector of fixed length. During the calculation process, the word frequency is used as the weight coefficient of each word, and the word vector embeddings of all words in each set are weighted and calculated. The vector calculation process of the set S of the i-th character is shown in formula (2).

[0084]

[0085] in, Let S be the vector representation of the generated set, Z be the sum of the frequencies of all words in the four word sets in the dictionary, z(w) be the frequency of word w in the dictionary, and x be the vector representation of the generated set. w This represents the word vector embedding of a word w in set S.

[0086] Finally, the vectors of the four word sets corresponding to a character are concatenated into the BERT word vector corresponding to that character, thus obtaining the new X1 matrix.

[0087] Subsequently, this invention uses vector representations to represent three features related to the logical relationship of events: event trigger word features, event sequence features, and relational connector features. These three multi-dimensional feature vectors are then fused with different weights to obtain matrix X2, the calculation process of which is shown in formula (3).

[0088]

[0089] Where α, β, and γ are the weights assigned to each feature. This indicates that the matrix is ​​concatenated.

[0090] After obtaining the fused multi-dimensional feature matrix X2, it is concatenated with the X1 matrix to obtain the final text feature matrix.

[0091] 2. Lexical and sentence-level feature extraction.

[0092] The input to the convolutional layer is the text feature matrix X. The convolution operation is the product of the convolution kernel and the input matrix. This is achieved by using a convolution kernel W (W∈R) with a window size of w. w×n ( ), slide a window over the input matrix X to obtain each word x i Local context features d i If there are no other words before or after a given word, add zeros to complete the sentence.

[0093] Set x i,i+j The vector x is from i to i+j. i ,x i+1 ,...,x i+j Each word x i Local context features d i The calculation process is shown in formula (4).

[0094] d i =σ(W*x i-w / 2:i+w / 2 +b) (4)

[0095] Where σ(·) is a nonlinear activation function, x i-w / 2:i+w / 2 b represents the input vector within the range of iw / 2 to i+w / 2 in the word window w, and b is the bias parameter.

[0096] For the input matrix X, after passing through the convolutional network, the output feature vector d is as shown in Equation (5), where n represents the length of the sentence.

[0097] d = [d1,...,d i ,...,d n (5)

[0098] To capture local features at different granularities, this invention uses a two-layer CNN structure to obtain important information from the input sequence. The first CNN layer has a single convolutional kernel, which reduces the dimensionality of the output features. The second layer has two CNNs with 2 and 3 convolutional kernels respectively. By inputting the computation results of the first CNN layer into the two CNNs with different kernel sizes in the second layer, more abstract features can be obtained. Finally, the outputs of the two CNNs are concatenated to obtain the final feature representation D of the multi-layer convolutional layers. CNN .

[0099] Let the number of convolution kernels be m, then D CNN ∈R n×m D CNN Each row represents the vocabulary-level features extracted from each word through multiple convolutional layers.

[0100] For each word w, the feature representation d is passed through a multi-layer convolutional network. w Perform max pooling to reduce its dimensionality, obtaining p. i Vector. Then, after performing max pooling on all words, we obtain matrix P, as shown in formula (6).

[0101] P = [p1, p2, ..., p n (6)

[0102] Finally, multiple convolutional layers have two output representations, representing matrix D. CNN As input to the subsequent self-attention layer, matrix P is concatenated into the output matrix of the global attention layer, and then used as input to the event relationship recognition layer.

[0103] The self-attention mechanism assigns different weights to words based on their importance, taking into account the relationships between words globally, and ultimately obtains word-level features D'. CNN D' CNN ∈R n×m .

[0104] Because traditional recurrent neural networks (RNNs) suffer from vanishing and exploding gradients when processing long sentences, this invention introduces gated recurrent units (GRUs). GRUs can effectively solve the vanishing gradient problem and obtain long-term dependencies.

[0105] To obtain the output at time t, the hidden state h at the current time is... t Based on this, perform calculations to obtain y t As shown in formula (7).

[0106] y t =σ(W y h t (7)

[0107] Since the state in a unidirectional GRU is calculated from front to back, without considering the influence of the following text on the state of the preceding text, this invention feeds the input matrix X into a bidirectional gated recurrent unit (BiGRU) to extract the contextual semantic features of the text. BiGRU consists of a forward GRU and a backward GRU, and its principle is as follows... Figure 2 As shown. Output matrix H GRU It is represented as shown in formula (8).

[0108]

[0109] in, This indicates the output of the forward GRU. This indicates the output of the inverted GRU.

[0110] Finally, the BiGRU layer has two output representations, which represent the matrix H. GRU As input to the subsequent self-attention layer; the output matrix H of the last one-dimensional hidden layer is used as input. l This is concatenated into the output matrix of the global attention layer and used as the input to the event relationship recognition layer. Similarly, H... GRU The input is fed into the attention layer, and the sentence-level features H' are finally obtained after calculation. GRU .

[0111] The results from the left and right channels are then input into the global attention mechanism layer for further processing to obtain the final representation features. The attention weights α between the two matrices are utilized. i,j The characteristic matrix H' of the BiGRU channel output GRU Perform weighted operations to obtain the output feature matrix G of the global attention layer, as shown in formula (9).

[0112]

[0113] Finally, matrices P and H are... l Concatenate these matrices into the output matrix of the global attention layer to obtain the input matrix of the event relationship recognition layer.

[0114] 3. Layer for recognizing logical relationships between events.

[0115] Conditional Random Field (CRF) models use transition matrices to consider the correlations and constraints between labels, thereby obtaining a globally optimal label sequence. For an input sentence, the global and local feature representation matrices Z = [z1, z2, ..., z2] are obtained in the previously introduced feature extraction layer. n Let L = [l1, l2, ..., l] be a sequence of tags output by the CRF.n The total score of a label sequence L is calculated as shown in formula (10).

[0116]

[0117] Where A is the transition score matrix, Indicates from label l i To tag l i+1 The transition probability, This indicates that the i-th character is in label l i The score below.

[0118] The purpose of the CRF optimization function is to ensure that the correct label sequence L... * The higher the weight of P(L) among all labels, the better; that is, the higher the weight of P(L). * |Z) should be as large as possible to maximize the correct label sequence L. * The log-likelihood estimation function and the objective function of the model are calculated as shown in formula (11).

[0119]

[0120] Finally, the model's loss function is defined as loss = -log(P(L)). * |Z)),, the parameters are optimized through backpropagation.

[0121] After identifying the event relationships in a sentence, this invention constructs a joint event extraction model based on a graph attention network for each event to extract the event, as shown in the following diagram. Figure 3 As shown.

[0122] 1) Text feature representation and extraction.

[0123] This invention uses the ERNIE pre-trained model for sentence encoding. This model incorporates Chinese corpora from Baidu Encyclopedia, Baidu Tieba, Baidu News, and other sources, integrating a great deal of external knowledge, resulting in better performance on Chinese NLP tasks. The ERNIE model is used to generate a word vector representation matrix X1.

[0124] To obtain more semantic information about Chinese words, this invention performs word segmentation and part-of-speech tagging on the input text. Then, each word is tagged with its part of speech according to the BIO tagging rules. The tags include B-pos, I-pos, E-pos, and words composed of a single character are represented by S-pos. pos refers to the part of speech of each word. Then, through random initialization and backpropagation optimization, a trained part-of-speech vector is obtained, and the part-of-speech embedding representation corresponding to each part of speech is obtained. For sentence S, the part-of-speech embedding matrix X2 is obtained.

[0125] Furthermore, this invention annotates the text with entity categories according to the BIO tagging rules, then optimizes it through random initialization and backpropagation to obtain trained entity category vectors, thereby obtaining the entity category embedding representation corresponding to each word. For sentence S, the entity category embedding matrix X3 is obtained.

[0126] Finally, by concatenating the three embedding methods mentioned above, we obtain the text feature matrix X.

[0127] This invention uses a bidirectional long short-term memory network (Bi-LSTM) model to obtain information from the beginning to the end and from the end to the beginning of a sentence. For a unidirectional LSTM, the calculation process is shown in equations (12) to (14).

[0128]

[0129]

[0130] h t =o t ⊙tanh(c t (14)

[0131] Where W is the parameter matrix to be trained, b is the bias vector, σ represents the sigmoid function, and ≌ represents the dot product operation between vectors.

[0132] The output matrix of Bi-LSTM at time t is represented as follows: in, This indicates the output of the forward GRU. This indicates the output of the inverted GRU.

[0133] 2) Construction of syntactic dependency graph.

[0134] First, DDParser is used to perform dependency parsing on the Chinese sentence to obtain a syntactic dependency tree. The syntactic dependency tree is defined as an undirected graph G = (V, E), where V is a set of nodes containing two subsets V0. c and V w V c It is a set of n characters, where n is the sentence length, V w It is a set of k words after word segmentation. Each word is represented by a pre-trained word vector, and its dimension is the same as that of the character vector.

[0135] For edge set E, the two words w that have a grammatical relation in the analysis results i,j =S(c i ,...,c j ) and w u,v =S(c u ,...,c v) Edge relationships need to be established. In this invention, the first and last character vectors of one of the words are both connected to the vector representation of the other word. Additionally, an opposite edge needs to be added to each syntactic relationship edge. At the same time, adjacent characters also need to be added with adjacent edges, and a self-loop edge needs to be added to all nodes.

[0136] For example, for the sample sentence that has undergone dependency parsing before, Figure 4 shows a partial construction result of its syntactic dependency graph. The relationship between "defeat" and "Medvedev" is a VOB relationship. There are 5 edges connected to the character "fu", namely the relationship between it and the word "defeat", the reverse relationship edge, the self-loop edge, and the two edges connected to adjacent characters.

[0137] 3) Graph Attention Network Algorithm.

[0138] Define that for any two word vector nodes v i , v j with a path between them, the shortest path is p ij . Define the edge between any two adjacent word vector nodes as (w m , w m+1 ). Then the calculation method of p ij is shown in formula (15).

[0139] p ij = [(v i , w1), (w1, w2),..., (w n , v j )] (15)

[0140] where w i refers to the word vector node.

[0141] After obtaining the shortest path between two nodes, this invention still uses the BiGRU network to fuse the features of all nodes on this path. The outputs of the forward and backward GRUs are respectively and Concatenate the two vectors together to obtain the fused feature vector h, which is the output of BiGRU at time t The h vector fuses the features of all other nodes on the shortest path between the two nodes and uses it as the surrounding nodes of these two nodes respectively.

[0142] At this time, after the above calculations, the original syntactic dependency graph G = (V, E) is expanded. Here, V is the set of nodes and now contains three subsets V c , V w and V b . V c is the set of n character vector nodes, where n is the sentence length, and Vw It is the set of k word vector nodes after word segmentation, V b The set of surrounding nodes of each word vector node is calculated by the shortest path algorithm, and its size is m. Therefore, the set of all nodes V of the syntactic dependency graph is obtained by summing these three subsets, as shown in formula (16).

[0143] V = {v1, v2, ..., v} n ,v n+1 ,...,v n+k ,v n+k+1 ,...,v n+k+m} (16)

[0144] Using the feature nodes and relation edges of the syntactic dependency graph as input to the m-th layer of an N-order graph attention neural network, the graph attention network processes the features v of each node in the graph. i Aggregate calculations are performed to obtain aggregated features v i The process is shown in formula (17).

[0145]

[0146] Where K is the number of attention heads, W k It is the weight matrix of the k-th attention head relative to the node. It calculates the weight coefficients for the k-th attention, N. i It is node v i In the syntactic dependency graph, all neighbor nodes v j The set of σ, where σ is a nonlinear activation function.

[0147] Based on the above calculations, this invention obtains the output set V' of the graph attention network layer. The number of nodes in set V' is n+k+m. However, in the subsequent classification process, we do not need to classify the k word vector nodes and the m surrounding nodes, so we discard them and only keep the first n character nodes, which are then converted into a matrix representation O.

[0148] 4) Trigger word and argument classification.

[0149] Finally, trigger words and arguments are jointly extracted in the trigger word and argument recognition layer. The BIO annotation method is used for multi-class classification. The output matrix O of the previous layer is first input into a fully connected layer, and then a softmax layer is connected to normalize the vectors of all types, thereby realizing the classification of event trigger words. The calculation process is shown in formulas (18) and (19).

[0150] O'=tanh(W O O+b O (18)

[0151]

[0152] in, W is the probability distribution of the trigger word type for the i-th entity. T It is a parameter matrix for classifying event-triggered words. Where n T n is the number of event types. c Indicates the size of the vector dimension.

[0153] Through the above calculations, we obtained candidate trigger words. Next, we used the output matrix O' to classify the arguments of the entity list in the sentence. Then, we performed average pooling on the word vectors contained in the trigger words to obtain the vector representation T of the candidate trigger words. i Then we put T i The vector E of each other word j The components are concatenated and input into a fully connected network, and then a softmax layer is added to achieve argument classification. The calculation process is shown in formula (20).

[0154]

[0155] in, W is the probability distribution of the role played by the j-th entity in the event triggered by the i-th candidate trigger word. A It is the parameter matrix for event argument classification. n A It is the number of argument types.

[0156] For ease of understanding, the present invention provides a more specific embodiment:

[0157] In the task of extracting logical relationships, addressing the problem of ambiguous and difficult-to-determine event boundaries, this invention uses the SoftLexicon method to introduce external word information based on the character vectors obtained from the pre-trained model. By constructing four word sets for each word, external word information is integrated into the character vectors, enhancing the semantics of the characters. Simultaneously, to address the issue of insufficient feature selection, this invention considers both lexical-level features of different granularities and sentence-level features containing contextual semantic information. It also uses vector representations for various features such as trigger word features and event sequence features, and constructs dependencies between words using an attention mechanism, calculating different attention weights for different features. This results in more comprehensive feature extraction, contributing to improved performance in extracting logical relationships.

[0158] In event extraction tasks, there are challenges in extracting multiple events from the same sentence, and in feature selection where only sentence sequence features are considered while ignoring sentence syntactic features, leading to difficulties in capturing the relevance of multiple events within a sentence. This invention addresses these issues by converting the syntactic dependency tree from dependency parsing into a syntactic dependency graph using different methods for Chinese and English. This graph is then input into a graph attention network, which learns syntactic structure features. The graph attention network aggregates features from the syntactic dependency graph, finds the shortest path between entities, concatenates the corresponding vectors to extract features, and finally extracts event trigger words, arguments, and their corresponding roles through joint extraction. The proposed method significantly improves the effectiveness of event extraction.

[0159] The Att-GRCNN model is the causal relationship extraction model proposed in this invention. This invention uses three evaluation metrics to assess the performance of various benchmark models and the Att-GRCNN model in causal relationship extraction. The results on the DuEE1.0 and CEC datasets are shown in Table 1. The Att-GRCNN model achieved good results on both the DuEE1.0 and CEC datasets, surpassing the experimental results of the benchmark models in most metrics. Specifically, the Att-GRCNN model improved the F1 score by 2% on the CEC dataset compared to the best-performing GAN-BiGRU-CRF model, and also improved the F1 score by 0.3% on the DuEE1.0 dataset.

[0160] Table 1 Comparative experiments of the Att-GRCNN model in causal relationship extraction.

[0161]

[0162]

[0163] Compared to six benchmark models, the Att-GRCNN model proposed in this invention achieves the best performance in recall and F1 score. The main reasons are as follows: First, the model uses the SoftLexicon method to introduce external word information, enhancing the semantic meaning of characters. Furthermore, this model simultaneously considers word-level features of different granularities and sentence-level features containing contextual semantic information, and uses vector representations for various features such as trigger word features and event sequence features, enriching the semantic features of event relationships. In addition, the model constructs dependencies between words through an attention mechanism, calculating different attention weights for different features, thereby making the extracted features more comprehensive and helping to improve the extraction of logical relationships.

[0164] The DEP-GAT model is the joint event extraction model proposed in this invention. Since event extraction includes two sub-tasks, this invention evaluates the performance of each model in the two tasks of trigger word recognition and classification, and argument recognition and classification. The results on the ACE-2005 English dataset are shown in Tables 2 and 3.

[0165] Table 2 Comparison of DEP-GAT model in trigger word recognition and classification

[0166]

[0167]

[0168] Table 3 Comparative experiments of the DEP-GAT model in argument recognition and classification.

[0169]

[0170] As shown in Tables 2 and 3, the proposed DEP-GAT model performs well in both trigger word recognition and classification, and argument recognition and classification tasks, surpassing the experimental results of the baseline model in most of the above metrics. Specifically, in the trigger word recognition task, the DEP-GAT model improves the F1 score by nearly 2% compared to the best-performing HPNet model, and by 9.2% compared to the classic JRNN model. Furthermore, in the argument recognition and classification tasks, the DEP-GAT model also shows significant improvements over the baseline model, outperforming the best-performing JMEE model by 3.2% and 5.4% in F1 scores for each task, respectively. The improvement in F1 scores is even more significant for the classic CNN-based DMCNN model. Overall, the experimental results in the two tables clearly demonstrate the effectiveness of the proposed DEP-GAT model.

[0171] Experimental results from the DEP-GAT model also demonstrate, to some extent, that joint event extraction methods have advantages over pipeline-based methods. For the StagedMaxent model, based on a two-stage pipeline approach, while its performance in the first stage (trigger word recognition and classification) is acceptable, its recall in the second stage (argument recognition and classification) is low, at only 20.3% and 19.3%, respectively. This indicates that errors from the first stage propagate to the second stage, severely impacting the performance of argument extraction. Furthermore, the experimental results show that the DEP-GAT model achieves a 7.6% improvement in F1 score for trigger word recognition compared to pipeline-based models such as DMCNN, and a 12.5% ​​improvement in F1 score for argument recognition, thus proving the effectiveness of the joint-based DEP-GAT model in event extraction tasks. In addition to simply comparing the F1 scores of multiple models, we can also compare the F1 differences of these models in trigger word classification and argument classification tasks. For example, the pipeline-based StagedMaxent model has an F1 difference of 36.2%, and the DMCNN model has an F1 difference of 15.6%. However, the joint JRNN model has an F1 difference of 13.9%, the JMEE model has an F1 difference of 13.4%, and the proposed DEP-GAT model has an F1 difference of 13.4%. The performance difference of the DEP-GAT model in the two tasks is relatively small, indicating that the joint event extraction method can alleviate some error propagation because this method does not use the results of trigger word extraction when extracting arguments, and errors in trigger word recognition will not be propagated to the argument extraction task.

[0172] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for extracting joint events based on event logic, characterized in that, include: Input the sentence into the logical relationship extraction model to obtain the event relationship pairs in the sentence; Each event in the event relationship pair in the sentence is input into a joint event extraction model based on a graph attention network to obtain the extraction result corresponding to the sentence; wherein, the extraction result includes trigger word classification and argument classification; The logic relationship extraction model includes an encoding layer, a feature extraction layer, and an event relationship recognition layer; The process of inputting sentences into a logical relationship extraction model to obtain extraction results corresponding to the sentences includes: The sentence is input into the encoding layer to obtain the text feature matrix corresponding to the sentence output by the encoding layer; The text feature matrix is ​​input into the feature extraction layer to obtain the global and local feature representation matrices output by the feature extraction layer; The global and local feature representation matrices are input into the event relationship recognition layer to identify event relationship pairs in the sentence; The step of inputting the sentence into the encoding layer to obtain the text feature matrix corresponding to the sentence output by the encoding layer includes: The sentence is input into the embedding layer in the encoding layer to convert each word in the sentence into a word vector, which is then encoded by the BERT model to generate a word vector representation matrix. The SoftLexicon method is used to import an external dictionary, which matches the characters in the sentence with the dictionary to obtain the words corresponding to the characters. Based on the position of the character in the word, the word is placed into four word sets: B, M, E, and S. These word sets represent the position of the character at the beginning, middle, end of the word, and when it forms a word on its own. After obtaining the four word sets for each character in the sentence, each word set is represented as a fixed-length vector. The word frequency is used as the weight coefficient for each word. The word vector embeddings of all words in each set are weighted and calculated to obtain the vector of the word set for each character. By concatenating the vectors of the four word sets corresponding to a character into the BERT word vector corresponding to that character, a new word vector representation matrix is ​​obtained. ; By assigning different weights to the event trigger word features, event sequence features, and relational connective features, a multi-dimensional feature matrix is ​​obtained. ; Will and By concatenating the components, we obtain the final text feature matrix. ; The step of inputting the text feature matrix into the feature extraction layer to obtain the global and local feature representation matrices output by the feature extraction layer includes: The text feature matrix is ​​input into the convolutional layer of the feature extraction layer to obtain the final feature representation of the multiple convolutional layers. ; Each row in the table represents the vocabulary-level features extracted from each word through multiple convolutional layers; m is the number of convolutional kernels, and n is the number of words in the sentence; After performing max pooling on all words, we obtain matrix P. , This is the vector obtained after performing max pooling on the i-th word; Will The self-attention layer of the feature extraction layer is input to obtain vocabulary-level features. , ; The text feature matrix is ​​input into the bidirectional gated recurrent unit of the feature extraction layer to obtain the output matrix. The bidirectional gated loop unit consists of a forward GRU and a reverse GRU. The number of hidden units is set to 's'. ; Each row in the text represents the sentence-level features extracted from each word through a bidirectional gated recurrent unit; matrix Inputting the feature extraction layer into another self-attention layer yields sentence-level features. ; Will , The global attention mechanism layer of the feature extraction layer is input to obtain the output feature matrix G; Put matrix P and matrix The features are concatenated into the output matrix of the global attention layer to output global and local feature representation matrices. ;in, This is the output matrix of the last one-dimensional hidden layer of the bidirectional gated recurrent unit layer; The process involves placing all events in the event relationship pairs within the sentence into a set, forming a text set, and then inputting this text set into a joint event extraction model based on a graph attention network to obtain the extraction results corresponding to the sentence, including: Word vector representation matrix Part-of-speech embedding matrix Entity category embedding matrix By concatenating them together, the text feature matrix can be obtained. ; Text feature matrix Input a Bi-LSTM network model and obtain the output matrix. ; DDParser is used to perform dependency parsing on sentences to obtain a syntactic dependency graph, which is then expanded. Using the feature nodes and relation edges of the syntactic dependency graph as input to the m-th layer of an N-order graph attention neural network, the graph attention network processes the features of each node in the graph. Aggregate calculations are performed to obtain aggregated features. Finally, the output of the graph attention network layer is obtained. gather, The number of nodes in the set is n+k+m; The trigger word and argument recognition layer in the joint event extraction model based on graph attention networks jointly extracts trigger words and arguments. It uses BIO annotation for multi-class classification tasks, inputting the output matrix O from the previous layer into a fully connected layer, and obtaining the matrix after passing through an activation function. Then a softmax layer is added to normalize all types of vectors, thereby achieving event-triggered word classification; After obtaining candidate trigger words, use This output matrix performs argument classification on the entity list in the sentence; it also performs average pooling on the word vectors contained in the trigger word to obtain the vector representation of the candidate trigger word. , and then Vectors of each other word The data is concatenated and fed into a fully connected network, followed by a softmax layer to achieve argument classification. The process of expanding the syntactic dependency graph is as follows: Define any two word vector nodes that have a path between them. The shortest path is The edge between any two adjacent word vector nodes is defined as ; This refers to the i-th word vector node; A BiGRU network is used to fuse the features of all nodes on the shortest path between two word vector nodes. The outputs of the forward and backward GRUs are respectively and ,Will and By concatenating the features together, we obtain the fused feature vector h, which is the output of BiGRU at time t. , and regard them as the surrounding nodes of each of these two nodes; Finally, the expanded syntactic dependency graph is obtained. Where V is a set of nodes, containing three subsets. , and , It is a set of n character vector nodes, where n is the sentence length. It is the set of k word vector nodes after word segmentation. It is the set of surrounding nodes of each word vector node calculated by the shortest path algorithm, and its size is m.

2. The method according to claim 1, characterized in that, The event relationship identification layer adopts a Conditional Random Field (CRF) model. Suppose a tag sequence output by CRF is Then the total score of a label sequence L is: Where A is the transition score matrix, Indicates from the label To tag The transition probability, This indicates that the i-th character is in the label. The score below; Maximize the correct label sequence The log-likelihood estimation function and the objective function of the logical relation extraction model are calculated as follows: The model's loss function is defined as follows: The parameters are optimized through backpropagation.

3. The method according to claim 1, characterized in that, The word vector representation matrix Part-of-speech embedding matrix Entity category embedding matrix By concatenating them together, the text feature matrix can be obtained. Previously, the method also included: The word vector representation matrix is ​​generated by encoding the ERNIE model in the joint event extraction model based on the graph attention network. ; The joint event extraction model based on graph attention network performs word segmentation and part-of-speech tagging on the event text in the input sentence, and finally obtains the part-of-speech embedding matrix corresponding to sentence S. ; The text is labeled with entity categories according to the BIO tagging rules. Then, through random initialization and backpropagation optimization, a trained entity category vector is obtained, yielding the entity category embedding representation for each word. Finally, the entity category embedding matrix for sentence S is obtained. .

4. The method according to claim 3, characterized in that, The joint event extraction model based on graph attention network performs word segmentation and part-of-speech tagging on the event text in the input sentence, and finally obtains the part-of-speech embedding matrix corresponding to sentence S. ,include: The joint event extraction model based on graph attention networks performs word segmentation and part-of-speech tagging on the event text in the input sentence. Then, it tags each word according to the BIO tagging rule, with tags including B-pos, I-pos, E-pos, and S-pos for words consisting of a single character. pos refers to the part-of-speech tag of each word. Then, through random initialization and backpropagation optimization, it obtains trained part-of-speech vectors, acquiring the part-of-speech embedding representation corresponding to each part of speech, and finally obtaining the part-of-speech embedding matrix corresponding to sentence S. .

5. The method according to claim 1, characterized in that, The graph attention network focuses on the features of each node in the syntactic dependency graph. Aggregate calculations are performed to obtain aggregate features. The calculation method is shown in the following formula: Where K is the number of attention heads, It is the weight matrix of the k-th attention head relative to the node. It calculates the weight coefficient of the k-th attention. It is a node All neighbor nodes in the syntactic dependency graph The set, It is a non-linear activation function; Based on the above calculations, the output of the graph attention network layer is obtained. gather, The set contains n+k+m nodes, but in the subsequent classification process, it is not necessary to classify the k word vector nodes and the m surrounding nodes, so they are discarded, leaving only the first n character nodes, which are then converted into a matrix representation O.

6. The method according to claim 1, characterized in that, The following formula is used to classify event trigger words: in, This is the probability distribution of the trigger word type for the i-th entity. It is a parameter matrix for classifying event trigger words. , , ,in It is the number of event types. Indicates the size of the vector dimension; Argument classification is achieved using the following formula: in, It is the probability distribution of the role played by the j-th entity in the event triggered by the i-th candidate trigger word. It is the parameter matrix for event argument classification. , It is the number of argument types.

Images