Heterogeneous graph interaction event extraction method fusing position embedding and attention matrix

Abstract

The application relates to the technical field of natural language processing, and discloses a heterogeneous graph interaction event extraction method fusing position embedding and attention matrix. The method first performs entity recognition on a document, taking the document as a candidate argument; then constructs a heterogeneous graph network fusing position embedding and attention matrix, to globally capture context entity relationship and semantic sequence information; then performs event type detection to detect a specific event type expressed by the document; and finally performs event argument extraction to find suitable arguments for the expressed event from the entities. The method fuses position embedding and attention matrix, can better combine global context semantic relationship and semantic sequence information in the document, does not need to identify event trigger words, reduces the pressure of manual labeling, improves the accuracy of event extraction, and has a wider application scenario. Verification on a financial data set and a contagious disease news data set also proves that the method is superior to other existing methods.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, specifically to a method for extracting heterogeneous graph interactive events by fusing location embedding and attention matrix. Background Technology

[0002] With the rapid development of information technology, the internet has become the primary medium for people to obtain information. The data carried by the internet is growing explosively, leading to a "data explosion" that leaves people lost in a sea of ​​data, unable to distinguish between useful knowledge and irrelevant noise. This makes manual data analysis and processing extremely difficult, thus giving rise to automated information extraction technology. Information extraction is the process of automatically extracting structured information from data such as text, audio, video, and images using computers. Event extraction, as an important research direction in the field of information extraction, has wide application value in business, finance, medicine, public health, and other fields. Its purpose is to extract event information from massive amounts of plain text and present it in a structured manner. The elements that make up an event include: event type, trigger words, arguments, and argument roles.

[0003] In the early stages of event extraction research, researchers used pattern matching algorithms. However, this method was time-consuming and laborious in modeling, and often relied too heavily on knowledge bases or dictionaries. In recent years, with the rapid development of deep learning, researchers have widely applied convolutional neural networks, recurrent neural networks, and Transformers to sentence-level event extraction. While achieving significant success, this approach has overlooked the prevalent issues of multiple events per document and scattered arguments in real-world scenarios. Consequently, document-level event extraction has received widespread attention. However, since documents often mention many entities with complex and intersecting logical relationships, extracting these relationships from complex multi-sentence scenarios requires reading, memorization, and reasoning to discover the relationships between multiple sentences and entities. Therefore, document-level event extraction faces two main challenges: 1) Arguments are scattered across different sentences in the text, requiring the model to comprehensively understand the context across sentences. 2) A document may contain multiple events, and an entity mention may belong to multiple event types, requiring the model to make comprehensive judgments about arguments and extract them by combining global information. Summary of the Invention

[0004] To address the issues of existing technologies only considering the information between words and sentences, neglecting the importance of contextual semantic relationships for document-level event extraction performance, and failing to capture latent semantic order information from multiple levels, thus hindering further improvement in event extraction accuracy, this invention provides a heterogeneous graph interactive event extraction method that integrates positional embeddings and attention matrices. This method first performs entity recognition on the document, identifying entities as candidate arguments. Next, it constructs a heterogeneous graph network integrating positional embeddings and attention matrices. The heterogeneous graph explicitly models the interactions between entity mention nodes and sentence nodes in the document. An entity attention relation matrix is ​​fused to aggregate information, and the adjacency matrix of the heterogeneous graph is used for joint filtering, enhancing or weakening the importance of edges in the graph and emphasizing the association between contextual entities. A graph convolutional neural network integrating positional embeddings is used to model the graph information, enabling the graph convolutional neural network to capture semantic order information. Then, event type detection is performed to detect the specific event type expressed in the document. Finally, event argument extraction is performed, finding suitable arguments from the entities for the expressed event.

[0005] To achieve the above objectives, the present invention employs the following technical solutions:

[0006] This invention provides a method for extracting heterogeneous graph interaction events by fusing location embedding and attention matrix, the method comprising the following steps:

[0007] Step 1: Identify entities from the document and use them as candidate arguments;

[0008] Step 2: Construct a heterogeneous graph network that integrates location embedding and attention matrix to capture contextual entity relationships and semantic order information from a global perspective. The heterogeneous graph G includes entity mention nodes and sentence nodes in the document. Model the interaction between the two. How to enable the heterogeneous graph to capture the order information of words and sentences is the most challenging aspect and the focus of this invention.

[0009] Step 3, Event Type Detection: Detect the specific event type expressed in the document;

[0010] Step 4, event argument extraction, finding suitable arguments for the event being expressed.

[0011] Furthermore, step 1 specifically includes:

[0012] Given document It is a sequence containing N input labels, where w i It is the i-th input, and D is converted using RoFormer. n Encode into a vector set

[0013] {g1,...g N} = RoFormer({w1,...,w N}) (1)

[0014] Entities are extracted from this vector set, and the scope and type of the entities are directly labeled using the BIO pattern. A conditional random field layer is then used to identify the entities. The loss function for this stage is:

[0015]

[0016] In the formula, y n is the golden label for the n-sequence, where N is the number of sequences of input labels.

[0017] Furthermore, step 2 specifically includes:

[0018] Step 2.1: Construct a heterogeneous graph using entity mention nodes and sentence nodes in document D. Sentence nodes are constructed by average pooling the representations of all words contained in a sentence. Simultaneously, connect sentence nodes with other sentence nodes, sentence nodes with entity mention nodes, different entity mention nodes in the same sentence, and the same entity mention nodes in different sentences using four edge construction methods to construct the heterogeneous graph. Graph Convolutional Neural Network (GCN) is then used to model global interactions.

[0019] Step 2.2: Integrate the entity attention relation matrix to aggregate information, and combine it with the heterogeneous graph adjacency matrix for joint filtering to enhance or weaken the importance of edges in the graph and emphasize the association between contextual entities;

[0020] Step 2.3: Use a graph convolutional neural network with fused position embeddings to model heterogeneous graph information, enabling the graph convolutional neural network to capture semantic order information.

[0021] Furthermore, the specific process of step 2.2 is as follows:

[0022] Attention features are obtained from the attention distribution generated by the pre-trained RoFormer language model encoder. An entity attention relation matrix with entity relation enhancement is used to aggregate information and supplement the GCN node features. For document D, the first tag of the entity is taken to represent the entity, given the input sequence. Suppose the RoFormer language model has L layers, each with H attention heads. Each attention head h generates an attention matrix at layer l, and the attention matrix at layer l is denoted as h. Next, we will focus on the attention head dimension d. h The attention representation of this layer is obtained by averaging.

[0023]

[0024] By jointly filtering using the attention relationship matrix and the heterogeneous graph adjacency matrix, the heterogeneous graph is enhanced. When the attention similarity value is greater than or less than a certain threshold, the weights of the edges between nodes are strengthened or weakened, thereby reducing the noise and bias that may naturally occur during the construction of the heterogeneous graph. The specific details are as follows:

[0025]

[0026] In the formula, A is the adjacency matrix enhancement matrix representation of the heterogeneous graph, and α and β define the close relationships between nodes. To enhance the weighting coefficients, To weaken the weighting coefficients, W l，i，j Let be the weights of the edges corresponding to nodes i and j in the l-th layer GCN. Attention similarity values ​​between nodes i and j in layer l.

[0027] Furthermore, the specific process of step 2.3 is as follows:

[0028] To enable graph neural networks to better capture sequential features, this invention learns a set of relative position embeddings. Where x is the size of the position window, d p It is the position embedding dimension, and its size is the same as d. h Equal; before aggregating the representations of adjacent nodes, relative position embeddings are added to the node representations, formalizing the relative linear positions to the current word, as follows:

[0029]

[0030]

[0031] clip(a,x)=max(-x,min(x,a)) (7)

[0032] In the formula, σ represents the ReLU activation function, M represents different types of edges, and N... i This represents the neighboring nodes of the current node in edge class m, where d represents the degree of the current node, and h represents the edge class m. j W represents the feature representation corresponding to the j-th node. m The transformation weight parameter represents the node under edge class m; in equation (6), the clip function returns the embedding index. For the current node i, if node j is within the range of size x, then P is used. j-i Position embedding, otherwise use the position embedding of the window boundary;

[0033] Since heterogeneous graphs contain sentence nodes and entity nodes, after passing through multiple layers of GCN, we obtain a sentence node embedding matrix S and an entity node embedding matrix E with rich semantic relations and semantic order information, as follows:

[0034]

[0035]

[0036] In equation (8), |D| represents the number of sentences in the document, and |δ| represents the number of entities in the document.

[0037] Furthermore, step 3 specifically includes:

[0038] Using the sentence feature matrix S, a multi-head attention mechanism is employed to detect event types.

[0039] M = MultiHead(Q,S,S) (10)

[0040] R = σ(M) T W t (11)

[0041] In the above formula, Q and W t Both represent weight parameters, and T represents the number of possible event types;

[0042] The loss function for this stage is:

[0043]

[0044] In the formula, This represents the label obtained after a multi-label classification task, y. t This represents the gold label of the t sequence.

[0045] Furthermore, step 4, event argument extraction, comprises two parts: event record extraction and argument extraction, specifically:

[0046] Step 4.1: Construct the event record in the form of an ordered tree. First, manually define the order of event roles, i.e., the order of argument extraction. Then, starting from a virtual root node, the tree expands sequentially according to the constrained order of predicted arguments. Since an argument role may have multiple qualified entities in the document, the current node may branch, and each path is uniquely identified. Finally, for the i-th record path consisting of a series of entities, assign a sequence U to its corresponding entity representation. i =[E i1 E i2 The hidden state G is obtained after LSTM encoding. i And add event type embeddings to it, storing them in the global memory module, E i1 This represents the path of the i-th record;

[0047] Step 4.2: The tracker continuously and dynamically tracks the extracted records. When extracting the next argument J of the current path, an entity embedding matrix that fuses specific role information is used. Sentence feature matrix S, current entity path U i And the global memory G, through the Transformer, obtains a new entity feature matrix. Updated sentence node embedding matrix representation Updated entity sequence representation and the updated global memory representation Specifically as follows:

[0048]

[0049]

[0050] In the formula, Role J This represents the embedding of the J-th role information;

[0051] Based on The binary classification task serves as the basis for path expansion, namely, predicting whether the i-th entity is the next argument role of the current record and expanding the path accordingly;

[0052] The loss function for this stage is:

[0053]

[0054] In the formula, N D ξ represents the nodes in the event record tree set, and ξ represents all nodes in the current path. This represents the gold label of the s sequence.

[0055] Compared with the prior art, the present invention has the following advantages:

[0056] The heterogeneous graph interaction event extraction method proposed in this invention, which integrates location embedding and attention matrix, can better combine global contextual semantic relationships and semantic order information in documents. It also eliminates the need to identify event trigger words, reducing the burden of manual annotation and improving the accuracy of event extraction. Furthermore, its application scenarios are broader. Validation on financial datasets and infectious disease news datasets demonstrates that this method outperforms other existing methods. Attached Figure Description

[0057] Figure 1 This is a graph of entity attention relationships.

[0058] Figure 2 Example diagram for initializing relative position encoding.

[0059] Figure 3 This is a diagram illustrating the event record extraction process.

[0060] Figure 4 This is a schematic diagram of the overall architecture of the method of the present invention. Detailed Implementation

[0061] The technical solution of the present invention will be described in detail below with reference to the embodiments and accompanying drawings. It should be noted that those skilled in the art can make several modifications and improvements without departing from the principle of the present invention, and these should also be considered to fall within the protection scope of the present invention.

[0062] A method for extracting heterogeneous graph interaction events by fusing location embedding and attention matrix includes the following steps:

[0063] Step 1: Identify entities from the document and use them as candidate arguments; specifically:

[0064] Given document It is a sequence containing N input labels, where w i It is the i-th input, and D is converted using RoFormer. n Encode into a vector set

[0065] {g1,...g N} = RoFormer({w1,...,w N}) (1)

[0066] Entities are extracted from this vector set, and the scope and type of the entities are directly labeled using the BIO pattern. A conditional random field layer is then used to identify the entities. The loss function for this stage is:

[0067]

[0068] In the formula, y n is the golden label for the n-sequence, where N is the number of sequences of input labels.

[0069] Step 2: Construct a heterogeneous graph network that integrates location embeddings and attention matrices to capture contextual entity relationships and semantic order information globally; specifically:

[0070] Step 2.1: Construct a heterogeneous graph using entity mention nodes and sentence nodes in document D. Sentence nodes are constructed by average pooling the representations of all words contained in a sentence. Simultaneously, connect sentence nodes with other sentence nodes, sentence nodes with entity mention nodes, different entity mention nodes in the same sentence, and the same entity mention nodes in different sentences using four edge construction methods to construct the heterogeneous graph. Graph Convolutional Neural Network (GCN) is then used to model global interactions.

[0071] Step 2.2: Integrate the entity attention relation matrix to aggregate information, and combine it with the heterogeneous graph adjacency matrix for joint filtering to enhance or weaken the importance of edges in the graph, emphasizing the association between contextual entities; the specific process is as follows:

[0072] Attention features are obtained from the attention distribution generated by the pre-trained RoFormer language model encoder. An entity attention relation matrix with entity relation enhancement is used to aggregate information and supplement the GCN node features. For document D, the first tag of the entity is taken to represent the entity, given the input sequence. Suppose the RoFormer language model has L layers, each with H attention heads. Each attention head h generates an attention matrix at layer l, and the attention matrix at layer l is denoted as h. Next, we will focus on the attention head dimension d. h The attention representation of this layer is obtained by averaging.

[0073]

[0074] Figure 1 This demonstrates a real-world example of generating an attention relation matrix, where chunks on the attention map represent the similarity between different entities, with colors gradually deepening as the entities become more related.

[0075] By jointly filtering using the attention relationship matrix and the heterogeneous graph adjacency matrix, the heterogeneous graph is enhanced. When the attention similarity value is greater than or less than a certain threshold, the weights of the edges between nodes are strengthened or weakened, thereby reducing the noise and bias that may naturally occur during the construction of the heterogeneous graph. The specific details are as follows:

[0076]

[0077] In the formula, A is the adjacency matrix enhancement matrix representation of the heterogeneous graph, and α and β define the close relationships between nodes. To enhance the weighting coefficients, To weaken the weighting coefficients, W l，i，j Let be the weights of the edges corresponding to nodes i and j in the l-th layer GCN. Attention similarity values ​​between nodes i and j in layer l.

[0078] Step 2.3 involves using a graph convolutional neural network with fused positional embeddings to model heterogeneous graph information, enabling the graph convolutional neural network to capture semantic order information; the specific process is as follows:

[0079] To enable graph neural networks to better capture sequential features, this invention learns a set of relative position embeddings. Where x is the size of the position window, d p It is the position embedding dimension, and its size is the same as d. h Equal; before aggregating the representations of adjacent nodes, relative position embeddings are added to the node representations, formalizing the relative linear positions to the current word, as follows:

[0080]

[0081]

[0082] clip(a,x)=max(-x,min(x,a)) (7)

[0083] In the formula, σ represents the ReLU activation function, M represents different types of edges, and N... i This represents the neighboring nodes of the current node in edge class m, where d represents the degree of the current node, and h represents the edge class m. j W represents the feature representation corresponding to the j-th node. m The transformation weight parameter represents the node under edge class m; in equation (6), the clip function returns the embedding index. For the current node i, if node j is within the range of size x, then P is used. j-i Position embedding, otherwise use the position embedding of the window boundary;

[0084] Since heterogeneous graphs contain sentence nodes and entity nodes, after passing through multiple layers of GCN, we obtain a sentence node embedding matrix S and an entity node embedding matrix E with rich semantic relations and semantic order information, as follows:

[0085]

[0086]

[0087] In equation (8), |D| represents the number of sentences in the document, and |δ| represents the number of entities in the document.

[0088] Step 3, Event Type Detection: Detects the specific event type expressed in the document; specifically:

[0089] Using the sentence feature matrix S, a multi-head attention mechanism is employed to detect event types.

[0090] M = MultiHead(Q,S,S) (10)

[0091] R = σ(M) T W t (11)

[0092] In the above formula, Q and W t Both represent weight parameters, and T represents the number of possible event types;

[0093] The loss function for this stage is:

[0094]

[0095] In the formula, This represents the label obtained after a multi-label classification task, y. tThis represents the gold label of the t sequence.

[0096] Step 4, event argument extraction, involves finding suitable arguments for the event being represented. This includes two parts: event record extraction and argument extraction, specifically:

[0097] Step 4.1: Construct the event record in the form of an ordered tree. First, manually define the order of event roles, i.e., the order of argument extraction. Then, starting from a virtual root node, the tree expands sequentially according to the constrained order of predicted arguments. Since an argument role may have multiple qualified entities in the document, the current node may branch, and each path is uniquely identified. Finally, for the i-th record path consisting of a series of entities, assign a sequence U to its corresponding entity representation. i =[E i1 E i2 The hidden state G is obtained by LSTM encoding. i And add event type embeddings to it, storing them in the global memory module, E i1 This represents the path of the i-th record;

[0098] Step 4.2: The tracker continuously and dynamically tracks the extracted records. When extracting the next argument J of the current path, an entity embedding matrix that fuses specific role information is used. Sentence feature matrix S, current entity path U i And the global memory G, through the Transformer, obtains a new entity feature matrix. Updated sentence node embedding matrix representation Updated entity sequence representation and the updated global memory representation Specifically as follows:

[0099]

[0100]

[0101] In the formula, Role J This represents the embedding of the J-th role information;

[0102] Based on The binary classification task serves as the basis for path expansion, namely, predicting whether the i-th entity is the next argument role of the current record and expanding the path accordingly;

[0103] The loss function for this stage is:

[0104]

[0105] In the formula, N Dξ represents the nodes in the event record tree set, and ξ represents all nodes in the current path. This represents the gold label of the s sequence.

[0106] To verify the effectiveness and advancement of the technical solution proposed in this invention, experiments were conducted on the following two datasets: 1) A financial dataset published by the Institute for Interdisciplinary Information Sciences, Tsinghua University, containing financial announcements published by listed companies in the Chinese stock market, comprising 32,040 documents. This dataset mainly focuses on five event types: equity freeze (EF), equity repurchase (ER), equity underweight (EU), equity overweight (EO), and equity pledge (EP), containing 32,040 documents. The training set, validation set, and test set are divided into an 8:1:1 ratio according to the given event elements. The number of event types included in this dataset is shown in Table 1. 2) An infectious disease news dataset, which contains infectious disease news data published by various media outlets and websites. This dataset focuses on six infectious disease event types: type A, type B, type C, type D, type E, and type F. It contains 6,644 documents and is divided into training, validation, and test sets in a ratio of 8:1:1 according to the event elements. The number of event types in this dataset is shown in Table 2.

[0107] Table 1. Types of Financial Data Events

[0108]

[0109] Table 2 Types of Infectious Disease News Events

[0110]

[0111] The hardware used in this invention is a computer equipped with Ubuntu 18.04, and the experimental environment includes the GPU parallel computing framework CUDA 11.0 and the deep learning framework torch 1.10.0.

[0112] 1. Evaluation Indicators

[0113] For the event extraction task, precision (P), recall (R), and F1 score are used as metrics to measure the performance of the GPAIT model. The formulas for calculating these three metrics are as follows:

[0114]

[0115]

[0116]

[0117] In this context, TP refers to the number of correctly drawn positive samples (actually positive, predicted positive). FP refers to the number of incorrectly drawn positive samples (actually negative, predicted positive). FN refers to the number of incorrectly drawn negative samples (actually positive, predicted negative). TN refers to the number of correctly drawn negative samples (actually negative, predicted negative).

[0118] 2. Experimental Setup

[0119] In the implementation of this invention, the encoding and decoding modules use 8 layers of RoFormer and 4 layers of Transformer, respectively, with the dimensions of the hidden layer and feedforward layer set to 768 and 1024, respectively. Two layers of GCN are used, and the dropout rate is set to 0.1. The attention matrix dimension d... h With position embedding dimension d p All values ​​are set to 768. In the adjacency matrix formula, μ is set to 1.3, κ to 0.8, α to 0.2, and β to 0.05. Adam is used as the optimizer, batch size is set to 4, and epoch is set to 100. The method proposed in this invention is compared with the following existing named entity recognition methods:

[0120] 1) DCFEE model, which identifies the central sentence related to the event from the whole text, extracts arguments from it, and queries the context to fill in the missing arguments.

[0121] 2) Doc2EDAG model: This method uses document-level context to encode entities, transforming the event table into an entity-based directed acyclic graph. It can transform the originally difficult event table filling problem into a more manageable sequential path expansion task, thereby extracting multiple events.

[0122] 3) GreedyDec model, which validates the necessity of end-to-end modeling by using recognized entity roles, populates only one event table entry, and shares the same architecture as Doc2EDAG.

[0123] 4) DE-PPN model, which uses a multi-granularity decoder to determine the role of elements in parallel, and introduces a matching loss function for training the end-to-end model to enhance global optimization.

[0124] 5) The SCDEE model models sentences as graph structures through entity co-occurrence relationships and uses graph attention networks to strengthen the relationships between sentences, representing events as clauses within them.

[0125] 6) The GIT model constructs a heterogeneous graph interaction neural network that jointly models entities and sentences in a document from a global perspective, thereby capturing event arguments scattered across different sentences.

[0126] 3. Experimental Results

[0127] This invention conducts experiments on the aforementioned financial dataset and infectious disease news dataset, and evaluates the performance of the GPAIT model.

[0128] Table 3 presents the overall performance results of the GPAIT model of this invention. As shown in Table 3, because the GPAIT model combines the entity attention relation matrix to enhance or weaken the heterogeneous graph accordingly, and injects position embeddings into the graph neural network to model graph information, thereby better capturing the contextual semantic relations and semantic order information in the document, the GPAIT model consistently outperforms other models. Compared with the previous state-of-the-art GIT model, the F1 score of the GPAIT model in this paper is improved by 1.2 percentage points on the financial domain dataset and by 3.6 percentage points on the infectious disease news dataset.

[0129] Table 3 Comparison of F1 scores between financial data and infectious disease news data

[0130]

Claims

1. A method for extracting interactive events from heterogeneous graphs by fusing location embedding and attention matrix, characterized in that, The method includes the following steps: Step 1: Identify entities from the document and use them as candidate arguments; Step 2: Construct a heterogeneous graph network that integrates location embedding and attention matrix to capture contextual entity relationships and semantic order information from a global perspective; Step 3, Event Type Detection: Detect the specific event type expressed in the document; Step 4: Event argument extraction, finding suitable arguments for the event being expressed; Step 2 specifically involves: Step 2.1: Construct a heterogeneous graph using entity mention nodes and sentence nodes in document D. Sentence nodes are constructed by average pooling the representations of all words contained in a sentence. Simultaneously, connect sentence nodes with other sentence nodes, sentence nodes with entity mention nodes, different entity mention nodes in the same sentence, and the same entity mention nodes in different sentences using four edge construction methods to construct the heterogeneous graph. Graph Convolutional Neural Network (GCN) is then used to model global interactions. Step 2.2: Integrate the entity attention relation matrix to aggregate information, and combine it with the heterogeneous graph adjacency matrix for joint filtering to enhance or weaken the importance of edges in the graph and emphasize the association between contextual entities; Step 2.3: Use a graph convolutional neural network with fused position embeddings to model heterogeneous graph information, enabling the graph convolutional neural network to capture semantic order information; Step 3 specifically involves: Using sentence feature matrix S Detecting event types using a multi-head attention mechanism: （10）； （11）； In the above formula, , All represent weight parameters. Indicates the number of possible event types; The loss function for this stage is: （12）； In the formula, This refers to the labels obtained after a multi-label classification task. express The golden tag of the sequence; Step 4, event argument extraction, comprises two parts: event record extraction and argument extraction. Specifically: Step 4.1: Construct the event record in the form of an ordered tree. First, manually define the order of event roles, i.e., the order of argument extraction. Then, starting from a virtual root node, the tree expands sequentially according to the constrained order of predicted arguments. Since an argument role may have multiple qualified entities in the document, the current node may branch, and each path is uniquely identified. Finally, for the first event role consisting of a series of entities... Each record path, and its corresponding entity representation sequence The hidden state is obtained after LSTM encoding. And add event type embeddings to it, storing them in the global memory module. Indicates the first i A single record represents a path; Step 4.2: Continuously and dynamically track the extracted records using a tracker, extracting the next argument of the current path. At the same time, an entity embedding matrix that integrates specific role information is used. Sentence feature matrix Current entity path and global memory A new entity feature matrix is ​​obtained through Transformer. Updated sentence node embedding matrix representation Updated entity sequence representation and the updated global memory representation The details are as follows: (13); （14）； In the formula, Indicates the first Embedding of individual character information; Based on The binary classification task serves as the basis for path expansion, namely, predicting the first... If an entity is the next argument role in the current record, expand the path accordingly; The loss function for this stage is: (15); In the formula, This represents a node in the event record tree collection. This represents all nodes in the current path. express The golden tag of the sequence.

2. The method for extracting heterogeneous graph interaction events by fusing location embedding and attention matrix according to claim 1, characterized in that, Step 1 specifically involves: Given document It is a collection A sequence of input tags, where It is the first Input, using RoFormer Encode into a vector set : （1）； Entities are extracted from this vector set, and the scope and type of the entities are directly labeled using the BIO pattern. A conditional random field layer is then used to identify the entities. The loss function for this stage is: （2） ； In the formula, for The golden tag of the sequence.

3. The method for extracting heterogeneous graph interaction events by fusing location embedding and attention matrix according to claim 1, characterized in that, The specific process of step 2.2 is as follows: Attention features are obtained from the attention distribution generated by the pre-trained RoFormer language model encoder. An entity attention relation matrix with entity relation enhancement is used to aggregate information and supplement the GCN node features. For document D, the first tag of the entity is taken to represent the entity, given the input sequence. Assume the RoFormer language model has L layers, each layer has H attention heads, and each attention head... h exist l The layer generates an attention matrix. l The attention matrix of the layer is denoted as Next, we'll focus on the attention head dimension. d h The attention representation of this layer is obtained by averaging. : （3）； By jointly filtering using the attention relationship matrix and the heterogeneous graph adjacency matrix, the heterogeneous graph is enhanced. When the attention similarity value is greater than or less than a certain threshold, the weights of the edges between nodes are strengthened or weakened, thereby reducing the noise and bias that may naturally occur during the construction of the heterogeneous graph. The specific details are as follows: （4）； In the formula, To enhance the matrix representation of the adjacency matrix of heterogeneous graphs, Define the close relationships between nodes. To enhance the weighting coefficients, To weaken the weighting coefficients, For the first Nodes in layer GCN , The weight of the corresponding edge. No. Layer nodes , Attention similarity value between , 4. The method for extracting heterogeneous graph interaction events by fusing location embedding and attention matrix according to claim 1, characterized in that, The specific process of step 2.3 is as follows: To enable graph neural networks to better capture sequential features, this invention learns a set of relative position embeddings. ,in It is the size of the location window. It is the position embedding dimension, and its size is... Equal; before aggregating the representations of adjacent nodes, relative position embeddings are added to the node representations, formalizing the relative linear positions to the current word, as follows: （5）； （6）； （7）； In the formula, Represents the ReLU activation function. Representing different types of edges, Indicates that the current node is Neighbor nodes of the edge class, This represents the degree of the current node. Indicates the first The feature representation corresponding to each node. express Transformation weight parameters of nodes under the edge class; in equation (6) The function returns the embedded index for the current node. If node exist Within the size range, then use Position embedding, otherwise use the position embedding of the window boundary; Since heterogeneous graphs contain sentence nodes and entity nodes, after passing through multiple layers of GCN, a sentence node embedding matrix with rich semantic relations and semantic order information is obtained. and entity node embedding matrix The details are as follows: （8）； （9）； In formula (8) This indicates the number of sentences in the document. This indicates the number of entities in the document.

Images