A document-level event extraction method based on heterogeneous graph interaction and masked multi-head attention mechanism

Abstract

This invention relates to a document-level event extraction method based on heterogeneous graph interaction and masked multi-head attention mechanism. The method includes the following steps: S1: Preprocessing the text to obtain word segments of the target text; S2: Encoding the target text using BERT to obtain word vectors; S3: Inputting the word vectors into a conditional random field for named entity recognition; S4: Constructing a document heterogeneous graph, initializing sentence node vectors and entity mention node vectors, and obtaining adjacency matrices corresponding to different edges; S5: Using different adjacency matrices as mask matrices, learning node features of the heterogeneous graph representation using a multi-head attention mechanism; S6: Performing event detection based on the multi-head attention mechanism; S7: Filling in arguments corresponding to predefined event roles sequentially. This method improves the accuracy of event extraction by modifying the masking mechanism of the Transformer's multi-head attention, enabling multi-head attention to focus on the structural information of the heterogeneous graph more efficiently.

Description

Technical Field

[0001] This invention belongs to the field of information processing technology, particularly the field of document-level event extraction methods, and relates to a document-level event extraction method based on heterogeneous graph interaction and masked multi-head attention mechanism. Background Technology

[0002] With the increasing diversity of data sources and the explosive growth of data volume on the internet, the rational utilization of valuable information contained in massive amounts of data for decision-making has become increasingly important. However, a large amount of internet data is complex or unstructured, with text data being one of the most representative data types. Relying entirely on manual analysis and processing of this massive amount of text data would consume enormous human and material resources. Therefore, how to use computers to quickly and automatically extract structured information from unstructured data has become particularly important and a significant challenge. In recent years, with the rapid development of technologies such as natural language processing and artificial intelligence, the possibility of using computers to extract structured information from unstructured text has been continuously increasing. Therefore, information extraction from text data has become an important research direction in natural language processing, profoundly impacting downstream applications such as knowledge graph construction and content recommendation.

[0003] Event extraction studies the extraction of structured event information from unstructured text. It has enormous application potential in information retrieval, intelligent question answering, and other scenarios, and is widely used in processing large and rapidly growing volumes of documents in fields such as finance, law, healthcare, and government. Compared to general information extraction tasks, event extraction often faces challenges such as more complex text structures and longer text lengths. Furthermore, documents typically contain multiple events, with their elements scattered and mixed throughout the document, making it more difficult than general structured information extraction. This makes it a current research hotspot and a challenging problem in the field of natural language processing.

[0004] Event extraction can be categorized into template-based methods, feature engineering-based methods, and neural network-based methods, depending on the extraction method used. Template-based methods primarily utilize manually written rules for extraction, such as syntax trees or regular expressions. While they perform well in specific domains, they require significant manpower to construct different templates for different domains. Furthermore, portability is poor because new templates need to be reconstructed when transferring from one domain to another. Based on the scope of features used, event extraction methods can also be divided into sentence-level and document-level methods. Generally, sentence-level methods use features extracted from within a sentence, while document-level methods incorporate features extracted across sentences and documents. Sentence-level features are typically universal across all event extraction methods, while document-level features are specialized features for specific tasks. Because document-level event extraction usually has far greater application value than sentence-level extraction, it has become a hot research topic in natural language processing in recent years. Many works rely on feature-based classifiers to distinguish candidate event arguments from ordinary text, achieving better performance.

[0005] In recent years, with the rapid development of deep learning technology, more and more researchers have begun to use various neural network-based methods to solve this task. Document-level event extraction faces two challenges: first, event arguments may be scattered across different sentences; second, a document may contain multiple events. To address these issues, the Doc2EDAG model attempts to encode entities at the document level using three Transformer encoders. However, it not only ignores the interactions between events but also suffers from excessive model size and slow training. Therefore, this invention uses heterogeneous graphs to represent document information. Compared to traditional sequence-related text features, heterogeneous graphs can reflect more spatial features between documents, features between sentences, and features between words, thus exhibiting better performance in practical applications. However, traditional neural networks, such as convolutional neural networks and recurrent neural networks, struggle to handle graph data structures. Graph neural networks (GNNs) suffer from long training times and oversmoothing due to their local propagation characteristics. Therefore, compared to using traditional RNNs and GNNs, document heterogeneous graph information can be extracted using Transformers. Leveraging the global propagation characteristics of Transformers, training speed can be accelerated while ensuring the accuracy of event extraction. Although a typical Transformer treats heterogeneous graphs as fully connected and enables global interactions, it still models sequences and cannot distinguish the edge information of the graph, resulting in low accuracy in event extraction. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide a document-level event extraction method based on heterogeneous graph interaction and masked multi-head attention mechanism. This method improves the accuracy of event extraction by changing the masking mechanism of multi-head attention in Transformer, enabling multi-head attention to focus on the structural information of heterogeneous graphs in a more efficient way.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A document-level event extraction method based on heterogeneous graph interaction and masked multi-head attention mechanism includes the following steps: S1: Preprocess the text to obtain word segments of the target text; S2: Encode the target text using BERT to obtain word vectors of the target text; S3: Input the word vectors into a conditional random field for named entity recognition; S4: Construct a document heterogeneous graph, initialize sentence node vectors and entity mention node vectors, and obtain adjacency matrices corresponding to different edges; S5: Use different adjacency matrices as mask matrices and learn the node features of the heterogeneous graph representation using a multi-head attention mechanism; S6: Perform event detection based on the multi-head attention mechanism; S7: Fill the arguments corresponding to the roles in a predefined event role queue in sequence.

[0009] Furthermore, in this method, event extraction requires extracting the arguments corresponding to event types and event roles from the text. Unstructured text refers to documents containing multiple sets of sentences. Where |D| represents the number of sentences in the document, and s represents the number of sentences. i Composed of a series of words Where w j Sentence s i The j-th word in the text; the event extraction task is decomposed into three sub-tasks: entity extraction, event type detection, and event role filling. Among them, entity extraction aims to extract entities that can be used as argument candidate words. Event type detection aims to identify the event type T contained in a document, while a document may contain multiple event types; event role filling aims to fill the extracted entities into the corresponding event roles.

[0010] Furthermore, in this method, the event extraction requires a named entity recognition task. This task serves two purposes: first, to extract the argument entities to be filled, and second, to perform initial word vector context representation. This method treats the entity recognition task as a BIO (Begin, Inside, Other) sequence labeling task, labeling each character as "BX", "IX", or O. "BX" indicates that the entity to which the character belongs belongs to role X and the character is at the beginning of the entity; "IX" indicates that the entity to which the character belongs belongs to role X and the character is in the middle or at the end of the entity; and O indicates that the character does not belong to any entity.

[0011] By preprocessing the text sentence, the word segmentation sequence of the text sentence is obtained. Using the Bert model to s i Encoded as a vector sequence Using Conditional Random Fields (CRF) for named entity recognition, the loss for named entity recognition is obtained as follows:

[0012]

[0013] Where s represents the word segmentation sequence of the target text sentence, y s Let P(y) be the tag sequence corresponding to the segmented word sequence s. s |s) represents the probability of the most likely label sequence for a word sequence. To obtain the most likely label sequence, this invention employs the Viterbi algorithm to decode the label sequence with the highest probability.

[0014] Furthermore, in this method, documents are mapped to a document heterogeneous graph. By learning the representation of the heterogeneous graph, multiple sets of sentences that fuse document information are used as the feature matrix for event detection. The document heterogeneous graph used contains two types of nodes and five types of edges. Nodes include entity mention nodes and sentence nodes. Edges include same-entity edges, entity mention edges, sentence-entity mention edges, sentence-sentence edges, and fully connected edges. Among them, same-entity edges represent connections between entity mentions of the same entity, entity mention edges represent connections between multiple entity mentions of the same sentence, sentence-entity mention edges represent connections between sentence nodes and entity mentions, sentence-sentence edges represent connections between sentence nodes in sequence, and fully connected edges represent any two nodes connected to each other, including self-connections of nodes.

[0015] Initialize sentence node feature vector h s =max(H i ), where max(·) represents max pooling, H i The hidden layer representation represents the word sequence; for all hidden layer representations of an entity mention, max pooling is used to obtain the entity mention representation.

[0016]

[0017] Where max(·) represents max pooling, N m Represents a set of word vectors for entity mentions; transforms discrete entity mention types into type embeddings. j This leads to the final entity representation. in,

[0018] Reconstruct the sentence node representations and all entity representations of the document into a node feature matrix.

[0019] The original masking mechanism of Transformer was a padding mask, which could lead to deviations in attention score calculation when the input sentence lengths varied. Zero-padding was used to ensure consistent sentence lengths, but these zeros were meaningless and still had corresponding probabilities after Softmax, affecting the global probability. To avoid this effect, Transformer designed a masking mechanism to conceal the zero values. After Softmax, these values ​​had a probability of 0, thus not affecting the prediction of the global probability.

[0020] Inspired by the masking mechanism of Transformer, this method, in order to allow the model to focus on the edge information of the heterogeneous graph, splits the document heterogeneous graph into 5 subgraphs with different edges according to the defined edge types between nodes, obtaining 5 adjacency matrices representing different edges:

[0021] A =<A1,A2,A3,A4,A5>

[0022] A special masked attention mechanism is employed, using different adjacency matrices as mask matrices to enable the model to focus on the edge information of heterogeneous graphs. To extract multiple distinct subgraphs, a multi-head attention mechanism is used, where different attention heads are responsible for focusing on different subgraphs.

[0023] Y=MultiHead(Q,K,V,A)=Concat(head1,…,head5)W O

[0024] in

[0025] in, All are learnable parameter matrices; MASK(·) represents the masking operation; if there is no edge between two nodes in the subgraph, in order to make the output probability of no edge in the subgraph after Softmax 0, the corresponding value in the matrix obtained after masking is assigned to negative infinity;

[0026]

[0027] This step allows different attention heads to focus on different edges of the heterogeneous graph;

[0028] Since a text document may contain multiple events of different types, this method treats event type detection as a multi-label classification task and performs multi-label classification based on a multi-head attention mechanism.

[0029] A = MultiHead(Q,S,S)

[0030] R = σ(A) T W t )

[0031] Among them, Q and W t For trainable parameters, σ(·) represents the sigmoid activation function; MultiHead() represents the standard multi-head attention mechanism; cross-entropy loss is used as the loss function for the event type detection task.

[0032]

[0033] Where T represents the number of event types, y t This indicates the event type label. When the event type is t, y t =1, otherwise, y t =0.

[0034] Furthermore, in this method, for each event type, an event role sequence is defined; then, each event record is converted into arguments in the order of this sequence, where each argument node is either an entity or a special empty node; therefore, the event record filling task is regarded as a path expansion task starting from the event type node; according to the predefined event role queue, the arguments corresponding to the roles are filled in sequence.

[0035] Each path expansion subtask is treated as multiple binary classification tasks, predicting whether each entity can be used as an event role filler; for ease of training, the cross-entropy loss function is used as the loss function for the event record filler task.

[0036]

[0037] Where, N D The set of nodes representing the event logging path. This indicates that argument filling is used for classification task labels. If the t-th entity can be filled as the next argument of node n, then... otherwise

[0038] The weighted sum of the loss functions of the three sub-tasks yields the overall loss function L. all :

[0039] L all =λ1L ner +λ2L det +λ3L rec

[0040] Where λ1, λ2, and λ3 are hyperparameters.

[0041] The beneficial effects of this invention are as follows:

[0042] Currently, document-level event extraction models based on sequence models such as RNNs and LSTMs can only model sequential text features, failing to extract heterogeneous graph features and ignoring inter-sentence interactions, resulting in poor performance. While graph neural networks (GNNs) can extract heterogeneous graph information, their local propagation characteristics make them prone to oversmoothing when dealing with documents containing only a few events. Therefore, to address these issues, this invention provides a document-level event extraction method based on heterogeneous graph interaction and masked multi-head attention mechanisms. This method modifies the masking mechanism of Transformer's multi-head attention, enabling it to focus on the structural information of heterogeneous graphs more efficiently, thus improving the accuracy of event extraction. This invention utilizes a masked multi-head attention mechanism to fully learn the feature representations of heterogeneous graph nodes, enhancing the model's understanding of document-level information and thereby improving the accuracy of event extraction. Furthermore, compared to models using multiple Transformers, such as Doc2EDAG, the approach adopted in this invention converges faster and has a shorter training time.

[0043] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0044] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0045] Figure 1 This is a flowchart illustrating the method described in this invention;

[0046] Figure 2 This is a model framework diagram of the present invention. Detailed Implementation

[0047] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0048] Figure 1The flowchart of the method described in this invention is shown in the figure. The method includes the following steps: S1: Preprocessing the text to obtain word segments of the target text; S2: Encoding the target text using BERT to obtain word vectors of the target text; S3: Inputting the word vectors into a conditional random field for named entity recognition; S4: Constructing a document heterogeneous graph, initializing sentence node vectors and entity mention node vectors, and obtaining adjacency matrices corresponding to different edges; S5: Using different adjacency matrices as mask matrices, learning the node features of the heterogeneous graph representation using a multi-head attention mechanism; S6: Performing event detection based on the multi-head attention mechanism; S7: Filling the arguments corresponding to the roles in a predefined event role queue in sequence. The technical solution of this invention will be described in detail below with reference to specific embodiments.

[0049] Event extraction studies the extraction of structured event information from unstructured text. As shown in Table 1, the document contains three events. The event type is equity increase, the event roles are "holder", "trading volume", "shares held", etc., and the event arguments are the entities corresponding to the event roles, such as "Ni Zhanggen" and "1,084,584 shares".

[0050] Table 1. Examples of documents and results extracted from an event.

[0051]

[0052] Event extraction involves extracting the event type and the arguments corresponding to the event roles from the text. Generally, unstructured text refers to documents containing multiple sets of sentences. Where |D| represents the number of sentences in the document, and s represents the number of sentences. i Composed of a series of words Where w j Sentence s i The j-th word in the context. Generally, event extraction tasks can be decomposed into three sub-tasks: entity extraction, event type detection, and event role filling. Among them, entity extraction aims to extract entities that can be used as argument candidate words. Event type detection aims to identify the event type T contained in a document, and a document may contain multiple event types. Event role population aims to populate the extracted entities with the corresponding event roles.

[0053] Entity recognition:

[0054] Event extraction first requires a named entity recognition task. This task serves two purposes: first, to extract the argument entities to be filled, and second, to generate initial word vector context representations. This invention treats the entity recognition task as a BIO (Begin, Inside, Other) sequence labeling task. Each character is labeled as "BX", "IX", or O. "BX" indicates that the character belongs to entity X and is at the beginning of that entity; "IX" indicates that the character belongs to entity X and is in the middle or at the end of that entity; and O indicates that the character does not belong to any entity.

[0055] This invention obtains the word segmentation sequence of a text sentence by preprocessing the text sentence. Using the Bert model to s i Encoded as a vector sequence Using Conditional Random Fields (CRF) for named entity recognition, the loss for named entity recognition is obtained as follows:

[0056]

[0057] Where s represents the word segmentation sequence of the target text sentence, y s Let P(y) be the tag sequence corresponding to the segmented word sequence s. s |s) represents the probability of the most likely label sequence for a word sequence. To obtain the most likely label sequence, this invention employs the Viterbi algorithm to decode the label sequence with the highest probability.

[0058] Event type detection:

[0059] Event type detection is used to identify the event type to which an event belongs. Since documents often contain multiple events, and the same event may span multiple sentences, the accuracy of event type detection using only sentence information is low. This invention maps documents to a document heterogeneous graph. By learning the representation of the heterogeneous graph, multiple sets of sentences incorporating document information are used as the feature matrix for event detection. The document heterogeneous graph used in this invention contains two types of nodes and five types of edges. Nodes include entity mention nodes and sentence nodes. Edges include same-entity edges, entity mention edges, sentence-entity mention edges, sentence-sentence edges, and fully connected edges. Among them, the same entity edge represents the connection between entity mentions of the same entity. Taking the example in Table 1, the entity mention "Ni Zhanggen" appears in multiple sentences of the document. The constructed heterogeneous graph contains multiple nodes of this entity mention, and each pair of them is connected by the same entity edge; the entity mention edge represents the connection between multiple entity mentions of the same sentence. In the example, the first sentence contains the entity mentions "Ni Zhanggen" and the entity mention "July 17, 2017", which are connected by the entity mention edge; the sentence-entity mention edge represents the connection between sentence nodes and entity mentions; the sentence-sentence edge represents the connection between sentence nodes in chronological order; and the fully connected edge represents any pair of nodes connected, including self-connections of nodes.

[0060] This invention initializes the sentence node feature vector h s =max(H i ), where max(·) represents max pooling, H i This represents the hidden layer representation of a word sequence. For all hidden layer representations of an entity mention, max pooling is used to obtain the entity mention representation.

[0061]

[0062] Where max(·) represents max pooling, N m This represents the set of word vectors associated with the same entity mention. It transforms discrete entity mention types into type embeddings. j This leads to the final entity representation. in,

[0063] Reconstruct the sentence node representations and all entity representations of the document into a node feature matrix.

[0064] The original masking mechanism of Transformer was a padding mask, which could lead to deviations in attention score calculation when the input sentence lengths varied. Zero-padding was used to ensure consistent sentence lengths, but these zeros were meaningless and still had corresponding probabilities after Softmax, affecting the global probability. To avoid this effect, Transformer designed a masking mechanism to conceal the zero values. After Softmax, these values ​​had a probability of 0, thus not affecting the prediction of the global probability.

[0065] Inspired by the masking mechanism of Transformer, this invention allows the model to focus on the edge information of the heterogeneous graph. Based on the defined edge types between nodes in the document heterogeneous graph, the document heterogeneous graph is split into 5 subgraphs with different edges, resulting in 5 adjacency matrices representing different edges.

[0066] A =<A1,A2,A3,A4,A5>

[0067] This invention employs a unique masked attention mechanism, using different adjacency matrices as mask matrices to enable the model to focus on the edge information of heterogeneous graphs. To extract multiple distinct subgraphs, this invention utilizes a multi-head attention mechanism, assigning different attention heads to focus on different subgraphs.

[0068] Y=MultiHead(Q,K,V,A)=Concat(head1,…,head5)W O

[0069] in

[0070] in, All are learnable parameter matrices. MASK(·) represents the masking operation. If there is no edge between two nodes in the subgraph, in order to make the output probability of no edge in the subgraph after Softmax 0, the corresponding value in the matrix obtained after masking is assigned negative infinity.

[0071]

[0072] This step allows different attention heads to focus on different edges of the heterogeneous graph.

[0073] Since a text document may contain multiple events and the event types may be different, this invention treats the event type detection task as a multi-label classification task and performs multi-label classification based on a multi-head attention mechanism.

[0074] A = MultiHead(Q,S,S)

[0075] R = σ(A) T W t )

[0076] Among them, Q and W t These are trainable parameters. σ(·) represents the sigmoid activation function. MultiHead() represents the standard multi-head attention mechanism. Cross-entropy loss is used as the loss function for the event type detection task.

[0077]

[0078] Where T represents the number of event types, y t This indicates the event type label. When the event type is t, y t =1, otherwise, y t =0.

[0079] Event log population:

[0080] For each event type, a sequence of event roles is defined. Then, each event record is converted into arguments and connected in the order of this sequence, where each argument node is either an entity or a special empty node. Therefore, the event record population task can be viewed as a path expansion task starting from the event type node. Arguments corresponding to roles are populated sequentially according to the predefined event role queue.

[0081] Each path expansion subtask is treated as multiple binary classification tasks. Predict whether each entity can be used as a filler for an event role.

[0082] To facilitate training, the cross-entropy loss function is used as the loss function for the event record filling task.

[0083]

[0084] Where, N D The set of nodes representing the event logging path. This indicates that argument filling is used for classification task labels. If the t-th entity can be filled as the next argument of node n, then... otherwise

[0085] The weighted sum of the loss functions of the three sub-tasks yields the overall loss function L. all .

[0086] L all =λ1L ner +λ2L det +λ3L rec

[0087] Where λ1, λ2, and λ3 are hyperparameters.

[0088] The dataset used in this embodiment is the Chinese financial dataset ChFinAnn. This dataset is the largest event extraction dataset to date, containing 32,040 financial announcement documents with 5 different event types, including equity pledge, equity increase, equity reduction, equity repurchase, and equity freeze, and a total of 35 event roles. Among them, 29% of the documents contain multiple events.

[0089] The model was trained using two Nvidia RTX Titan GPUs with the ALBERT-ZN BERT model. The word embedding vector dimension was 768, and the Adam optimizer was used with a learning rate of 5e. -4 Set 100 epochs, loss function L all The parameters for the Chinese Super League are λ1 = λ2 = λ3 = 1.0.

[0090] Table 2 shows a comparison of training times for the models. Due to the large size of the Doc2EDAG model (using three Transformers) and device limitations, the maximum configurable batch size is 2. Under the same memory constraints, the maximum configurable batch size for this invention is 8. Compared to Doc2EDAG, the training time of this invention is only 27.4% of the time required for a single epoch, and only 22.4% of the total time required to reach the optimal F1 score.

[0091] Table 2 Comparison of Model Training Time

[0092] Model Best Epoch Time per Epoch (minutes) Total time (hours) Doc2EDAG 82 201.4 275.2 This invention 67 55.2 61.6

[0093] Table 3 Comparison of macro F1 and micro F1 scores of the model

[0094]

[0095] Table 3 compares the macro F1 score and micro F1 score of the model. Single-record comparisons represent data with only one event record in the document, multi-record comparisons represent data with two or more event records in the document, and "all" represents the overall comparison. This invention uses both macro and micro F1 scores to evaluate the model's accuracy. The experimental results show that this invention outperforms the Doc2EDAG model in both single-record and multi-record scenarios.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications should be covered within the scope of the claims of the present invention.

Claims

1. A document-level event extraction method based on heterogeneous graph interaction and mask multi-head attention mechanism, characterized in that: The method comprises the following steps: S1: preprocessing the text to obtain the word segmentation group of the target text; S2: using Bert to encode the target text to obtain the word vector of the target text; S3: inputting the word vector into the conditional random field for named entity recognition; S4: constructing a document heterogeneous graph, initializing the sentence node vector and the entity mention node vector, and obtaining the adjacency matrix corresponding to different edges; S5: using different adjacency matrices as mask matrices, and learning the node features of the heterogeneous graph representation by using a multi-head attention mechanism; S6: event detection based on the multi-head attention mechanism; S7: filling the argument corresponding to the role according to the pre-defined event role queue; In the present method, event extraction needs to extract event types and argument corresponding to event roles from text, and unstructured text refers to a document containing multiple groups of sentences Where |D| represents the number of sentences in the document, and s i Consists of a series of words Where w j represents the jth word in the sentence s i The event extraction task is divided into three subtasks: entity extraction, event type detection, and event role filling. Entity extraction aims to extract entities as argument candidates Event type detection aims to identify the event types T contained in the document, and a document may contain multiple event types; event role filling aims to fill the extracted entities into the corresponding event roles; In the method, the event extraction needs to perform a named entity recognition task, which is used to extract the argument entity to be filled and to perform an initial word vector context representation; the method regards the entity recognition task as a BIO sequence labeling task, and labels each word as "B-X", "I-X" or O, "B-X" indicates that the word belongs to the role X and the word is at the beginning of the entity, "I-X" indicates that the word belongs to the role X and the word is at the middle or end position of the entity, and O indicates that the word does not belong to any entity; By preprocessing the text sentence, the text sentence segmentation sequence is obtained Using the Bert model, s i Encoding into a vector sequence Using conditional random field for named entity recognition, the named entity recognition loss is obtained wherein s represents a word segmentation sequence of a target text sentence, y s is a label sequence corresponding to the word segmentation sequence s, P(y s |s) represents a probability of the most likely label sequence of the word sequence. In the method, the document is mapped into a document heterogeneous graph, and a plurality of groups of sentences fused with document information are used as a feature matrix for event detection by learning the representation of the heterogeneous graph; the document heterogeneous graph used comprises two types of nodes and five types of edges, wherein the nodes comprise entity mention nodes and sentence nodes, and the edges comprise same entity edges, entity mention edges, sentence-entity mention edges, sentence-sentence edges and full connection edges; the same entity edges represent the connection between entity mentions of the same entity, the entity mention edges represent the connection between a plurality of entity mentions of the same sentence, the sentence-entity mention edges represent the connection between the sentence nodes and the entity mentions, the sentence-sentence edges represent the connection between the sentence nodes in the order, and the full connection edges represent the connection between any two nodes, including self-connection of the nodes; Initialize sentence node feature vector h s = max(H i ), where max(·) denotes max-pooling, H i denotes the hidden layer representation of the word sequence; max-pooling is used to obtain the entity mention representation for all hidden layer representations of an entity mention where max(·) denotes max-pooling, N m denotes the set of word vectors for the entity mention; the discrete entity mention type is converted into a type embedding l j and the final entity representation is obtained as where, reorganizing the sentence node representations of the document with all entity representations into a node feature matrix In the method, according to the types of edges between nodes of the defined document heterogeneous graph, the document heterogeneous graph is divided into five subgraphs with different edges, and five adjacency matrices representing different edges are obtained: A=<A1,A2,A3,A4,A5> A special mask attention mechanism is used to use different adjacency matrices as mask matrices, so that the model focuses on the edge information of the heterogeneous graph; in order to extract a plurality of different subgraphs, a multi-head attention mechanism is used, and different attention heads are responsible for focusing on different subgraphs: Y = MultiHead(Q, K, V, A) = Concat(head1,..., head5) W O wherein wherein, are learnable parameter matrices; MASK(·) represents a mask operation; if there is no edge between two nodes in a subgraph, in order to make the output probability of the subgraph without edges to be 0 after Softmax, the corresponding value in the matrix obtained after the mask is assigned to negative infinity; This step makes different attention heads focus on different edges of the heterogeneous graph; Since a text document may contain multiple events and the event types may be different, the method regards the event type detection task as a multi-label classification task, and performs multi-label classification based on a multi-head attention mechanism: A=MultiHead(Q,S,S) R = σ(A T W t ) where Q and W t are trainable parameters, σ(·) denotes the sigmoid activation function; MultiHead() denotes the standard multi-head attention mechanism; and cross-entropy loss is adopted as the loss function for the event type detection task: where T represents the number of event types, y t represents the event type label, and y t = 1 when the event type is t. Otherwise, y t = 0; In the method, for each event type, an event role sequence is defined; then, each event record is converted into a sequential connection of arguments according to the sequence, wherein each argument node is either an entity or a special empty node; therefore, the event record filling task is regarded as a path expansion task starting from an event type node; and according to a pre-defined event role sequence, arguments corresponding to roles are sequentially filled; Each path expansion sub-task is regarded as a plurality of binary classification tasks, and whether each entity can be filled as an event role is predicted; in order to facilitate training, a cross-entropy loss function is used as a loss function of the event record filling task: where N D a set of nodes representing the event record path, denotes the argument filling classification task label, if the t-th entity can be filled as the next argument of node n, then otherwise The three sub-task loss functions are weighted and summed to obtain the overall loss function L all : L all = λ1L ner + λ2L det + λ3L rec Wherein, λ1, λ2, λ3 are hyperparameters.

Images