Document-level event extraction method and system based on hypergraph neural network

Abstract

The present invention belongs to the field of document-level event extraction and provides a document-level event extraction method and system based on a hypergraph neural network. The method comprises obtaining document text data; extracting document-level events using a trained document-level joint extraction model based on the document text data; and designing an improved affine graph parser to enable the model to more fully capture grammatical dependency information and longer-range dependency grammatical information for local feature extraction. Furthermore, a hypergraph convolutional network is introduced for global feature extraction, while sentence and word association matrices are constructed. An effective fusion mechanism for local features and the association matrix is ​​designed to enhance the feature extraction capabilities of the hypergraph convolutional network.

Description

Technical Field

[0001] The present invention belongs to the technical field of document-level event extraction, and specifically relates to a document-level event extraction method and system based on a hypergraph neural network. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Event extraction is a key task in text mining and an effective method for constructing knowledge from unstructured text. In recent years, event extraction technology has been widely applied in numerous fields, such as extracting information about proteins, genes, and chromosomes in the biomedical field. However, due to the large number of nested events in documents, event extraction requires considering the information transfer between multiple sentences, making it difficult to effectively extract relevant event information at the document level.

[0004] In the early days, event extraction models were mostly applied at the sentence level. For example, Zhang et al. designed an extraction model based on probabilistic grammar forests, which can fully utilize the dependent grammatical information of sentences. However, compared with sentence-level event extraction, document-level event extraction can better obtain the information transmission between sentences and the latent semantics of the text. Nowadays, document-level event extraction models are mainly divided into two categories: traditional machine learning models and deep learning models. Most traditional machine learning models require a large number of manually designed features, and their model algorithms themselves cannot effectively extract and utilize internal feature information. Subsequently, researchers introduced deep learning models such as CNN, RNN, and LSTM into document-level event extraction tasks. Although deep learning models have shown stronger capabilities in extracting word sequence information and dependent grammatical information, they still cannot effectively obtain the global contextual information between sentences in a document.

[0005] In recent years, graph convolutional networks (GCNs) have attracted the attention of researchers. However, simple graphs in GCNs can only represent pairwise relationships, which means that an edge can only connect two vertices. Therefore, a learning method for hypergraph representation has been proposed, which can effectively model complex relationships between more than two objects. In order to obtain global context information in documents, Zhao et al. used the concept of hypergraph to propose a joint extraction model HANN, which uses GCN and hypergraph aggregation to extract local features and global features respectively. Although HANN enriches global context information with relatively accurate results, the local extraction layer cannot fully learn local features using a single GCN model, and may also lead to feature sparsity problems. In addition, HANN's multiple hypergraph aggregation easily leads to over-smoothing of node features. Summary of the Invention

[0006] In order to solve the above problems, with the emergence of massive texts, manual document-level event extraction will become increasingly laborious. The present invention proposes a document-level event extraction method and system based on a hypergraph neural network. The present invention can automatically extract events and related parameter information, greatly reducing labor costs.

[0007] According to some embodiments, a first solution of the present invention provides a document-level event extraction method based on a hypergraph neural network, which adopts the following technical solutions:

[0008] The document-level event extraction method based on hypergraph neural network includes:

[0009] Get the text data of the document;

[0010] Based on the document text data, document-level event extraction is performed using the trained document-level joint extraction model;

[0011] The document-level event extraction using the trained document-level joint extraction model includes:

[0012] Obtain word vector representations of text data through word vector embedding, and use the BiLSTM network to identify word sequence features represented by word vectors;

[0013] Parse the dependency grammatical information of each sentence in the text data to obtain a soft adjacency matrix;

[0014] Generate local feature information based on learning word sequence features and soft adjacency matrix using graph convolutional network;

[0015] Based on the local feature information, the sentence vector representation is obtained, and the association matrix is ​​constructed based on the word vector representation and the sentence vector representation;

[0016] The association matrix and the local feature matrix are fused using a hypergraph convolutional network to obtain a fused feature matrix. Vertex convolution and hyperedge convolution are performed based on the fused feature matrix to obtain the conceptual representation of words in sentences and realize document-level event extraction.

[0017] Furthermore, the word vector representation of the text data is obtained by word vector embedding, including:

[0018] Each word is converted into a real-valued vector x including word embedding, position embedding and entity type embedding by embedding the following vector representation ki , the real-valued vector x ki is represented as follows:

[0019]

[0020] Among them, x w ,x p and xt They represent word embedding, position embedding, and entity type embedding respectively.

[0021] Furthermore, the use of the BiLSTM network to identify word sequence features represented by word vectors includes:

[0022] The BiLSTM network learns word sequence information by simulating the long-term dependency between memory units and forgetting units, and obtains the encoded sequence representation of words, that is, the word sequence feature.

[0023] Furthermore, the soft adjacency matrix obtained by parsing the dependency grammatical information of each sentence in the text data is obtained using an improved bi-affine graph parser, specifically including:

[0024] Embed the words in each sentence into a vector representation to obtain a word embedding vector representation including word embedding and dependency embedding;

[0025] Based on the word embedding vector representation, grammatical parsing is performed to obtain strong dependency and weak dependency grammatical information and store them in the soft adjacency matrix;

[0026] The grammatical information of the words in the sentence is obtained through self-loop operations to fill the diagonal elements of the soft adjacency matrix and obtain the final soft adjacency matrix.

[0027] Furthermore, based on the local feature information, a sentence vector representation is obtained. Based on the word vector representation and the sentence vector representation, an association matrix is ​​constructed, including:

[0028] Based on the local feature matrix, the maximum pooling layer of the graph convolutional network is used to map the local feature representation to the sentence vector;

[0029] The word vector representation and sentence vector representation are represented as vertices and hyperedges of a hypergraph, and an association matrix is ​​constructed based on the relationship between the feature representations of the same word in different sentences.

[0030] Furthermore, the hypergraph convolutional network is used to fuse the correlation matrix and the local feature matrix to obtain a fused feature matrix, including:

[0031] According to the principle of node consistency, the correlation matrix and the local feature matrix are fused to construct a fused feature matrix;

[0032] The output of the fusion matrix I h as follows:

[0033] I h =join(H,I T )

[0034] Among them, H represents the local features of the graph convolutional network, I TRepresents the transpose of the correlation matrix, join() is a fusion function.

[0035] Furthermore, the vertex convolution and hyperedge convolution are performed based on the fused feature matrix to obtain the conceptual representation of the word in the sentence and realize document-level event extraction, including:

[0036] Based on the fusion feature matrix, the fusion feature information is aggregated into super-edge features through vertex convolution;

[0037] Hyperedge convolution uses an attention mechanism to aggregate hyperedge feature information into corresponding vertices to obtain vertex feature representation;

[0038] Connect the representation of each vertex feature to the corresponding sentence to obtain the conceptual representation of the word in the sentence;

[0039] Based on the conceptual representation of words in sentences, the maximum score is selected through the corresponding scores to extract event types and parameters, thus realizing document-level event extraction.

[0040] According to some embodiments, a second solution of the present invention provides a document-level event extraction system based on a hypergraph neural network, which adopts the following technical solutions:

[0041] A document-level event extraction system based on a hypergraph neural network, including:

[0042] A data acquisition module is configured to acquire text data of a document;

[0043] The document-level event extraction module is configured to extract document-level events based on the text data of the document using the trained document-level joint extraction model;

[0044] The document-level event extraction using the trained document-level joint extraction model includes:

[0045] Obtain word vector representations of text data through word vector embedding, and use the BiLSTM network to identify word sequence features represented by word vectors;

[0046] Parse the dependency grammatical information of each sentence in the text data to obtain a soft adjacency matrix;

[0047] Generate local feature information based on learning word sequence features and soft adjacency matrix using graph convolutional network;

[0048] Based on the local feature information, the sentence vector representation is obtained, and the association matrix is ​​constructed based on the word vector representation and the sentence vector representation;

[0049] The association matrix and the local feature matrix are fused using a hypergraph convolutional network to obtain a fused feature matrix. Vertex convolution and hyperedge convolution are performed based on the fused feature matrix to obtain the conceptual representation of words in sentences and realize document-level event extraction.

[0050] According to some embodiments, a third aspect of the present invention provides a computer-readable storage medium.

[0051] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the document-level event extraction method based on a hypergraph neural network as described in the first aspect above.

[0052] According to some embodiments, a fourth aspect of the present invention provides a computer device.

[0053] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the document-level event extraction method based on a hypergraph neural network as described in the first aspect above are implemented.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] This paper utilizes an improved dual affine graph parser to fully capture the grammatical information of the text. It also utilizes BiLSTM (bidirectional long short-term memory) and GCN (graph convolutional neural network) networks to enhance the ability to extract local contextual features of the text. A hypergraph convolutional network is introduced for global contextual feature extraction, and is improved through an effective fusion mechanism of local features and association matrices. This allows the extraction of hyperedge features, thereby capturing the interaction information between words and sentences. Furthermore, the hypergraph convolution operation can alleviate the problem of oversmoothing node features caused by hypergraph aggregation. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0057] Figure 1 is a flowchart of a document-level event extraction method based on a hypergraph neural network according to an embodiment of the present invention;

[0058] Figure 2 is a structural diagram of the improved dual affine graph parser according to an embodiment of the present invention;

[0059] Figure 3 Schematic diagram of the construction process of the correlation matrix and the fusion matrix according to an embodiment of the present invention;

[0060] Figure 4 2 is a schematic diagram of the hypergraph convolution process described in an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0062] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0063] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0064] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0065] Example 1

[0066] like Figure 1 As shown, this embodiment provides a document-level event extraction method based on a hypergraph neural network. This embodiment uses the method applied to a server as an example for illustration. It is understandable that the method can also be applied to a terminal, and can also be applied to a system including a terminal, a server, and a server, and is implemented through the interaction between the terminal and the server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communications, middleware services, domain name services, security services CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected by wired or wireless communication, which is not limited in this application. In this embodiment, the method includes the following steps:

[0067] Get the text data of the document;

[0068] Based on the document text data, document-level event extraction is performed using the trained document-level joint extraction model;

[0069] The document-level event extraction using the trained document-level joint extraction model includes:

[0070] Obtain word vector representations of text data through word vector embedding, and use the BiLSTM network to identify word sequence features represented by word vectors;

[0071] Parse the dependency grammatical information of each sentence in the text data to obtain a soft adjacency matrix;

[0072] Generate local feature information based on learning word sequence features and soft adjacency matrix using graph convolutional network;

[0073] Based on the local feature information, the sentence vector representation is obtained, and the association matrix is ​​constructed based on the word vector representation and the sentence vector representation;

[0074] The association matrix and the local feature matrix are fused using a hypergraph convolutional network to obtain a fused feature matrix. Vertex convolution and hyperedge convolution are performed based on the fused feature matrix to obtain the conceptual representation of words in sentences and realize document-level event extraction.

[0075] This embodiment proposes a document-level joint event extraction model BGHGCN based on a hypergraph convolutional network. The flowchart of the model algorithm is as follows: Figure 1 shown.

[0076] Data processing occurs only before the embedding layer. The entire process framework diagram shows feature extraction after the embedding layer. The final result is selected by calculating probabilities in the joint extraction layer. In other words, features are extracted before the joint extraction layer. The joint extraction layer calculates feature scores and selects the highest score, thereby extracting event types and parameters.

[0077] The entire model mainly consists of 7 modules: word vector embedding, word sequence information recognition, dependency grammar embedding, GCN extraction of word sequence and dependency grammar information, association matrix embedding, HGCN extraction of local and global context information, and a joint extraction layer of triggers and parameters.

[0078] First, the text data needs to be represented by word vectors through the word embedding layer, and then input into the BiLSTM layer to identify word sequence features.

[0079] At the same time, an improved BiAffine Graph Parser is used to parse the dependency grammatical information of each sentence and output a soft adjacency matrix.

[0080] The word sequence features of the BiLSTM and the soft adjacency matrix generated by the improved affine graph parser are then input into the GCN layer. The GCN layer extracts local feature information through convolution operations and outputs sentence vectors through pooling operations. In addition, the model of this embodiment uses word and sentence representations to construct an association matrix to learn the information interaction between words and sentences.

[0081] The association matrix is ​​constructed after the sentence vector is obtained by the pooling operation of the graph convolutional neural network, and is constructed by sentence vectors and word vectors.

[0082] Graph convolutional neural networks cannot learn the information interaction between words and sentences. The relationship between sentences and words is represented by constructing an association matrix, and the information interaction between the two is learned after the hypergraph neural network learns.

[0083] The local feature matrix and the association matrix are then fed into the hypergraph convolution layer. Through the fusion mechanism and hypergraph convolution operation, the model can extract global features and finer-grained local features. Finally, this embodiment can identify triggers and parameters through the joint extraction layer.

[0084] Specific implementation process:

[0085] Word Embedding Layer

[0086] In this embodiment, it is assumed that a document D consists of L sentences D = {s1, s2, ..., s L}, each sentence S consists of n words, s k ={x k1 ,x k2 ,...,x kn}, where the i-th word in the k-th sentence is represented by x ki Indicates that s k Represents the kth sentence. In the input layer, this embodiment first converts each word into a real-valued vector x including word embedding, position embedding, and entity type embedding by embedding the following vector representation: ki , its real value vector x ki The representation is as follows.

[0087]

[0088] Among them, x w ,x p and x t They represent word embedding, position embedding, and entity type embedding respectively.

[0089] It can be understood that the real-valued vector is the word vector representation, where x kiThe meaning is the vector representation of the i-th word in the k-th sentence in the text. The following word vector x i Represents the vector representation of the i-th word in the sentence. (For BiLSTM, sequence information is only obtained for the current sentence and cannot be obtained across sentences, so it is not written as x ki form.)

[0090] Word sequence recognition layer (BiLstm Layer)

[0091] BGHGCN uses BiLSTM in local feature extraction to extract sequence features and enrich local features. For word vector x i , BiLSTM learns word sequence information by simulating the long-term dependencies between memory units and forgetting units.

[0092] The bidirectional LSTM is defined as follows:

[0093]

[0094]

[0095] Among them, x i represents the vector representation of the i-th word, Represents the word sequence information of the i-1th word. This embodiment then connects the forward LSTM and reverse LSTM representations to obtain a bidirectional feature representation:

[0096] Then the encoding sequence of the word is expressed as:

[0097]

[0098] Among them, hd represents the hidden layer size.

[0099] Improved BiAffine Graph Parser

[0100] In order to fully utilize relevant grammatical information, this embodiment designs an improved bi-affine graph parser (IBGP) to generate a soft adjacency matrix during the grammatical parsing process.

[0101] The improved dual-affine graph parser is used for syntactic parsing. It includes four components: Embedding, BiGRU, MLP, and Decode. Compared to the traditional dual-affine graph parser, this embodiment introduces a BiGRU to replace the BiLSTM in the dual-affine graph parser, abandons the attention mechanism, and adds a self-loop operation to improve the affine graph parser. The BiGRU can handle longer dependency grammatical information, thereby obtaining more complete dependency grammatical information when parsing longer sentences. It also has the advantages of low complexity and fast response time. Furthermore, this embodiment does not use the attention mechanism during syntactic parsing. Therefore, the strong and weak dependency grammatical information obtained through parsing can be better stored simultaneously in a soft adjacency matrix, where elements are represented by probability values ​​and divided into strong and weak dependency information with a threshold of 0.5. Furthermore, the self-loop operation can also obtain grammatical information about the words themselves to fill the diagonal elements of the soft adjacency matrix. Through these improvements, this embodiment can fully utilize the grammatical information of strong and weak dependencies to enrich local features. Generating a soft adjacency matrix through IBGP parsing The process is as follows Figure 2 shown.

[0102] For sentence s i ={x1,x2,...,x n The improved affine graph parser embeds words into vector representations, which include word embeddings and dependency embeddings. BiGRU is then used to obtain long word sequence features, and the vectors output by the BiGRU are re-encoded through the MLP. Next, the soft adjacency matrix is ​​generated by operating on the affine matrix U. The calculation process is as follows:

[0103] {h'1,h'2,...,h' n}=BiGRU({x1,x2,...,x n}) (5)

[0104] {y1,y2,...,y n}=MLP({h'1,h'2,...h' n}) (6)

[0105]

[0106] Where Y={y1,y2,...,y n}, Represents the probability score from the i-th word to the j-th word, which is the element (probability score) in the matrix.

[0107] Most studies on reliance on grammatical information use only the probability score A i,j>0.5 to obtain a hard adjacency matrix, while ignoring the weak dependency information. Instead, in order to combine the strong dependency and weak dependency grammatical information, the soft adjacency matrix is ​​established in the local feature extraction using equation (7)

[0108] Compared to the word sequence features identified by the word sequence recognition layer, the sequence information obtained here is not much different. Due to the different networks used, the BiGRU can learn sequence information for longer sentences. Compared to the BiLSTM, this affine graph parser can, to a certain extent, parse grammatical information over longer distances.

[0109] Graph Neural Network (GCN Layer)

[0110] GCN is a feature extraction neural network based on graph data structure, which can be used to extract relevant grammatical information from the adjacency matrix. Since the number of GCN layers limits the ability to learn graph information, the word sequence features of BiLSTM and the soft adjacency matrix generated by IBGP (Improved Affine Graph Parser) are input into the GCN layer together to enrich the local feature extraction. Before inputting into GCN, the soft adjacency matrix It should be normalized by applying the softmax function: Then, the local feature representation is obtained through multi-layer convolution of GCN. The output of the lth layer in GCN is as follows:

[0111]

[0112] Among them, ReLU is a nonlinear activation function, H l-1 represents the feature output of the l-1 layer, and W is the weight matrix.

[0113] In order to build an association matrix for the Hypergraph Convolutional Network (HGCN) to better learn global information, it is necessary to derive a sentence vector from the GCN. The sentence vector is as follows:

[0114] s k =MAX pool (H l ) (9)

[0115] Among them, MAX pool It is the maximum pooling layer in GCN, which maps word information to a sentence vector s k middle.

[0116] The graph convolutional neural network learns word sequence and soft adjacency matrix feature information to generate a local feature matrix. After generating sentence vectors through the pooling operation of the graph convolutional neural network, the sentence vectors and word vectors can be used to construct the association matrix.

[0117] Incidence Embedding

[0118] For each sentence in the document, the local feature representation of each sentence vector is obtained through the GCN layer. To better learn the relationship between words and sentences, this embodiment uses the concept of hyperedges in the hypergraph to establish an association matrix. Unlike simple graphs, hypergraphs define hyperedges that can connect more than two vertices and use an association matrix to represent the pairing relationship between vertices and hyperedges. Therefore, this embodiment can represent words and sentences in a document as vertices and hyperedges of the hypergraph.

[0119] In order to describe the complex relationship between words and sentences in the document, this embodiment defines a hypergraph structure G = (V, E) to represent the word and sentence information in the document, where V represents a vertex set consisting of N words and E represents a hyperedge set consisting of L sentences. The association matrix of sentences and words reflects the feature representation of the same word in different sentences, so that the information transfer between sentences can be studied. The construction process of the association matrix is ​​as follows: Figure 3 Shown on the left.

[0120] For the sentence vector s k and word vector x i , this embodiment defines that if the hyperedge s k ∈E is connected to vertex v i ∈V, then I k,i =1, otherwise I k,i = 0. Note that the word vector x i Position embedding is not included. The calculation process of the correlation matrix I is as follows:

[0121]

[0122] Hypergraph Convolutional Network layer (HGCN layer)

[0123] To effectively extract global contextual information and alleviate the problem of over-smoothing node features, this embodiment introduces a hypergraph convolutional network for event extraction. In the hypergraph convolution layer of the BGHGCN, hypergraph convolution can obtain information about multi-hop neighbor nodes and hyperedges, thereby more effectively extracting global features.

[0124] In order to obtain node representations with local and global information in the HGCN layer, this embodiment designs a fusion mechanism that fuses the local feature matrix and the correlation matrix. The fusion process is as follows: Figure 3 as shown on the right side of .

[0125] Based on the principle of node consistency, this embodiment fuses the corresponding association matrix with the local feature matrix to construct a new feature matrix for use in the hypergraph convolution layer, as shown in sentence s3. During the fusion process, there is a problem of filling empty coordinates. To address this problem, this embodiment uses 0 to fill empty coordinates for two reasons. First, 0 represents the true connection between sentences and words. Second, 0 can prevent the model from overfitting due to dense matrix features. The output of the fusion matrix I h as follows:

[0126] I h =join(H,I T ) (11)

[0127] Among them, H represents the local features of GCN, I T Represents the transpose of the correlation matrix, join() is a fusion function.

[0128] In the hypergraph convolution layer, the hypergraph convolution process is divided into two parts: vertex convolution and edge convolution. The process of hypergraph convolution is as follows: Figure 4 shown.

[0129] Vertex convolution: The vertex features in the fusion matrix are aggregated into hyperedge features through convolution operations. In order to better learn the feature representation between vertices, this embodiment uses MLP to obtain the vertex transformation matrix T from the vertex features to represent the weight of the vertex features. One-dimensional convolution is used for feature extraction, and multi-hop neighbor node features can also be obtained by increasing the depth of the convolution layer, such as Figure 4 shown.

[0130] The calculation process is as follows:

[0131] T=MLP(I h ) (12)

[0132] x e =conv(T·I h ) (13)

[0133] Among them, x e represents the super edge feature, I h Represents the fusion matrix.

[0134] Hyperedge convolution: Aggregate hyperedge feature information into corresponding vertices to enrich the output node representation, such as Figure 4 As shown in Figure 2. Hyperedge convolution uses an attention mechanism to aggregate hyperedge features and uses MLP to generate hyperedge weight scores. The vertex feature representation is calculated jointly by hyperedge features and weight scores. The calculation process is as follows:

[0135] w=softmax(x e W+b) (14)

[0136] Where W and b represent the weight matrix and bias vector.

[0137]

[0138] Where L represents the size of the adjacent hyperedge set, x v represents the vertex feature, and w represents the weight score of each hyperedge.

[0139] After the hyperedge convolution, each vertex feature needs to be connected to the corresponding sentence to obtain the conceptual representation of the word in the sentence. The calculation is as follows:

[0140]

[0141] in, represents the connection, W and b represent the weight matrix and bias vector respectively.

[0142] The above process only completes the feature extraction. The feature vector that can be used for subsequent recognition is obtained through formula (16). The final extraction and classification is performed in the joint extraction layer.

[0143] Joint extraction layer

[0144] To train the BGHGCN network and reduce error propagation, this embodiment jointly learns the loss function of triggers and parameters and minimizes them.

[0145] Trigger recognition:

[0146] For sentence s k Each word x in k,i , the context vector of the candidate trigger is represented as Among them, K represents the K-th hypergraph convolution layer. Then C k,i Input a fully connected network and predict word x k,i The trigger tag is as follows:

[0147] O k,i =softmax(W t C k,i +b t ) (17)

[0148] Among them, W t and b t are the weight matrix and bias vector in the softmax function respectively.

[0149] Parameter detection:

[0150] In this embodiment, the candidate trigger t and the candidate parameter a are respectively expressed as and Then, this embodiment connects their representations as And input it into a fully connected layer to predict the parameter role as follows:

[0151] O t,a =softmax(W a C t,a +b a ) (18)

[0152] Among them, W a and b a Denote the weight matrix and bias vector respectively. t,a It represents the probability that candidate parameter a plays a role in the event triggered by candidate trigger t.

[0153] Joint loss function:

[0154] This example minimizes the joint negative log-likelihood loss function of triggers and parameters to train BGHGCN. The final loss function is as follows:

[0155]

[0156] Where D represents the training set including N documents, T d and A d Represents trigger set and parameter set respectively. d represents the number of sentences, x di Represents sentence s d The number of words in . f is an indicator function. It outputs a number greater than 1 when its argument is true, and 1 otherwise. β is a hyperparameter that adjusts the joint loss function.

[0157] Example 2

[0158] This embodiment provides a document-level event extraction system based on a hypergraph neural network, including:

[0159] A data acquisition module is configured to acquire text data of a document;

[0160] The document-level event extraction module is configured to extract document-level events based on the text data of the document using the trained document-level joint extraction model;

[0161] The document-level event extraction using the trained document-level joint extraction model includes:

[0162] Obtain word vector representations of text data through word vector embedding, and use the BiLSTM network to identify word sequence features represented by word vectors;

[0163] Parse the dependency grammatical information of each sentence in the text data to obtain a soft adjacency matrix;

[0164] Generate local feature information based on learning word sequence features and soft adjacency matrix using graph convolutional network;

[0165] Based on the local feature information, the sentence vector representation is obtained, and the association matrix is ​​constructed based on the word vector representation and the sentence vector representation;

[0166] The association matrix and the local feature matrix are fused using a hypergraph convolutional network to obtain a fused feature matrix. Vertex convolution and hyperedge convolution are performed based on the fused feature matrix to obtain the conceptual representation of words in sentences and realize document-level event extraction.

[0167] The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the contents disclosed in the above embodiment 1. It should be noted that the above modules as part of the system can be executed in a computer system such as a set of computer executable instructions.

[0168] The descriptions of the various embodiments in the above embodiments have different focuses. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0169] The proposed system can be implemented in other ways. For example, the system embodiment described above is merely illustrative. For example, the above module division is only a logical function division. In actual implementation, other division methods may be used. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not implemented.

[0170] Example 3

[0171] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the document-level event extraction method based on a hypergraph neural network as described in the first embodiment above are implemented.

[0172] Example 4

[0173] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the document-level event extraction method based on a hypergraph neural network as described in the first embodiment above are implemented.

[0174] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0175] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0176] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0177] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0178] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0179] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without any creative work are still within the scope of protection of the present invention.

Claims

1. A document-level event extraction method based on hypergraph neural network, characterized by: include: Get the text data of the document; Based on the document text data, document-level event extraction is performed using the trained document-level joint extraction model; The document-level event extraction using the trained document-level joint extraction model includes: Obtain word vector representations of text data through word vector embedding, and use the BiLSTM network to identify word sequence features represented by word vectors; Parse the dependency grammatical information of each sentence in the text data to obtain a soft adjacency matrix; Use graph convolutional networks to learn word sequence features and soft adjacency matrices to generate local feature information; Based on the local feature information, the sentence vector representation is obtained, and the association matrix is ​​constructed based on the word vector representation and the sentence vector representation; The hypergraph convolutional network is used to fuse the association matrix and the local feature matrix to obtain a fused feature matrix. Vertex convolution and hyperedge convolution are performed based on the fused feature matrix to obtain the conceptual representation of words in sentences and realize document-level event extraction.

2. The document-level event extraction method based on a hypergraph neural network according to claim 1, characterized in that: The word vector representation of the text data obtained by word vector embedding includes: Each word is converted into a real-valued vector x including word embedding, position embedding and entity type embedding by embedding the following vector representation ki , the real-valued vector x ki is represented as follows: Among them, x w ,x p and x t They represent word embedding, position embedding, and entity type embedding respectively.

3. The document-level event extraction method based on a hypergraph neural network according to claim 1, characterized in that: The method of using the BiLSTM network to identify word sequence features represented by word vectors includes: The BiLSTM network learns word sequence information by simulating the long-term dependency between memory units and forgetting units, and obtains the encoded sequence representation of words, that is, the word sequence feature.

4. The document-level event extraction method based on a hypergraph neural network according to claim 1, wherein: The soft adjacency matrix obtained by parsing the dependency grammatical information of each sentence in the text data is obtained by using an improved bi-affine graph parser, specifically including: Embed the words in each sentence into a vector representation to obtain a word embedding vector representation including word embedding and dependency embedding; Based on the word embedding vector representation, grammatical parsing is performed to obtain strong dependency and weak dependency grammatical information and store them in the soft adjacency matrix; The grammatical information of the words in the sentence is obtained through self-loop operations to fill the diagonal elements of the soft adjacency matrix and obtain the final soft adjacency matrix.

5. The document-level event extraction method based on hypergraph neural network according to claim 1, characterized in that: Based on the local feature information, the sentence vector representation is obtained. Based on the word vector representation and the sentence vector representation, the association matrix is ​​constructed, including: Based on the local feature matrix, the maximum pooling layer of the graph convolutional network is used to map the local feature representation to the sentence vector; The word vector representation and sentence vector representation are represented as vertices and hyperedges of a hypergraph, and an association matrix is ​​constructed based on the relationship between the feature representations of the same word in different sentences.

6. The document-level event extraction method based on hypergraph neural network according to claim 1, characterized in that: The hypergraph convolutional network is used to fuse the correlation matrix and the local feature matrix to obtain a fused feature matrix, including: According to the principle of node consistency, the correlation matrix and the local feature matrix are fused to construct a fused feature matrix; The output of the fusion matrix I h as follows: I h =join(H,I T ) Among them, H represents the local features of the graph convolutional network, I T Represents the transpose of the correlation matrix, join() is a fusion function.

7. The document-level event extraction method based on hypergraph neural network according to claim 1, characterized in that: The method performs vertex convolution and hyperedge convolution based on the fused feature matrix to obtain the conceptual representation of words in sentences and implement document-level event extraction, including: Based on the fusion feature matrix, the fusion feature information is aggregated into super-edge features through vertex convolution; Hyperedge convolution uses an attention mechanism to aggregate hyperedge feature information into corresponding vertices to obtain vertex feature representation; Connect the representation of each vertex feature to the corresponding sentence to obtain the conceptual representation of the word in the sentence; Based on the conceptual representation of words in sentences, the maximum score is selected through the corresponding scores to extract event types and parameters, thus realizing document-level event extraction.

8. A document-level event extraction system based on a hypergraph neural network, characterized by: include: A data acquisition module is configured to acquire text data of a document; The document-level event extraction module is configured to extract document-level events based on the text data of the document using the trained document-level joint extraction model; The document-level event extraction using the trained document-level joint extraction model includes: Obtain word vector representations of text data through word vector embedding, and use the BiLSTM network to identify word sequence features represented by word vectors; Parse the dependency grammatical information of each sentence in the text data to obtain a soft adjacency matrix; Use graph convolutional networks to learn word sequence features and soft adjacency matrices to generate local feature information; Based on the local feature information, the sentence vector representation is obtained, and the association matrix is ​​constructed based on the word vector representation and the sentence vector representation; The hypergraph convolutional network is used to fuse the association matrix and the local feature matrix to obtain a fused feature matrix. Vertex convolution and hyperedge convolution are performed based on the fused feature matrix to obtain the conceptual representation of words in sentences and realize document-level event extraction.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the document-level event extraction method based on a hypergraph neural network as described in any one of claims 1 to 7 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the document-level event extraction method based on a hypergraph neural network as described in any one of claims 1 to 7 are implemented.

Images