Document event extraction method and device, terminal equipment and storage medium

Abstract

The invention discloses a document event extraction method and device, terminal equipment and a storage medium, and belongs to the field of natural language process.The method comprises the steps that a first graph is generated according to a to-be-processed document, the first graph comprises a plurality of entity nodes, a plurality of sentence nodes and a plurality of event nodes, the event node is respectively and fully connected with other event nodes, entity nodes and sentence nodes, and each sentence node corresponds to a plurality of entity nodes; on the basis of the first atlas, according to the first embedded representation vectors of the entity nodes, constructing a connection relationship between the entity nodes, and obtaining a second atlas; on the basis of the second atlas, according to sentence nodes corresponding to the entity nodes, entity sentence nodes corresponding to the entity nodes are added, and a third atlas is obtained; and through a preset graph neural network, encoding and decoding each event node in the third graph to obtain an event corresponding to each event node. Through the method and the device, the problem of low event extraction accuracy of the document can be solved.

Description

Technical Field

[0001] This application relates to the field of natural language processing, and in particular to a method, apparatus, terminal device, and storage medium for extracting document events. Background Technology

[0002] Event extraction aims to extract structured information from unstructured text, including event type and argument entities such as time, location, and participants. It provides a knowledge base for extracting key information and quickly associating and retrieving related information in business intelligence such as e-commerce, stock trading, and technological advancements, and plays a crucial role in applications such as intelligent question answering, public opinion analysis, and recommendation systems.

[0003] Graph neural networks have gained widespread attention in recent years due to their ability to capture rich semantic information in documents, thus becoming a mainstream method for event extraction. In this method, graph construction is the core step affecting the final event extraction performance. Existing techniques primarily focus on the local dense connections between events and entities, as well as sentences, during graph construction. This locality ignores the rich prior semantic relationships between entity mentions and fails to fully model the direct connections between sentences and events, resulting in the loss of some key semantic information during the modeling process. Furthermore, while this indiscriminate dense connection aims to cover all possible relationships, it inevitably introduces a large amount of irrelevant or noisy information, interfering with the model's identification and learning of truly relevant information, thereby limiting the improvement of event extraction accuracy.

[0004] Therefore, improving the accuracy of event extraction from documents is a technical problem that needs to be solved. Summary of the Invention

[0005] This application provides a document event extraction method, apparatus, terminal device, and storage medium, which can solve the problem of how to improve the accuracy of document event extraction in the prior art.

[0006] One embodiment of this application provides a document event extraction method, including:

[0007] A first graph is generated based on the document to be processed; the first graph includes several entity nodes, several sentence nodes, and several event nodes; each entity node and each sentence node has its own corresponding embedding representation vector; each event node is fully connected to each of the other event nodes, entity nodes, and sentence nodes; each sentence node corresponds to several entity nodes.

[0008] Based on the first graph, the connection relationship between the entity nodes is constructed according to the first embedding representation vector corresponding to each entity node to obtain the second graph;

[0009] Based on the second graph, according to the sentence nodes corresponding to each entity node, add entity sentence nodes corresponding to each entity node, and fully connect each entity sentence node with each event node to obtain the third graph;

[0010] By using a preset graph neural network, each event node in the third graph is encoded and decoded to obtain the event corresponding to each event node.

[0011] Compared with existing technologies, the above embodiments have the following beneficial effects: To alleviate the problem of scattered argument entities in documents, connection relationships between entity nodes are constructed based on the first embedding representation vector of each entity. This allows for the structural association of scattered arguments of an event, effectively solving the problem that a single sentence cannot cover the complete arguments of an event, and improving the accuracy of subsequent event extraction. Furthermore, based on the original dense connections, entity sentence nodes that integrate entities and their corresponding sentences are introduced. This strengthens the feature aggregation of entity sentences to reduce the noise introduced by dense connections between entities, sentences, and events, while retaining the positive role of dense connections in extracting features related to entity co-operation relationships and event relationships, thereby improving the accuracy of subsequent event extraction.

[0012] Further, the step of constructing the connection relationships between the entity nodes and obtaining the second graph based on the first graph and according to the first embedding representation vector corresponding to each entity node includes:

[0013] For two entity nodes that correspond to the same sentence node, establish a connection between the two entity nodes;

[0014] Based on the first embedded representation vector corresponding to each of the two entity nodes, predict whether the two entity nodes correspond to the same event node;

[0015] For two entity nodes that correspond to the same event node, establish a connection relationship between the two entity nodes;

[0016] For two entity nodes that have a connection relationship with the same entity node, establish the connection relationship between the two entity nodes.

[0017] Compared with the prior art, the above embodiments have the following beneficial effects: by establishing connection relationships between entity nodes, the graph neural network can learn the features of co-reference, co-occurrence and other relationships between entity nodes when encoding event nodes, thereby strengthening the semantic association between entity mentions and reducing the negative impact of argument dispersion on the accuracy of document event extraction.

[0018] Further, the step of predicting whether two entity nodes correspond to the same event node based on the first embedding representation vectors corresponding to each of the two entity nodes includes:

[0019] By strengthening the first embedding representation vectors corresponding to the two entity nodes through the self-attention mechanism layer, the second embedding representation vectors corresponding to the two entity nodes are obtained.

[0020] After scaling the second embedding representation vectors corresponding to the two entity nodes, a dot product operation is performed, and the result of the dot product operation is input into a preset activation function to obtain the probability value of the two entity nodes corresponding to the same event node.

[0021] When the probability value exceeds a preset threshold, it is determined that the two entity nodes correspond to the same event node.

[0022] Compared with the prior art, the above embodiments have the following beneficial effects: by introducing a self-attention mechanism to enhance the embedded expression of entity nodes, deeper semantic features are mined, and the contextual semantic relationship between entities is better reflected. Furthermore, by predicting the probability value of two entity nodes corresponding to the same event node, two entity nodes that are not related to the same event node can be effectively distinguished, avoiding blind connections between entity nodes and reducing noise in the second graph.

[0023] Further, the step of scaling the second embedding representation vectors corresponding to the two entity nodes, performing a dot product operation, and inputting the dot product result into a preset activation function to obtain the probability value of the two entity nodes corresponding to the same event node includes:

[0024] The specific formula for calculating the probability value is as follows:

[0025]

[0026] In the formula, These are the learning weights and bias vectors, respectively. The dimension of the vector is d; and These are the second embedding vectors corresponding to the i-th entity node and the j-th entity node, respectively; and is the scaled second embedding representation vector; · represents the dot product operation of vectors; φ represents the sigmoid function; This is the probability value for predicting that two entity nodes correspond to the same event node.

[0027] Further, the step of adding entity sentence nodes corresponding to each entity node based on the sentence nodes corresponding to each entity node, and fully connecting each entity sentence node with each event node to obtain the third graph, includes:

[0028] For each of the aforementioned entity nodes, construct the corresponding entity sentence node;

[0029] Extract the entities corresponding to each entity node and the sentences corresponding to each sentence node from the document to be processed.

[0030] Based on the position of the entity in the corresponding sentence, obtain the attention submatrix of the entity node corresponding to the entity;

[0031] Based on the third embedding representation vector of the sentence node corresponding to the entity node and the attention submatrix, the fourth embedding representation vector corresponding to the entity sentence node is obtained.

[0032] Compared with existing technologies, the above embodiments have the following beneficial effects: by obtaining the corresponding attention sub-matrix based on the position of the entity in the sentence, and combining the third embedding representation vector and the attention sub-matrix to obtain the fourth embedding representation vector of the entity sentence node, the association between the entity and the sentence in which it is located and the event is strengthened, and the context in which the entity is located is expressed more accurately. This effectively eliminates the noise effect caused by dense connections, and while retaining the learning ability of fully connected features, it further realizes local semantic constraints and feature selection, and finally improves the ability of the third graph to model the relationship between events and context, thereby improving the robustness and accuracy of event extraction.

[0033] Further, obtaining the fourth embedding representation vector corresponding to the entity sentence node based on the third embedding representation vector of the entity node and the attention submatrix includes:

[0034]

[0035] A″ m =softmax(A′) m )

[0036] h u =A″m×H s

[0037] Among them, A m Let a be the attention submatrix of entity node m; ij For A m The element in the i-th row and j-th column; A′ m Let be the attention vector of entity node m relative to its corresponding sentence node s; |m| represents the number of words in the entity corresponding to entity node m; |s| represents the number of entities in the sentence corresponding to the sentence node; a j ′ is A′ m The j-th element in; A″ mThe normalized attention vector; softmax(*) is the softmax layer; h u H is the fourth embedding vector of entity sentence node u; s This is the third embedding vector of sentence node s.

[0038] Another embodiment of this application also provides a document event extraction device, including: a first graph extraction module, a second graph construction module, a third graph construction module, and an event extraction module;

[0039] The first graph extraction module is used to generate a first graph based on the document to be processed. The first graph includes several entity nodes, several sentence nodes, and several event nodes. Each entity node and each sentence node has its own corresponding embedding representation vector. Each event node is fully connected to each of the other event nodes, entity nodes, and sentence nodes. Each sentence node corresponds to several entity nodes.

[0040] The second graph construction module is used to construct the connection relationship between the entity nodes based on the first graph and according to the first embedding representation vector corresponding to each entity node, and obtain the second graph.

[0041] The third graph construction module is used to add entity sentence nodes corresponding to the entity nodes according to the sentence nodes corresponding to the entity nodes based on the second graph, and to fully connect each entity sentence node with each event node to obtain the third graph;

[0042] The event extraction module is used to encode and decode each event node in the third graph using a preset graph neural network to obtain the event corresponding to each event node.

[0043] Further, the second graph construction module includes: a first connection relationship construction unit, a collaboration prediction unit, a second connection relationship construction unit, and a third connection relationship construction unit; the second graph construction module is used to construct the connection relationships between the entity nodes based on the first graph and according to the first embedding representation vector corresponding to each entity node, to obtain the second graph, including:

[0044] The first connection relationship construction unit is used to construct a connection relationship between two entity nodes that correspond to the same sentence node;

[0045] The co-op prediction unit is used to predict whether two entity nodes correspond to the same event node based on the first embedded representation vector corresponding to each of the two entity nodes.

[0046] The second connection relationship construction unit is used to construct a connection relationship between two entity nodes that correspond to the same event node.

[0047] The third connection relationship construction unit is used to construct a connection relationship between two entity nodes that have a connection relationship with the same entity node.

[0048] Another embodiment of this application also provides a terminal device, including: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements the steps of the document event extraction method of this application.

[0049] Another embodiment of this application also provides a computer-readable storage medium item, including: a stored computer program, which, when the computer program is running, controls the device where the computer-readable storage medium is located to perform the steps of the document event extraction method of this application. Attached Figure Description

[0050] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0051] Figure 1 This is a flowchart illustrating a document event extraction method provided in some embodiments of this application;

[0052] Figure 2 This is another flowchart illustrating a document event extraction method provided in some embodiments of this application.

[0053] Figure 3 This is a bar chart showing the experimental results of F1 scores grouped by the number of document entities, provided in some embodiments of this application.

[0054] Figure 4 This is a bar chart showing the experimental results of F1 scores grouped by the number of document events, provided in some embodiments of this application.

[0055] Figure 5 This is a schematic diagram of the structure of a document event extraction device provided in some embodiments of this application. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0058] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0059] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0060] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0061] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0062] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in the embodiments of this application according to the specific circumstances.

[0063] Graph neural networks have gained widespread attention in recent years due to their ability to capture rich semantic information in documents, thus becoming a mainstream method for event extraction. In this method, graph construction is the core step affecting the final event extraction performance. Existing techniques primarily focus on the local dense connections between events and entities, as well as sentences, during graph construction. This locality ignores the rich prior semantic relationships between entity mentions and fails to fully model the direct connections between sentences and events, resulting in the loss of some key semantic information during the modeling process. Furthermore, while this indiscriminate dense connection aims to cover all possible relationships, it inevitably introduces a large amount of irrelevant or noisy information, interfering with the model's identification and learning of truly relevant information, thereby limiting the improvement of event extraction accuracy.

[0064] Please refer to Figure 1 To address the problem of low accuracy in document event extraction in existing technologies, this application provides a document event extraction method, including steps S101 to S104, specifically including:

[0065] S101: Generate a first graph based on the document to be processed; the first graph includes several entity nodes, several sentence nodes, and several event nodes; each entity node and each sentence node has its own corresponding embedding representation vector; each event node is fully connected to each of the other event nodes, entity nodes, and sentence nodes; each sentence node corresponds to several entity nodes.

[0066] For example, the specific forms of words, entities, sentences, arguments, roles, and events mentioned in some embodiments of this application are as follows: Figure 2 As shown in document D, "Equity Pledge 1" is... Figure 2The event shown in the text is "Share Pledge," which is the event type. S5 and S7 are sentences. Fields such as "Mr. A," "B Corporation," and "17,750,000" are entities. "Limited Company," "Shares," and "B" are words in the entity "B Corporation." "Pledger" is the role of entity "Mr. A" in the event "Share Pledge 1." And {"Mr. A," "Pledger"} together form an argument in the event "Share Pledge 1."

[0067] It should be noted that the entities and sentences extracted from the document to be processed each form the corresponding entity nodes and sentence nodes in the first graph. The event nodes in the first graph are initialized according to preset parameters. That is, when step S101 is executed, the specific events corresponding to each event node are unknown, and the initial embedding representation vector corresponding to the event node needs to be modulated to obtain an accurate embedding representation vector.

[0068] It should be further noted that the entities, events, sentences, and corresponding mathematical modeling symbols for entity sentences mentioned in the following embodiments of this application are also equivalent to the data modeling symbols for their respective nodes. For example, the entity set M also represents the entity node set, m i This represents the i-th entity in document D, and also the i-th entity node in the set of entity nodes M. Furthermore, the embedding vectors for entities, events, and sentences are equivalent to the embedding vectors of their respective nodes.

[0069] It should be further noted that in the following embodiments of this application, the symbols... This represents the dimension of the corresponding matrix or vector as *. Furthermore, from... Figure 2 It can be seen that document-level event extraction faces two challenges: argument dispersion and the coexistence of multiple events. The former refers to the distribution of arguments for a single event across multiple sentences. For example, ... Figure 2 As shown, the arguments for the event "Equity Pledge 1" are scattered across sentences labeled S5 and S7. Extracting argument information from a single sentence is incomplete, requiring the event extraction model to have a holistic understanding of the document. The latter refers to the possibility that an article may contain more than one event; for example, Mr. A may have pledged equity in another company, thus creating two events with unclear boundaries.

[0070] Furthermore, in some embodiments of this application, the mathematical modeling of the relationship between words, entities, sentences, arguments, roles, and events can be represented as follows: The input document is decomposed into conceptual relationships; a document D consists of multiple sentences s, and a sentence s consists of multiple words t. The relationship between these three can be represented by a set: Where |*| represents the number of elements in the set, s it represents the i-th sentence in document D. ij Let represent the j-th word in the i-th sentence. Furthermore, for an event v, belonging to event type c, an event has multiple entities m, each entity m playing a role r in a specific event v. Therefore, an event can be represented as: Where, m i and r i Let R be the i-th entity in event v and the corresponding role of that entity in the event. c The set of roles defined for event type c. Meanwhile, an entity m is contained within a sentence s, and an entity m consists of multiple words t; therefore, an entity m can be mathematically modeled as... The entire document's event set V is modeled as The entity set M is modeled as Where v i For the i-th event in the document, m i Let m be the i-th entity in the document. It should be noted that in this embodiment, entity m is an entity mention, meaning sentences mentioned in different sentences are considered different entity mentions, such as... Figure 2 In S5 and S7, "Mr. A" appears simultaneously as a reference to two different entities.

[0071] Furthermore, in some embodiments of this application, in addition to the explicit conceptual relationships in the mathematical modeling described above, there are also implicit conceptual relationships between events and between entities, such as causal, sequential, hierarchical, etc., relationships between events, and coreference, subordinate, etc., relationships between entities.

[0072] Preferably, in some embodiments of this application, the step of constructing the first embedding representation vector for each entity node includes:

[0073] Each entity includes: several words;

[0074] Convert the words from text representation to a fifth embedding representation vector;

[0075] For any entity, the average of the fifth embedding representation vectors of all the words it includes is taken as the first embedding representation vector of the corresponding entity node.

[0076] Preferably, in some embodiments of this application, the document entity extraction and embedding process is learned as an independent sequence labeling task. BIO (Beginning, Inside, Outside) is used to label each word in the document, representing the beginning, inside, and outside of the entity, respectively. Then, a pre-trained BERT model under the RoBERTa (Robustly Optimized BERT Pre-training Approach) setting is used for sentence-level sequence labeling. The loss value of this process is denoted as... Through the above process, we can obtain: the entity set M and the word order interval of each entity in the document; the third embedding representation vector of each word t∈D. in The vector with dimension d represents the latent state vector, where d denotes the embedding dimension, a configurable hyperparameter; the fourth embedding representation vector for each sentence s∈D. That is, the Classification Language Substitution (CLS) vector; and the attention matrix for each sentence s∈D. The matrix represents a dimension of |s|×|s|, where A s This refers to the last attention matrix output by the BERT model.

[0077] Preferably, in some embodiments of this application, after obtaining the sixth embedding representation vector of each word, the first embedding representation vector h of each entity node is... m Calculated using the following formula:

[0078]

[0079] in, Let be the third embedding representation vector of the i-th word in entity m.

[0080] As can be seen from the above embodiments, this application constructs the first embedding representation vector of the corresponding entity node based on the fifth embedding representation vector of the words within the entity, retains the semantic information inside the entity, avoids the loss of word-level processing, and improves the calculation speed by taking the average of the fifth embedding representation vectors of all words to determine the first embedding representation vector of the corresponding entity node.

[0081] Preferably, in some embodiments of this application, such as Figure 2 The graph shown is mathematically modeled as follows: the first graph mainly consists of event nodes v, entity nodes m, and sentence nodes s, as shown in the figure. Figure 2The dashed lines connecting the two parts represent the structure of the first graph. It should be noted that the event nodes in the first graph have no specific meaning during construction; they only serve as placeholders, and the number of event nodes is a configurable hyperparameter. Furthermore, when the set of event nodes in the first graph is... The set of entity nodes is and the set of sentence nodes is At that time, the node set P of the first graph b It can be represented as P b =V∪M∪D, and further, the edge set of the first graph can be represented as The first subset, which is the full connection between event nodes, represents implicit causal, sequential, hierarchical, and other event relationships; the second subset, which is the full connection from entity nodes to event nodes, represents the pre-defined relationship between events and entities; and the third subset, which is the full connection from sentence nodes to event nodes, represents the semantic connection between sentences and event relationships.

[0082] Furthermore, as can be seen from the embodiments of the first atlas described above, the first atlas still has the following shortcomings:

[0083] (1) No connection is established between entity nodes, which means that features such as co-reference and co-occurrence of entities cannot be extracted during the learning process.

[0084] (2) Establishing full connections from entity nodes to event nodes makes it possible to extract the relationships between entities, but unfiltered full connections inevitably introduce noise into the learning process.

[0085] (3) Establishing full connections from sentence nodes to event nodes can indirectly play a role in the feature extraction of event relationships, but the importance of sentences is not distinguished, and noise will be introduced during the learning process.

[0086] S102: Based on the first graph, according to the first embedding representation vector corresponding to each entity node, construct the connection relationship between the entity nodes to obtain the second graph.

[0087] Furthermore, in some embodiments of this application, the step of constructing the connection relationship between the entity nodes and obtaining the second graph based on the first graph and according to the first embedding representation vector corresponding to each entity node includes:

[0088] For two entity nodes that correspond to the same sentence node, establish a connection between the two entity nodes;

[0089] Based on the first embedded representation vector corresponding to each of the two entity nodes, predict whether the two entity nodes correspond to the same event node;

[0090] For two entity nodes that correspond to the same event node, establish a connection relationship between the two entity nodes;

[0091] For two entity nodes that have a connection relationship with the same entity node, establish the connection relationship between the two entity nodes.

[0092] Preferably, in some embodiments of this application, the step of establishing a connection relationship between two entity nodes corresponding to the same sentence node includes: if two different entities m i and m j Being mentioned in the same sentence means representing two entity nodes m. i and m j For the same sentence node, if there is a co-occurrence relationship between the two entity nodes, then a connection should be established between them. The edge between these two entity nodes is called an entity co-occurrence edge, denoted as...

[0093] Preferably, in some embodiments of this application, for two entity nodes corresponding to the same event node, establishing a connection relationship between the two entity nodes includes: if two different entities m i and m j Belonging to the same event, that is, representing two different entity nodes m i and m j For the same event node, the two entity nodes have a collaborative relationship. Therefore, a connection should be established between the two entity nodes. The edge between the two entity nodes is called a collaborative edge, denoted as... It should be noted that the working relationship between entity nodes is implicit. A learning task needs to be constructed based on the labeled working relationship samples to predict whether there is a working relationship between two entity nodes.

[0094] Preferably, in some embodiments of this application, the step of establishing a connection between two entity nodes that simultaneously have a connection relationship with the same entity node includes: if two different entity nodes m i and m j With the third entity node m k If two entities co-occur or co-occur, then a bridging relationship exists between them. Therefore, a connection should be established between the two entity nodes. The edge between the two entity nodes is called an entity bridging edge, denoted as E. mr ={ <m i ,m j >| <m i ,m k >∈E ms ∪E mv , <m k,m j >∈E ms ∪E mv ,i≠j≠k}.

[0095] As can be seen from the above embodiments, this application establishes connections between entity nodes, enabling the graph neural network to learn the characteristics of co-reference, co-occurrence, and other relationships between entity nodes when encoding event nodes. This strengthens the semantic association between entity mentions and reduces the negative impact of argument dispersion on the accuracy of document event extraction.

[0096] Furthermore, in some embodiments of this application, the step of predicting whether two entity nodes correspond to the same event node based on the first embedding representation vectors corresponding to the two entity nodes includes:

[0097] By strengthening the first embedding representation vectors corresponding to the two entity nodes through the self-attention mechanism layer, the second embedding representation vectors corresponding to the two entity nodes are obtained.

[0098] After scaling the second embedding representation vectors corresponding to the two entity nodes, a dot product operation is performed, and the result of the dot product operation is input into a preset activation function to obtain the probability value of the two entity nodes corresponding to the same event node.

[0099] When the probability value exceeds a preset threshold, it is determined that the two entity nodes correspond to the same event node.

[0100] Preferably, in some embodiments of this application, the step of strengthening the first embedding representation vectors corresponding to the two entity nodes through a self-attention mechanism layer to obtain the second embedding representation vectors corresponding to the two entity nodes includes:

[0101] Entity co-operation relationship prediction uses the first embedding representation vectors corresponding to the two entity nodes. and (i≠j) as input, predict m i and m j Determining the probability of belonging to the same event is a binary classification problem. To better understand the contextual semantic relationships between entities, a Transformer layer is first introduced to enhance the first embedding representation vector of each entity node, as shown in the equation:

[0102]

[0103] in, m i The enhanced embedding representation, Transformer(*) is the Transformer layer.

[0104] Furthermore, in some embodiments of this application, the step of scaling the second embedding representation vectors corresponding to the two entity nodes, performing a dot product operation, and inputting the dot product result into a preset activation function to obtain the probability value of the two entity nodes corresponding to the same event node includes:

[0105] The specific formula for calculating the probability value is as follows:

[0106]

[0107] In the formula, These are the learning weights and bias vectors, respectively. The dimension of the vector is d; and These are the second embedding vectors corresponding to the i-th entity node and the j-th entity node, respectively; and is the scaled second embedding representation vector; · represents the dot product operation of vectors; φ represents the sigmoid function; This is the probability value for predicting that two entity nodes correspond to the same event node.

[0108] If the scaled second embedding representation vector If the threshold γ is exceeded, then entity node m is considered to be... i and m j There is a collaborative relationship between them. This task uses binary cross-entropy as the loss function, denoted as .

[0109] By introducing a self-attention mechanism to enhance the embedded representation of entity nodes, we can mine deeper semantic features and better reflect the contextual semantic relationships between entities. Furthermore, by predicting the probability values ​​of two entity nodes corresponding to the same event node, we can effectively distinguish two entity nodes that are not related to the same event node, avoid blind connections between entity nodes, and reduce noise in the second graph.

[0110] S103: Based on the second graph, according to the sentence nodes corresponding to each entity node, add entity sentence nodes corresponding to each entity node, and fully connect each entity sentence node with each event node to obtain the third graph.

[0111] Furthermore, in some embodiments of this application, the step of adding entity sentence nodes corresponding to the entity nodes according to the sentence nodes corresponding to the entity nodes, and fully connecting each entity sentence node with each event node to obtain a third graph, based on the second graph, includes:

[0112] For each of the aforementioned entity nodes, construct the corresponding entity sentence node;

[0113] Extract the entities corresponding to each entity node and the sentences corresponding to each sentence node from the document to be processed.

[0114] Based on the position of the entity in the corresponding sentence, obtain the attention submatrix of the entity node corresponding to the entity;

[0115] Based on the third embedding representation vector of the sentence node corresponding to the entity node and the attention submatrix, the fourth embedding representation vector corresponding to the entity sentence node is obtained.

[0116] Preferably, in some embodiments of this application, constructing the corresponding entity sentence node for each entity node includes: for Then s is called an entity sentence. After fusing the features of m and s, an entity sentence node u is formed, denoted as u = fusion(m, s). The fusion function is constructed by embedding an inference subtask into the learning model. The set U of entity sentence nodes is represented as... Where u i Let |U| represent the i-th entity sentence node. Note that |U| = |M|. A fully connected entity sentence node and event node form an edge called the entity sentence-event table, denoted as E. uv ={<u,v> |u∈U,v∈V}. Based on the second graph above, the third graph is represented as G=(P,E), where P=P b ∪U, E=E b ∪E ms ∪E mv ∪E mr ∪E uv .

[0117] Preferably, in some embodiments of this application, the step of obtaining the attention sub-matrix of the entity node corresponding to the entity based on the position of the entity in the corresponding sentence includes: for an entity m, For the attention matrix A corresponding to sentence s s Extract the attention submatrix A of m from it. m That is, the row vector of the word order interval of sentence s containing m, denoted as A. m =(a ij ) |m|×|s| ,a ij ∈A s ,t i ∈m, where a ij Representing A s The elements in.

[0118] Furthermore, in some embodiments of this application, obtaining the fourth embedding representation vector corresponding to the entity sentence node based on the third embedding representation vector of the sentence node corresponding to the entity node and the attention submatrix includes:

[0119]

[0120] A″ m =softmax(A′) m )

[0121] h u =A″m×H s

[0122] Among them, A m Let a be the attention submatrix of entity node m; ij For A m The element in the i-th row and j-th column; A′ m Let be the attention vector of entity node m relative to its corresponding sentence node s; |m| represents the number of words in the entity corresponding to entity node m; |s| represents the number of entities in the sentence corresponding to the sentence node; a j ′ is A′ m The j-th element in; A″ m The normalized attention vector; softmax(*) is the softmax layer; H s Let be the third embedding vector of sentence node s. For sentence s, the third embedding vectors corresponding to the words it contains are concatenated to form a matrix. The matrix represents a dimension of |s|×d; h u The fourth embedding representation vector of entity sentence node u is fused with key semantic information of (m,s) based on the attention matrix. Entity sentence nodes will establish dense connections with event nodes, working together with entities to construct a third graph, and participating in the information aggregation of the event encoding and decoding process.

[0123] As can be seen from the above embodiments, this application obtains the corresponding attention sub-matrix based on the position of the entity in the sentence, and combines the third embedding representation vector and the attention sub-matrix to obtain the fourth embedding representation vector of the entity sentence node. This strengthens the association between the entity and the sentence in which it is located with the event, more accurately expresses the context in which the entity is located, and effectively eliminates the noise influence caused by dense connections. While retaining the learning ability of fully connected features, it further realizes local semantic constraints and feature selection, and finally improves the ability of the third graph to model the relationship between the event and the context, and improves the robustness and accuracy of event extraction.

[0124] S104: Encode and decode each event node in the third graph using a preset graph neural network to obtain the event corresponding to each event node.

[0125] Preferably, in some embodiments of this application, the step of encoding and decoding each event node in the third graph using a preset graph neural network to obtain the event corresponding to each event node includes:

[0126] The event includes: an event type and several arguments; each argument includes: an entity and the role of that entity in the event to which the argument belongs;

[0127] The graph neural network is used to perform feature modulation information propagation operation on the third graph to obtain the fifth embedding representation vector corresponding to each event node.

[0128] The event type corresponding to the event is determined based on the fifth embedding representation vector;

[0129] The role is determined based on the first embedding representation vector corresponding to the entity of the argument and the fifth embedding representation vector of the event node corresponding to the event to which the argument belongs.

[0130] Preferably, in some embodiments of this application, the step of performing feature modulation information propagation operation on the third graph through the graph neural network to obtain the fifth embedding representation vector corresponding to each event node includes:

[0131] Event encoding involves aggregating information surrounding the event to form an embedded representation of the event node. First, the embedded representation vectors of each node in the third graph G are initialized. The acquisition of the embedded representation vectors corresponding to entity node m, sentence node s, and entity sentence node u has been described in detail in some embodiments of this application and will not be repeated here. The initial embedded representation of each event node v... Then a d-dimensional random initialization vector is used.

[0132] Based on the initial node embedding representation, a graph neural network with feature-wise linear modulation (GNN-Film) is used to propagate feature-modulated information on the third graph G to capture global structural information. The sixth embedding representation vector of the event node in G is updated using the following formula:

[0133]

[0134] in, For event v iThe fifth embedding representation vector after feature modulation, GNN-FiLM is a graph neural network, and P and E are the set of nodes and the set of edges of the third graph G, respectively.

[0135] Because an event is Therefore, event decoding is divided into two stages: first, determining the event type c, i.e., the event type classification; and then, defining the event type m for each entity m in the event. i Determine its corresponding role r i That is, the allocation of roles in the argument.

[0136] Preferably, in some embodiments of the application, determining the event type corresponding to the event based on the fifth embedding representation vector includes:

[0137] In the event type classification stage, the fifth embedding representation vector h is mainly used. v Perform multi-class classification prediction. Use a fully connected layer to predict the event type for each event. The specific process is as follows:

[0138]

[0139] in, Let be the learning weights and bias vectors, respectively, and C be the set of event types. The dimension of the matrix is ​​d×(|C|+1). The dimension of the vector is |C|+1. Let v represent the probability distribution vector of v over the event type set, and softmax(*) be the softmax layer. It's worth noting that "empty event" is also predicted as an event type, so the event type set here is |C|+1. If an event is classified as "empty event," then that event does not correspond to a real event, and no argument role assignment is needed. Finally, the type with the highest probability is chosen as the event type, i.e.:

[0140]

[0141] Here, argmax represents the operation of finding the maximum value. This represents the type with the highest probability.

[0142] Preferably, in some embodiments of this application, determining the role based on the first embedding representation vector corresponding to the entity of the argument and the sixth embedding representation vector corresponding to the event to which the argument belongs includes:

[0143] Entities with the same name generally have a coreference relationship, where "same name" means that two entities have the same textual form. In this application, entities with the same name are treated as a single physical entity for argument role assignment. First, a multi-head attention module is introduced to fuse multiple entities with the same name into a single physical entity. The event embedding representation is used as the query, and the connection matrix of the embedding representation of entities with the same name corresponding to a physical entity is used as the key and value, as shown below:

[0144]

[0145] in, Represents a physical entity. Indicates event v i The corresponding query They represent The corresponding keys and values. After multi-head attention weighting. In response to event v i The embedding representation is as follows:

[0146]

[0147] in, for In response to event v i The embedding representation is given by MHA, which stands for Multi-Head Attention Computation Function.

[0148] Next, the embedded representation of events will be combined with... By splicing them together, we get The role representation for the event node set is as follows:

[0149]

[0150] in, These are the learning weights and bias vectors, respectively. Represents physical entities Belongs to v i In the event type, the role type r i,k The probability distribution vector is important to note, but "empty role" is also predicted as a role type, representing an entity. Since it is unrelated to events, the dimension of the set of character types is |R|. c +1, c is v i The event type. The event node v obtained in the final prediction. i middle The type of character is As shown in the formula:

[0151]

[0152] If multiple physical entities belong to the same role in an event, the physical entity with the highest probability is selected.

[0153] It should be noted that the above-mentioned encoding and decoding process of event nodes is only one feasible embodiment, and this application does not limit the specific encoding and decoding process.

[0154] As can be seen from the above embodiments, after obtaining a more accurate third graph, this application fully utilizes the contextual association information in the third graph through a feature propagation mechanism to encode event nodes, thereby improving the accuracy and robustness of overall event recognition and understanding; furthermore, it determines the event type based on the sixth embedding representation vector corresponding to the event node, and determines the role based on the first embedding representation vector corresponding to the entity and the sixth embedding representation vector of the event to which the entity belongs, thereby improving the accuracy of event extraction.

[0155] Furthermore, in some embodiments of the application, the relevant parameters in the above-described document entity extraction and first embedding representation vector acquisition process, the prediction process of the working relationship between entities, and the encoding and decoding process of event nodes are trained and optimized in the following manner:

[0156] First, assume It is the set of predicted events represented by event nodes in G. It is a collection of true events. Corresponding node v i Predicting events, Corresponding node v i A true story, and These are event nodes v i Real event types and The real character type, and The distance between them is defined as shown in the following formula:

[0157]

[0158] Where CE represents the cross-entropy loss function, Indicates the size of the set of physical entities.

[0159] Since the goal of training is to minimize and To address the potential issues with the average Hausdorff distance during training—namely, the predicted event might be optimized towards multiple different real events simultaneously—a constraint is introduced to ensure that each pair... and The distance calculation between them is performed at most once, as shown below:

[0160]

[0161] Finally, combining the losses from the three processes mentioned above, the total loss for the entire training process is as follows:

[0162]

[0163] in, This represents the loss value for the encoding and decoding tasks of the event node.

[0164] To illustrate the effectiveness of the document event extraction method in this application, further analysis and explanation will be provided below based on experimental results.

[0165] The proposed method is implemented using Python based on the PyTorch open-source framework. During implementation, the input document is segmented into sentences with a maximum length of 512 BIO tags, and then fed into RoBERTa-wwm-ext as a sequence labeling model. When constructing the graph, the number of event nodes |V| is set to 16 and 8 on the ChFinAnn and DuEE-Fin datasets, respectively. During training, the Adam optimizer is used, with a batch size of 32, a word embedding dimension d of 512, and a threshold of 0.5 in the entity co-occurrence relationship prediction step. The learning rate of the pre-trained model is set to 10%. -5 The learning rate for other parameters is set to 10. -4 The model was trained using a 2-layer GNN-FiLM. It was trained for 100 epochs on an NVIDIA RTX 4090, and the best F1 score was selected for evaluation on the test set. ChFinAnn is a large-scale financial dataset consisting of annual reports, profit forecasts, and other official disclosures extracted from financial announcements. This dataset includes five event types: share pledge, share freeze, share repurchase, share reduction, and share increase, as well as 35 different argument roles. It contains 25,632 documents for model training, 3,204 documents for development evaluation, and 3,204 documents for testing. On average, each document contains approximately 20 sentences and 912 tags, with approximately 30% of documents involving multiple event records. DuEE-Fin is a Chinese document-level event extraction dataset built by Baidu through manual annotation. It contains 11,900 financial documents, covering 13 event types, with approximately 33% of documents containing multiple event records. It is worth noting that, since the correct annotation test set of DuEE-Fin is not publicly available, this application uses the original development set as the test set.

[0166] Table 1 shows the performance comparison results between the proposed method in this application embodiment and other existing models. The proposed method in this application embodiment is represented by CESG. The performance indicators in the table include: P represents accuracy, R represents recall, F1 represents the overall F1 score, F1(S) represents the F1 score for a single event, and F1(M) represents the F1 score for multiple events, all expressed as percentage values. The best results are marked in bold.

[0167]

[0168] Table 1

[0169] Overall, the CESG model proposed in this application outperforms all baseline models. For the most comprehensive evaluation metric, F1, CESG achieves the best performance on both datasets. Of all metrics, only the accuracy on the DuEE-Fin dataset fails to achieve the best performance, falling slightly below ProCNet. This verifies the effectiveness of the proposed global entity semantic graph in document-level event extraction tasks.

[0170] Regarding graph models, performance can be broadly summarized as follows: dense graph models > local graph models > graph-free sequence models. Averaging the F1 scores by graph type, dense graph models (GIT, ProCNet, CESG), local graph models (PTPCG, Doc2EDAG, Greedy-Dec, ReDEE), and graph-free sequence models (DCFEE-O, DCFEE-M, DE-PPN) scored 77.65, 70.41, and 60.78, respectively, showing significant differences among the three. While DE-PPN achieved a good F1 score on the ChFinAnn dataset, it performed the weakest on the DuEE-Fin dataset, exhibiting severe instability. This may be because graph-free sequence models rely solely on the order information of words and entities for event extraction, resulting in a serious problem of ignoring long-distance dependencies and poor performance in extracting document-level events with scattered arguments; while local graph models only construct limited entity relationships, insufficiently expressing the contextual semantic relationships between sentences and events, leading to poorer aggregation effects for scattered arguments compared to dense graphs. This also confirms the importance of building graph models based on prior information for document-level event extraction.

[0171] ProCNet and CESG with event nodes outperform GIT without event nodes across all performance metrics. In terms of F1 scores on both datasets, CESG improves upon GIT by 4.23% and 23.45%, respectively, demonstrating the crucial role of event nodes in semantic association of entity context and information aggregation. The introduction of event nodes and the construction of dense relationships express implicit event relationships and enhance the aggregation of sentence and entity semantic information, thereby improving the performance of event extraction.

[0172] A comparison of CESG and ProCNet shows that CESG outperforms ProCNet on F1, F1(S), and F1(M). The main difference lies in the fact that CESG's graph model extends entity relations and entity sentence relations, validating the effectiveness of graph extension in CESG. By enhancing the semantics surrounding entities, the information convergence weights of dense connections around events in the base graph are diluted, thus reducing the risk of noise introduction.

[0173] To evaluate the performance of CESG in handling argument dispersion, experiments were conducted on the ChFinAnn dataset with different entity count settings. As shown in Table 2, the test set was divided into five groups based on the number of entities in each document, and the F1 score of CESG in each group was evaluated. Local graph model PTPCG and dense graph model ProCNet were selected for comparison to enhance the reliability of the results. The results are as follows: Figure 3 As shown.

[0174]

[0175] Table 2

[0176] Figure 3Experimental results show that the performance of all models decreases as the number of arguments in a document increases. However, CESG consistently outperforms PTPCG and ProCNet across all five groups, demonstrating a superior event extraction performance across different argument dispersion levels. In the ChFinAnn test set, approximately 91% of documents contain no more than 16 arguments, and CESG achieves an F1 score of 92.3% on these documents, surpassing ProCNet's 91.1% and PTPCG's 88.2%. When the number of arguments exceeds 16, CESG's performance advantage becomes even more pronounced, with an average F1 score improvement of approximately 2.1% compared to ProCNet and approximately 5.2% compared to PTPCG. This indicates that an increased number of entities in a document poses a challenge to the event extraction task. While PTPCG performs local mapping of entities in the document and models relationships between entities using a similarity matrix, its Bi-LSTM word embeddings are weaker than those of BERT or Transformer models, resulting in lower accuracy in entity extraction and similarity matrix representation, leading to a weaker ability to handle argument dispersion. While the densely constructed ProCNet doesn't explicitly build relationships between entities, it dynamically aggregates entity mention representations at event nodes using a graph neural network through dense connections between entity mention nodes and event nodes. Furthermore, ProCNet employs BERT strong word embeddings, resulting in better representation capabilities for entity mentions. Therefore, ProCNet outperforms PTPCG in addressing the argument dispersion problem. In contrast, CESG, building upon ProCNet, locally constructs co-occurrence, co-occurrence, and bridging relationships between entity mentions, enabling information to propagate among more relevant entity mentions and optimizing the representations of entity mention nodes and event nodes, effectively addressing the challenge of argument dispersion.

[0177] Table 1 shows that all methods consistently outperform F1(S) in F1(M) scores across both datasets, indicating that multi-event extraction remains a significant challenge in document-level event extraction. In the ChFinAnn dataset, approximately 30% of documents contain multiple events. To evaluate CESG's ability to address this challenge, this paper divides the ChFinAnn test set into multiple subsets based on the number of events, as shown in Table 3. Subsequently, this paper evaluates the performance of CESG and baseline models on these subsets, using PTPCG and ProCNet as controls. The experimental results are as follows: Figure 4 As shown.

[0178]

[0179] Table 3

[0180] from Figure 4The results show that PTPCG performs poorly in multi-event scenarios, mainly for two reasons. First, as the number of events increases, pseudo-trigger words in PTPCG may be shared across multiple events, making it difficult for the model to accurately predict entity-based local graph structures, thus leading to performance degradation. Second, the event record decoding process in PTPCG heavily relies on the correct prediction of pseudo-trigger words. As the number of events increases, the number of pseudo-trigger words also increases, exacerbating error propagation and further reducing performance. In contrast, ProCNet introduces event proxy nodes, enabling interaction modeling between different events, which makes it outperform PTPCG in multi-event tasks. However, not all sentences and entity mentions in a document contain event records. ProCNet's dense connection graph construction increases the risk of noise introduction, making it difficult for event proxy nodes to accurately decode into events.

[0181] CESG consistently outperforms ProCNet across all multi-event subsets, improving the average F1 score by 0.7%. When the document contains 3 event records, CESG's performance improves by 2.1%, highlighting its effectiveness in handling multi-event challenges. This performance improvement can be attributed to the introduction of entity sentence nodes. In documents containing multiple events, semantic information becomes increasingly complex, and noise also increases. ProCNet directly and densely connects the semantic information of all sentences to event nodes, undoubtedly introducing noise and reducing the performance of the event extraction task. CESG mitigates the noise introduced by dense connections between sentences and event agents by strengthening the feature aggregation of entity sentences, reducing interference from irrelevant semantics. This design allows each event node to better represent its corresponding event type, improving the model's ability to distinguish multiple events.

[0182] In summary, the document event extraction method provided in this application has the following advantages compared to existing technologies: To alleviate the problem of scattered argument entities in a document, a connection relationship between entity nodes is constructed based on the first embedding representation vector of each entity. This allows for the structural association of scattered arguments of an event, effectively solving the problem that a single sentence cannot cover the complete arguments of an event, and improving the accuracy of subsequent event extraction. Furthermore, based on the original dense connections, entity sentence nodes that integrate entities and their corresponding sentences are introduced. This strengthens the feature aggregation of entity sentences to reduce the noise introduced by dense connections between entities, sentences, and events, while retaining the positive role of dense connections in extracting features related to entity co-operation relationships and event relationships, thereby improving the accuracy of subsequent event extraction.

[0183] like Figure 5As shown, based on the above method embodiments, an embodiment of this application provides a document event extraction device, including: a first graph extraction module 201, a second graph construction module 202, a third graph construction module 203, and an event extraction module 204.

[0184] Further, in some embodiments of this application, the first graph extraction module 201 is used to generate a first graph based on the document to be processed; the first graph includes several entity nodes, several sentence nodes, and several event nodes; each entity node and each sentence node has its own corresponding embedding representation vector; each event node is fully connected to each other event node, each entity node, and each sentence node; each sentence node corresponds to several entity nodes; the second graph construction module 202 is used to construct the connection relationship between the entity nodes based on the first graph and according to the first embedding representation vector corresponding to each entity node, to obtain a second graph; the third graph construction module 203 is used to add entity sentence nodes corresponding to each entity node based on the second graph and according to the sentence nodes corresponding to each entity node, and fully connect each entity sentence node with each event node, to obtain a third graph; the event extraction module 204 is used to encode and decode each event node in the third graph through a preset graph neural network to obtain the event corresponding to each event node.

[0185] Further, in some embodiments of this application, the second graph construction module 202 includes: a first connection relationship construction unit, a collaboration prediction unit, a second connection relationship construction unit, and a third connection relationship construction unit; the second graph construction module 202 is used to construct connection relationships between entity nodes based on the first graph and according to the first embedding representation vector corresponding to each entity node, to obtain a second graph, including: the first connection relationship construction unit is used to construct a connection relationship between two entity nodes corresponding to the same sentence node; the collaboration prediction unit is used to predict whether two entity nodes correspond to the same event node according to the first embedding representation vector corresponding to each of the two entity nodes; the second connection relationship construction unit is used to construct a connection relationship between two entity nodes corresponding to the same event node; the third connection relationship construction unit is used to construct a connection relationship between two entity nodes that simultaneously have a connection relationship with the same entity node.

[0186] Furthermore, in some embodiments of this application, the co-op prediction unit is used to predict whether two entity nodes correspond to the same event node based on the first embedding representation vectors corresponding to the two entity nodes, including: strengthening the first embedding representation vectors corresponding to the two entity nodes through a self-attention mechanism layer to obtain the second embedding representation vectors corresponding to the two entity nodes; scaling the second embedding representation vectors corresponding to the two entity nodes, performing a dot product operation, and inputting the dot product operation result into a preset activation function to obtain a probability value that the two entity nodes correspond to the same event node; when the probability value exceeds a preset threshold, it is determined that the two entity nodes correspond to the same event node.

[0187] Furthermore, in some embodiments of this application, the step of scaling the second embedding representation vectors corresponding to the two entity nodes, performing a dot product operation, and inputting the dot product result into a preset activation function to obtain the probability value of the two entity nodes corresponding to the same event node includes:

[0188] The specific formula for calculating the probability value is as follows:

[0189]

[0190] In the formula, These are the learning weights and bias vectors, respectively. The dimension of the vector is d; and These are the second embedding vectors corresponding to the i-th entity node and the j-th entity node, respectively; and is the scaled second embedding representation vector; · represents the dot product operation of vectors; φ represents the sigmoid function; This is the probability value for predicting that two entity nodes correspond to the same event node.

[0191] Further, in some embodiments of this application, the third graph construction module 203 includes: an entity sentence node construction unit, an entity sentence extraction unit, an attention sub-matrix acquisition unit, and a fourth embedding representation vector acquisition unit; the third graph construction module 203 is used to, based on the second graph, add entity sentence nodes corresponding to the entity nodes according to the sentence nodes corresponding to each entity node, and fully connect each entity sentence node with each event node to obtain the third graph, including: the entity sentence node construction unit is used to construct a corresponding entity sentence node for each entity node; the entity sentence extraction unit is used to extract the entity corresponding to each entity node and the sentence corresponding to each sentence node from the document to be processed; the attention sub-matrix acquisition unit is used to obtain the attention sub-matrix of the entity node corresponding to the entity node according to the position of the entity in the corresponding sentence; the fourth embedding representation vector acquisition unit is used to obtain the fourth embedding representation vector corresponding to the entity sentence node according to the third embedding representation vector of the sentence node corresponding to the entity node and the attention sub-matrix.

[0192] Further, in some embodiments of this application, the fourth embedding representation vector acquisition unit is used to acquire the fourth embedding representation vector corresponding to the entity sentence node based on the third embedding representation vector of the sentence node corresponding to the entity node and the attention submatrix, including:

[0193]

[0194] A″ m =softmax(A′) m )

[0195] h u =A″m×H s

[0196] Among them, A m Let a be the attention submatrix of entity node m; ij For A m The element in the i-th row and j-th column; A′ m Let be the attention vector of entity node m relative to its corresponding sentence node s; |m| represents the number of words in the entity corresponding to entity node m; |s| represents the number of entities in the sentence corresponding to the sentence node; a j ′ is A′ m The j-th element in; A″ m The normalized attention vector; softmax(*) is the softmax layer; h u H is the fourth embedding vector of entity sentence node u; s This is the third embedding vector of sentence node s.

[0197] In summary, the document event extraction device provided in this application has the following advantages compared to the prior art: To alleviate the problem of scattered argument entities in a document, a connection relationship between entity nodes is constructed based on the first embedding representation vector of each entity. This allows for the structural association of scattered arguments of an event, effectively solving the problem that a single sentence cannot cover the complete arguments of an event, and improving the accuracy of subsequent event extraction. Furthermore, based on the original dense connections, entity sentence nodes that integrate entities and their corresponding sentences are introduced. This strengthens the feature aggregation of entity sentences to reduce the noise introduced by dense connections between entities, sentences, and events, while retaining the positive role of dense connections in extracting features related to entity co-operation relationships and event relationships, thereby improving the accuracy of subsequent event extraction.

[0198] It is understood that the above-described device embodiments correspond to the method embodiments of this application, and can implement the document event extraction method provided by any of the above-described method embodiments of this application.

[0199] It should be noted that the device embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided in this application, the connection relationships between modules indicate that they have communication connections, which can specifically be implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0200] Based on the above-described embodiments of the document event extraction method, another embodiment of this application provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the document event extraction method of any embodiment of this application.

[0201] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete this application. The one or more module units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the terminal device.

[0202] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0203] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0204] Based on the above-described method embodiments, another embodiment of this application provides a computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the document event extraction method described in any of the above-described method embodiments of this application.

[0205] The modules / units integrated in the device / terminal equipment, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

Claims

1. A document event extraction method, characterized in that, include: A first graph is generated based on the document to be processed; the first graph includes several entity nodes, several sentence nodes, and several event nodes; each entity node and each sentence node has its own corresponding embedding representation vector; each event node is fully connected to each of the other event nodes, entity nodes, and sentence nodes; each sentence node corresponds to several entity nodes. Based on the first graph, the connection relationship between the entity nodes is constructed according to the first embedding representation vector corresponding to each entity node to obtain the second graph; Based on the second graph, according to the sentence nodes corresponding to each entity node, add entity sentence nodes corresponding to each entity node, and fully connect each entity sentence node with each event node to obtain the third graph; By using a preset graph neural network, each event node in the third graph is encoded and decoded to obtain the event corresponding to each event node.

2. The document event extraction method as described in claim 1, characterized in that, The step of constructing the connection relationships between the entity nodes and obtaining the second graph based on the first graph and the first embedding representation vector corresponding to each entity node includes: For two entity nodes that correspond to the same sentence node, establish a connection between the two entity nodes; Based on the first embedded representation vector corresponding to each of the two entity nodes, predict whether the two entity nodes correspond to the same event node; For two entity nodes that correspond to the same event node, establish a connection relationship between the two entity nodes; For two entity nodes that have a connection relationship with the same entity node, establish the connection relationship between the two entity nodes.

3. The document event extraction method as described in claim 2, characterized in that, The step of predicting whether two entity nodes correspond to the same event node based on the first embedding representation vector corresponding to each of the two entity nodes includes: By strengthening the first embedding representation vectors corresponding to the two entity nodes through the self-attention mechanism layer, the second embedding representation vectors corresponding to the two entity nodes are obtained. After scaling the second embedding representation vectors corresponding to the two entity nodes, a dot product operation is performed, and the result of the dot product operation is input into a preset activation function to obtain the probability value of the two entity nodes corresponding to the same event node. When the probability value exceeds a preset threshold, it is determined that the two entity nodes correspond to the same event node.

4. The document event extraction method as described in claim 3, characterized in that, The step of scaling the second embedding representation vectors corresponding to the two entity nodes, performing a dot product operation, and inputting the dot product result into a preset activation function to obtain the probability value of the two entity nodes corresponding to the same event node includes: The specific formula for calculating the probability value is as follows: In the formula, These are the learning weights and bias vectors, respectively. The dimension of the vector is d; and These are the second embedding vectors corresponding to the i-th entity node and the j-th entity node, respectively; and is the scaled second embedding representation vector; · represents the dot product operation of vectors; φ represents the sigmoid function; d represents the probability that two entity nodes correspond to the same event node; d is a preset distance parameter.

5. The document event extraction method as described in claim 1, characterized in that, Based on the second graph, the third graph is obtained by adding entity sentence nodes corresponding to each entity node according to the sentence nodes corresponding to each entity node, and making full connections between each entity sentence node and each event node. For each of the aforementioned entity nodes, construct the corresponding entity sentence node; Extract the entities corresponding to each entity node and the sentences corresponding to each sentence node from the document to be processed. Based on the position of the entity in the corresponding sentence, obtain the attention submatrix of the entity node corresponding to the entity; Based on the third embedding representation vector of the sentence node corresponding to the entity node and the attention submatrix, the fourth embedding representation vector corresponding to the entity sentence node is obtained.

6. The document event extraction method as described in claim 5, characterized in that, The step of obtaining the fourth embedding representation vector corresponding to the entity sentence node based on the third embedding representation vector of the entity node and the attention submatrix includes: A″ m =softmax(A ′ m ) A u =A″m×H s Among them, A m Let a be the attention submatrix of entity node m; ij for h m The element in the i-th row and j-th column; A ′ m Let be the attention vector of entity node m relative to its corresponding sentence node s; |m| represents the number of words in the entity corresponding to entity node m; |s| represents the number of entities in the sentence corresponding to the sentence node; a j ′ For A ′ m The j-th element in; A″ m The normalized attention vector; softmax(*) is the softmax layer; h u H is the fourth embedding vector of entity sentence node u; s This is the third embedding vector of sentence node s.

7. A document event extraction device, characterized in that, include: The system comprises a first graph extraction module, a second graph construction module, a third graph construction module, and an event extraction module. The first graph extraction module is used to generate a first graph based on the document to be processed. The first graph includes several entity nodes, several sentence nodes, and several event nodes. Each entity node and each sentence node has its own corresponding embedding representation vector. Each event node is fully connected to each of the other event nodes, entity nodes, and sentence nodes. Each sentence node corresponds to several entity nodes. The second graph construction module is used to construct the connection relationship between the entity nodes based on the first graph and according to the first embedding representation vector corresponding to each entity node, and obtain the second graph. The third graph construction module is used to add entity sentence nodes corresponding to the entity nodes according to the sentence nodes corresponding to the entity nodes based on the second graph, and to fully connect each entity sentence node with each event node to obtain the third graph; The event extraction module is used to encode and decode each event node in the third graph using a preset graph neural network to obtain the event corresponding to each event node.

8. The document event extraction device as described in claim 7, characterized in that, The second graph construction module includes: a first connection relationship construction unit, a collaboration prediction unit, a second connection relationship construction unit, and a third connection relationship construction unit; the second graph construction module is used to construct the connection relationships between the entity nodes based on the first graph and according to the first embedding representation vector corresponding to each entity node, to obtain the second graph, including: The first connection relationship construction unit is used to construct a connection relationship between two entity nodes that correspond to the same sentence node; The co-op prediction unit is used to predict whether two entity nodes correspond to the same event node based on the first embedded representation vector corresponding to each of the two entity nodes. The second connection relationship construction unit is used to construct a connection relationship between two entity nodes that correspond to the same event node. The third connection relationship construction unit is used to construct a connection relationship between two entity nodes that have a connection relationship with the same entity node.

9. A terminal device, characterized in that, The system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements a document event extraction method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform a document event extraction method as described in any one of claims 1 to 6.

Images