A causal event extraction method based on a causal event extraction model

Abstract

The application discloses a kind of based on causal event extraction model's causal event extraction method, comprising the following steps: based on event extraction and causal relationship identification two subtasks, construct causal event extraction model;Acquire input sentence;Based on pre-training language model, input sentence is encoded, based on sequence labeling decoder, the coded input sentence is decoded, and the event of existence causal correlation is extracted;Based on causal event extraction model, construct initial background graph;Event is inserted into initial background graph, and updated background graph is obtained;Using graph neural network, the representation of extracted event is obtained by encoding and updating to updated background graph;Based on classifier and the representation of extracted event, the causal relationship between events is obtained, realizes the extraction of causal event.The experimental results of the sentence of different causal event pairs prove the ability of the method of the application to extract complex causal relationship.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of sentence causal event extraction, and particularly relates to a causal event extraction method based on a causal event extraction model. BACKGROUND

[0002] The causal event extraction task aims to extract causal event pairs from text. The causal event extraction task supports a wide range of important downstream tasks, such as future event prediction and machine reading comprehension. Most existing methods use a pipeline framework to process the causal event extraction task, including two subtasks: 1) event extraction, extracting events that may have a causal relationship with other events in the input sentence; and event causal relationship identification, identifying the causal relationship between events in the input sentence. However, the pipeline framework ignores the interdependence between the two subtasks, and the performance is not ideal. First, if the event causal relationship identification model cannot pair the extracted events with any other extracted events, the event extraction model needs to be prompted to some information, that is, either this causal event pair should not be extracted, or the cause event or the result event needs to be ignored; second, the event causal relationship identification model provides the event extraction model with rich knowledge of events that may have a causal relationship. In addition, due to the error propagation from event extraction to event causal relationship identification in the pipeline framework, the errors of early event extraction cannot be corrected in the later event causal relationship identification task.

[0003] Therefore, it is necessary to propose a causal event extraction method based on a causal event extraction model. SUMMARY

[0004] The present application aims to solve the problems of the prior art and proposes a causal event extraction method based on a causal event extraction model. The carefully designed subtask interaction and knowledge fusion mechanism can effectively improve the performance of the task and knowledge fusion network.

[0005] To achieve the above-mentioned purpose, the present application provides the following scheme:

[0006] A causal event extraction method based on a causal event extraction model, comprising the following steps:

[0007] Based on the two subtasks of event extraction and causal relationship identification, a causal event extraction model is constructed;

[0008] An input sentence is obtained;

[0009] Based on a pre-trained language model, the input sentence is encoded, and based on a sequence labeling decoder, the encoded input sentence is decoded to extract events with causal relationships;

[0010] Based on the causal event extraction model, obtain the knowledge graph node of the input sentence and the preset hop number neighbor of the knowledge graph node, and construct an initial background graph;

[0011] Insert the event into the initial background graph to obtain an updated background graph;

[0012] Using a graph neural network, the updated background graph is encoded and updated to obtain the representation of the extracted event;

[0013] Based on the classifier and the representation of the extracted event, the causal relationship between the events is obtained, and the causal event extraction is realized.

[0014] Preferably, the method for extracting events with causal relationships is:

[0015] Based on the causal event extraction model, obtain the token representation of the input sentence;

[0016] The token representation of the input sentence is input into the pre-trained language model to obtain a pre-fused token representation;

[0017] The pre-fused token representation is input into the sequence labeling decoder to extract events with causal relationships in the input sentence.

[0018] Preferably, the method for constructing an initial background graph is:

[0019] Retrieving the external knowledge graph of the input sentence to obtain the knowledge graph node of the input sentence;

[0020] Adding the preset hop number neighbors of the knowledge graph node and any knowledge graph node on the path between any pair of knowledge graph nodes to obtain a node set;

[0021] The connection between the input sentence and the knowledge graph node is input into the pre-trained language model to output a node relevance score;

[0022] Based on the node relevance score, the node set is pruned;

[0023] In the knowledge graph, all edges connecting the pruned node set and any node pair of the knowledge graph node are used as edges of the initial background graph to realize the construction of the initial background graph.

[0024] Preferably, the method for obtaining an updated background graph is:

[0025] The extracted event is added to the initial background graph as an event node;

[0026] obtaining a union node based on the union of the event nodes and the knowledge graph nodes added to the initial background graph

[0027] computing a joint representation of any pair of the union nodes based on an input representation of the union nodes, a weight matrix, and a linear transformation with a hyperbolic tangent activation function;

[0028] assigning a non-negative score to edges of a link structure among the union nodes in the union to obtain an adjacency matrix;

[0029] computing a root score of the union nodes as a root of any of the syntactic dependency trees based on an assumption that the event nodes and the knowledge graph nodes are connected by the syntactic dependency trees;

[0030] obtaining a Laplacian matrix of the link structure and variants thereof based on the adjacency matrix and the root score;

[0031] obtaining a weighted adjacency matrix based on the Laplacian matrix, the adjacency matrix, a Kronecker function, and a matrix tree theorem;

[0032] inserting the extracted event dynamically into the initial background graph based on the weighted adjacency matrix to obtain the updated background graph.

[0033] Preferably, the method for obtaining the representation of the extracted event is:

[0034] initializing input embeddings of the knowledge graph nodes and nodes in the node set based on the fused knowledge graph node embeddings;

[0035] obtaining a pre-fused event context representation based on the pre-fused token representation;

[0036] initializing an input embedding of the event node based on the pre-fused event context representation;

[0037] inputting the initialized input embeddings of the nodes in the updated background graph into the graph neural network to obtain pre-fused node embeddings;

[0038] jointly encoding the retrieved external knowledge and the extracted event based on the pre-fused node embeddings and the initialized input embedding of the event node to obtain the representation of the event.

[0039] Preferably, there is a mutual dependency relationship between the causal relationship identification and the event extraction.

[0040] The dependency relationship includes: a dependency of the causal relationship identification on the event extraction; and a dependency of the event extraction on the causal relationship identification.

[0041] Preferably, the method for realizing the dependence of the event extraction on the causal relationship identification is:

[0042] The pre-fusion event context representation output by the pre-trained language model is connected with the pre-fusion node embedding output by the graph neural network, and is input into a sub-task information aggregator to obtain a fused representation;

[0043] The pre-fusion event context representation is replaced with the fused representation, and information about the result of the causal relationship identification is provided to the event extraction, thereby realizing the dependence of the event extraction on the causal relationship identification.

[0044] Preferably, the dependence of the causal relationship identification on the event extraction uses a stack propagation framework to reduce error propagation.

[0045] Compared with the prior art, the present application has the beneficial effects that: the present application proposes a causal event extraction method based on a causal event extraction model, which can simultaneously solve three key problems of causal event extraction, namely complex causal relationship extraction, sub-task interaction and knowledge fusion. For the problem of complex causal relationship extraction, the present application uses a separate sub-task decoder as a basic framework. For the other two problems, the present application first designs a sub-task interaction mechanism, uses stack propagation learning and a sub-task information aggregator to realize the mutual interaction between the two sub-tasks of causal event extraction. In addition, the present application also designs a knowledge fusion mechanism, which uses an insertion deduction module and a knowledge information aggregator to effectively fuse two different modes of knowledge fusion mechanism. The sub-task interaction and knowledge fusion mechanism are both multi-layer and bidirectional working modes, which ensure the deep exchange of information. Experimental results show that the model of the present application always achieves better performance than the baseline method on three widely used data sets. In addition, the ablation experiment proves the effectiveness of the multi-layer sub-task interaction and knowledge fusion mechanism proposed by the present application. The experimental results on sentences of different causal event pairs prove the ability of the method of the present application to extract complex causal relationships. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the present application, the following briefly introduces the drawings needed to be used in the embodiments. Obviously, the drawings described in the following embodiments are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0047] Figure 1 is a method flowchart of the causal event extraction method based on the causal event extraction model according to the first embodiment of the present application;

[0048] Figure 2is a structural schematic diagram of a causal event extraction system based on a causal event extraction model according to an embodiment two of the present application;

[0049] Figure 3 is an experimental result graph for inter-model analysis according to an embodiment four of the present application;

[0050] Figure 4 is a model test result graph according to an embodiment four of the present application;

[0051] Figure 5 is an experimental result graph for testing performance of a causal event extraction method according to an embodiment four of the present application. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0053] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0054] Embodiment one

[0055] As shown in Figure 1 , a causal event extraction method based on a causal event extraction model comprises the following steps:

[0056] Based on two sub-tasks of event extraction and causal relationship identification, a causal event extraction model is constructed;

[0057] An input sentence is obtained;

[0058] Based on a pre-trained language model, the input sentence is encoded, and based on a sequence labeling decoder, the encoded input sentence is decoded to extract events with causal correlation;

[0059] Based on the causal event extraction model, knowledge graph nodes of the input sentence and preset hop neighbors of the knowledge graph nodes are obtained to construct an initial background graph;

[0060] The events are inserted into the initial background graph to obtain an updated background graph;

[0061] A graph neural network is used to encode and update the updated background graph to obtain a representation of the extracted events;

[0062] Based on a classifier and the representation of the extracted events, the causal relationship between the events is obtained to realize the extraction of causal events.

[0063] The method for extracting events with causal correlations comprises the following steps:

[0064] Based on the causal event extraction model, a token representation of the input sentence is obtained.

[0065] The token representation of the input sentence is input into a pre-trained language model to obtain a pre-fused token representation.

[0066] The pre-fused token representation is input into a sequence labeling decoder to extract events with causal correlations in the input sentence.

[0067] The method for constructing an initial background graph comprises the following steps:

[0068] An external knowledge graph of the input sentence is retrieved to obtain knowledge graph nodes of the input sentence.

[0069] A preset hop neighbor of the knowledge graph nodes and any knowledge graph node on a path between any pair of knowledge graph nodes are added to obtain a node set.

[0070] The input sentence and the connection of the knowledge graph nodes are input into a pre-trained language model to output a node correlation score.

[0071] The node set is pruned based on the node correlation score.

[0072] All edges of any pair of nodes connecting the pruned node set and the knowledge graph nodes in the knowledge graph are taken as edges of the initial background graph to realize the construction of the initial background graph.

[0073] The method for obtaining an updated background graph comprises the following steps:

[0074] The extracted events are added to the initial background graph as event nodes.

[0075] Based on the union set of the event nodes and the knowledge graph nodes added to the initial background graph, a union node set is obtained.

[0076] Based on the input representation of the union node set, a weight matrix and a linear transformation with a hyperbolic tangent activation function, a joint representation of any pair of union nodes is calculated.

[0077] A non-negative score is assigned to the edges of the link structure between the union nodes in the union set to obtain an adjacency matrix.

[0078] Assuming that the event nodes and the knowledge graph nodes are connected by a syntax dependency tree, the non-standardized probability of the union nodes as the root of any syntax dependency tree is calculated to obtain a root score.

[0079] Based on the adjacency matrix and the root score, a Laplacian matrix of the link structure and its variants is obtained.

[0080] The weighted adjacency matrix is obtained based on a Laplacian matrix, an adjacency matrix, a Kronecker function and a matrix tree theorem;

[0081] The extracted event is dynamically inserted into the initial background graph based on the weighted adjacency matrix to obtain an updated background graph.

[0082] Preferably, the method for obtaining the representation of the extracted event is:

[0083] The input embedding of the knowledge graph node and the nodes in the node set is initialized based on the fused knowledge graph node embedding;

[0084] The pre-fused event context representation is obtained based on the pre-fused token representation;

[0085] The input embedding of the event node is initialized based on the pre-fused event context representation;

[0086] The input embedding of the node in the initialized updated background graph is input into the graph neural network to obtain a pre-fused node embedding;

[0087] The retrieved external knowledge and the extracted event are jointly encoded based on the pre-fused node embedding and the initialized input embedding of the event node to obtain the representation of the event.

[0088] There is a mutual dependency relationship between the causal relationship identification and the event extraction;

[0089] The dependency relationship includes: the dependency of the causal relationship identification on the event extraction; and the dependency of the event extraction on the causal relationship identification.

[0090] The method for realizing the dependency of the event extraction on the causal relationship identification is:

[0091] The pre-fused node embedding output by the graph neural network is connected with the pre-fused event context representation output by the pre-trained language model to input into the sub-task information aggregator to obtain a fused representation;

[0092] The pre-fused event context representation is replaced by the fused representation to obtain The pre-fused event context representation is replaced by the fused representation to obtain

[0093] The dependency of the causal relationship identification on the event extraction adopts a stack propagation framework to reduce error propagation.

[0094] Embodiment two

[0095] ​​The causal event extraction model proposed in this invention includes: an event module and a relationship module;

[0096] The event module is used to acquire the input statement; based on the pre-trained language model, it encodes the input statement; based on the sequence label decoder, it decodes the encoded input statement and extracts events with causal relationships.

[0097] The relationship module is used to obtain the knowledge graph nodes of the input statement and their preset hop count neighbors to construct an initial background graph; insert events into the initial background graph to obtain an updated background graph; use a graph neural network to encode and update the updated background graph to obtain the representation of the extracted events; based on the classifier and the representation of the extracted events, obtain the causal relationships between the events to achieve the extraction of causal events. The relationship module also includes an insertion derivation submodule.

[0098] The causal event extraction model is called Task and Knowledge Fusion Network (TKNet).

[0099] Figure 2 This invention presents an overview of the TKNet architecture, which consists of two main modules: one with... The event module of the layer and a with Layer relational modules. Given a sentence The first few N layers of the event module encode each token in the sentence using the first few N layers of a pre-trained language model. Additionally, the relation module retrieves relevant knowledge from an external knowledge graph of the knowledge graph nodes mentioned in the sentence, constructing an initial background graph. In each of the following In the layer, the first event module The layer first updates the representation of each token in sentence S using a pre-trained language model layer, and then uses a sequence label decoder to extract events that may have causal relationships. Finally, an insertion inference module inserts these extracted events into... In the middle, the updated background image is obtained. Then, the relation module uses GNN to... The representation of the middle node is encoded and updated (including by the first node). The event module layer extracts events, and then a classifier is used to determine the causal relationships between the extracted events. At the end of each subsequent M layer, the invention combines the annotations in sentence S with... The pre-fusion representation of the nodes is input into two information aggregators to fuse information between the two subtasks and the two knowledge patterns. The fused representation is then presented. and which is then used as the input of the next layer of the TKNet. After the iterative information fusion of M layers, the output of the last layer is used as the final prediction of the TKNet.

[0100] Specifically, the event module:

[0101] The event module aims to extract the causal events in the sentence S that may have a causal relationship. The present application uses the implicit semantic knowledge stored in the pre-trained language model to enhance the model of the present application, which has been proven to be very effective for causal event extraction in previous work. In addition, a sequence labeling decoder is also used to extract potential causal events.

[0102] Specifically, the present application uses the BERT (pre-trained language model) model to make a fair comparison with previous work on causal event extraction. For the first layer of the event module, the present application inputs the token representation generated by the previous layer into the BERT layer to obtain the pre-fused token representation . In the first N layers of the event module, is equal to , where is the token representation generated by the previous layer, and is the pre-fused token representation. However, in the next layers, is calculated by two information aggregators designed by the present application, which fuse and exchange information between the two sub-tasks and two types of knowledge. In addition, according to the paper of BERT, the present application initializes the labeling representation by summing the token embedding, segment embedding and position embedding corresponding to each token in the sentence S.

[0103] Then, in each layer of the last layer, the present application inputs the pre-fused token representation into a sequence labeling decoder to extract events in the input sentence, and the decoder is shared in all layers. In practice, the present application uses a conditional random field (CRF) decoder for event extraction:

[0104] (1)

[0105] where is the BIO label sequence predicted by the CRF decoder. Finally, the present application uses the representation of the last token in each event as the pre-fused event context representation, denoted as .

[0106] Specifically, the relationship module:

[0107] The relation module is designed to identify the causal relationships between events extracted by the event module. To reduce the error propagation from event extraction to causal event identification, the present invention adopts a stack propagation framework that utilizes the event context representation produced by the event module as the input of the relation module in each of the last M layers. In addition, most of the causal relationships in the text are implicitly expressed (i.e., without causal cue words), and external knowledge is essential for effectively determining such implicit causal relationships. Therefore, the relation module of the present invention first retrieves a background graph containing relevant knowledge of the input sentence from the external knowledge graph. Then, in each layer, the present invention designs an insertion deduction module to dynamically connect the extracted events with useful knowledge in the graph, avoiding irrelevant knowledge that may reduce the quality of event representation. The GNN is then used to encode all nodes in the updated graph, including the extracted events. Finally, the event node representation produced by the GNN, which contains knowledge from the pre-trained language model and the external knowledge graph, is input into a classifier to identify the causal relationships between the extracted events.

[0108] Construction of the background graph:

[0109] Initial background graph: Given an input sentence , the present invention first retrieves the knowledge graph to obtain an initial background graph , which contains external knowledge useful for extracting causal event pairs in the sentence S. Previous causal event extraction methods only use manually annotated events to perform the retrieval process, and such event annotation labels are difficult to obtain in practice. At the same time, non-event elements (such as entities) in the sentence S can also be associated with useful knowledge. Therefore, the present invention retrieves knowledge related to each element in the sentence S, rather than relying solely on the true annotated events. Following previous work, the present invention first retrieves the external knowledge graph to obtain the knowledge graph nodes mentioned in the input sentence as basic nodes, denoted as . Then, the present invention adds the 2-hop neighbors of and any knowledge graph node on the shortest path between any pair of to obtain the set of relevant nodes . The maximum length of the shortest path is limited to 10. The present invention uses the calculated relevance score to prune . Specifically, the present invention inputs the connection between the sentence S and a knowledge graph node into a pre-trained language model, and uses the output score of the node as its relevance score. contains the top-scoring nodes. The maximum number of nodes in the initial background graph is set to 50. Finally, the present invention utilizes the knowledge graph to connect the pruned and All edges of any node pair are used as the initial background graph. The edges in.

[0110] Dynamic update (update background image): in the first... In the layer, the event module extracts events from the input sentence. Then, the present invention adds the extracted events as event nodes to the graph. In the middle, it is denoted as and in ∪ Weighted edges are constructed between nodes. Then, an insertion sensing module is designed to perform the above insertion process using the updated background. Enter the first A GNN with a relational module layer.

[0111] Causal reasoning based on graph neural networks:

[0112] In the In the layer, this invention uses GNN to obtain the first layer. Information propagated across a background graph is used to jointly encode retrieved external knowledge and extracted events. Finally, the invention can obtain event node representations containing knowledge from a pre-trained language model and an external knowledge graph.

[0113] For the In this invention, the fused node embedding generated in the previous layer is used to initialize the layer. and The input embedding of the nodes in the [the dataset]. The input embedding of the event node is then used by the first The pre-fusion event context generated by each event module layer represents Perform initialization. For the first layer, and The provided pre-trained knowledge graph embeddings are used for initialization.

[0114] In each of the M layers, the present invention will Input embedding of middle nodes Input GNN to obtain pre-fused node embeddings . yes The number of nodes in the GNN. Constructing a GNN, although other GNN variants can also be used, where... yes The neighborhood of node t It is a 2-layer multilayer perceptron (multilayer perceptron). and This indicates that attention should be paid to the weight of the message passed from node s to node t, and the calculation result is:

[0115] (2)

[0116] (3)

[0117] (4)

[0118] (5)

[0119] (6)

[0120] wherein, , is a learned embedding of the node type (i.e. the knowledge graph node mentioned in the sentence S, other knowledge graph nodes and event nodes). is a 2-layer multi-layer perceptron, , , f k is a linear transformation. is and dimension. Wherein, , , and are intermediate variables; represents the edge weight in, the calculation method is:

[0121] (8)

[0122] wherein, is an adjacency matrix, which represents the weighted edge constructed by the insertion derivation module (insertion derivation sub-module) of the application, used to dynamically insert the extracted event into the initial background graph.

[0123] Causal relationship classifier:

[0124] In each layer of layer, the application uses a classifier to identify the causal relationship between each pair of extracted events , whose GNN generated node embedding and , which contains knowledge from pre-trained language models and external knowledge graphs:

[0125] (9)

[0126] wherein is a 2-layer multi-layer perceptron with a softmax activation function, represents the causal relationship predicted by the th relationship module layer.

[0127] Insert a derivation submodule (insert a derivation module or insert an induction module):

[0128] In practice, the truly labeled events are unavailable, while non-event elements in sentence S, such as entities, are also related to valuable knowledge. To avoid losing useful information, this invention uses retrieved knowledge associated with each element in sentence S to initialize a background graph. Then, this invention uses a Generative Neural Network (GNN) to obtain event representations enhanced by external knowledge to identify causal relationships. However, integrating knowledge from all elements may introduce irrelevant and redundant information, as not all elements in sentence S are closely related to the causal relationships between events. To address this issue, this invention designs an insertion inference module that dynamically connects the extracted events to each element in sentence S to help the GNN avoid irrelevant information.

[0129] This invention will ∪ The link structure between nodes is represented as a weighted adjacency matrix. .value Represents node pairs The edge weights between them, where Since the extracted events and the mentioned knowledge graph nodes are in the same sentence, it is reasonable to assume that they are connected by some kind of syntactic dependency tree, which indicates the relationship between them. Based on this inductive bias, this invention designs an insertion derivation module that uses a variant of Kirchhoff's matrix tree theorem to derive the expected probabilities of the link structure as different possible dependency trees.

[0130] For the layer, This represents the input representation of the i-th node. The present invention first calculates the input representation of any pair of nodes. Joint statement :

[0131] (10)

[0132] Among them, W c It is a weight matrix. and It is a linear transformation with a tanh activation function. Then, the invention assigns non-negative scores to... Given the edges, we obtain the following adjacency matrix:

[0133] (11)

[0134] in, This is the score of the edge between the i-th node and the j-th node. Then, this invention calculates the root score. , which represents the non-normalized probability of the ith node being the root of any dependency tree:

[0135] (12)

[0136] where, is a weight matrix. Then, the link structure and its variants the Laplacian matrix is defined as:

[0137] (13)

[0138] (13)

[0139] Finally, according to the matrix tree theorem, the computation of is:

[0140] (15)

[0141] where is the Kronecker function, which can be formalized as:

[0142] (16)

[0143] Usage of the subtask information aggregator:

[0144] The interdependence between the two subtasks (i.e., event extraction and causal (relation) event identification) is crucial for the causal event extraction task. For the dependence of causal event identification on event extraction, the present invention employs a stack propagation framework to mitigate error propagation from causal event identification to event extraction. However, the dependence of event extraction on causal event identification is not simple either. First, if the causal event identification model cannot pair the extracted event with any other extracted event, the event extraction model needs to be reminded of two facts: one is that the event should not be extracted, or the cause or result event might need to be ignored. In addition, the causal relationship determined by the causal event identification model provides useful knowledge for the event extraction model to extract events that may have causal relationships. To help event extraction benefit from the results of causal event identification, the present embodiment designs a subtask information aggregator (T-Aggregator) that uses the extracted events as a bridge to provide information about the results of causal event identification to the event extraction of the TKNet final layer of the present invention.

[0145] At the first layer of the relation module of the present invention, the GNN outputs the embedding of the extracted events, which are then input into a simple classifier to identify the causal relationship. Therefore, the information about the results of causal event identification has been implicitly embedded in For the extracted th event, the present invention first concatenates the pre-fused event node embedding of the output of the GNN with the pre-fused event context representation output by the pre-trained language model Then, the present invention inputs them into a T-pooler to obtain the fused representation Then, the present invention inputs them into a T-pooler to obtain the fused representation

[0146]

[0147] where the T-pooler is a 2-layer multi-layer perceptron, although other fusion functions can be used. Finally, the present invention replaces the pre-fused representation in the th event module layer with the pre-fused representation to obtain , , which is the input of the next event module layer and provides information about the causal event identification results for event extraction.

[0148] Use of the knowledge information aggregator:

[0149] The effective and interactive fusion of knowledge from two modalities (i.e., expressive pre-trained language models and structured knowledge graphs) is important for the causal event extraction task. They can provide complementary information about events to help extract implicit causal relationships that have no causal clue word expressions. Through a well-designed knowledge graph retrieval process and an insertion inference module, the present invention has been able to effectively obtain event representations containing knowledge from pre-trained language models and external knowledge graphs. However, the two knowledge modalities can also enhance each other's encoding process, for example, pre-trained language models help select which knowledge in the knowledge graph is important, while the knowledge graph provides additional knowledge to help the pre-trained language model understand sentence semantics. For this purpose, the present invention designs a knowledge information aggregator (K-pooler) that uses the above knowledge graph node as a bridge to information to facilitate the encoding process of the two modalities.

[0150] At the th layer, for the th knowledge graph node mentioned in the input sentence , the corresponding pre-fused context embedding and knowledge embedding are and , respectively. is calculated by the th event module layer The pre-fused context representation of the last token in the middle. It is the first The pre-fused node embeddings are computed at each relation module layer. The pre-fused representation. and The input is concatenated and fed into a K-aggregator. The output is then split into fused embeddings. and :

[0151]

[0152] The K-aggregator is a two-layer multilayer perceptron, but other fusion functions can also be used. Finally, this invention will... and Embedded before fusion and Replace with the merged node embedding and To obtain separately and These are the inputs to the next-level event and relation modules, respectively. Therefore, the information from the knowledge graph and the pre-trained language model crosses their respective modules. Interacting during the propagation of layers.

[0153] The process of training a causal event extraction model:

[0154] During training, the event module and the relationship module perform their final... The layers perform sequence labeling and relation classification. This invention performs subtask interaction and knowledge fusion in all the final M layers to ensure that all three key problems of causal event extraction can be solved. This invention utilizes cross-entropy loss respectively. and To optimize the first of the two modules Layers. The loss function of TKNet is defined as the average of the last M layers:

[0155]

[0156] Table 1 shows the statistical data of the three causal event extraction datasets. For convenience, this invention uses pairs to represent causal event pairs.

[0157] Table 1

[0158]

[0159] During inference, this invention uses only the output of the last layer to obtain all predicted causal event pairs in the input sentence.

[0160] Example 3

[0161] The present application is based on an experimental setting of a causal event extraction model:

[0162] (1) Set the data set and evaluation index:

[0163] The present application conducts experiments on three widely used data sets: 1) EventStoryLine v0.9 (ESC), containing 258 files, 5334 events and 1770 pairs of causal event pairs; 2) SCIFI, containing 5236 sentences, 1866 pairs of causal event pairs; 3) Causal-TimeBank (Causal-TB), containing 184 documents, 6813 events and 318 pairs of causal event pairs. Detailed statistical data are listed in Table 1.

[0164] In the three causal event extraction data sets, the ESC data set labels all the causal event pairs in each document, which is consistent with the natural distribution of causal relationships expressed in the text. In causal event extraction, 93.4% of the causal relationships have no causal cue words. However, the SCIFI data set uses a different method, i.e. the sentences are pre-filtered with causal cue words before being labeled. Therefore, most of the labeled causal relationships in SCIFI are explicitly expressed, which deviates from the natural distribution of causal relationships in the text. In addition, most of the sentences in SCIFI contain only one causal event pair. Compared with the other two data sets, Causal-TB is a much smaller data set that can be used to test the performance of the causal event extraction model in a data-scarce scenario. For the following experiments, the ESC data set is used as the experimental setting by default unless otherwise specified.

[0165] For validation experiments, the present application uses precision (P), recall (R) and F1 value (F1) at the event pair level as evaluation indexes for the causal event extraction task:

[0166]

[0167] wherein, is the set of all sentences in the data set, and a predicted causal event pair is considered correct only when it exactly matches a labeled causal event pair. The present application performs 5-fold, 5-fold and 10-fold cross-validation on the ESC, SCIFI and Causal-TimeBank (Causal-TB) data sets, respectively. As in previous methods, the present application uses the last two topics of ESC as the validation set of Causal-TB data set. In addition, the present application corrects the SCIFI data set by deleting duplicate negative samples that appear in the training set from the test set. All results are the average of three independent experiments.

[0168] Parameter setting: the present application will and The values ​​are set to 9 and 3 respectively. This invention uses a basic BERT-Based-Uncased architecture to implement the event module, which has 12 layers, 768 hidden layer dimensions, and 12 heads. The hidden layer dimension of other parameters is set to 200. This invention chooses ConceptNet as the external knowledge graph, and the maximum number of nodes in the initial background graph is set to 50. In each relation module layer, the number of GNN layers is set to 1. The model's dropout is set to 0.2. This invention uses an early stopping mechanism and the Adam optimizer with a linear warmup mode to optimize the model. This invention sets the batch size to 20 and uses different learning rates for the LM encoder (lr=1e-5) and other parameters (lr=1e-4). Similar to previous methods, this invention uses a negative sampling strategy on the ESC and Causal-TB datasets, taking into account the sparsity of positive samples. The negative sampling is set to 0.6.

[0169] Baseline Methods: To evaluate the effectiveness of the method of the present invention, it was compared with four sets of baseline methods. Following previous work, the present invention first compares its method with two earlier feature-based baseline methods:

[0170] Rules-Bayesian extracts causal event pairs through pattern matching and a Bayesian classifier.

[0171] CausalNet: Measures the causal strength between phrases and filters causal event pairs with thresholds.

[0172] The present invention then compares its method with several pipeline-based state-of-the-art (SOTA) baseline methods. However, these methods only report their performance on causal event recognition tasks based on real-world labeled events, not the entire causal event extraction task. Furthermore, they only perform binary classification to determine whether a causal relationship exists between two events, without determining the order of causality. To evaluate their performance on the causal event extraction task, the present invention first trains a BERT-CRF model to extract events from sentences, and then feeds the extracted events into the following baseline methods for three-class classification:

[0173] BERT-Pipeline: A typical pipeline-based model that first uses the BERT-CRF model to extract events from sentences, and then uses the BERT model with a classification layer to identify the causal relationships between the extracted events.

[0174] KMMG: A BERT-based model that leverages external knowledge to enhance the representation of events and uses a mask mechanism to guide the model to focus on event-independent textual patterns that express causal relationships.

[0175] DPJL: A BERT-based model that designs two prompt-based derivative tasks to include information about the semantic relationship between causal cues and events. Compared with other pipeline-based methods, DPJL achieves SOTA results.

[0176] In addition, the present application compares the method of the present application with previous sequence labeling-based causal event extraction methods that can extract multiple causal event pairs in one sentence:

[0177] Nearest-BERT-CRF: First uses a BERT-CRF model to extract causal relationships, and then pairs each cause with the closest result in the sentence.

[0178] SCITE: First extracts causal events using an LSTM-CRF model, and then matches causal event relationships into pairs using a set of artificial rules. To make a fair comparison, the present application replaces their LSTM encoder with BERT.

[0179] Table 2 Experimental results of the models of the present application and the baselines on the dataset. All results in this table use the same dataset setting. P, R, and F1 represent precision, recall, and F1 value, respectively. Bold indicates the best effect. † indicates using BERT as the encoder of the model.

[0180] Table 2

[0181]

[0182] In addition, considering the formal similarity between causal event extraction and JERE tasks, the present application compares the method of the present application with recent JERE baseline methods:

[0183] CasRel: A BERT-based model that uses a cascade framework for general relationship extraction, first extracts all possible subjects, and then identifies the corresponding objects for each subject.

[0184] PRGC: A BERT-based model that pre-predicts potential relationships to reduce the redundancy of relationships in the cascade framework and filters out low-confidence entity pairs to improve model performance.

[0185] RFBFN: A BERT-based model that extracts entity pairs by filling in the blanks with the semantic information of the target relationship through a target relationship template.

[0186] Example Four:

[0187] Based on the experimental setup, experiments were conducted:

[0188] To verify the effectiveness of the method of the present application, the present application was experimented on three datasets and compared the method of the present application with four groups of baseline methods, including an early feature-based causal event extraction method, a pipeline-based causal event extraction method, a sequence labeling-based causal event extraction method and a recent JERE method. Table 3 shows the results of the ESC, SCIFI and Causal-TB datasets. The ESC dataset directly annotates each entire document, and the annotation results are consistent with the natural distribution of causal relationships in the text. The SCIFI dataset uses causal event cues to filter the sentences to be pre-annotated, including more explicit causal relationships. Compared with the other two datasets, the Causal-TB dataset is much smaller and can be used to test the performance of the causal event extraction model in the case of data scarcity.

[0189] Firstly, the present application can find that the model of the present application achieves the most advanced performance compared with all the baselines. On the ESC dataset, the F1 value of the method of the present application is 5.26% higher than the best baseline method DPJL. On the SCIFI dataset, the F1 value of the method is 4.83% higher than the best baseline method SCITE. On the Causal-TB dataset, the F1 value of the method of the present application is 2.45% higher than the best baseline method DPJL. These experiences show that the method proposed by the present application can effectively capture the causal event pairs in the text by promoting three main problems, such as complex causal relationship extraction, sub-task interaction and knowledge fusion.

[0190] Secondly, the present application can find that the sequence labeling-based method SCITE performs best on the SCIFI dataset, except for the model of the present application itself, but its performance on the ESC and Causal-TB datasets is not satisfactory. This is mainly because SCITE uses a set of hand-crafted rules to match the extracted causes and effects into pairs. However, their hand-crafted rules are designed specifically for the SCIFI dataset, so it is difficult to generalize to other datasets.

[0191] In addition, the present application can find that the performance of DPJL on the ESC dataset is better than that of KMMG, but not on the SCIFI dataset. This is mainly because DPJL is better at integrating various event-related knowledge, which helps to capture implicit causal relationships. However, KMMG can capture event-independent language templates that express causal relationships, which helps to capture explicit causal relationships. In ESC, there are mainly implicit causal relationships, while in SCIFI, there are more explicit causal relationships.

[0192] In addition, the present application finds that, although the JERE methods achieve acceptable performance on the causal event extraction task, they cannot outperform the state-of-the-art causal event extraction baseline methods. This indicates that directly applying the JERE methods to the causal event extraction task is not satisfactory, although they can be used to extract complex causal relation patterns and eliminate error propagation. This is mainly because causal event extraction is a knowledge-dependent reasoning task, and it also needs to model the interdependence between event extraction and causal event identification.

[0193] In addition, the present application also observes the performance of the baseline models. It can be found that the BERT-based methods are generally superior to the early feature-based methods. This is mainly because the BERT encoder can provide better representations for the input text.

[0194] Effect of subtask interaction and knowledge fusion:

[0195] As shown in Table 3, the present application studies the effectiveness of the subtask interaction and knowledge fusion mechanism designed by the present application. In the following, w / o is the abbreviation of without, which means that the module is missing. "w / o subtask interaction" means that in the last layer, the present application extracts the real labeled events instead of the extracted events into the relation module for training, and deletes the subtask information aggregator. "w / o knowledge fusion" means that there is no node in the initial background graph and the knowledge information aggregator is deleted. "w / o both" means that the two settings described above are applied at the same time. Table 3: Experimental results of subtask interaction and knowledge fusion mechanism in ECE task.

[0196] Table 3

[0197]

[0198] The present application can find that when the subtask interaction and knowledge fusion mechanism are removed, the F1 value of the model of the present application decreases by 4.63% and 5.84% respectively. This indicates the importance of subtask interaction and knowledge fusion to the causal event extraction task. This also indicates that the model of the present application can realize deep information exchange across two subtasks and two knowledge modes in the multi-layer framework of the present application. In addition, when the subtask interaction and knowledge fusion mechanism are removed at the same time, the F1 value decreases from 48.34% to 37.25%. This indicates that the interaction of subtasks and the fusion mechanism of knowledge can provide complementary contributions to the method of the present application. Table 4: Comparison of different direction interactions between two subtasks of the causal event extraction task. Table 5: Effect of different knowledge fusion components on the performance of the ECE task.

[0199] Table 4

[0200]

[0201] Table 5

[0202]

[0203] As shown in Table 4, the present application analyzes the effectiveness of subtask interaction in different directions on the causal event extraction task. "w / o causal event recognition to event extraction" means that the present application deletes the subtask information aggregator. "w / o event extraction to causal event extraction" means that the relationship module is trained with the true labeled events, rather than the events extracted by the event module. "w / o both" means that the present application applies the above two settings at the same time.

[0204] Compared with TKNet, the present application observes that the F1 values of "w / o causal event recognition to event extraction" and "w / o event extraction to causal event extraction" decrease by 1.88% and 2.53% respectively. Their similar performance loss levels show that both directions of subtask interaction are important for the causal event extraction task. This also shows that the subtask interaction mechanism designed by the present application enables effective information exchange between the two subtasks.

[0205] In addition, compared with "w / o both", the precision of the model of the present application increases by 7.48%, and the recall increases by 2.56%. This shows that the interaction between subtasks mainly helps the model of the present application to reduce the false extracted causal event pairs. In addition, the present application can observe that the "w / o both" model shows further performance degradation compared with "w / o causal event recognition to event extraction" and "w / o event extraction to causal event extraction". This is mainly because the two directions of subtask interaction can provide complementary benefits for the causal event extraction task.

[0206] As shown in Table 5, the present application analyzes the effectiveness of different knowledge fusion components on the performance of TKNet in the causal event extraction task. "w / o pre-trained language model to knowledge graph" means that in the last M layers, the fused node embedding (knowledge information aggregator output) is not used to update , i.e. information is not transferred from the pre-trained language model to the knowledge graph. Similarly, "w / o knowledge graph to pre-trained language model" means that the fused token representation, is not used to update , i.e. information is not transferred from the knowledge graph to the pre-trained language model. "w / o bidirectional relationship between knowledge graph and pre-trained language model" means that the present application deletes these knowledge information aggregators. "w / o insert inference" means that the present application does not insert the extracted events into the background graph, but directly inputs the event representation output by the pre-trained language model layer into the causal relationship classifier of the relationship module. "w / o all" means that the present application applies all the above settings at the same time.

[0207] First, this invention reveals that the TKNet of this invention outperforms the "w / o pre-trained language model to knowledge graph" and "w / o pre-trained language model to knowledge graph" approaches in F1 score by 1.84% and 1.08%, respectively. This indicates that both directions of knowledge fusion are important for causal event extraction tasks, primarily because the knowledge in the pre-trained language model and the knowledge graph can improve each other's encoding processes. For example, the pre-trained language model can help determine which knowledge in the external knowledge graph is important, while the knowledge graph can help improve the semantic understanding ability of the pre-trained language model. Furthermore, the further performance degradation of the "w / o knowledge graph to pre-trained language model bidirectional relationship" model suggests that these two knowledge fusion directions can provide complementary advantages for causal event extraction tasks.

[0208] Secondly, by removing the insertion derivation module, this invention reveals that the "w / o insertion derivation" model achieves a 3.38% lower F1 score compared to the TKNet of this invention. This indicates that event representations incorporating knowledge from both the pre-trained language model and the knowledge graph are more effective for causal event extraction than event representations containing only contextual information from the pre-trained language model. The main reason is that implicit causal relationships constitute the majority of causal relationships in the text, typically requiring the use of external knowledge from the knowledge graph for reasoning.

[0209] Finally, this invention reveals that the "w / o All" and "w / o Insertion Derivation" models exhibit similar performance to causal event extraction tasks. Compared to the "w / o All" and "w / o Insertion Derivation" models, the "w / o All" model shows a further decrease in its F1 score. This indicates that integrating both knowledge models, both in the knowledge encoding process and in the event representation, is crucial for causal event extraction tasks.

[0210] As shown in Table 6, Table 6 presents the initial background graphs of TKNet for different insertion extraction events; this invention analyzes the effectiveness of the proposed insertion derivation module. Specifically, this invention compares the proposed method with four other variants, as shown below:

[0211] No connection: In this method, the extracted events are not inserted into the background graph, and the event representations output by the pre-trained language model layer are directly input into the causal classifier of the relation module.

[0212] Fragment matching: If the extracted event and Nodes in The token fragments overlap in the input sentence. This invention establishes an edge between them, namely... It equals 1.

[0213] Full connection: the present invention establishes edges between all extracted events and knowledge graph nodes in , i.e. any value in is equal to 1.

[0214] Dot-product attention mechanism: the present invention replaces the dot-product attention mechanism with the insertion inference module to calculate the edge weight between the extracted event i and the knowledge graph node j .

[0215] Table 6

[0216]

[0217] Table 7

[0218]

[0219] Table 7 represents the performance of different layers of TKNet; from Table 7, the present invention can observe that the model of the present invention is significantly better than all four variants. For the two rule-based variants, the insertion inference module of the present invention has a 2.83% and 2.46% higher F1 value than "fragment matching" and "full connection". This shows that artificial rules cannot establish appropriate connections between extracted events and knowledge graph nodes mentioned in the input sentence. On the one hand, "fragment matching" relies on event token fragments to link relevant knowledge graph nodes; however, fragment matching of events usually only overlaps with a small number of nodes in the knowledge graph. On the other hand, "full connection" connects all knowledge graph nodes, which can introduce too many irrelevant knowledge and lead to an unreliable inference process.

[0220] In addition, the present invention can find that the insertion inference module of the present invention has a 2.21% higher F1 value than the "dot-product attention mechanism". This is mainly because the method of the present invention reasonably assumes that the extracted events and the knowledge graph nodes mentioned in the same sentence are subject to a certain syntactic dependency tree, and introduces this inductive bias according to Kirchhoff's matrix tree theorem.

[0221] In addition, the present invention can find that the method of the present invention is 3.39% better than the "no connection" model in F1 value. This shows that the insertion inference module designed by the present invention can effectively connect the extracted events with valuable knowledge stored in the external knowledge graph, and improve the performance of the causal event extraction task.

[0222] In order to facilitate sufficient information exchange between the two knowledge patterns and two sub-tasks of causal event extraction, the present invention adopts a multi-layer framework in the design of the model. In this section, the present invention will further explore the effectiveness of changing the number of fusion layers on the performance of TKNet. In order to conduct a more comprehensive analysis, the present invention considers both the intra-model and inter-model perspectives.

[0223] From an intra-model perspective, the present application conducts experiments to test the performance of the last M layers in TKNet ( ).

[0224] First, the present application observes that the performance of the model improves as the number of layers increases. This is because higher levels enable more comprehensive information exchange between the two knowledge patterns and the two sub-tasks of causal event extraction, thus helping to obtain causal relationships. In addition, higher layers have more parameters, which provide them with stronger encoding capabilities.

[0225] In addition, the F1 value of the second-to-last layer is 7.83% stronger than that of the third layer, and the last layer is 3.81% stronger than the second-to-last layer. In other words, as the number of fusion layers increases, the marginal benefit of adding an additional fusion layer decreases. This is mainly because as the number of fusion layers increases, the performance loss caused by excessive fusion becomes apparent. In addition, as the performance improves, the remaining error examples become more difficult to predict, reducing the marginal improvement in performance.

[0226] From an inter-model perspective, the present application conducts experiments to observe the performance of TKNet models with different hyperparameters . In the experiments, the present application synchronously adjusts the hyperparameters to ensure that the hyperparameters are equal to the number of layers in the pre-trained language model, i.e., 12 layers for BERT-Base. As Figure 3 shown.

[0227] First, the present application can find that the method of the present application achieves the best performance when using three fusion layers, and continuously improves as the hyperparameter increases from 1 layer to 3 layers. This is mainly because increasing the value of the hyperparameter can provide more sufficient information fusion, which helps to effectively capture causal relationships in text.

[0228] Second, when the number of fusion layers is too large ( ), the performance of the model of the present application will decrease. This is mainly because a too large hyperparameter M will cause excessive fusion, and the performance loss caused by excessive fusion exceeds the benefits. On the one hand, excessive information exchange between the pre-trained language model and the knowledge graph may damage their respective encoding processes, thereby reducing their ability to understand input sentences and integrate external knowledge. On the other hand, when the hyperparameter M is too large, the causal events extracted by the lower layers may contain too much noise, and providing these lower-level prediction results to the upper layers may interfere with their reasoning ability.

[0229] Finally, a large performance improvement can be observed between the best setting and the worst setting. This shows that the depth of information interaction is crucial to effectively improve the performance of the causal event extraction task. In addition, exploring how to reasonably integrate the two knowledge modes and two sub-tasks of causal event extraction is a promising direction for future research.

[0230] In order to analyze the complex causal event relationship extraction capability of the method of the present application, the present application tests the model on sentences containing different numbers of annotated causal event pairs. Figure 4 The performances of the best causal event extraction method based on pipeline DPJL, the method based on optimal sequence labeling SCITE and the TKNet proposed by the present application are shown.

[0231] First of all, the present application can find that the method of the present application is always superior to DPJL and SCITE in the number of all causal event pairs in each sentence. This strongly proves the effectiveness of the framework proposed by the present application in dealing with the problem of complex causal event relationship extraction.

[0232] Secondly, although both TKNet and DPJL use separate event extraction and causal event recognition decoders, which can theoretically handle all forms of multiple causal event pairs, TKNet of the present application is more effective than DPJL in extracting complex causal relationships. This is mainly because DPJL cannot simulate the interaction between the two sub-tasks, nor can it effectively combine the knowledge in the external knowledge graph with the pre-trained language model.

[0233] In addition, the present application can also find that the performance of DPJL on sentences with one or two causal event pairs is similar to that of SCITE, but the performance on sentences with more than three causal event pairs is better than that of SCITE. This is mainly because the artificial rules of SCITE cannot match the extracted causal relationships into pairs, which limits the performance of SCITE on sentences with multiple causal event pairs.

[0234] Finally, the present application can find that the performance of all three models decreases significantly as the number of causal event pairs increases. This shows that complex causal relationship extraction is a key problem of the causal event extraction task, and there is still much room for improvement. In addition, it also shows that if the present application can better solve this problem, the present application will make a breakthrough in causal event extraction.

[0235] In order to introduce external knowledge, the present application retrieves a subgraph from the external knowledge graph related to the input sentence and then uses it to initialize the background graph. Therefore, the size of the initial background graph is an important hyperparameter of the method of the present application. In this section, the present application uses initial background graphs with different numbers of nodes to experiment to test the performance of the method of the present application. The experimental results are shown in Figure 5 .

[0236] First, the present application can find that as the number of nodes increases from 0 to 50, the performance of the cause and effect event extraction task of the method of the present application is gradually improved. This is mainly because when the number of nodes is too low, the model cannot obtain enough external knowledge from the background graph. In addition, increasing the number of nodes introduces more useful external knowledge, thereby improving the causal reasoning ability of the model of the present application.

[0237] Second, the present application observes that as the number of nodes increases from 50 to 125, the performance of the model of the present application continues to decline. The model with 125 nodes even performs worse than the model with 0 nodes. This is because when the number of nodes is too large, the amount of irrelevant information in the background graph increases, increasing the difficulty of reasoning. In addition, as the number of nodes increases, the performance loss caused by redundant knowledge nodes becomes more and more significant. Finally, the present application finds that when the number of nodes is set to 50, the model of the present application achieves the best performance, which is significantly improved compared to the worst setting. This indicates that selecting an appropriate amount of external knowledge is crucial to the method of the present application, because too little knowledge will result in insignificant impact, and too much knowledge will bring unacceptable noise.

[0238] In the design of the model of the present application, the present application uses two information aggregators in the last M layers to exchange information between the two sub-tasks and the two knowledge modes. These aggregators are simply designed as two-layer multi-layer perceptrons. In this section, the present application will conduct experiments to analyze the effectiveness of different aggregators on the performance of the method of the present application. Table 8 shows the performance of different fusion methods, and the experimental results are shown in Table 8.

[0239] Table 8

[0240]

[0241] Variants of the aggregator include:

[0242] Mean aggregator: the present application simply takes the average of the pre-fusion representation as the post-fusion representation.

[0243] Single-layer and three-layer multi-layer perceptron aggregators: 1-layer and 3-layer multi-layer perceptron.

[0244] Self-attention mechanism aggregator: the present application inputs the pre-fusion representation into a self-attention mechanism module for information fusion, and uses the output vector obtained as the post-fusion representation.

[0245] First, the present application finds that the performance of the “mean aggregator” is significantly worse than other methods. This indicates that simply combining two pre-fusion representations can make it difficult for the present application to retain their respective semantics before aggregation, thereby disrupting the encoding process of the next layer of the TKNet of the present application.

[0246] The performance of the two-layer multi-layer perception machine in F1 is 1.85% better than that of the one-layer multi-layer perception machine, and the performance of the three-layer multi-layer perception machine in F1 is 0.53% lower than that of the two-layer multi-layer perception machine. On the one hand, the encoding ability of the one-layer multi-layer perception machine is weak due to the shallow encoding depth, and on the other hand, the three-layer multi-layer perception machine has an overfitting problem due to the excessive encoding depth.

[0247] Finally, it can be found that the F1 value of the self-attention mechanism aggregator is slightly higher than that of the two-layer multi-layer perception machine by 0.40%. This indicates that although the encoding ability of the aggregator is weak, it will reduce the performance of the method of the present application, but the use of different, stronger aggregators has a relatively small impact.

[0248] The above-described embodiments are only descriptions of the preferred modes of the present application and do not limit the scope of the present application. Without departing from the design spirit of the present application, various modifications and improvements to the technical solutions of the present application made by those skilled in the art shall fall within the protection scope determined by the claims of the present application.

Claims

1. A cause-effect event extraction method based on a cause-effect event extraction model, characterized in that, The method comprises the following steps: Based on event extraction and causal relationship identification two subtasks, build a causal event extraction model; Obtain the input sentence; Based on the pre-training language model, encode the input sentence, decode the encoded input sentence based on the sequence labeling decoder, and extract events with causal relationships; Based on the causal event extraction model, obtain the knowledge graph nodes of the input sentence and the preset hop neighbors of the knowledge graph nodes, and construct an initial background graph; Insert the event into the initial background graph to obtain an updated background graph; Using a graph neural network, encode and update the updated background graph to obtain the representation of the extracted event; Based on the classifier and the representation of the extracted event, obtain the causal relationship between the events, and realize the extraction of the causal event; The method for extracting events with causal relationships is: Based on the causal event extraction model, obtain the token representation of the input sentence; Input the token representation of the input sentence into the pre-training language model to obtain a pre-fused token representation; Input the pre-fused token representation into the sequence labeling decoder to extract events with causal relationships in the input sentence; The method for constructing an initial background graph is: Retrieving the external knowledge graph of the input sentence to obtain the knowledge graph nodes of the input sentence; Adding the preset hop neighbors of the knowledge graph nodes and any knowledge graph nodes on the path between any pair of knowledge graph nodes to obtain a node set; Input the connection between the input sentence and the knowledge graph nodes into the pre-training language model to output a node relevance score; Prune the node set based on the node relevance score; Connect all edges between the pruned node set and any node pair in the knowledge graph to the edges of the initial background graph to realize the construction of the initial background graph. 2.The cause-effect event extraction method based on the cause-effect event extraction model according to claim 1, wherein, The method for obtaining an updated background graph is: Add the extracted event as an event node to the initial background graph; Based on the union of the event nodes added to the initial background graph and the knowledge graph nodes, obtain a union node Based on the input representation of the union node, the weight matrix, and the linear transformation with the hyperbolic tangent activation function, calculate the joint representation of any pair of union nodes; Assign a non-negative score to the edges of the link structure between the union nodes in the union set to obtain an adjacency matrix; Assuming that the event nodes and the knowledge graph nodes are connected by a syntax dependency tree, calculate the non-normalized probability of the union nodes as the root of any syntax dependency tree to obtain a root score; Based on the adjacency matrix and the root score, obtain the Laplacian matrix of the link structure and its variants; Based on the Laplacian matrix, the adjacency matrix, the Kronecker function, and the matrix tree theorem, obtain a weighted adjacency matrix; Based on the weighted adjacency matrix, dynamically insert the extracted event into the initial background graph to obtain the updated background graph. 3.The cause-effect event extraction method based on the cause-effect event extraction model according to claim 2, characterized in that, The method for obtaining the representation of the extracted event is: Based on the fused knowledge graph node embedding, initializing the input embedding of the knowledge graph node and the node in the node set; Based on the pre-fused token representation, obtaining a pre-fused event context representation; Based on the pre-fused event context representation, initializing the input embedding of the event node; Inputting the initialized input embedding of the node in the updated background graph into the graph neural network to obtain a pre-fused node embedding; Based on the pre-fused node embedding and the initialized input embedding of the event node, jointly encoding the retrieved external knowledge and the extracted event to obtain the representation of the event. 4.The cause-effect event extraction method based on the cause-effect event extraction model according to claim 3, characterized in that, There is a mutual dependency relationship between the causal relationship identification and the event extraction; The dependency relationship includes: the dependency of the causal relationship identification on the event extraction; The event extraction depends on the causal relationship identification. 5.The cause-effect event extraction method based on the cause-effect event extraction model according to claim 4, characterized in that, The method for realizing the dependency of the event extraction on the causal relationship identification is: Connecting the pre-fused node embedding output by the graph neural network and the pre-fused event context representation output by the pre-trained language model to input a sub-task information aggregator to obtain a fused representation; Replacing the pre-fused event context representation with the fused representation to obtain a token representation that provides information about the results of the causal relationship identification for the event extraction, thereby realizing the dependency of the event extraction on the causal relationship identification. 6.The cause-effect event extraction method based on the cause-effect event extraction model according to claim 4, characterized in that, The dependency of the causal relationship identification on the event extraction uses a stack propagation framework to reduce error propagation.

Images