Event causal relationship identification method and system based on prompt learning
Patent Information
- Application Number
- CN202410484648.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-22
- Publication Date
- 2026-11-03
- Estimated Expiration
- 2044-04-22
AI Technical Summary
Existing technologies suffer from the problems of diverse causal expression forms and scarce training data in identifying causal relationships in events, resulting in large errors in the identification results. Furthermore, existing methods have failed to effectively address the loss of structured information and knowledge bias during knowledge introduction.
We employ a prompt-based learning approach, constructing modules through encoders and prompt-learning units, combining knowledge graphs and graph convolutional neural networks to perform knowledge completion and interactive fusion, constructing a causal memory space, and fine-tuning the representation vectors to improve recognition accuracy.
It enhances the accuracy of event causal relationship identification by integrating external knowledge and internal memory, thereby mitigating the semantic inconsistency between knowledge representation and event representation and improving the recognition effect.
Smart Images
Figure CN118396027B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of natural language processing technology, specifically relating to a method and system for identifying event causal relationships based on prompting learning. Background Technology
[0002] Event Causality Recognition (ECI) is a task that identifies causal relationships between events from text. In the field of Natural Language Processing (NLP), ECI is an important task that supports many NLP applications, such as machine reading comprehension, process extraction, and future event prediction. However, research on ECI faces challenges such as the diversity of causal expressions and the scarcity of training data. While the diversity of causal expressions can often be addressed by introducing end-to-end neural network models to achieve global autonomous learning, the relatively small size of existing ECI datasets is insufficient to train a neural network capable of fully understanding the context of events.
[0003] Therefore, existing technologies can utilize Trainable Language Models (PLMs) to address the aforementioned problems. Inspired by the human ability to combine existing knowledge related to two events when reasoning about causality, PLM models can incorporate external knowledge to improve causal relationship identification. However, a gap exists between the pre-training and fine-tuning stages of a PLM model. Specifically, the low-level features of the ECI-related data input into the PLM model during fine-tuning differ from the final top-level features learned from the data during pre-training. Consequently, existing technologies do not consider the loss of structured information or knowledge bias when introducing external knowledge, leading to potentially large errors in the final causal relationship identification process. Summary of the Invention
[0004] This invention provides a method and system for identifying causal relationships between events based on cue learning, in order to solve the problem of large errors in the identification results in the task of identifying causal relationships between events.
[0005] In a first aspect, the present invention provides a method for identifying event causal relationships based on cue learning, the method comprising the following steps:
[0006] Obtain a target text instance for event causality identification, wherein the target text instance contains a target event pair consisting of a first event representation and a second event representation, and both the first event representation and the second event representation carry prompt information;
[0007] A prompting learning module is constructed using an encoder and a prompting learning unit based on reinforcement understanding context representation. The target text instance is then input into the prompting learning module to obtain the basic content representation vector of the target text instance, the comprehensive prompting learning representation vector, and the initial event causal relationship recognition result of the target text instance.
[0008] Based on the initial event causal relationship identification results, all related event pairs with causal relationships in the target text instance are stored in a preset causal event memory space;
[0009] Using a pre-defined knowledge graph and knowledge base, knowledge nodes are searched and completed for the first event representation and the second event representation, respectively, and a first knowledge network corresponding to the first event representation and a second knowledge network corresponding to the second event representation are constructed.
[0010] The first knowledge network and the second knowledge network are convolutionally computed by a preset graph convolutional neural network to obtain the first external knowledge encoding vector corresponding to the first event representation and the second external knowledge encoding vector corresponding to the second event representation. The first external knowledge encoding vector and the second external knowledge encoding vector are concatenated to obtain the comprehensive knowledge representation vector of the target event pair.
[0011] Based on a preset transformation matrix, the basic content representation vector and the comprehensive knowledge representation vector are debiased and fused using a linear interactive fusion method. The debiased and fused representation vector is then concatenated with the comprehensive prompt learning representation vector to obtain the fused representation vector.
[0012] Based on the event similarity between the target event pair and each of the associated event pairs in the causal event memory space, a first causal similarity memory graph and a second causal similarity memory graph corresponding to the first event representation and the second event representation are constructed by combining the target event pair and the causal event memory space, respectively.
[0013] By combining the first causal similarity memory map and the second causal similarity memory map, and using information aggregation to fine-tune the representation vector parts corresponding to the first event representation and the second event representation in the fused representation vector, the fine-tuned final fused representation vector is obtained;
[0014] The final fused representation vector is input into a preset binary classifier to obtain the final event causal relationship recognition result of the target text instance.
[0015] Optionally, the cue learning unit based on reinforced understanding of context representation includes a first cue learning unit and a second cue learning unit. The first cue learning unit is used to establish a mapping relationship between the target event pairs in the target text instance and the context representation. The second cue learning unit is used to infer cue words in the target text instance that represent the causal relationship of events and to analyze the semantic role of each event representation in the target text instance.
[0016] Optionally, the step of constructing a cue learning module using an encoder and a cue learning unit based on reinforcement understanding context representation, and inputting the target text instance into the cue learning module to obtain the basic content representation vector, the comprehensive cue learning representation vector, and the initial event causality recognition result of the target text instance includes the following steps:
[0017] A prompting learning module is constructed by combining the first prompting learning unit, the second prompting learning unit, and the encoder;
[0018] After inserting preset start and end tags into the target text instance, the target text instance is input into the prompt learning module;
[0019] The encoder encodes the target text instance into a basic content representation vector.
[0020] The first prompt learning unit processes the target text instance to obtain a first prompt learning instance containing a mask region.
[0021] The range of values for all events represented in the mask region is limited by a preset set of causal results;
[0022] The first prompt learning unit constructs a causal mapping relationship between the causal result set and the preset causal labels, and calculates the causal relationship values of all events in the mask area based on the causal mapping relationship. The causal relationship values of all events in the mask area are used as the initial event causal relationship identification results of the target text instance.
[0023] The target text instance is processed by the second prompt learning unit to obtain a second prompt learning instance;
[0024] The first cue learning instance and the second cue learning instance are concatenated to form a comprehensive cue learning instance, and the comprehensive cue learning instance is encoded into a comprehensive cue learning representation vector by the encoder.
[0025] Optionally, the step of using a preset knowledge graph and knowledge base to find and complete the knowledge nodes of the first event representation and the second event representation, and constructing a first knowledge network corresponding to the first event representation and a second knowledge network corresponding to the second event representation, includes the following steps:
[0026] Use a pre-defined knowledge graph to find the knowledge nodes of the first event representation and the second event representation;
[0027] If there are knowledge nodes that cannot be found using the knowledge graph, then the knowledge base is used to complete the knowledge nodes of the first event representation and / or the second event representation;
[0028] A first knowledge network is constructed by combining the first event representation and all knowledge nodes corresponding to the first event representation, and a first adjacency matrix is constructed based on the connection relationship of each node in the first knowledge network.
[0029] A second knowledge network is constructed by combining the second event representation and all knowledge nodes corresponding to the second event representation, and a second adjacency matrix is constructed based on the connection relationships of each node in the second knowledge network.
[0030] Optionally, the step of de-biasing and fusing the basic content representation vector and the comprehensive knowledge representation vector based on a preset transformation matrix and using a linear interactive fusion method, and then concatenating the de-biased and fused representation vector with the comprehensive prompt learning representation vector to obtain the fused representation vector includes the following steps:
[0031] The affinity matrix is calculated based on the preset transformation matrix and in combination with the basic content representation vector and the comprehensive knowledge representation vector;
[0032] Based on the affinity matrix, the basic content representation vector and the comprehensive knowledge representation vector are respectively debiased and fused using a linear transformation method to obtain the debiased basic content representation vector and the debiased comprehensive knowledge representation vector.
[0033] The bias-free basic content representation vector, the bias-free comprehensive knowledge representation vector, and the comprehensive prompt learning representation vector are concatenated into a fusion representation vector.
[0034] Optionally, the affinity matrix is calculated using the following formula:
[0035]
[0036] In the formula: T represents the affinity matrix, Indicates matrix transpose. The basic content represents a vector, and W represents the transformation matrix. This represents the comprehensive knowledge representation vector. This represents the first external knowledge encoding vector. This represents the second external knowledge encoding vector;
[0037] The formula for calculating the basic content representation vector after bias removal is as follows:
[0038]
[0039] In the formula: This represents the basic content representation vector after the bias removal process.
[0040] The formula for calculating the bias-reduced comprehensive knowledge representation vector is as follows:
[0041]
[0042] In the formula: The vector represents the biased comprehensive knowledge representation.
[0043] Optionally, the transformation matrix contains multiple learnable parameters, and the method further includes the following steps:
[0044] The first knowledge network and the second knowledge network are combined into a comprehensive knowledge network, and the comprehensive knowledge network is textualized into a knowledge network representation.
[0045] The knowledge network representation and the bias-free comprehensive knowledge representation vector are combined to construct an event context training sequence with knowledge prompts based on the target text instance;
[0046] The knowledge nodes in the comprehensive knowledge network are predicted by training sequences through the event context, and all the learnable parameters are optimized during the prediction process.
[0047] Optionally, each of the associated event pairs includes a first associated event representation and a second associated event representation. The step of constructing a first causal similarity memory map and a second causal similarity memory map corresponding to the first event representation and the second event representation respectively, based on the event similarity between the target event pair and each of the associated event pairs in the causal event memory space, and combining the target event pair and the causal event memory space, includes the following steps:
[0048] Calculate the first event similarity between the first event representation and each of the first associated event representations in the causal event memory space;
[0049] The representations of each of the first associated events in the causal event memory space are sorted in descending order of the similarity of the first events;
[0050] A predetermined number of the first associated event representations are read from the causal event memory space according to the sorted order;
[0051] The first event is represented as the first root node, all the first associated events are represented as the first child nodes, and the first event similarity between the first root node and each of the first child nodes is used as the first edge weight to construct the node edge between the first root node and each of the first child nodes, thus forming a first causal similarity memory graph.
[0052] Calculate the second event similarity between the second event representation and each of the second associated event representations in the causal event memory space;
[0053] The representations of each of the second related events in the causal event memory space are sorted in descending order of the similarity of the second events;
[0054] A predetermined number of the second associated event representations are read from the causal event memory space according to the sorted order;
[0055] The second event is represented as the second root node, and all the second associated events are represented as the second child nodes. The node edges between the second root node and each of the second child nodes are constructed using the similarity of the second events between the second root node and each of the second child nodes as the second edge weights, thus forming a second causal similarity memory graph.
[0056] Optionally, the step of combining the first causal similarity memory map and the second causal similarity memory map, and fine-tuning the representation vector portions corresponding to the first event representation and the second event representation in the fused representation vector using an information aggregation method to obtain the fine-tuned final fused representation vector includes the following steps:
[0057] The first symmetric weight matrix of the first causal similarity memory graph is calculated based on the first edge weight.
[0058] The second symmetric weight matrix of the second causal similarity memory graph is calculated based on the second edge weight.
[0059] The first fine-tuned representation vector corresponding to the first event representation is calculated by combining the first causal similarity memory graph and the first symmetric weight matrix, and the first fine-tuned representation vector is used to replace the representation vector part corresponding to the first event representation in the fused representation vector.
[0060] The second fine-tuned representation vector corresponding to the second event representation is calculated by combining the second causal similarity memory graph and the second symmetric weight matrix. The second fine-tuned representation vector is then used to replace the representation vector part corresponding to the second event representation in the fused representation vector to obtain the fine-tuned final fused representation vector.
[0061] In a second aspect, the present invention also provides an event causality recognition system based on cue learning, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the event causality recognition method based on cue learning as described in the first aspect.
[0062] The beneficial effects of this invention are:
[0063] The event causal relationship recognition method based on cue learning provided in this invention mainly offers a cue learning framework with knowledge fusion and causal memory. It enhances the content and representation of events by integrating external knowledge and internal causal memory. To address biases in the introduction of external knowledge, this invention designs an interactive fusion mechanism. Compared with traditional methods, this mechanism has a strong enhancement effect on event causal relationship recognition, thus considering the loss of structured information while introducing external knowledge. Furthermore, a causal memory space is constructed based on known events with embedded causal relationships to adjust event representations, mitigating the semantic inconsistency between knowledge representation and event representation, ultimately improving the accuracy of event causal relationship recognition. Attached Figure Description
[0064] Figure 1 This is a flowchart illustrating the steps of an event causal relationship recognition method based on prompting learning in one embodiment of this application.
[0065] Figure 2 This is a schematic diagram of the framework of an event causal relationship identification method based on prompting learning in one embodiment of this application. Detailed Implementation
[0066] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.
[0067] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.
[0068] Figure 1 This is a flowchart illustrating an event causality recognition method based on cue learning in one embodiment. It should be understood that, although... Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps. For example Figure 1 As shown, the event causal relationship recognition method based on cue learning disclosed in this invention specifically includes the following steps:
[0069] S101. Obtain the target text instance for which event causality identification is required.
[0070] The target text instance S contains a target event pair consisting of a first event representation e1 and a second event representation e2, both of which carry prompt information.
[0071] S102. Construct a prompt learning module using an encoder and a prompt learning unit based on reinforcement understanding context representation, and input the target text instance into the prompt learning module to obtain the basic content representation vector of the target text instance, the comprehensive prompt learning representation vector, and the initial event causal relationship recognition result of the target text instance.
[0072] S103. Based on the initial event causal relationship identification results, store all related event pairs with causal relationships in the target text instance in the preset causal event memory space.
[0073] S104. Use the preset knowledge graph and knowledge base to find and complete the knowledge nodes of the first event representation and the second event representation respectively, and construct the first knowledge network corresponding to the first event representation and the second knowledge network corresponding to the second event representation.
[0074] S105. Perform convolution calculations on the first knowledge network and the second knowledge network respectively through a preset graph convolutional neural network to obtain the first external knowledge encoding vector corresponding to the first event representation and the second external knowledge encoding vector corresponding to the second event representation. Concatenate the first external knowledge encoding vector and the second external knowledge encoding vector to obtain the comprehensive knowledge representation vector of the target event pair.
[0075] S106. Based on the preset transformation matrix, the basic content representation vector and the comprehensive knowledge representation vector are de-biased and fused using a linear interactive fusion method. The de-biased and fused representation vector is then concatenated with the comprehensive prompt learning representation vector to obtain the fused representation vector.
[0076] S107. Based on the event similarity between the target event pair and each associated event pair in the causal event memory space, construct the first causal similarity memory map and the second causal similarity memory map corresponding to the first event representation and the second event representation, respectively.
[0077] S108. Combining the first causal similarity memory map and the second causal similarity memory map, and using information aggregation to fine-tune the representation vector parts corresponding to the first event representation and the second event representation in the fused representation vector, the final fused representation vector after fine-tuning is obtained.
[0078] S109. Input the final fused representation vector into a preset binary classifier to obtain the final event causal relationship recognition result of the target text instance.
[0079] In one embodiment, the cue learning unit based on reinforced understanding of context representation includes a first cue learning unit and a second cue learning unit. The first cue learning unit is used to establish a mapping relationship between target event pairs and context representations in the target text instance. The second cue learning unit is used to infer cue words in the target text instance that represent causal relationships between events and to analyze the semantic role of each event representation in the target text instance.
[0080] In this embodiment, step S102 specifically includes the following steps:
[0081] A prompting learning module is constructed by combining the first prompting learning unit, the second prompting learning unit, and the encoder;
[0082] After inserting preset start and end tags into the target text instance, input the target text instance into the prompt learning module;
[0083] The target text instance is encoded into a basic content representation vector using an encoder;
[0084] The target text instance is processed by the first prompt learning unit to obtain the first prompt learning instance containing the mask region;
[0085] The range of values for all events represented in the mask region is limited by a pre-defined set of causal results;
[0086] The first prompt learning unit constructs a causal mapping relationship between the causal result set and the preset causal labels, and calculates the causal relationship values of all events in the mask area based on the causal mapping relationship. The causal relationship values of all events in the mask area are used as the initial event causal relationship recognition results of the target text instance.
[0087] The target text instance is processed by the second prompt learning unit to obtain the second prompt learning instance;
[0088] The first cue learning instance and the second cue learning instance are concatenated into a comprehensive cue learning instance, and the comprehensive cue learning instance is encoded into a comprehensive cue learning representation vector by an encoder.
[0089] In this embodiment, the preset causal label is Y = {yes, no}, and the preset causal result set is V: {is, isn't}. (Refer to...) Figure 2 To ensure the encoder in the cue learning module fully understands the context of events, a cue learning unit based on reinforced understanding of contextual representations is used to construct the module. This helps the encoder better understand the task to be completed during the fine-tuning phase. The cue learning unit consists of two parts: a first cue learning unit (PT1) and a second cue learning unit (PT2). The first cue learning unit (PT1) establishes a mapping between target event pairs and contextual representations in the target text instance during training, helping to obtain better contextual representations. The second cue learning unit (PT2) infers cue words representing causal relationships in the target text instance, thus fully utilizing explicit cues in the sentence. It is also used to analyze the semantic role of each event representation in the target text instance, thereby fully uncovering hidden cues within the target text instance.
[0090] After inserting the preset start tag [CLS] and end tag [SEP] into the target text instance, you get the target text instance containing the start tag and end tag. target text instance The input is fed into the prompting learning module, where the target text instance is processed by the first prompting learning unit to obtain a first prompting learning instance PT1(X) containing a mask region [Mask]. The values of the mask region [Mask] in the first prompting learning instance PT1(X) are restricted to the causal result set V: {is, isn't}, and causal labels are associated with the causal result set using a mapping f: Y→V. Then, the causal relationship values representing all events in the mask region are calculated based on the causal mapping relationship, using the following formula:
[0091]
[0092] In the formula: v represents an element in the set of causal results.
[0093] Then, the target text instance is processed through the second prompting learning unit PT2. The second cue learning instance PT2(X) is obtained, and a comprehensive cue learning instance is obtained through concatenation. Finally, the target text instance is... The combined cue learning instances are input into the encoder (such as RoBERTa) for training and encoding, resulting in the following representation of the cue learning phase:
[0094]
[0095] in, This is the encoded result of the target text instance, i.e., the basic content representation vector. and These are the encoding results for the first and second prompt learning examples, respectively.
[0096] In one embodiment, step S104 specifically includes the following steps:
[0097] Use a pre-defined knowledge graph to find knowledge nodes representing the first and second events;
[0098] If there are knowledge nodes that cannot be found using the knowledge graph, then the knowledge nodes representing the first event and / or the second event are completed using the knowledge base;
[0099] A first knowledge network is constructed by combining the first event representation and all knowledge nodes corresponding to the first event representation, and a first adjacency matrix is constructed based on the connection relationship of each node in the first knowledge network.
[0100] A second knowledge network is constructed by combining the second event representation and all the knowledge nodes corresponding to the second event representation, and a second adjacency matrix is constructed based on the connection relationship of each node in the second knowledge network.
[0101] In this embodiment, taking the first event representation as an example, the system first searches for knowledge nodes related to the first event representation in a preset knowledge graph (such as ConceptNet). Specifically, it searches for knowledge nodes in the knowledge graph based on 19 semantic relationships related to causal judgments. If a knowledge node cannot be found in the knowledge graph, it can be supplemented using a preset knowledge base (such as COMET) based on the given event and known semantic relationships. Finally, the first knowledge network is constructed by combining the first event representation e1 and all the knowledge nodes corresponding to the first event representation. set up Composed of n nodes, an n×n first adjacency matrix can be constructed based on the connection relationships between the nodes in the first knowledge network. in Each element in is:
[0102]
[0103] The second event represents the second knowledge network corresponding to e2. and the second adjacency matrix The construction methods are all the same as those for the first event representation.
[0104] In one implementation, corresponding to step S105, in order to preserve the structural information of the external knowledge network for each event, this implementation uses graph convolutional neural networks (GCNs). For the first knowledge network... Or the Second Knowledge Network Node i in the graph convolutional neural network performs the following convolution calculation in the l-th layer:
[0105]
[0106] In the formula: and Let be the weight matrix and bias of the l-th layer, respectively, and σ(·) be the activation function (e.g., ReLU). Let j be the representation vector of the adjacent node j at the (l-1)th layer. The initial representation vector of node i can be obtained by encoding using a pre-trained model (such as BERT).
[0107] Finally, the first external knowledge encoding vector representing the first event can be obtained using the above convolution calculation formula. The second external knowledge encoding vector of the second event representation The two are concatenated to obtain the comprehensive knowledge representation vector of the target event pair (e1, e2). Right now This indicates a splicing operation.
[0108] In one embodiment, step S106 specifically includes the following steps:
[0109] The affinity matrix is calculated based on the preset transformation matrix and the basic content representation vector and the comprehensive knowledge representation vector.
[0110] Based on the affinity matrix and using a linear transformation method, the basic content representation vector and the comprehensive knowledge representation vector are respectively debiased and fused to obtain the debiased basic content representation vector and the debiased comprehensive knowledge representation vector.
[0111] The bias-free basic content representation vector, the bias-free comprehensive knowledge representation vector, and the comprehensive prompt learning representation vector are concatenated into a fused representation vector.
[0112] In this embodiment, the formula for calculating the affinity matrix is as follows:
[0113]
[0114] In the formula: T represents the affinity matrix, Indicates matrix transpose. The basic content is represented by a vector, and W represents the transformation matrix. Representing a vector as a representation of comprehensive knowledge. This represents the first external knowledge encoding vector. This represents the second external knowledge encoding vector.
[0115] The formula for calculating the bias-reduced basic content representation vector is as follows:
[0116]
[0117] In the formula: This represents a vector with the basic content removed from the bias representation. The basic content representation vectors are the semantically debiased representation vectors.
[0118] The formula for calculating the partial-reduction comprehensive knowledge representation vector is as follows:
[0119]
[0120] In the formula: This represents a vector representing the knowledge obtained through partial decomposition and synthesis.
[0121] Finally, the bias-free basic content representation vector, the bias-free comprehensive knowledge representation vector, and the comprehensive prompt learning representation vector are concatenated into a fused representation vector.
[0122] In one implementation, the transformation matrix W contains multiple learnable parameters Θ. W The method also includes the following steps:
[0123] The first and second knowledge networks are combined into a comprehensive knowledge network, and the comprehensive knowledge network is textualized into a knowledge network representation.
[0124] By combining knowledge network representation and bias-removed comprehensive knowledge representation vectors, an event context training sequence with knowledge prompts based on target text instances is constructed.
[0125] Predict knowledge nodes in a comprehensive knowledge network by training sequences based on event contexts, and optimize all learnable parameters during the prediction process.
[0126] In this embodiment, although the linear transformation method based on the transformation matrix can eliminate the ambiguity between cue embedding and knowledge embedding, the introduction of a transformation matrix containing some learnable parameters necessitates maximizing the optimization of these learnable parameters. Therefore, the first and second knowledge networks are first combined into a comprehensive knowledge network, and the comprehensive knowledge network is then textualized into a knowledge network representation. Then, combined with the knowledge network representation And biased comprehensive knowledge representation vector Constructing event context training sequences with knowledge hints based on target text instances. The prediction probability formula for knowledge node g in the comprehensive knowledge network is as follows: The sequence is trained using event context to predict knowledge nodes.
[0127]
[0128] In the formula: Through pooling Learn from the contextual representations of all symbols in p g It is the fused representation of knowledge node g, Θ PLM These are fixed parameters.
[0129] In one embodiment, each pair of associated events includes a first associated event representation and a second associated event representation. Step S107 specifically includes the following steps:
[0130] Calculate the first event similarity between the first event representation and each first associated event representation in the causal event memory space;
[0131] Sort the representations of each first associated event in the causal event memory space according to the order of first event similarity from largest to smallest;
[0132] Read a predetermined number of first related event representations from the causal event memory space according to the sorted order;
[0133] The first event is represented as the first root node, and all first associated events are represented as the first child nodes. The first event similarity between the first root node and each first child node is used as the first edge weight to construct the node edges between the first root node and each first child node, forming the first causal similarity memory graph.
[0134] Calculate the second event similarity between the second event representation and each second associated event representation in the causal event memory space;
[0135] The representations of each second-related event in the causal event memory space are sorted in descending order of the similarity of the second event;
[0136] A predetermined number of second related event representations are read from the causal event memory space according to the sorted order;
[0137] The second event is represented as the second root node, and all the second related events are represented as the second child nodes. The node edges between the second root node and each second child node are constructed using the similarity of the second events between the second root node and each second child node as the weight of the second edge, thus forming the second causal similarity memory graph.
[0138] In this embodiment, a causal event memory space is defined in step S103. This space records the event representations that have appeared in a causal relationship. Specifically, it stores the representation vectors of event pairs whose mask region [Mask] value is "is" in the initial event causal relationship identification results.
[0139] After knowledge fusion, to further refine and adjust the event representation, taking the first event representation as an example, the similarity between the first event representation and each first associated event representation in the causal event memory space is calculated. The first associated event representations in the causal event memory space are then sorted in descending order of first event similarity. Based on this sorted order, m (a preset number) first associated event representations are read from the causal event memory space. The first event representation is then used as the first root node, and the m first associated event representations are used as the first child nodes, resulting in the first node set. Construct node edges between the first root node and each of its first child nodes using the similarity of the first event between the first root node and each of its first child nodes as the weight of the first edge. Let be the set of edges between the first root node and each of its first child nodes, based on... and The definition forms the first causal similarity memory map The first causal similarity memory graph is a weighted graph, and its weight matrix is... The calculation method is as follows:
[0140]
[0141] In the formula: Representing nodes in the first causal similarity memory graph and nodes The cosine similarity between them is calculated using the following formula:
[0142]
[0143] The second causal similarity memory map corresponding to the second event can be obtained using the same process described above.
[0144] In one embodiment, step S108 specifically includes the following steps:
[0145] The first symmetric weight matrix of the first causal similarity memory graph is calculated based on the first edge weight.
[0146] The second symmetric weight matrix of the second causal similarity memory graph is calculated based on the weight of the second side.
[0147] The first fine-tuned representation vector corresponding to the first event representation is calculated by combining the first causal similarity memory graph and the first symmetric weight matrix. The first fine-tuned representation vector is then used to replace the representation vector part corresponding to the first event representation in the fused representation vector.
[0148] The second fine-tuned representation vector corresponding to the second event representation is calculated by combining the second causal similarity memory graph and the second symmetric weight matrix. The second fine-tuned representation vector is then used to replace the representation vector part corresponding to the second event representation in the fused representation vector to obtain the fine-tuned final fused representation vector.
[0149] In this embodiment, based on the first causal similarity memory graph and the second causal similarity memory graph, the information of the first associated event representation and the second associated event representation are aggregated into the first event representation and the second event representation, respectively, so as to obtain the final fine-tuning result of the event representation.
[0150] Specifically, taking the representation of the first event as an example, the first symmetric weight matrix of the first causal similarity memory graph is first calculated based on the weight of the first edge. The calculation formula is as follows:
[0151]
[0152] In the formula: It is a diagonal matrix, and the diagonal elements are calculated in a way similar to that of a degree matrix:
[0153]
[0154] Then, by combining the first causal similarity memory graph and the first symmetric weight matrix, and using a GCN with two layers, the first fine-tuned representation vector corresponding to the first event representation is calculated. First fine-tuning representation vector The calculation formula is as follows:
[0155]
[0156] In the formula: X1 is the representation vector of each node in the first causal similarity memory graph, W0 is the first layer training weight vector of GCN, W1 is the second layer training weight vector of GCN, and ReLU(·) is the activation function, whose expression is:
[0157] ReLU(x) = max(0, x)
[0158] The second fine-tuned representation vector corresponding to the second event representation can be obtained using the same process described above. Finally, the first fine-tuned representation vector replaces the representation vector part corresponding to the first event representation in the fused representation vector, and the second fine-tuned representation vector replaces the representation vector part corresponding to the second event representation in the fused representation vector, resulting in the final fine-tuned fused representation vector.
[0159] The present invention also discloses an event causality recognition system based on cue learning, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the event causality recognition method based on cue learning as described in any of the above embodiments.
[0160] The processor can be a central processing unit (CPU). Of course, depending on the actual use, it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc., and this application does not limit it in this regard.
[0161] The memory can be an internal storage unit of a computer device, such as a hard disk or RAM, or an external storage device, such as a plug-in hard disk, smart memory card (SMC), secure digital card (SD), or flash memory card (FC) provided on the computer device. Furthermore, the memory can be a combination of internal storage units and external storage devices of a computer device. The memory is used to store computer programs and other programs and data required by the computer device. The memory can also be used to temporarily store data that has been output or will be output. This application does not limit this.
[0162] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of protection of this application is limited to these examples; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of one or more embodiments of this application as described above, which are not provided in detail for the sake of brevity.
[0163] One or more embodiments in this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of this application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments in this application should be included within the protection scope of this application.
Claims
1. A method for identifying event causal relationships based on cue learning, characterized in that, Includes the following steps: Obtain a target text instance for event causality identification, wherein the target text instance contains a target event pair consisting of a first event representation and a second event representation, and both the first event representation and the second event representation carry prompt information; A prompting learning module is constructed using an encoder and a prompting learning unit based on reinforcement understanding context representation. The target text instance is then input into the prompting learning module to obtain the basic content representation vector of the target text instance, the comprehensive prompting learning representation vector, and the initial event causal relationship recognition result of the target text instance. Based on the initial event causal relationship identification results, all related event pairs with causal relationships in the target text instance are stored in a preset causal event memory space; Using a pre-defined knowledge graph and knowledge base, knowledge nodes are searched and completed for the first event representation and the second event representation, respectively, and a first knowledge network corresponding to the first event representation and a second knowledge network corresponding to the second event representation are constructed. The first knowledge network and the second knowledge network are convolutionally computed by a preset graph convolutional neural network to obtain the first external knowledge encoding vector corresponding to the first event representation and the second external knowledge encoding vector corresponding to the second event representation. The first external knowledge encoding vector and the second external knowledge encoding vector are concatenated to obtain the comprehensive knowledge representation vector of the target event pair. Based on a preset transformation matrix, the basic content representation vector and the comprehensive knowledge representation vector are debiased and fused using a linear interactive fusion method. The debiased and fused representation vector is then concatenated with the comprehensive prompt learning representation vector to obtain the fused representation vector. Based on the event similarity between the target event pair and each of the associated event pairs in the causal event memory space, a first causal similarity memory graph and a second causal similarity memory graph corresponding to the first event representation and the second event representation are constructed by combining the target event pair and the causal event memory space, respectively. By combining the first causal similarity memory map and the second causal similarity memory map, and by using information aggregation to fine-tune the representation vector parts corresponding to the first event representation and the second event representation in the fused representation vector, the fine-tuned final fused representation vector is obtained; The final fused representation vector is input into a preset binary classifier to obtain the final event causal relationship recognition result of the target text instance.
2. The event causal relationship recognition method based on cue learning according to claim 1, characterized in that, The cue learning unit based on reinforced understanding of context representation includes a first cue learning unit and a second cue learning unit. The first cue learning unit is used to establish a mapping relationship between the target event pairs in the target text instance and the context representation. The second cue learning unit is used to infer the cue words in the target text instance that represent the causal relationship of events and to analyze the semantic role of each event representation in the target text instance.
3. The event causal relationship recognition method based on cue learning according to claim 2, characterized in that, The step of constructing a prompting learning module using an encoder and a prompting learning unit based on reinforced understanding of context representation, and inputting the target text instance into the prompting learning module to obtain the basic content representation vector of the target text instance, the comprehensive prompting learning representation vector, and the initial event causality recognition result of the target text instance includes the following steps: A prompting learning module is constructed by combining the first prompting learning unit, the second prompting learning unit, and the encoder; After inserting preset start and end tags into the target text instance, the target text instance is input into the prompt learning module; The encoder encodes the target text instance into a basic content representation vector. The first prompt learning unit processes the target text instance to obtain a first prompt learning instance containing a mask region. The range of values for all events represented in the mask region is limited by a preset set of causal results; The first prompt learning unit constructs a causal mapping relationship between the causal result set and the preset causal labels, and calculates the causal relationship values of all events in the mask area based on the causal mapping relationship. The causal relationship values of all events in the mask area are used as the initial event causal relationship identification results of the target text instance. The target text instance is processed by the second prompt learning unit to obtain a second prompt learning instance; The first cue learning instance and the second cue learning instance are concatenated to form a comprehensive cue learning instance, and the comprehensive cue learning instance is encoded into a comprehensive cue learning representation vector by the encoder.
4. The event causal relationship recognition method based on cue learning according to claim 1, characterized in that, The step of using a preset knowledge graph and knowledge base to find and complete the knowledge nodes of the first event representation and the second event representation, and constructing the first knowledge network corresponding to the first event representation and the second knowledge network corresponding to the second event representation, includes the following steps: Use a pre-defined knowledge graph to find the knowledge nodes of the first event representation and the second event representation; If there are knowledge nodes that cannot be found using the knowledge graph, then the knowledge base is used to complete the knowledge nodes of the first event representation and / or the second event representation; A first knowledge network is constructed by combining the first event representation and all knowledge nodes corresponding to the first event representation, and a first adjacency matrix is constructed based on the connection relationship of each node in the first knowledge network. A second knowledge network is constructed by combining the second event representation and all knowledge nodes corresponding to the second event representation, and a second adjacency matrix is constructed based on the connection relationships of each node in the second knowledge network.
5. The event causal relationship recognition method based on cue learning according to claim 1, characterized in that, The process of fusing the basic content representation vector and the comprehensive knowledge representation vector based on a preset transformation matrix and using a linear interactive fusion method to remove bias and then concatenating the fused representation vector with the comprehensive prompting learning representation vector to obtain the fused representation vector includes the following steps: The affinity matrix is calculated based on the preset transformation matrix and in combination with the basic content representation vector and the comprehensive knowledge representation vector; Based on the affinity matrix, the basic content representation vector and the comprehensive knowledge representation vector are respectively debiased and fused using a linear transformation method to obtain the debiased basic content representation vector and the debiased comprehensive knowledge representation vector. The bias-free basic content representation vector, the bias-free comprehensive knowledge representation vector, and the comprehensive prompt learning representation vector are concatenated into a fusion representation vector.
6. The event causal relationship recognition method based on cue learning according to claim 5, characterized in that, The formula for calculating the affinity matrix is as follows: In the formula: T represents the affinity matrix, Indicates matrix transpose. The basic content represents a vector, and W represents the transformation matrix. This represents the comprehensive knowledge representation vector. This represents the first external knowledge encoding vector. This represents the second external knowledge encoding vector; The formula for calculating the basic content representation vector after bias removal is as follows: In the formula: This represents the basic content representation vector after the bias removal process. The formula for calculating the bias-reduced comprehensive knowledge representation vector is as follows: In the formula: The vector represents the biased comprehensive knowledge representation.
7. The event causal relationship recognition method based on cue learning according to claim 5, characterized in that, The transformation matrix contains multiple learnable parameters, and the method further includes the following steps: The first knowledge network and the second knowledge network are combined into a comprehensive knowledge network, and the comprehensive knowledge network is textualized into a knowledge network representation. The knowledge network representation and the bias-free comprehensive knowledge representation vector are combined to construct an event context training sequence with knowledge prompts based on the target text instance; The knowledge nodes in the comprehensive knowledge network are predicted by training sequences through the event context, and all the learnable parameters are optimized during the prediction process.
8. The event causal relationship recognition method based on cue learning according to claim 1, characterized in that, Each of the aforementioned associated event pairs includes a first associated event representation and a second associated event representation. The step of constructing a first causal similarity memory map and a second causal similarity memory map corresponding to the first event representation and the second event representation respectively, based on the event similarity between the target event pair and each of the associated event pairs in the causal event memory space, and combining the target event pair with the causal event memory space, includes the following steps: Calculate the first event similarity between the first event representation and each of the first associated event representations in the causal event memory space; The representations of each of the first associated events in the causal event memory space are sorted in descending order of the similarity of the first events; A predetermined number of the first associated event representations are read from the causal event memory space according to the sorted order; The first event is represented as the first root node, all the first associated events are represented as the first child nodes, and the first event similarity between the first root node and each of the first child nodes is used as the first edge weight to construct the node edge between the first root node and each of the first child nodes, thus forming a first causal similarity memory graph. Calculate the second event similarity between the second event representation and each of the second associated event representations in the causal event memory space; The representations of each of the second related events in the causal event memory space are sorted in descending order of the similarity of the second events; A predetermined number of the second associated event representations are read from the causal event memory space according to the sorted order; The second event is represented as the second root node, and all the second associated events are represented as the second child nodes. The node edges between the second root node and each of the second child nodes are constructed using the similarity of the second events between the second root node and each of the second child nodes as the second edge weights, thus forming a second causal similarity memory graph.
9. The event causal relationship recognition method based on cue learning according to claim 8, characterized in that, The step of combining the first causal similarity memory map and the second causal similarity memory map, and fine-tuning the representation vector portions corresponding to the first event representation and the second event representation in the fused representation vector using an information aggregation method to obtain the fine-tuned final fused representation vector includes the following steps: The first symmetric weight matrix of the first causal similarity memory graph is calculated based on the first edge weight. The second symmetric weight matrix of the second causal similarity memory graph is calculated based on the second edge weight. The first fine-tuned representation vector corresponding to the first event representation is calculated by combining the first causal similarity memory graph and the first symmetric weight matrix, and the first fine-tuned representation vector is used to replace the representation vector part corresponding to the first event representation in the fused representation vector. The second fine-tuned representation vector corresponding to the second event representation is calculated by combining the second causal similarity memory graph and the second symmetric weight matrix. The second fine-tuned representation vector is then used to replace the representation vector part corresponding to the second event representation in the fused representation vector to obtain the fine-tuned final fused representation vector.
10. A cue-based learning-based event causality recognition system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the event causality recognition method based on cue learning as described in any one of claims 1 to 9.