A method and apparatus for causal relationship recognition based on graph convolutional networks and contrastive learning

Abstract

This application relates to a method and apparatus for causal relationship recognition based on graph convolutional networks and contrastive learning. The method includes: fusing the positional and semantic relationships of initial word vectors using a graph convolutional network to obtain relationship representations; learning relationship representations for sentences containing event pairs using a BiLSTM model to obtain hidden relationship representations; enhancing the relationship representations using contrastive learning; fusing the hidden relationship representations and the enhanced relationship representations; calculating the fused relationship representations using fully connected layers and SoftMax layers to obtain an initial probability of whether a causal relationship exists in the event pair; constructing a cross-entropy loss function based on the initial probability and the true causal relationships in the pre-labeled event pairs; training the causal relationship recognition model using the cross-entropy loss function; and then performing causal relationship recognition on the text to be recognized. This method can improve the accuracy of causal relationship recognition for event pairs.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus, computer device, and storage medium for causal relationship recognition based on graph convolutional networks and contrastive learning. Background Technology

[0002] Causal relationship identification is a challenging task in natural language processing (NLP), attracting increasing attention from the field. Causal relationship identification methods primarily involve mining causal event pairs within text, or identifying whether a causal relationship exists between given event pairs. In human society, events serve as carriers of social activities and a means of information expression. If machines could understand the causal relationships between events, it would undoubtedly make machines more intelligent and human-computer interaction more convenient, potentially making significant contributions to fields such as aviation and medicine.

[0003] However, current methods for identifying causal relationships suffer from problems such as low efficiency and low accuracy. Summary of the Invention

[0004] Therefore, it is necessary to provide a causal relationship identification method, apparatus, computer device, and storage medium based on graph convolutional networks and contrastive learning that can improve the accuracy of event-to-causal relationship identification, addressing the aforementioned technical problems.

[0005] A causal relationship recognition method based on graph convolutional networks and contrastive learning, the method comprising:

[0006] Obtain the text to be identified; the text contains multiple sentences; a portion of the sentences in the text contains multiple event pairs;

[0007] The sentences are preprocessed to obtain initial word vectors;

[0008] Construct a causal relationship recognition model; the causal relationship recognition model includes graph convolutional networks, BiLSTM models, fully connected layers, and SoftMax layers;

[0009] The graph convolutional network fuses the positional and semantic relationships of the initial word vectors to obtain a relation representation;

[0010] Hidden relation representations are obtained by learning relation representations for sentences containing event pairs using the BiLSTM model.

[0011] The relation representation is augmented using a contrastive learning method to obtain the augmented relation representation.

[0012] The hidden relation representation and the enhanced relation representation are fused to obtain the fused relation representation;

[0013] The initial probability of whether a causal relationship exists in an event pair is obtained by using a fully connected layer and a SoftMax layer to calculate the fusion relationship representation.

[0014] Based on the initial probability of whether there is a causal relationship in the event pair and the true causal relationship in the pre-labeled event pair, a cross-entropy loss function is constructed. The pre-constructed causal relationship recognition model is trained using the cross-entropy loss function and the loss function in contrastive learning to obtain the trained causal relationship recognition model.

[0015] A trained causal relationship recognition model is used to identify causal relationships in the text to be identified.

[0016] In one embodiment, the sentences are preprocessed to obtain initial text vectors, including: taking the length of the sentence containing the most words in the sentence containing all event pairs as the maximum length, and padding sentences that are less than the maximum length with zeros to obtain multiple sentences of equal length; and preprocessing the sentences according to the BERT pre-trained model to obtain initial text vectors.

[0017] In one embodiment, the positional and semantic relationships of the initial word vectors are fused using a graph convolutional network to obtain a relation representation, including: adding positional encoding to the initial word vectors to obtain position-encoded word vectors; the position-encoded word vectors represent the positional relationships between word vectors; using the position-encoded word vectors as graph nodes, the relationship between the position-encoded word vectors is calculated according to a metric function to obtain semantic relationships; and the semantic relationships are fused using a graph convolutional neural network to obtain a relation representation.

[0018] In one embodiment, semantic relationships are obtained by calculating the relationship between position-encoded word vectors based on a metric function, including:

[0019] The semantic relationships are calculated based on the metric function used to analyze the relationships between position-encoded word vectors.

[0020] S i,j =ReLU(W i v i ) T ReLU(W j v j )

[0021] Where ReLU(x) = max(0,x) represents the metric function, v i and v j W represents the position-encoded word vector representation of the i-th and j-th nodes, respectively. i and W j This represents the trainable parameter matrix.

[0022] In one embodiment, semantic relations are fused using a graph convolutional neural network to obtain a relation representation, including:

[0023] The semantic relationships are fused using a graph convolutional neural network to obtain the relationship representation as follows:

[0024]

[0025] in, Let W represent the adjacency matrix of the graph. (l) Let represent the trainable parameter matrix, and σ refer to the non-linear activation function, here using the ReLU activation function. I is the identity matrix. express The degree matrix.

[0026] In one embodiment, a BiLSTM model is used to learn relational representations of sentences containing event pairs to obtain hidden relational representations, including:

[0027] Using a BiLSTM model to learn relation representations for sentences containing event pairs, the hidden relation representations are obtained as follows:

[0028]

[0029]

[0030] Where X = [x1, x2, ..., x n ] represents the text vector of the sentence containing the event pair, x n Representing word vectors, and This represents the hidden relationship.

[0031] In one embodiment, the relation representation is enhanced using a contrastive learning method to obtain an enhanced relation representation, including:

[0032] From sentences containing event pairs, extract event pairs of the same category as the current relation representation as positive examples, and event pairs of different relation categories as negative examples. By comparison, the similarity between the current relation representation and the positive example relation representation is brought closer, while the similarity with the negative example relation representation is distanced, thus obtaining the enhanced relation representation.

[0033] The loss function that narrows the similarity between the current relation representation and the relation representation of positive examples, and distances it from the relation representation of negative examples, is:

[0034] Among them, V (l+1) V represents the relational representation of the currently identified event pairs. jV represents the relationship between positive samples. k Excluding V (l+1) The relationship between samples in the total sample set is represented by N, where N is the total number of samples, sim refers to the L2 distance function used to calculate the similarity between features, and τ is used to adjust the similarity weight.

[0035] In one embodiment, a fully connected layer and a SoftMax layer are used to calculate the fusion relationship representation to obtain the initial probability of whether a causal relationship exists in the event pair, including:

[0036] By using fully connected layers and SoftMax layers to calculate the fusion relationship representation, the initial probability of whether a causal relationship exists in the event pairs is obtained.

[0037]

[0038] Among them, F r =Relu(W T F) = max(0, W) T F) represents the relational representation after passing through the fully connected layer, W T F = concat(P,T) is the transpose of the trainable parameter matrix, fusing relation representations, where P represents the enhanced relation representation, and p j This represents the probability that the current correct event pair is the j-th event pair. Let j be the j-th feature parameter of the concatenated features. is the i-th feature parameter of the concatenated features.

[0039] In one embodiment, a cross-entropy loss function is constructed based on the initial probability of whether a causal relationship exists in the event pair and the pre-labeled true causal relationships in the event pairs, including:

[0040] Based on the initial probability of whether a causal relationship exists in the event pairs and the pre-labeled true causal relationships in the event pairs, a cross-entropy loss function is constructed as follows:

[0041]

[0042] Where y represents the true causal relationship in the pre-labeled event pairs, x represents the event pair label, n represents the number of event pairs, and i represents the event pair sequence number.

[0043] A causal relationship recognition device based on graph convolutional networks and contrastive learning, the device comprising:

[0044] The preprocessing module is used to obtain the text to be recognized; the text contains multiple sentences; a portion of the sentences in the text contains multiple event pairs; the sentences are preprocessed to obtain initial word vectors;

[0045] The relation fusion module is used to construct a causal relation recognition model. The causal relation recognition model includes a graph convolutional network, a BiLSTM model, a fully connected layer, and a SoftMax layer. The positional and semantic relations of the initial word vectors are fused based on the graph convolutional network to obtain the relation representation.

[0046] The relation representation learning and enhancement module is used to learn relation representations for sentences containing event pairs using a BiLSTM model to obtain hidden relation representations; and to enhance the relation representations using a contrastive learning method to obtain enhanced relation representations.

[0047] The relation representation fusion and probability calculation module is used to fuse the hidden relation representation and the enhanced relation representation to obtain the fused relation representation; the fully connected layer and the SoftMax layer are used to calculate the fused relation representation to obtain the initial probability of whether there is a causal relationship in the event pair;

[0048] The causal relationship identification module is used to construct a cross-entropy loss function based on the initial probability of whether a causal relationship exists in the event pair and the true causal relationship in the pre-labeled event pair. The pre-constructed causal relationship identification model is trained using the cross-entropy loss function and the loss function in contrastive learning to obtain a trained causal relationship identification model. The trained causal relationship identification model is then used to identify the causal relationship in the text to be identified.

[0049] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0050] Obtain the text to be identified; the text contains multiple sentences; a portion of the sentences in the text contains multiple event pairs;

[0051] The sentences are preprocessed to obtain initial word vectors;

[0052] Construct a causal relationship recognition model; the causal relationship recognition model includes graph convolutional networks, BiLSTM models, fully connected layers, and SoftMax layers;

[0053] The graph convolutional network fuses the positional and semantic relationships of the initial word vectors to obtain a relation representation;

[0054] Hidden relation representations are obtained by learning relation representations for sentences containing event pairs using the BiLSTM model.

[0055] The relation representation is augmented using a contrastive learning method to obtain the augmented relation representation.

[0056] The hidden relation representation and the enhanced relation representation are fused to obtain the fused relation representation;

[0057] The initial probability of whether a causal relationship exists in an event pair is obtained by using a fully connected layer and a SoftMax layer to calculate the fusion relationship representation.

[0058] Based on the initial probability of whether there is a causal relationship in the event pair and the true causal relationship in the pre-labeled event pair, a cross-entropy loss function is constructed. The pre-constructed causal relationship recognition model is trained using the cross-entropy loss function and the loss function in contrastive learning to obtain the trained causal relationship recognition model.

[0059] A trained causal relationship recognition model is used to identify causal relationships in the text to be identified.

[0060] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0061] Obtain the text to be identified; the text contains multiple sentences; a portion of the sentences in the text contains multiple event pairs;

[0062] The sentences are preprocessed to obtain initial word vectors;

[0063] Construct a causal relationship recognition model; the causal relationship recognition model includes graph convolutional networks, BiLSTM models, fully connected layers, and SoftMax layers;

[0064] The graph convolutional network fuses the positional and semantic relationships of the initial word vectors to obtain a relation representation;

[0065] Hidden relation representations are obtained by learning relation representations for sentences containing event pairs using the BiLSTM model.

[0066] The relation representation is augmented using a contrastive learning method to obtain the augmented relation representation.

[0067] The hidden relation representation and the enhanced relation representation are fused to obtain the fused relation representation;

[0068] The initial probability of whether a causal relationship exists in an event pair is obtained by using a fully connected layer and a SoftMax layer to calculate the fusion relationship representation.

[0069] Based on the initial probability of whether there is a causal relationship in the event pair and the true causal relationship in the pre-labeled event pair, a cross-entropy loss function is constructed. The pre-constructed causal relationship recognition model is trained using the cross-entropy loss function and the loss function in contrastive learning to obtain the trained causal relationship recognition model.

[0070] A trained causal relationship recognition model is used to identify causal relationships in the text to be identified.

[0071] The aforementioned method, apparatus, computer device, and storage medium for causal relationship recognition based on graph convolutional networks and contrastive learning first preprocesses the sentence to obtain initial word vectors. Each word vector contains semantic information of the context. The initial word vectors are used as nodes, and the relationships between them are used as edges to construct a graph. Then, a graph convolutional network is used to fuse the positional and semantic relationships of the initial word vectors to obtain a relation representation. This relation representation includes the causal relationship between event pairs. A BiLSTM model is used to learn the relation representation of sentences containing event pairs to obtain hidden relation representations. By mining the hidden relationships between event pairs in the sentence, the fused relation obtained in subsequent relation fusion is more accurate. To improve the accuracy of model training, the relation representation is enhanced using a contrastive learning method, resulting in enhanced relation representations of event pairs. These enhanced representations are then fused with hidden relation representations. Furthermore, contextual and relational information is supplemented to the event pairs using the sentences they belong to, making the resulting event pairs more conducive to subsequent causal relationship judgments and yielding more accurate initial probabilities. A cross-entropy loss function is constructed based on the initial probability of causal relationships in the event pairs and the true causal relationships in the pre-labeled event pairs. Training the pre-built causal relationship recognition model using the cross-entropy loss function and the loss function from contrastive learning improves the model training accuracy, thereby enhancing the accuracy of causal relationship recognition. Attached Figure Description

[0072] Figure 1 This is a flowchart illustrating a causal relationship recognition method based on graph convolutional networks and contrastive learning, as an example.

[0073] Figure 2 This is a block diagram of a causal relationship recognition device based on graph convolutional networks and contrastive learning in one embodiment;

[0074] Figure 3 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0075] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0076] In one embodiment, such as Figure 1 As shown, a causal relationship recognition method based on graph convolutional networks and contrastive learning is provided, including the following steps:

[0077] Step 102: Obtain the text to be identified; the text contains multiple sentences; a portion of the sentences in the text contains multiple event pairs; perform data preprocessing on the sentences to obtain initial word vectors.

[0078] The maximum sentence length is set as the length of the sentence containing the most words in all event pairs. Sentences shorter than the maximum length are padded with zeros to achieve equal sentence length. Then, the words and event pairs in the equal-length sentences are converted into initial word vectors using a BERT pre-trained model. Each word vector contains contextual semantic information, which facilitates the subsequent identification of causal relationships in event pairs.

[0079] Step 104: Construct a causal relationship recognition model; the causal relationship recognition model includes a graph convolutional network, a BiLSTM model, a fully connected layer, and a SoftMax layer; the positional relationship and semantic relationship of the initial word vectors are fused according to the graph convolutional network to obtain the relationship representation.

[0080] The initial word vectors are used as nodes, and the relationships between word vectors are used as edges to construct a graph. Then, a graph convolutional network is used to fuse the positional and semantic relationships of the initial word vectors to obtain a relation representation, which includes the causal relationship between event pairs.

[0081] Step 106: Use the BiLSTM model to learn relation representations for sentences containing event pairs to obtain hidden relation representations; use the contrastive learning method to enhance the relation representations to obtain enhanced relation representations.

[0082] A BiLSTM model is used to learn relation representations for sentences containing event pairs, resulting in hidden relation representations. By mining the hidden relations of event pairs in sentences, the fused relations obtained during subsequent relation fusion are more accurate, thereby improving the accuracy of model training. Then, the relation representations are augmented using a contrastive learning method to obtain augmented relation representations of event pairs. The augmented relation representations and hidden relation representations are fused together. By supplementing the event pairs with scene and relation information through the sentences containing the event pairs, the obtained event pairs are more conducive to subsequent causal relationship judgments and obtain more accurate initial probabilities.

[0083] Step 108: Fuse the hidden relation representation and the enhanced relation representation to obtain the fused relation representation; use the fully connected layer and the SoftMax layer to calculate the fused relation representation to obtain the initial probability of whether there is a causal relationship in the event pair.

[0084] The output of the BiLSTM on the sentence containing the event pair is concatenated with the event pair relation representation enhanced by contrastive learning. This is done to supplement the event pair with contextual and relational information through the sentence containing the event pair. The computation expression is:

[0085] F = concat(P, T)

[0086] Where T represents the output of the sentence containing the event pair after passing through the BiLSTM model, and P represents the event pair relation representation enhanced by contrastive learning. The concatenated feature F is input into the fully connected layer and the ReLU activation function layer, as shown in the following expression:

[0087] F r =Relu(W T F) = max(0, W) T F)

[0088] Among them W T This is the transpose of the trainable parameter matrix. Finally, the softmax function outputs the probability distribution of the corresponding category (in this application, the category is "yes" or "no," i.e., whether a causal relationship exists). The category with the highest probability is the obtained result.

[0089] Step 110: Construct a cross-entropy loss function based on the initial probability of whether a causal relationship exists in the event pair and the true causal relationship in the pre-labeled event pair. Use the cross-entropy loss function and the loss function in contrastive learning to train the pre-constructed causal relationship recognition model to obtain the trained causal relationship recognition model. Use the trained causal relationship recognition model to perform causal relationship recognition on the text to be recognized.

[0090] Based on the initial probability of causal relationship in event pairs and the true causal relationship in pre-labeled event pairs, a cross-entropy loss function is constructed. Using the cross-entropy loss function and the loss function in contrastive learning to train the pre-constructed causal relationship recognition model can improve the accuracy of model training, thereby improving the accuracy of causal relationship recognition.

[0091] In the aforementioned causal relationship recognition method based on graph convolutional networks and contrastive learning, the sentence is first preprocessed to obtain initial word vectors. Each word vector contains semantic information of the context. The initial word vectors are used as nodes, and the relationships between them are used as edges to construct a graph. Then, a graph convolutional network is used to fuse the positional and semantic relationships of the initial word vectors to obtain a relation representation. This relation representation contains the causal relationship between event pairs. A BiLSTM model is then used to learn the relation representation of sentences containing event pairs to obtain hidden relation representations. By mining the hidden relationships between event pairs in the sentence, the fused relation obtained in subsequent relation fusion is more accurate, thereby improving the model training performance. To improve accuracy, the relation representation is enhanced using a contrastive learning method, resulting in an enhanced relation representation of the event pair. This enhanced relation representation is then fused with the hidden relation representation. By supplementing the event pair with contextual and relational information through the sentence containing the event pair, the resulting event pairs become more conducive to subsequent causal relationship judgments, leading to more accurate initial probabilities. Based on the initial probability of causal relationships in the event pairs and the true causal relationships in the pre-labeled event pairs, a cross-entropy loss function is constructed. Training the pre-constructed causal relationship recognition model using the cross-entropy loss function and the loss function in contrastive learning can improve the accuracy of model training, thereby improving the accuracy of causal relationship recognition.

[0092] In one embodiment, the sentences are preprocessed to obtain initial text vectors, including: taking the length of the sentence containing the most words in the sentence containing all event pairs as the maximum length, and padding sentences that are less than the maximum length with zeros to obtain multiple sentences of equal length; and preprocessing the sentences according to the BERT pre-trained model to obtain initial text vectors.

[0093] In one embodiment, the positional and semantic relationships of the initial word vectors are fused using a graph convolutional network to obtain a relation representation, including: adding positional encoding to the initial word vectors to obtain position-encoded word vectors; the position-encoded word vectors represent the positional relationships between word vectors; using the position-encoded word vectors as graph nodes, the relationship between the position-encoded word vectors is calculated according to a metric function to obtain semantic relationships; and the semantic relationships are fused using a graph convolutional neural network to obtain a relation representation.

[0094] In one embodiment, semantic relationships are obtained by calculating the relationship between position-encoded word vectors based on a metric function, including:

[0095] The semantic relationships are calculated based on the metric function used to analyze the relationships between position-encoded word vectors.

[0096] S i,j =ReLU(W i v i )T ReLU(W j v j )

[0097] Where ReLU(x) = max(0,x) represents the metric function, v i and v j W represents the position-encoded word vector representation of the i-th and j-th nodes, respectively. i and W j This represents the trainable parameter matrix.

[0098] In one embodiment, semantic relations are fused using a graph convolutional neural network to obtain a relation representation, including:

[0099] The semantic relationships are fused using a graph convolutional neural network to obtain the relationship representation as follows:

[0100]

[0101] in, Let W represent the adjacency matrix of the graph. (l) Let represent the trainable parameter matrix, and σ refer to the non-linear activation function, here using the ReLU activation function. I is the identity matrix. express The degree matrix.

[0102] In a specific embodiment, since graph structures cannot capture the temporal relationships between words, positional encoding is added to the text vector before converting it into a graph structure to capture the input temporal information. The calculation process is as follows:

[0103]

[0104]

[0105] Where, pos s d represents the position of the word vector in the sequence. s The dimension of the word vector is represented by , and i represents the i-th element in each word vector.

[0106] The word vectors with added positional encoding are used as nodes in the graph, and the semantic relationships between words are calculated using a metric function as edges in the graph. Then, the positional and semantic relationships between word vectors are fused through a graph convolutional network to obtain a relational representation.

[0107] In one embodiment, a BiLSTM model is used to learn relational representations of sentences containing event pairs to obtain hidden relational representations, including:

[0108] Using a BiLSTM model to learn relation representations for sentences containing event pairs, the hidden relation representations are obtained as follows:

[0109]

[0110]

[0111] Where X = [x1, x2, ..., x n ] represents the text vector of the sentence containing the event pair, x n Representing word vectors, and This represents the hidden relationship.

[0112] In one embodiment, the relation representation is enhanced using a contrastive learning method to obtain an enhanced relation representation, including:

[0113] From sentences containing event pairs, extract event pairs of the same category as the current relation representation as positive examples, and event pairs of different relation categories as negative examples. By comparison, the similarity between the current relation representation and the positive example relation representation is brought closer, while the similarity with the negative example relation representation is distanced, thus obtaining the enhanced relation representation.

[0114] The loss function that narrows the similarity between the current relation representation and the relation representation of positive examples, and distances it from the relation representation of negative examples, is:

[0115] Among them, V (l+1) V represents the relational representation of the currently identified event pairs. j V represents the relationship between positive samples. k Excluding V (l+1) The relationship between samples in the total sample set is represented by N, where N is the total number of samples, sim refers to the L2 distance function used to calculate the similarity between features, and τ is used to adjust the similarity weight.

[0116] In a specific embodiment, other event pairs are sampled from the dataset. Event pairs with the same relationship category as the current event pair are taken as positive examples (not necessarily having a causal relationship), while event pairs with different relationship categories are taken as negative examples (not necessarily not having a causal relationship).

[0117] Event pairs with the same relationship category exhibit similar relationship representations. Therefore, by comparing these representations, the similarity between the event pair relationship representation and positive example relationship representations is reduced, while the similarity with negative example relationship representations is reduced. This makes the relationship representations more closely reflect the currently identified event pair relationships and better distinguishes the feature representations of different relationships, thereby enhancing the feature representation. This application constructs a loss function and continuously trains the system to gradually reduce the similarity with positive examples and reduce the similarity with negative examples, resulting in enhanced relationship representations.

[0118] In one embodiment, a fully connected layer and a SoftMax layer are used to calculate the fusion relationship representation to obtain the initial probability of whether a causal relationship exists in the event pair, including:

[0119] By using fully connected layers and SoftMax layers to calculate the fusion relationship representation, the initial probability of whether a causal relationship exists in the event pairs is obtained.

[0120]

[0121] Among them, F r =Relu(W T F) = max(0, W) T F) represents the relational representation after passing through the fully connected layer, W T F = concat(P,T) is the transpose of the trainable parameter matrix, fusing relation representations, where P represents the enhanced relation representation, and p j This represents the probability that the current correct event pair is the j-th event pair. Let j be the j-th feature parameter of the concatenated features. is the i-th feature parameter of the concatenated features.

[0122] In one embodiment, a cross-entropy loss function is constructed based on the initial probability of whether a causal relationship exists in the event pair and the pre-labeled true causal relationships in the event pairs, including:

[0123] Based on the initial probability of whether a causal relationship exists in the event pairs and the pre-labeled true causal relationships in the event pairs, a cross-entropy loss function is constructed as follows:

[0124]

[0125] Where y represents the true causal relationship in the pre-labeled event pairs, x represents the event pair label, n represents the number of event pairs, and i represents the event pair sequence number.

[0126] In a specific embodiment, the loss function in contrastive learning will be... cons Combining this with the cross-entropy loss function, and then continuously updating the parameters through stochastic gradient descent to minimize the loss function, the expression is as follows:

[0127] loss θ =loss cons +loss crE

[0128]

[0129] Where θ represents all trainable parameters in the model, and η is the learning rate during model training.

[0130] After training is complete, the positive and negative examples and the contrastive learning module in the model are removed, and the remaining model framework can be used to identify whether event pairs have a causal relationship.

[0131] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0132] In one embodiment, such as Figure 2 As shown, a causal relationship recognition device based on graph convolutional networks and contrastive learning is provided, including: a preprocessing module 202, a relationship fusion module 204, a relationship representation learning and enhancement module 206, a relationship representation fusion and probability calculation module 208, and a causal relationship recognition module 210, wherein:

[0133] The preprocessing module 202 is used to acquire the text to be recognized; the text contains multiple sentences; a portion of the sentences in the text contains multiple event pairs; the sentences are preprocessed to obtain initial word vectors;

[0134] The relation fusion module 204 is used to construct a causal relation recognition model. The causal relation recognition model includes a graph convolutional network, a BiLSTM model, a fully connected layer, and a SoftMax layer. The positional and semantic relations of the initial word vectors are fused based on the graph convolutional network to obtain the relation representation.

[0135] The relation representation learning and enhancement module 206 is used to learn relation representations for sentences containing event pairs using a BiLSTM model to obtain hidden relation representations; and to enhance the relation representations using a contrastive learning method to obtain enhanced relation representations.

[0136] The relation representation fusion and probability calculation module 208 is used to fuse the hidden relation representation and the enhanced relation representation to obtain the fused relation representation; the fully connected layer and the SoftMax layer are used to calculate the fused relation representation to obtain the initial probability of whether there is a causal relationship in the event pair;

[0137] The causal relationship identification module 210 is used to construct a cross-entropy loss function based on the initial probability of whether a causal relationship exists in the event pair and the true causal relationship in the pre-labeled event pair, and to train the pre-constructed causal relationship identification model using the cross-entropy loss function and the loss function in contrastive learning to obtain a trained causal relationship identification model; and to use the trained causal relationship identification model to identify the causal relationship of the text to be identified.

[0138] In one embodiment, the preprocessing module 202 is further configured to perform data preprocessing on the sentences to obtain an initial text vector, including: taking the length of the sentence containing the most words in the sentence containing all event pairs as the maximum length, and padding sentences that are less than the maximum length with zeros to obtain multiple sentences of equal length; and preprocessing the sentences according to the BERT pre-trained model to obtain the initial text vector.

[0139] In one embodiment, the relation fusion module 204 is further configured to fuse the positional and semantic relationships of the initial word vectors according to the graph convolutional network to obtain a relation representation, including: adding positional encoding to the initial word vectors to obtain position-encoded word vectors; the position-encoded word vectors represent the positional relationships between word vectors; using the position-encoded word vectors as graph nodes, calculating the relationship between the position-encoded word vectors according to the metric function to obtain semantic relationships; and fusing the semantic relationships according to the graph convolutional neural network to obtain a relation representation.

[0140] In one embodiment, the relation fusion module 204 is further configured to calculate the relationship between position-encoded word vectors according to a metric function to obtain semantic relations, including:

[0141] The semantic relationships are calculated based on the metric function used to analyze the relationships between position-encoded word vectors.

[0142] S i,j =ReLU(W i v i ) T ReLU(W j v j )

[0143] Where ReLU(x) = max(0,x) represents the metric function, v i and v j W represents the position-encoded word vector representation of the i-th and j-th nodes, respectively. i and W j This represents the trainable parameter matrix.

[0144] In one embodiment, the relation fusion module 204 is further configured to fuse semantic relations according to a graph convolutional neural network to obtain relation representations, including:

[0145] The semantic relationships are fused using a graph convolutional neural network to obtain the relationship representation as follows:

[0146]

[0147] in, Let W represent the adjacency matrix of the graph. (l) Let represent the trainable parameter matrix, and σ refer to the non-linear activation function, here using the ReLU activation function. I is the identity matrix. express The degree matrix.

[0148] In one embodiment, the relation representation learning and enhancement module 206 is further configured to perform relation representation learning on sentences containing event pairs using a BiLSTM model to obtain hidden relation representations, including:

[0149] Using a BiLSTM model to learn relation representations for sentences containing event pairs, the hidden relation representations are obtained as follows:

[0150]

[0151]

[0152] Where X = [x1, x2, ..., x n ] represents the text vector of the sentence containing the event pair, x n Representing word vectors, and This represents the hidden relationship.

[0153] In one embodiment, the relation representation learning and enhancement module 206 is further configured to enhance the relation representation according to a contrastive learning method to obtain an enhanced relation representation, including:

[0154] From sentences containing event pairs, extract event pairs of the same category as the current relation representation as positive examples, and event pairs of different relation categories as negative examples. By comparison, the similarity between the current relation representation and the positive example relation representation is brought closer, while the similarity with the negative example relation representation is distanced, thus obtaining the enhanced relation representation.

[0155] The loss function that narrows the similarity between the current relation representation and the relation representation of positive examples, and distances it from the relation representation of negative examples, is:

[0156] Among them, V (l+1) V represents the relational representation of the currently identified event pairs. j V represents the relationship between positive samples. k Excluding V (l+1) The relationship between samples in the total sample set is represented by N, where N is the total number of samples, sim refers to the L2 distance function used to calculate the similarity between features, and τ is used to adjust the similarity weight.

[0157] In one embodiment, the relation representation fusion and probability calculation module 208 is further configured to calculate the fused relation representation using a fully connected layer and a SoftMax layer to obtain an initial probability of whether a causal relationship exists in the event pair, including:

[0158] By using fully connected layers and SoftMax layers to calculate the fusion relationship representation, the initial probability of whether a causal relationship exists in the event pairs is obtained.

[0159]

[0160] Among them, F r =Relu(W T F) = max(0, W) T F) represents the relational representation after passing through the fully connected layer, W T F = concat(P,T) is the transpose of the trainable parameter matrix, fusing relation representations, where P represents the enhanced relation representation, and p j This represents the probability that the current correct event pair is the j-th event pair. Let j be the j-th feature parameter of the concatenated features. is the i-th feature parameter of the concatenated features.

[0161] In one embodiment, the causal relationship identification module 210 is further configured to construct a cross-entropy loss function based on the initial probability of whether a causal relationship exists in the event pair and the true causal relationships in the pre-labeled event pairs, including:

[0162] Based on the initial probability of whether a causal relationship exists in the event pairs and the pre-labeled true causal relationships in the event pairs, a cross-entropy loss function is constructed as follows:

[0163]

[0164] Where y represents the true causal relationship in the pre-labeled event pairs, x represents the event pair label, n represents the number of event pairs, and i represents the event pair sequence number.

[0165] Specific limitations regarding the causal relationship recognition device based on graph convolutional networks and contrastive learning can be found in the limitations of the causal relationship recognition method based on graph convolutional networks and contrastive learning described above, and will not be repeated here. Each module in the aforementioned causal relationship recognition device based on graph convolutional networks and contrastive learning can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0166] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 3 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When executed by the processor, the computer program implements a causal relationship recognition method based on graph convolutional networks and contrastive learning. The display screen can be an LCD screen or an e-ink display screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0167] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0168] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the method described above.

[0169] In one embodiment, a computer storage medium is provided that stores a computer program, which, when executed by a processor, implements the steps of the method described above.

[0170] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0171] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0172] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A causal relationship recognition method based on graph convolutional networks and contrastive learning, characterized in that, The method includes: Obtain the text to be identified; the text contains multiple sentences; a portion of the sentences in the text contains multiple event pairs; The sentence is preprocessed to obtain initial word vectors; A causal relationship recognition model is constructed; the causal relationship recognition model includes a graph convolutional network, a BiLSTM model, a fully connected layer, and a SoftMax layer; The positional and semantic relationships of the initial word vectors are fused using a graph convolutional network to obtain a relation representation; The BiLSTM model is used to learn relational representations for sentences containing event pairs to obtain hidden relational representations. The relation representation is enhanced using a contrastive learning method to obtain an enhanced relation representation. The hidden relation representation and the enhanced relation representation are fused to obtain the fused relation representation; The initial probability of whether a causal relationship exists in the event pair is obtained by calculating the fusion relationship representation using a fully connected layer and a SoftMax layer. Based on the initial probability of whether there is a causal relationship in the event pair and the true causal relationship in the pre-labeled event pair, a cross-entropy loss function is constructed. The pre-constructed causal relationship recognition model is trained using the cross-entropy loss function and the loss function in contrastive learning to obtain the trained causal relationship recognition model. The trained causal relationship recognition model is used to identify causal relationships in the text to be identified. The initial word vectors are fused using a graph convolutional network to obtain relation representations, including: Positional encoding is added to the initial word vector to obtain position-encoded word vectors; the position-encoded word vectors represent the positional relationships between word vectors. Using position-encoded word vectors as graph nodes, semantic relationships are obtained by calculating the relationships between the position-encoded word vectors according to a metric function. The semantic relationships are fused using a graph convolutional neural network to obtain a relationship representation; The semantic relationships are calculated based on the relationship between the position-encoded word vectors using a metric function, including: The semantic relationship S is obtained by calculating the relationship between the position-encoded word vectors based on the metric function. i,j =ReLU(W i v i ) T ReLU(W j v j ) Where ReLU(x) = max(0,x) represents the metric function, v i and v j W represents the position-encoded word vector representation of the i-th and j-th nodes, respectively. i and W j This represents the trainable parameter matrix.

2. The method according to claim 1, characterized in that, The sentence is preprocessed to obtain an initial text vector, including: The maximum length is determined by using the length of the sentence containing the most words in the sentence containing all event pairs. Sentences shorter than the maximum length are padded with zeros to obtain multiple sentences of equal length. The sentence is preprocessed using the BERT pre-trained model to obtain an initial text vector.

3. The method according to claim 1, characterized in that, The semantic relationships are fused using a graph convolutional neural network to obtain relationship representations, including: The semantic relationships are fused using a graph convolutional neural network to obtain the relationship representation as follows: in, Let W represent the adjacency matrix of the graph. (l) Let represent the trainable parameter matrix, and σ refer to the non-linear activation function, here using the ReLU activation function. I is the identity matrix. express The degree matrix.

4. The method according to claim 3, characterized in that, The BiLSTM model is used to learn relation representations for sentences containing event pairs to obtain hidden relation representations, including: The BiLSTM model is used to learn relation representations for sentences containing event pairs, resulting in hidden relation representations. Where X = [x1, x2, ..., x n ] represents the text vector of the sentence containing the event pair, x n Representing word vectors, and This represents the hidden relationship.

5. The method according to claim 4, characterized in that, The relation representation is enhanced using a contrastive learning method to obtain an enhanced relation representation, including: From sentences containing event pairs, extract event pairs of the same category as the current relation representation as positive examples, and event pairs of different relation categories as negative examples. By comparison, the similarity between the current relation representation and the positive example relation representation is brought closer, while the similarity with the negative example relation representation is distanced, thus obtaining the enhanced relation representation. The loss function for narrowing the similarity between the current relation representation and the positive sample relation representation, and for distancing the similarity between the relation representation and the negative sample relation representation, is: Among them, V (l+1) V represents the relational representation of the currently identified event pairs. j V represents the relationship between positive samples. k Excluding V (l+1) The relationship between samples in the total sample set is represented by N, where N is the total number of samples, sim refers to the L2 distance function used to calculate the similarity between features, and τ is used to adjust the similarity weight.

6. The method according to claim 5, characterized in that, The fusion relationship representation is calculated using a fully connected layer and a SoftMax layer to obtain the initial probability of whether a causal relationship exists in the event pair, including: The fusion relationship representation is calculated using a fully connected layer and a SoftMax layer to obtain the initial probability of whether a causal relationship exists in the event pair. Among them, F r =Relu(W T F) = max(0, W) T F) represents the relational representation after passing through the fully connected layer, W T F = concat(P,T) is the transpose of the trainable parameter matrix, fusing relation representations, where P represents the enhanced relation representation, and p j This represents the probability that the current correct event pair is the j-th event pair. Let j be the j-th feature parameter of the concatenated features. is the i-th feature parameter of the concatenated features.

7. The method according to claim 6, characterized in that, The cross-entropy loss function is constructed based on the initial probability of whether a causal relationship exists in the event pairs and the pre-labeled true causal relationships in the event pairs, including: Based on the initial probability of whether a causal relationship exists in the event pairs and the pre-labeled true causal relationships in the event pairs, a cross-entropy loss function is constructed as follows: Where y represents the true causal relationship in the pre-labeled event pairs, x represents the event pair label, n represents the number of event pairs, and i represents the event pair sequence number.

8. A causal relationship recognition device based on graph convolutional networks and contrastive learning, characterized in that, The device includes: A preprocessing module is used to acquire the text to be recognized; the text contains multiple sentences; a portion of the sentences in the text contains multiple event pairs; the sentences are preprocessed to obtain initial word vectors; A relation fusion module is used to construct a causal relation recognition model; the causal relation recognition model includes a graph convolutional network, a BiLSTM model, a fully connected layer, and a SoftMax layer; the positional and semantic relations of the initial word vectors are fused using the graph convolutional network to obtain a relation representation; the relation representation obtained by fusing the positional and semantic relations of the initial word vectors using the graph convolutional network includes: Positional encoding is added to the initial word vector to obtain position-encoded word vectors; the position-encoded word vectors represent the positional relationships between word vectors. Using position-encoded word vectors as graph nodes, semantic relationships are obtained by calculating the relationships between the position-encoded word vectors according to a metric function. The semantic relationships are fused using a graph convolutional neural network to obtain a relationship representation; The semantic relationships are calculated based on the relationship between the position-encoded word vectors using a metric function, including: The semantic relationship S is obtained by calculating the relationship between the position-encoded word vectors based on the metric function. i,j =ReLU(W i v i ) T ReLU(W j v j ) Where ReLU(x) = max(0,x) represents the metric function, v i and v j W represents the position-encoded word vector representation of the i-th and j-th nodes, respectively. i and W j Represents the trainable parameter matrix; The relation representation learning and enhancement module is used to learn relation representations for sentences containing event pairs using the BiLSTM model to obtain hidden relation representations; and to enhance the relation representations using a contrastive learning method to obtain enhanced relation representations. The relation representation fusion and probability calculation module is used to fuse the hidden relation representation and the enhanced relation representation to obtain the fused relation representation; and to calculate the fused relation representation using a fully connected layer and a SoftMax layer to obtain the initial probability of whether there is a causal relationship in the event pair. The causal relationship identification module is used to construct a cross-entropy loss function based on the initial probability of whether a causal relationship exists in an event pair and the true causal relationship in the pre-labeled event pairs. The pre-constructed causal relationship identification model is trained using the cross-entropy loss function and the loss function in contrastive learning to obtain a trained causal relationship identification model. The trained causal relationship identification model is then used to identify the causal relationship in the text to be identified.

Images