A method for constructing a contextual contrast learning recognition model and application thereof

Abstract

The application provides a construction method of a context contrast learning recognition model and application thereof, including the following steps: using a self-defined template to template samples, and splicing to obtain input T i ; inputting T i into a pre-training language model for coding; in a contrast module, the model optimizes the representation of event pairs and extracts relationship information; a prediction module predicts the probability distribution of missing words based on event representation and selects the most possible answer label; the model is trained through a training set to realize accurate recognition of event causal relationships. The application introduces explicit guidance in the form of adding labeled context samples, further excavates the implicit causal knowledge in the pre-training language model, and organically combines context learning and contrast learning in the form of context contrast learning, more fully utilizes positive and negative samples in the context samples, and effectively improves the accuracy of event causal relationship recognition.

Description

Technical Field

[0001] This invention relates to the field of event causality recognition, and in particular to a method for constructing a contextual contrast learning recognition model and its application. Background Technology

[0002] Event causality identification (ECI) aims to detect whether there is a causal relationship between two event mentions in a document. This task is crucial for many downstream tasks in Natural Language Processing (NLP), such as question answering systems and information summarization, and also has wide applications in real-world scenarios, such as event prediction. Several techniques have been developed for ECI, with recent methods broadly categorized into graph-based learning paradigms and cue-based learning paradigms.

[0003] Traditional approaches to solving causal relationship identification tasks using graph structures mainly fall into two categories: node classification and edge prediction. Node classification methods focus on aggregating document-level semantic information on the graph structure using graph neural networks, continuously updating node representations to enrich their semantic information, and ultimately determining whether a causal relationship exists between two nodes based on their representations. Edge prediction methods, on the other hand, shift their focus to the study of edges in the graph, continuously updating the graph using graph neural networks and other methods, and finally determining whether two events have a causal relationship based on the state of the edges between nodes.

[0004] The basic idea of ​​the prompting learning paradigm is to utilize the semantic knowledge inherent in the pre-trained language model (MLM) to complete downstream tasks. Typically, the downstream task is transformed into a pre-training task of the MLM by designing appropriate prompt templates. Specifically, this involves designing suitable prompt templates based on the input of the downstream task, asking the pre-trained language model questions in a cloze test format, and then mapping the pre-trained language model's answers to corresponding labels to complete the downstream task. For event causality recognition tasks, some studies have designed complex prompt templates to better utilize the implicit semantic knowledge in the pre-trained language model. However, the design of these templates often consumes a lot of time and computational resources and is difficult to prove universally applicable to most downstream tasks. This invention emphasizes that instead of inefficiently exploring for efficient and universal templates, it introduces explicit guidance by splicing labeled contextual samples to mine the semantic knowledge of the pre-trained language model. Simultaneously, it introduces contrastive learning to better distinguish between positive and negative samples, emphasizing the different impacts of different labeled contextual samples on the samples to be predicted. Summary of the Invention

[0005] The main objective of this invention is to provide a method for constructing a contextual contrast learning recognition model and its application, thereby solving the problem that existing event causal relationship recognition methods based on prompt learning consume a lot of resources in prompt template design and have poor universality.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a method for constructing a contextual contrast learning recognition model, comprising the following steps:

[0007] S1, Template Building Module: Based on two custom templates T q (q), T g (d k ), and the context sample q and the sample to be predicted d k Templated and concatenated to obtain T i As input;

[0008] S2, Pre-trained network encoding: T i The pre-trained language model is input for encoding to obtain the representation vectors of events in each sample and [MASK] in the sample to be predicted, which are used as inputs to the contrastive learning module and the prediction module, respectively.

[0009] S3, Contrast Module and Prediction Module: In the contrast module, the representation of event pairs is optimized using a contrastive learning paradigm. The event representations obtained in step S3 serve as the initial input to the contrast module, extracting relational information, which is then used for contrastive learning. In the prediction module, based on h... [MASK] Predict the probability distribution of the missing word at [MASK] on the full vocabulary, map it to the corresponding answer space, and take the label of the answer word with the highest probability as the prediction result;

[0010] S4. Construct an event causal relationship recognition model based on contextual contrast learning: Through the above steps, construct an event causal relationship recognition model;

[0011] S5. Training and Testing the Network: Based on reducing the distance between positive samples and the event representations of the samples to be predicted, increasing the distance between negative samples and the event representations of the samples to be predicted, and minimizing the distance based on h. [MASK] The goal is to train an event causal relationship recognition model using the training set, with the difference between the predicted missing words and the true labels as the objective.

[0012] In the preferred embodiment, in step S1, according to the designed template T q (q), T g (d k ), the sample to be predicted q and the context sample d k Templated and concatenated to obtain T i Specifically, it includes the following steps:

[0013] S01. First, template the sample q to be predicted and the context sample d respectively according to the following template: k Template them respectively:

[0014]

[0015]

[0016] where CLS and SEP are special tokens in the pre-trained language model, representing the beginning and end of a sentence respectively, S1 and S2 represent the sentences where events E1 and E2 are located, and y k ∈ {<causal>, <none>} is the true label of the context sample d k , representing that there is a causal relationship or no causal relationship between event pairs respectively;

[0017] S02. Concatenate the templated context sample and the sample to be predicted. Among them, the context sample with the label of having a causal relationship is in the front, and then concatenate the context sample with the label of having no causal relationship and the sample to be predicted in turn. Finally, obtain the model input T i :

[0018]

[0019] where the context sample contains M causal samples and N non-causal samples To maintain consistency during the training process, non-causal samples are always concatenated after causal samples. And during the training process, for the same sample to be predicted, different samples will be randomly selected as context samples in different training rounds. While in the test phase, to maintain the consistency of the evaluation criteria between different rounds, the same context samples will be selected for the same sample to be predicted and concatenated in the same order as the model input.

[0020] In the preferred solution, in step S2, input T i into the pre-trained language model to obtain the representation vectors of each event in the input and the hidden layer vector of [MASK]. Among them, the event pair representation from is denoted as The event pair representation from is denoted as The event pair representation from q is denoted as The representation of the special token [MASK] is denoted as h [MASK] ;

[0021] The pre-trained language model is the RoBERTa model.

[0022] In the preferred embodiment, the comparison module in step S3 selects positive and negative samples from the selected context samples based on the label selection samples during the comparison learning process; that is, causal samples from the context samples. Positive samples, non-causal samples The negative samples are then used as the initial input for the comparison module to extract relational information. This extracted relational information is then used for comparisonal learning, specifically including the following sub-steps:

[0023] S11. The event representations obtained in step S2 are used as the initial input to the comparison module, and the relationship information between event pairs in the same sample is extracted by subtraction for comparison learning. The relationship extraction method is as follows:

[0024]

[0025]

[0026]

[0027] S12. The extracted relational information is used for contrastive learning, and the loss is calculated as shown in the following formula:

[0028]

[0029] in τ is the temperature coefficient.

[0030] In the preferred embodiment, the prediction module in step S3 specifically includes the following sub-steps:

[0031] S21. Input the hidden vector of the special label [MASK] into the classifier for prediction. First, obtain the distribution probability of the missing word at [MASK] in the full vocabulary V:

[0032] P([MASK]=v d ∈V|T i (x)) (8)

[0033] S22. Then, map the probability of the entire vocabulary to the answer space V, which consists of two virtual words {<causal>, <none>}. a :

[0034]

[0035] Finally, the missing word at [MASK] is found in the answer space V. a The probability distribution is used to select words with high probabilities as the final prediction, and these words are mapped to either a causal relationship {<causal>} or no causal relationship {<none>}.

[0036] In the preferred embodiment, in step S5, a loss function L is constructed between the predicted label in S3 and the corresponding true label. pre The loss function L pre And the loss function L in the comparison module con Joint training is performed to train the event causality recognition model in a way that minimizes the loss function;

[0037] The training samples in the training set mainly consist of event pairs, the sentences containing the events, and their corresponding real labels.

[0038] This invention also provides a contextual contrast learning recognition method, which selects appropriate contextual samples based on the data to be predicted and uses template T respectively. q (q) and T g (d k After being templated, the data is concatenated and used as model input. The input context is compared and the steps of the recognition model construction method are then used to obtain the recognition result.

[0039] The present invention also provides a system for constructing a recognition model based on contextual comparison learning, including a computer-readable storage medium and a processor;

[0040] The computer-readable storage medium is used to store executable instructions;

[0041] The processor is used to read executable instructions stored in the computer-readable storage medium and execute the steps of the method for constructing a contextual comparison learning recognition model.

[0042] This invention also provides a contextual comparison learning-based event causal relationship recognition system, which selects appropriate contextual samples based on the data to be predicted and uses template T respectively. q (q) and T g (d k After being templated, the data is concatenated and used as input to the model. This input is then fed into a recognition method based on contextual contrast learning to obtain the recognition result.

[0043] The present invention also provides a computer-readable storage medium storing computer instructions for causing a processor to execute a method for constructing a context-contrast learning recognition model or a context-contrast learning recognition method.

[0044] This invention provides a method for constructing a contextual contrastive learning recognition model and its application. It introduces explicit guidance to a pre-trained language model by splicing labeled contextual samples, thereby better mining and utilizing the implicit semantic knowledge within the pre-trained language model. Simultaneously, it introduces a contrastive learning paradigm using contextual samples as positive and negative samples, enhancing the model's focus on the key information of event pairs, improving the model's ability to model event pairs, and achieving better prediction results by performing classification and comparison of the introduced explicit guidance. Attached Figure Description

[0045] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0046] Figure 1 This is a schematic diagram of the structure of the event causality recognition model provided in the embodiments of the present invention;

[0047] Figure 2 This is a schematic diagram of the construction method of the event causal relationship recognition model based on contextual comparison learning provided in the embodiments of the present invention;

[0048] Figure 3 This describes the performance of the model provided in the embodiments of the present invention when using different numbers of causal and non-causal samples as contextual samples;

[0049] Figure 4 This describes the performance trends of the model provided in the embodiments of this invention and the ERGO model in low-resource scenarios.

[0050] Figure 5 This is a visualization of the event pair representation of the model provided in the embodiments of the present invention and other comparative models. Detailed Implementation

[0051] Example 1

[0052] like Figures 1-5 As shown, a method for constructing a contextual contrast learning recognition model includes the following steps:

[0053] S1, Template Building Module: Based on two custom templates T q (q), T g (d k ), and the context sample q and the sample to be predicted d k Templated and concatenated to obtain T i As input;

[0054] S2, Pre-trained network encoding: T i The pre-trained language model is input for encoding to obtain the representation vectors of events in each sample and [MASK] in the sample to be predicted, which are used as inputs to the contrastive learning module and the prediction module, respectively.

[0055] S3, Contrast Module and Prediction Module: In the contrast module, the representation of event pairs is optimized using a contrastive learning paradigm. The event representations obtained in step S3 serve as the initial input to the contrast module, extracting relational information, which is then used for contrastive learning. In the prediction module, based on h... [MASK] Predict the probability distribution of the missing word at [MASK] on the full vocabulary, map it to the corresponding answer space, and take the label of the answer word with the highest probability as the prediction result;

[0056] S4. Construct an event causal relationship recognition model based on contextual contrast learning: Through the above steps, construct an event causal relationship recognition model;

[0057] S5. Training and Testing the Network: Based on reducing the distance between positive samples and the event representations of the samples to be predicted, increasing the distance between negative samples and the event representations of the samples to be predicted, and minimizing the distance based on h. [MASK] The goal is to train an event causal relationship recognition model using the training set, with the difference between the predicted missing words and the true labels as the objective.

[0058] In the preferred embodiment, in step S1, according to the designed template T q (q), T g (d k ), the sample to be predicted q and the context sample d k Templated and concatenated to obtain T i Specifically, it includes the following steps:

[0059] S01. First, according to the following template, the sample to be predicted q and the context sample d are respectively... k Templated separately:

[0060]

[0061]

[0062] Here, CLS and SEP are special markers in the pre-trained language model, representing the beginning and end of a sentence, respectively; S1 and S2 represent the sentences containing events E1 and E2, respectively; and y k ∈{<causal>,<none>} represents the context sample d k The true labels represent whether there is a causal relationship or not between the event pairs;

[0063] S02. Concatenate the templated context samples and the samples to be predicted, with the context samples labeled as causal first, followed by the context samples labeled as non-causal and the samples to be predicted, to obtain the model input T. i :

[0064]

[0065] Among them, the context sample contains M causal samples. and N non-causal samples To maintain consistency during training, non-causal samples are always appended after causal samples. During training, different samples are randomly selected as context samples for the same sample to be predicted in different training rounds. In the testing phase, to maintain consistency of evaluation criteria between different rounds, the same context samples are selected for the same sample to be predicted and appended in the same order as the input to the model.

[0066] In the preferred embodiment, in step S2, T i Inputting the pre-trained language model yields the representation vector for each event in the input and the hidden layer vector of [MASK], where the vector comes from... The event is represented as From The event is represented as An event pair from q is denoted as The special symbol [MASK] is denoted as h. [MASK] ;

[0067] The pre-trained language model is the RoBERTa model.

[0068] In the preferred embodiment, the comparison module in step S3 selects positive and negative samples from the selected context samples based on the label selection samples during the comparison learning process; that is, causal samples from the context samples. Positive samples, non-causal samples The negative samples are then used as the initial input for the comparison module to extract relational information. This extracted relational information is then used for comparisonal learning, specifically including the following sub-steps:

[0069] S11. The event representations obtained in step S2 are used as the initial input to the comparison module, and the relationship information between event pairs in the same sample is extracted by subtraction for comparison learning. The relationship extraction method is as follows:

[0070]

[0071]

[0072]

[0073] S12. The extracted relational information is used for contrastive learning, and the loss is calculated as shown in the following formula:

[0074]

[0075] in τ is the temperature coefficient.

[0076] This loss function brings the event pair representation in the sample q to be predicted closer to the event pair representation of samples with the same label (positive samples) and further away from the event pair representation of samples with different labels (negative samples), making the model's encoding of event pairs more reasonable and enhancing the model's attention to and modeling ability for event pairs.

[0077] The contrast module is an event-based contrastive learning approach that further enhances the analogical relationship between contextual samples and samples to be predicted, thereby strengthening the event modeling capabilities of the RoBERTa model. Since label guidance is required during contrastive learning and the predicted labels are not directly output, the contrast module only takes effect during training.

[0078] In the preferred embodiment, the prediction module in step S3 specifically includes the following sub-steps:

[0079] S21. Input the hidden vector of the special label [MASK] into the classifier for prediction. First, obtain the distribution probability of the missing word at [MASK] in the full vocabulary V:

[0080] P([MASK]=v d ∈V|T i (x)) (8)

[0081] S22. Then, map the probability of the entire vocabulary to the answer space V, which consists of two virtual words {<causal>, <none>}. a :

[0082]

[0083] Finally, the missing word at [MASK] is found in the answer space V. a The probability distribution is used to select words with high probabilities as the final prediction, and these words are mapped to either a causal relationship {<causal>} or no causal relationship {<none>}.

[0084] In the preferred embodiment, in step S5, a loss function L is constructed between the predicted label in S3 and the corresponding true label. pre The loss function L pre And the loss function L in the comparison module con Joint training is performed to train the event causality recognition model in a way that minimizes the loss function;

[0085] The training samples in the training set mainly consist of event pairs, the sentences containing the events, and their corresponding real labels.

[0086] This invention also provides a contextual contrast learning recognition method, which selects appropriate contextual samples based on the data to be predicted and uses template T respectively. q (q) and T g (d k After being templated, the data is concatenated and used as model input. The input context is compared and the steps of the recognition model construction method are then used to obtain the recognition result.

[0087] Specifically, the method for establishing an event causal relationship recognition model based on contextual contrastive learning provided by the present invention includes: a text preprocessing step, a prompt template construction step, a pre-trained network encoding step, a contrastive learning step, a prediction step, and a network training and testing step.

[0088] Step 1: Text preprocessing steps: Preprocess the original input data in the dataset, extract event pairs, the sentences containing the event pairs, and select appropriate context samples.

[0089] Step 2, Constructing the Prompt Template: Based on two custom templates, the context sample and the sample to be predicted are templated and concatenated as input, including the following sub-steps:

[0090] Step 21: First, according to the template below, analyze the sample to be predicted q and the context sample d respectively. k Templated separately:

[0091]

[0092]

[0093] Here, CLS and SEP are special markers in the pre-trained language model, representing the beginning and end of a sentence, respectively; S1 and S2 represent the sentences containing events E1 and E2, respectively; and y k ∈{<causal>,<none>} represents the context sample d k The true labels represent whether there is a causal relationship or not between the event pairs.

[0094] Step 22: Concatenate the templated context samples and the samples to be predicted, with the context samples labeled as causal first, followed by the context samples labeled as non-causal and the samples to be predicted, to obtain the model input T. i :

[0095]

[0096] Step 3, Pre-trained network encoding steps: T i Inputting the pre-trained language model yields the representation vector for each event in the input and the hidden layer vector of [MASK], where the vector comes from... The event is represented as From The event is represented as An event pair from q is denoted as The special symbol [MASK] is denoted as h. [MASK] ;

[0097] Step 4, Contrastive Learning Step: We use contextual samples with the same label as the sample to be predicted as positive samples. Conversely, as negative samples The representation of event pairs is optimized using a contrastive learning paradigm. The event representations are used as initial input to the contrastive module to extract relational information, which is then used for contrastive learning. This process includes the following sub-steps:

[0098] Step 41: We use the event representations obtained in the template construction module as the initial input to the comparison module. Furthermore, we extract the relationship information between event pairs in the same sample by subtraction for comparative learning. The relationship extraction method is shown in the figure below:

[0099]

[0100]

[0101]

[0102] Step 42: Use the extracted relational information for contrastive learning. The loss is calculated as shown in the following formula:

[0103]

[0104] in τ is the temperature coefficient.

[0105] Step 5, Prediction Step: Used to predict based on h [MASK] Predict the probability distribution of the missing word at [MASK] on the full vocabulary and map it to the corresponding answer space. The label of the answer word with the highest probability is used as the prediction result. This includes the following sub-steps:

[0106] Step 51: Input the hidden vector of the special label [MASK] into the classifier for prediction. First, obtain the distribution probability of the missing word at [MASK] in the full vocabulary V:

[0107] P([MASK]=v d ∈V|T i (x)) (8)

[0108] Step 52: Then map the probabilities of the entire vocabulary to the answer space V, which consists of two virtual words {<causal>, <none>}. a :

[0109]

[0110] Finally, the missing word at [MASK] is found in the answer space V. a The probability distribution is used to select words with high probabilities as the final prediction and map them to either a causal relationship {<causal>} or no causal relationship {<none>}.

[0111] Step 6, Training and Testing the Network: The labels are determined based on the probability distribution of the predicted [MASK] in the answer space. A loss function is constructed using the true label y, and the cross-entropy loss function is adopted, which is expressed by the following formula:

[0112]

[0113] Among them, y (k) and λ and θ are the true and predicted labels of the k-th event pair in the document, respectively. λ and θ are regularization hyperparameters.

[0114] The comparison module and the prediction module respectively calculate two losses L. con and L pre During training, the two modules are trained jointly, and their losses are summed before gradient calculation to update the model parameters.

[0115] L total =L pre +β*L con (10)

[0116] Where β is the loss coefficient of the comparison module, the model is then trained by minimizing the loss function.

[0117] The dataset includes a training set, a validation set, and a test set. After training the model on the training set, the model is input into the validation and test sets. The model with the highest F1 score on the validation set is selected to obtain the event causality prediction results for the corresponding test samples.

[0118] The present invention also provides a system for constructing a recognition model based on contextual comparison learning, including a computer-readable storage medium and a processor;

[0119] The computer-readable storage medium is used to store executable instructions;

[0120] The processor is used to read executable instructions stored in the computer-readable storage medium and execute the steps of the method for constructing a contextual comparison learning recognition model.

[0121] This invention also provides a contextual comparison learning-based event causal relationship recognition system, which selects appropriate contextual samples based on the data to be predicted and uses template T respectively. q (q) and T g (d k After being templated, the data is concatenated and used as input to the model. This input is then fed into a recognition method based on contextual contrast learning to obtain the recognition result.

[0122] The present invention also provides a computer-readable storage medium storing computer instructions for causing a processor to execute a method for constructing a context-contrast learning recognition model or a context-contrast learning recognition method.

[0123] Example 2

[0124] To further illustrate this point, using two commonly used datasets—Event Storyline (ESL) and Causal Time Database (CTB)—widely applied to event causality recognition tasks, as examples, we will demonstrate the performance of the context-based contrastive learning-based event causality recognition method proposed in this invention. ESL contains 258 documents collected from news websites, divided into 22 topics, containing 5334 events. 5655 event pairs are labeled as causal relationships, including 1770 intra-sentence causal relationships and 3855 inter-sentence causal relationships. The data corresponding to the first 20 topics are used for 5-fold cross-validation, and the last two folds are used as the validation set. CTB contains 184 documents collected from English articles, labeled with 7608 events, of which 318 are labeled as causal relationships, including 300 intra-sentence causal relationships and 18 inter-sentence causal relationships. Following existing research methods, the CTB dataset requires a 10-fold cross-validation approach. In this experiment, we use precision (P), recall (R), and F1 score as performance metrics.

[0125] Experiments were conducted using a 768-dimensional pre-trained RoBERTa and BERT model provided by HuggingFace transformers, running on an NVIDIA GTX 3090Ti GPU with the PyTorch framework and CUDA. RoBERTa, proposed by Facebook, is a language model pre-trained in an unsupervised manner using a cloze test on a large amount of unlabeled text. The learning rate, β, of the main experimental model was set to {1e-5, 0.5}, where β represents the importance of contrastive learning during training. All trainable parameters were randomly initialized from a normal distribution. We used an L2-regularized Adam optimizer combined with dropout for model training. Other hyperparameters were used; when selecting contextual samples, we chose two causal samples and two non-causal samples. Table 1 shows the performance of the event causality recognition model and competing models on the ESL dataset for intra-sentence, inter-sentence, and overall event causality recognition. ICCL is an abbreviation for the event causality recognition model.

[0126] Table 1 compares the recognition performance of the model in this embodiment and competing models on the ESL dataset for intra-sentence, inter-sentence, and overall events.

[0127]

[0128]

[0129] Table 2 compares the recognition performance of the model in this embodiment with competing models on the ESL and CTB datasets within sentences.

[0130]

[0131] Tables 1 and 2 show a performance comparison between the proposed event causality recognition model and existing competing models in terms of accuracy and F1 score. Table 1 reports the performance of the proposed model and competing models in identifying intra-sentence, inter-sentence, and overall event causality on the ESL dataset. Table 2 shows a performance comparison between the proposed model and existing competing models in identifying intra-sentence causality on the ESL and CTB datasets. As can be seen from the tables, the proposed model significantly outperforms existing competing models. This indicates that the context-based contrastive learning-based event causality recognition model, by introducing contextual samples as explicit guidance and using contrastive learning to improve the model's attention to events, is helpful in identifying event causality.

[0132] Comparative Example 1

[0133] To compare the importance of the contrastive learning and contextual samples modules proposed in this invention, and to determine whether selecting event pair representations as input has a positive impact on the model's ability to better identify causal relationships, this invention conducted ablation experiments to test the event causal relationship identification performance of five different schemes:

[0134] (1) Prompt word scheme: This model only uses the prompt learning paradigm to complete the task of identifying the cause-and-effect relationship of events with a simple prompt template;

[0135] (2) Context-specific approach: Compared with the event causal relationship recognition model, this model does not use the contrastive learning module. In some experiments, this model is referred to as the PI model.

[0136] (3) No context sample scheme: This model does not introduce context samples as explicit guidance. When performing contrastive learning, positive and negative samples come from other samples in the same batch. The input of the contrast module uses the hidden layer vector of [MASK].

[0137] (4) Introducing contextual samples: Compared with the no-contextual-sample scheme, this model introduces contextual samples. The positive and negative samples for contrast learning come from contextual samples, and the hidden layer vector of [MASK] is used as the input of the contrast module.

[0138] (5) Event scheme: This model does not introduce contextual samples as explicit guidance. When performing contrastive learning, positive and negative samples come from other samples in the same batch. The input of the contrast module uses the hidden vector of the event pair.

[0139] Table 3 Comparison of ablation experiment results

[0140]

[0141] Comparing the results in Table 3, we can see that the model incorporating contextual samples as explicit guidance performs better. For example, the contextual sample approach outperforms the non-contextual sample approach in all metrics across various datasets. This demonstrates the correctness of our proposed method of providing explicit guidance through contextual samples to further explore causal knowledge in pre-trained language models. Furthermore, the model incorporating a contrastive learning module performs better than the model without it. For instance, the event causal relationship recognition model outperforms the specific contextual approach model in all metrics. This indicates that the contrastive learning paradigm does indeed help the model better focus on certain keywords (event pairs, [MASK]). Further, we found that using the event pair representation as input to the contrastive module is more effective than using the hidden vector of [MASK]. This is because the hidden vector of [MASK] is also the input to the prediction module, and the prediction and contrastive modules do not optimize the pre-trained language model in the same direction. Using the same input would cause the two modules to interfere with each other, thus degrading performance.

[0142] To further explore the impact of selecting different proportions and numbers of contextual samples on model performance, Figure 3 The performance of the event causality recognition model proposed in this invention is shown in the form of bar and line charts when using different numbers and proportions of context samples. The model uses RoBERTa as the encoder. The horizontal axis represents the ratio of causal samples to non-causal samples in the context samples used by the model. For example, 2 / 1 means that 2 causal samples and 1 non-causal sample are selected for each sample. The height of the bars represents the accuracy and recall of the model prediction, and the line chart represents the F1 score of the model prediction.

[0143] The comparison of the results shows that the performance of the event causality recognition model gradually improves as the total number of context samples increases. However, the PI model (an event causality recognition model without contrastive learning, detailed below) shows a performance decline after a large number of context samples. This is because as the number of context samples increases, the model's input becomes longer, and the pre-trained language model's attention to event pairs is diverted, leading to performance degradation. The event causality recognition model, by applying a contrastive learning paradigm, encourages the model to focus more on events, mitigating the problem of decreased attention to event pairs by the pre-trained language model, thus maintaining performance improvement. Furthermore, the model with a 2 / 1 ratio performs better than the model with a 1 / 2 ratio. This is because the number of causal samples in the dataset is far less than the number of non-causal samples. Selecting more causal samples as context samples during training helps the model better understand and model the features of causal samples, thereby improving overall performance.

[0144] Figure 4This paper compares the performance of the event causality recognition model and the ERGO model in low-resource scenarios. The ERGO model applies a prompting learning paradigm and designs complex prompt templates to solve the event causality recognition task. The figure shows the performance (F1 score) trends of the two models in low-resource scenarios for inter-sentence event causality recognition, intra-sentence event causality recognition, and overall event causality recognition.

[0145] The comparison results in the figure show that although our proposed event causality recognition model only uses simple prompt templates, the introduction of labeled contextual samples as explicit guidance and the use of contrastive learning to improve the model's attention to events result in a performance decrease of only about 10% as the training data decreases from 100% to 20%. In contrast, the ERGO model experiences a performance decrease of about 25% as the training data decreases from 100% to 20%. This demonstrates that our proposed model has good adaptability to low-resource scenarios.

[0146] Figure 5 This diagram visualizes the vector representations of event pairs modeled by the event causality identification model and several other comparative models. We use t-SEN to reduce the dimensionality of the vectors in the high-dimensional space to a 2D plane. Causal samples are represented as positive, and non-causal samples as negative. The four different colored points represent true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN), respectively.

[0147] The comparison results in the figure show that for the model using the comparison module, most event pairs with the same label cluster together, with only a small number of samples remaining outside the clusters. This indicates that the comparison module effectively adjusts the event pair representations, making event pairs with the same label more similar, and conversely, enhancing their differences. Furthermore, we can observe a clear trend of clustering among all event pairs predicted to have the same label. This suggests that the representation of event pairs plays a crucial role in determining whether there is a causal relationship between them. This finding aligns with our initial intention of using event pair representations as input to the comparison module and also with our intuitive understanding.

[0148] In summary, the method provided by this invention is based on a prompting learning paradigm. It introduces contextual samples as explicit guidance on top of simple prompt templates. At the same time, it uses a contrast module with event pairs as input to enhance the model's attention to event pairs and make fuller use of contextual samples. This not only improves the recognition of causal relationships between events, but also represents a new exploration of ways to mine the implicit knowledge of pre-trained language models.

[0149] The above embodiments are merely preferred technical solutions of the present invention and should not be considered as limitations on the present invention. The scope of protection of the present invention should be limited to the technical solutions described in the claims, including equivalent substitutions of the technical features described in the claims. That is, equivalent substitutions and improvements within this scope are also within the scope of protection of the present invention.

Claims

1. A method for constructing a contextual contrast learning recognition model, characterized by: Includes the following steps: S1, Template Building Module: Based on two types of custom templates , , context sample Samples to be predicted Templated and assembled As input; specifically including the following steps: S01、First, the samples to be predicted and the context samples are respectively templated as follows: （1） （2） in, and These are special markers used in pre-trained language models to represent the beginning and end of a sentence, respectively. and Representative event and The sentence in which it appears For context samples The true labels represent whether there is a causal relationship or not between the event pairs; S02. Concatenate the templated context samples and the samples to be predicted, with the context samples labeled as causal first, followed by the context samples labeled as non-causal and the samples to be predicted, to obtain the final model input. : （3） Among them, the context samples include causal samples and Non-causal samples To maintain consistency during training, non-causal samples are always appended to causal samples. During training, different samples are randomly selected as context samples for the same sample to be predicted in different training rounds. In the testing phase, to maintain consistency of evaluation criteria between different rounds, the same context samples are selected for the same sample to be predicted and appended in the same order as the input to the model. S2, Pre-trained network encoding: [The text abruptly ends here, likely due to an incomplete sentence or a format The input is encoded into a pre-trained language model to obtain the events in each sample and the events in the sample to be predicted. The representation vectors are used as inputs to the contrastive learning module and the prediction module, respectively; in step S2, the vectors are... Inputting the pre-trained language model yields a representation vector for each event in the input, and The hidden layer vector, where, comes from The event is represented as , from The event is represented as , from The event is represented as , The hidden layer vector is denoted as ; The pre-trained language model is a RoBERTa model; S3. Contrast Module and Prediction Module: In the contrast module, the representation of event pairs is optimized using a contrastive learning paradigm. The event representations obtained in step S3 serve as the initial input to the contrast module, extracting relational information, which is then used for contrastive learning. Specifically, it includes the following sub-steps: S11. The event representations obtained in step S2 are used as the initial input to the comparison module, and the relationship information between event pairs in the same sample is extracted by subtraction for comparison learning. The relationship extraction method is as follows: （4） （5） （6） S12. The extracted relational information is used for contrastive learning, and the loss is calculated as shown in the following formula: （7） in , Temperature coefficient; In the prediction module, according to predict The probability distribution of missing words on the full vocabulary is determined and mapped to the corresponding answer space. The label corresponding to the answer word with the highest probability is used as the prediction result. S4. Construct an event causal relationship recognition model based on contextual contrast learning: Through the above steps, construct an event causal relationship recognition model; S5. Training and Testing the Network: Based on reducing the distance between positive samples and the event representations of the samples to be predicted, increasing the distance between negative samples and the event representations of the samples to be predicted, and minimizing the distance between positive samples and the event representations of the samples to be predicted. The goal is to train an event causal relationship recognition model using the training set, with the difference between the predicted missing words and the true labels as the objective.

2. The method for constructing a contextual contrast learning recognition model according to claim 1, characterized in that: The prediction module in step S3 specifically includes the following sub-steps: S21, Special Marking The hidden layer vectors are input into the classifier for prediction, first obtaining... Missing words in the complete word list The probability distribution: （8） S22, then map the full vocabulary probability to the two virtual words. The answer space constituted : （9） Finally obtained Missing words in the answer space The probability distribution is used to select words with high probabilities as the final prediction, and these words are then mapped to words with causal relationships. Or no causal relationship .

3. The method for constructing a contextual contrast learning recognition model according to claim 1, characterized in that: In step S5, a loss function is constructed between the predicted label in S3 and the corresponding true label. , loss function loss function in comparison module Joint training is performed to train the event causality recognition model in a way that minimizes the loss function; The training samples in the training set mainly consist of event pairs, the sentences containing the events, and their corresponding real labels.

4. A contextual contrast learning-based recognition method, characterized by: Based on the data to be predicted, select appropriate context samples and apply templates accordingly. and After templated processing, the data is concatenated and used as model input. This data is then fed into the contextual contrast learning recognition model constructed by the method described in any one of claims 1-3 to obtain the recognition result.

5. A system for constructing a recognition model based on contextual contrast learning, characterized by: Includes computer-readable storage media and processors; The computer-readable storage medium is used to store executable instructions; The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the method as described in any one of claims 1-3.

6. A contextual contrast learning-based event causal relationship recognition system, characterized by: Based on the data to be predicted, select appropriate context samples and apply templates accordingly. and After templated processing, the data is concatenated and used as model input. This input is then fed into the contextual contrast learning recognition model constructed using the method described in claim 4 to obtain the recognition result.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions for causing a processor to execute the method for constructing a recognition model of contextual contrast learning as described in any one of claims 1-3 or the recognition method of contextual contrast learning as described in claim 4.

Images