Denoising causal relation extraction method and device based on multi-task cooperative learning

Abstract

This invention discloses a method for extracting causal relationships, specifically a method and apparatus for extracting denoised causal relationships based on multi-task collaborative learning. The method includes: inputting the text to be processed into a task-sharing layer of a denoised causal relationship extraction model based on multi-task collaborative learning for feature extraction, obtaining extraction results; inputting the extraction results into a specific task layer of the denoised causal relationship extraction model based on multi-task collaborative learning for part-of-speech tagging, chunking, and causal extraction tasks, respectively obtaining feature labels corresponding to the part-of-speech tagging task, the chunking task, and the causal extraction task; wherein the denoised causal relationship extraction model based on multi-task collaborative learning is trained in parallel for the part-of-speech tagging task, the chunking task, and the causal extraction task. This invention can improve the accuracy of causal relationship extraction.

Description

Technical Field

[0001] This invention relates to a causal relationship extraction method, specifically a denoising causal relationship extraction method and apparatus based on multi-task collaborative learning. Background Technology

[0002] Event Causality Extraction (CE), a crucial task in Natural Language Processing (NLP), plays an indispensable role in various downstream applications, such as knowledge graphs, event logic graphs, and question-answering systems. Extracting causal relationships is quite challenging because it requires capturing rich semantic information and complex linguistic phenomena. Traditional methods utilize sequence labeling tasks to perform CE, and these methods have achieved significantly better performance than conventional approaches. Furthermore, introducing pre-trained language models, such as ELMo and BERT, can further enhance the model's understanding capabilities, thereby improving its performance.

[0003] However, these methods often face the problem of noisy data. This is mainly because some words in the text are unlikely to be labeled as "cause" or "effect." Semantic components such as adjectives and prepositions cannot point to a specific entity, introducing noise into the labeling of causal entities. Traditional sequence labeling frameworks often make incorrect predictions, indiscriminately predicting some adjectives and prepositions as "cause" or "effect." For example, experiments were conducted on the SemEval-2010 Task 8 dataset, the Event StoryLine corpus, and Causal TimeBank, using a BiLSTM+CRF model for sequence labeling to study the impact of noisy data such as adjectives and prepositions on model performance. Experimental results show that in the first two samples of the SemEval-2010 Task8 dataset, BERT labeled "blind" as "B-Cause" without considering the part of speech of the words. A similar problem exists in the Causal TimeBank case. Furthermore, in the Event StoryLine sample, "off" was labeled as "B-Cause". Therefore, traditional sequence labeling frameworks often make incorrect predictions, indiscriminately predicting some adjectives and prepositions as "cause" or "effect".

[0004] Therefore, it can be seen that the commonly used causal extraction methods in the current technology are mainly limited to the extraction of causal relationships in a single domain, and cannot effectively handle causal relationships in multiple domains, resulting in low extraction accuracy. Summary of the Invention

[0005] The technical problem to be solved by the present invention is that the causal extraction methods commonly used in the prior art are mainly limited to the extraction of causal relationships in a single domain, and cannot effectively handle causal relationships in multiple domains, resulting in low extraction accuracy. In order to solve the above problems, the present invention provides a denoising causal relationship extraction method and apparatus based on multi-task collaborative learning.

[0006] The content of this invention includes:

[0007] In a first aspect, embodiments of the present invention provide a method for denoising causal relationship extraction based on multi-task collaborative learning, including:

[0008] The text to be processed is input into the task-sharing layer of a multi-task collaborative learning denoising causal relationship extraction model for feature extraction, and the extraction result is obtained.

[0009] The extraction results are input into a specific task layer of the denoising causal relationship extraction model based on multi-task collaborative learning to perform part-of-speech tagging, chunking analysis, and causal extraction tasks, respectively, to obtain the feature labels corresponding to the part-of-speech tagging task, the feature labels corresponding to the chunking analysis task, and the feature labels corresponding to the causal extraction task.

[0010] The denoising causal relationship extraction model based on multi-task collaborative learning is obtained by training the part-of-speech tagging task, the chunking analysis task, and the causal extraction task in parallel.

[0011] Optionally, the denoising causal relationship extraction model based on multi-task collaborative learning includes:

[0012] The part-of-speech tagging task subnet includes a first task-shared sublayer and a first specific task sublayer. The first task-shared sublayer is used to extract features from the text to be processed to obtain a first extraction result. The first specific task layer is used to perform part-of-speech tagging based on the first extraction result to obtain the feature label corresponding to the part-of-speech tagging task.

[0013] The chunking analysis task subnet includes a second task sharing sublayer and a second specific task sublayer. The second task sharing sublayer is used to extract features from the text to be processed to obtain a second extraction result. The second specific task sublayer is used to perform chunking analysis based on the second extraction result to obtain the feature label corresponding to the chunking analysis task.

[0014] The causal relationship extraction task subnet includes a third task sharing sublayer and a third specific task sublayer. The third task sharing sublayer is used to extract features from the text to be processed to obtain a third extraction result. The third feature task layer is used to extract causal relationships based on the third extraction result to obtain the feature labels corresponding to the causal relationship extraction task.

[0015] The task sharing layer includes a first task sharing sublayer, a second task sharing sublayer, and a third task sharing sublayer, and the specific task layer includes a first specific task sublayer, a second specific task sublayer, and a third specific task sublayer.

[0016] Optionally, the first task-sharing sublayer includes a first BERT semantic extraction layer and a first BiLSTM layer connected in sequence, the second task-sharing sublayer includes a second BERT semantic extraction layer and a second BiLSTM layer connected in sequence, and the third task-sharing sublayer includes a third BERT semantic extraction layer and a third BiLSTM layer connected in sequence.

[0017] Among them, the first BERT semantic extraction layer, the second BERT semantic extraction layer and the third BERT semantic extraction layer have some of the same parameters, and the first BiLSTM layer, the second BiLSTM layer and the third BiLSTM layer have some of the same parameters.

[0018] Optionally, the first specific task sub-layer includes a first feature transformation layer and a first label prediction layer connected in sequence, the second specific task sub-layer includes a second feature transformation layer and a second label prediction layer connected in sequence, and the third specific task sub-layer includes a third feature transformation layer and a third label prediction layer connected in sequence.

[0019] Optionally, the method further includes:

[0020] The denoised causal relationship extraction model based on multi-task collaborative learning is iteratively trained to obtain the denoised causal relationship extraction model based on multi-task collaborative learning.

[0021] During the iterative training process, some parameters of the part-of-speech tagging task subnet, the chunking analysis task subnet, and the causal relationship extraction task subnet are iteratively pruned.

[0022] Optionally, the loss function for iteratively training the denoising causal relationship extraction model based on multi-task collaborative learning is:

[0023] loss task ={loss CE loss pos tagging losschunk analysis};

[0024] loss task =-logp(Y true |X);

[0025]

[0026] Among them, Y true Y is used to represent the true labeled sequence. X Let s(·) represent all possible label sequences, and let s(·) represent the score of the label sequence. The loss function is used to represent the loss function. CE The loss function used to represent the loss generated by the causal extraction task. pos tagging The loss function used to represent the part-of-speech tagging task is loss. chunk analysis This is used to represent the loss function generated by the chunking analysis task.

[0027] Optionally, during the nth iteration of training, the pruning rate of some parameters of the part-of-speech tagging task subnet, the chunking analysis task subnet, and the causal relationship extraction task subnet satisfies the following:

[0028]

[0029] Where p represents the pruning rate for each iteration, and α represents the percentage of parameters retained in the part-of-speech tagging task subnet, the chunking analysis task subnet, and the causal relationship extraction task subnet after iterative training.

[0030] Secondly, embodiments of the present invention provide a denoising causal relationship extraction device based on multi-task collaborative learning, comprising:

[0031] The feature extraction module is used to extract features from the task-sharing layer of the denoising causal relationship extraction model based on multi-task collaborative learning, and obtain the extraction results.

[0032] The specific task module is used to input the extraction results into the specific task layer of the denoising causal relationship extraction model based on multi-task collaborative learning to perform part-of-speech tagging, chunking analysis and causal extraction tasks, respectively to obtain the feature labels corresponding to the part-of-speech tagging task, the feature labels corresponding to the chunking analysis task and the feature labels corresponding to the causal extraction task.

[0033] The denoising causal relationship extraction model based on multi-task collaborative learning is obtained by training the part-of-speech tagging task, the chunking analysis task, and the causal extraction task in parallel.

[0034] Thirdly, embodiments of the present invention provide an electronic device, including: a memory, a processor, and a program stored in the memory and executable on the processor; the processor is configured to read the program in the memory to implement the steps in the denoising causal relationship extraction method based on multi-task collaborative learning as described in the first aspect.

[0035] Fourthly, embodiments of the present invention provide a readable storage medium for storing a program, which, when executed by a processor, implements the steps in the denoising causal relationship extraction method based on multi-task collaborative learning as described in the first aspect.

[0036] In this embodiment of the invention, the text to be processed is input into the task-sharing layer of a denoising causal relationship extraction model based on multi-task collaborative learning for feature extraction, and the extraction result is obtained. The extraction result is input into a specific task layer of the denoising causal relationship extraction model based on multi-task collaborative learning for part-of-speech tagging, chunking, and causal extraction tasks, respectively, to obtain feature labels corresponding to the part-of-speech tagging, chunking, and causal extraction tasks. The denoising causal relationship extraction model based on multi-task collaborative learning is obtained by parallel training of the part-of-speech tagging, chunking, and causal extraction tasks.

[0037] The beneficial effects of this invention are that the multi-task collaborative learning denoising causal relation extraction model simultaneously performs part-of-speech tagging, chunking, and causal extraction tasks. Through parallel training of these three tasks, the model can learn information from the part-of-speech tagging and chunking tasks related to causal extraction. The trained multi-task collaborative learning denoising causal relation extraction model can utilize shared knowledge and complementary information between different but related tasks to alleviate noise in causal relation extraction, while simultaneously acquiring additional relevant semantic information, reducing unreasonable causal relationships, and improving the accuracy of causal relation extraction. Furthermore, multi-task learning enables the capture of more complex semantic interactions and the mastery of fine-grained semantic information, allowing the model to effectively adapt to different data domains and exhibit better adaptability. Attached Figure Description

[0038] Appendix Figure 1 A flowchart illustrating a denoising causal relationship extraction method based on multi-task collaborative learning provided in an embodiment of the present invention;

[0039] Appendix Figure 2 A schematic diagram of the structure of a multi-task collaborative learning denoising causal relationship extraction model provided in an embodiment of the present invention;

[0040] Appendix Figure 3 This shows how the model's performance changes with epochs on the SemEval-2010 Task 8 dataset.

[0041] Appendix Figure 4 The performance of the model on the MTL-CE dataset changes with epochs;

[0042] Appendix Figure 5 The model performance under different pruning iterations on the emEval-2010 Task 8 dataset;

[0043] Appendix Figure 6 The model performance on the MTL-CE dataset under different iterations of pruning;

[0044] Appendix Figure 7 A schematic diagram of a noise reduction and causal relationship extraction device based on multi-task collaborative learning provided in an embodiment of the present invention;

[0045] Appendix Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0046] In this application's embodiments, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship. In this application's embodiments, the term "multiple" refers to two or more, and other quantifiers are similar. The terms "first," "second," etc., in the specification and claims of this application are used to distinguish similar objects and are not used to describe a specific order or sequence. It should be understood that such terms can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, not limited to the number of objects. For example, the first object can be one or multiple.

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0048] Please see Figure 1 , Figure 1 This is a flowchart illustrating the denoising and causal relationship extraction method based on multi-task collaborative learning provided in an embodiment of the present invention. The method specifically includes the following steps:

[0049] Step 101: Extract features from the task-sharing layer of the denoising causal relationship extraction model based on multi-task collaborative learning, and obtain the extraction results.

[0050] Step 102: Input the extraction results into a specific task layer of the denoising causal relationship extraction model based on multi-task collaborative learning to perform part-of-speech tagging, chunking analysis, and causal extraction tasks, respectively, to obtain the feature labels corresponding to the part-of-speech tagging task, the feature labels corresponding to the chunking analysis task, and the feature labels corresponding to the causal extraction task.

[0051] The denoising causal relationship extraction model based on multi-task collaborative learning is obtained by training the part-of-speech tagging task, the chunking analysis task, and the causal extraction task in parallel.

[0052] The method provided in this application is mainly used for causal relationship extraction. The output result of the causal relationship extraction task is the result obtained from the causal relationship extraction. Part-of-speech tagging and chunking analysis are trained in parallel with the causal relationship extraction task as co-training tasks. This can remove noise from a large number of semantic components and improve the model's performance by learning information from other tasks related to the causal relationship extraction task. This helps the model locate causal entities and achieve causal relationship extraction.

[0053] Specifically, Part-of-Speech Tagging (POS) assigns grammatical labels to words in a sentence. Most causal relationships are nouns (NNs in POS labels). POS tagging helps the model eliminate semantically inconsistent tags when locating "cause" and "effect" entities. Chunk Analysis groups words in a sentence into meaningful chunks, segmenting different semantic components across several spans. This also helps to minimize unreasonable causal relationships based on POS tagging results. Furthermore, multi-task learning can capture more complex semantic interactions and grasp fine-grained semantic information, enabling the model to effectively adapt to different data domains and exhibit better adaptability.

[0054] Please see Figure 2 The Multi-task with POS and Chunk for Causal Relation (MPC-CE) model based on multi-task collaborative learning in this embodiment includes a task-sharing layer and a task-specific layer. First, the text to be processed is input into the task-sharing layer, which extracts common features for prediction across all tasks. Then, the task-specific layer extracts the features required for specific task label prediction, namely the label features corresponding to part-of-speech tagging, chunking analysis, and causal extraction tasks, thereby completing the part-of-speech tagging, chunking analysis, and causal extraction tasks and obtaining the output results. In this embodiment, the output results of each task are typically in the form of feature labels.

[0055] Optionally, in some embodiments, the denoising causal relationship extraction model based on multi-task collaborative learning includes:

[0056] The part-of-speech tagging task subnet includes a first task-shared sublayer and a first specific task sublayer. The first task-shared sublayer is used to extract features from the text to be processed to obtain a first extraction result. The first specific task layer is used to perform part-of-speech tagging based on the first extraction result to obtain the feature label corresponding to the part-of-speech tagging task.

[0057] The chunking analysis task subnet includes a second task sharing sublayer and a second specific task sublayer. The second task sharing sublayer is used to extract features from the text to be processed to obtain a second extraction result. The second specific task sublayer is used to perform chunking analysis based on the second extraction result to obtain the feature label corresponding to the chunking analysis task.

[0058] The causal relationship extraction task subnet includes a third task sharing sublayer and a third specific task sublayer. The third task sharing sublayer is used to extract features from the text to be processed to obtain a third extraction result. The third feature task layer is used to extract causal relationships based on the third extraction result to obtain the feature labels corresponding to the causal relationship extraction task.

[0059] The task sharing layer includes a first task sharing sublayer, a second task sharing sublayer, and a third task sharing sublayer, and the specific task layer includes a first specific task sublayer, a second specific task sublayer, and a third specific task sublayer.

[0060] It should be understood that, in this embodiment of the application, a part-of-speech tagging task subnet is provided for the part-of-speech tagging task, a chunking analysis task subnet is provided for the chunking analysis task, and a causal relationship extraction task subnet is provided for the causal extraction task. Each subnet has its own task-sharing sublayer and specific task sublayer. The task-sharing sublayers of the three subnets constitute the task-sharing layer, and the specific task sublayers of the three subnets constitute the specific task layer.

[0061] Optionally, in some embodiments, the first task-sharing sublayer includes a first BERT semantic extraction layer and a first BiLSTM layer connected in sequence, the second task-sharing sublayer includes a second BERT semantic extraction layer and a second BiLSTM layer connected in sequence, and the third task-sharing sublayer includes a third BERT semantic extraction layer and a third BiLSTM layer connected in sequence.

[0062] Among them, the first BERT semantic extraction layer, the second BERT semantic extraction layer and the third BERT semantic extraction layer have some of the same parameters, and the first BiLSTM layer, the second BiLSTM layer and the third BiLSTM layer have some of the same parameters.

[0063] The following sections introduce the BERT semantic extraction layers (first BERT semantic extraction layer, second BERT semantic extraction layer, and third BERT semantic extraction layer) and the BiLSTM layers (first BiLSTM layer, second BiLSTM layer, and third BiLSTM layer).

[0064] BERT is a language representation model that represents bidirectional encoder representations from Transformers. Its specific structure can be found in related technical descriptions and will not be elaborated upon here. In this embodiment, the BERT semantic extraction layer dynamically maps each word in the sentence to a feature vector based on contextual information.

[0065] For example, the text to be processed includes m words, and the text to be processed is denoted as the sequence x = (x1, x2, ..., xm). m The BERT semantic extraction layer takes the input sequence x = (x1, x2, ..., x...) as input. m The mapping is:

[0066] t = (t1, t2, ..., t1);

[0067] In its implementation, the BERT mapping process takes into account various factors, such as polysemous words and syntactic features of sentences, which will not be elaborated here.

[0068] Bidirectional Long Short-Term Memory (BiLSTM) is an improved Recurrent Neural Network (RNN) specifically designed for processing sequential data. For each input sequence in a BiLSTM layer, both the forward Long Short-Term Memory (LSTM) and backward LSTM layers encode it into a hidden vector. In this embodiment, the two directional hidden states are concatenated to produce a complete sequence, which helps capture the positional information of each word in the sentence.

[0069] The output sequence of the hidden state of the forward LSTM is:

[0070]

[0071] The output sequence of the backward LSTM hidden state is:

[0072]

[0073] Combining the output sequences of the forward LSTM hidden state and the backward LSTM hidden state yields the complete output sequence h of the BiLSTM hidden state:

[0074]

[0075] h = (h1, h2, ..., h) m )∈R m×n ;

[0076] Where t represents the position of a word in a sentence, m represents the number of words in the sentence, and n represents the dimension of the vector.

[0077] It should be understood that the task-sharing sublayers of the three subnets all include a BERT semantic extraction layer and a BiLSTM layer connected in sequence, but the specific parameters of the BERT semantic extraction layer and BiLSTM layer included in the three subnets are not exactly the same.

[0078] Optionally, in some embodiments, the first specific task sub-layer includes a first feature conversion layer and a first label prediction layer connected in sequence, the second specific task sub-layer includes a second feature conversion layer and a second label prediction layer connected in sequence, and the third specific task sub-layer includes a third feature conversion layer and a third label prediction layer connected in sequence.

[0079] The following sections will introduce the feature transformation layers (first feature transformation layer, second feature transformation layer and third feature transformation layer) and the label prediction layers (first label prediction layer, second label prediction layer and third label prediction layer).

[0080] Following the BiLSTM layer, this embodiment employs a feature transformation layer to map the n-dimensional hidden state vector h to k dimensions. Specifically, the feature transformation layer extracts sentence features using a matrix, denoted as P:

[0081] P = (p1, p2, ..., p) m )∈R m×k ;

[0082] Here, k represents the number of labels, which varies depending on the task. For example, the CE task involves five different labels: "B-cause, I-cause, B-result, I-result" and "O".

[0083] The goal of the feature transformation layer is to transform an n-dimensional hidden state vector into a k-dimensional low-dimensional vector, thereby effectively extracting high-level features. By minimizing information loss in this process, it enables simultaneous training of multiple tasks.

[0084] The label prediction layer uses the Conditional Random Field (CRF) algorithm for label prediction, which calculates the label of each word in a given sequence based on the overall maximum probability. This is suitable for sequence labeling tasks with a large number of annotations.

[0085] Given an input sentence X = (x1, x2, x3, ..., x...), ... n ) and the target label sequence Y = (y1, y2, y3, ..., y n ), where x i It is the i-th word of X, y i It is x i The labels are defined as follows, where n is the length of the input sentence and the target label sequence. In this embodiment, the score for the entire label is defined as follows:

[0086]

[0087] Where s(·) calculates the score for a given label sequence, It is the emission matrix, used to represent the label vector generated from the BiLSTM layer, indicating which label should be the current x. i Mapping. This is a transition matrix that stores the scores for transitions from one label to another; higher scores indicate a more likely transition. k is the number of labels, and y0 and y... n These are the beginning and end of the sentence. The predicted label sequence can be obtained based on the score of the entire label sequence.

[0088] In this embodiment, the part-of-speech tagging task subnet includes a first BERT semantic extraction layer, a first BiLSTM layer, a first feature transformation layer, and a first label prediction layer. The chunking analysis task subnet includes a second BERT semantic extraction layer, a second BiLSTM layer, a second feature transformation layer, and a second label prediction layer. The causal relationship extraction task subnet includes a third BERT semantic extraction layer, a third BiLSTM layer, a third feature transformation layer, and a third label prediction layer. The specific contents of the BERT semantic extraction layer, BiLSTM layer, feature transformation layer, and label prediction layer in each subnet can be found above. For different tasks, the relevant parameters of each network layer in different subnets may be the same or different.

[0089] Optionally, in some embodiments, the method further includes:

[0090] The denoised causal relationship extraction model based on multi-task collaborative learning is iteratively trained to obtain the denoised causal relationship extraction model based on multi-task collaborative learning.

[0091] During the iterative training process, some parameters of the part-of-speech tagging task subnet, the chunking analysis task subnet, and the causal relationship extraction task subnet are iteratively pruned.

[0092] In this embodiment, in the task sharing layer, the part-of-speech tagging task subnet, the chunking task subnet, and the causal relationship extraction task subnet first share the basic network architecture and basic parameters of the BERT and BiLSTM layers. Then, iterative Magnitude Pruning (IMP) is used to generate specific parameters for each subnet. That is, iteratively pruning the attention-related parameters in the BERT semantic extraction layer and the LSTM-related parameters in the BiLSTM layer for each of the three tasks of part-of-speech tagging, chunking, and causal relationship extraction, thereby obtaining the subnet parameters for each task.

[0093] Therefore, in this embodiment, the attention-related parameters in the BERT semantic extraction layer and the LSTM-related parameters in the BiLSTM layer are unique to each sub-network, while the remaining parameters are shared (i.e., identical). This approach promotes positive transfer learning while mitigating potential over-parameterization issues. Furthermore, in this embodiment, the feature transformation layer and label prediction layer of each sub-network are separate.

[0094] Through the above implementation methods, a multi-task parallel training strategy is adopted during model training, and parameter sparse sharing is performed on the task-sharing layer, which enables different tasks to obtain semantic complementary information. The MPC-CE model can learn to denoise semantic components that are irrelevant to the causal relationship extraction task, thus achieving better results in complex causal extraction scenarios. It can also handle noisy single-source data and improve model performance.

[0095] Optionally, for each task, the objective is to maximize the log-likelihood probability of the correctly labeled sequence. In this embodiment, the Softmax function is used to calculate the scores of the labeling results from two directions, yielding the conditional probability of the correctly labeled sequence as follows:

[0096]

[0097] Among them, Y true Y is used to represent the true labeled sequence. X s is used to represent all possible label sequences, and s(·) is used to represent the score of the label sequence.

[0098] To reduce computational costs, the logarithmic value of the sequence label predicted for each task is used as follows:

[0099]

[0100] In this application, the loss function for the three tasks is designed as the negative log-likelihood function, thus obtaining the loss function for iterative training of the denoising causal relationship extraction model based on multi-task collaborative learning:

[0101] loss task ={loss CE loss pos tagging loss chunk analysis};

[0102] loss task =-logp(Y true |X);

[0103] loss CE The loss function used to represent the loss generated by the causal extraction task. pos tagging The loss function used to represent the part-of-speech tagging task is loss. chunk analysis This is used to represent the loss function generated by the chunking analysis task.

[0104] During model training, iterative pruning techniques are used to generate task subnets. The base network of the model is denoted as ξ, and the corresponding model parameters are denoted as θ. ξ The masking matrix is ​​denoted as M. t Assume the binary value is 0 or 1, where 0 indicates parameter masking and 1 indicates parameter preservation.

[0105] During training, for each of the three tasks—part-of-speech tagging, chunking, and causal relation extraction—we iteratively prune the attention-related parameters in the BERT semantic extraction layer and the LSTM-related parameters in the BiLSTM layer. This allows us to obtain the Mask matrix for each task, as well as the ξ and Mask matrices of the base network. Then, through M... t and θ ξ Multiplying the elements together yields the parameters of the subnet corresponding to task t, thus obtaining the subnet structure, represented as:

[0106] ξ t =(M t ☉θ ξ );

[0107] To effectively reduce the parameter size, this embodiment employs an iterative order-of-magnitude pruning technique. This technique involves repeatedly pruning the LSTM and Attention-related model parameters during training. Specifically, we perform n iterations of pruning, executing one pruning operation after each training epoch, and subsequently generating a subnet for the corresponding task.

[0108] Optionally, in some embodiments, during the nth iteration of training, the pruning rate of some parameters of the part-of-speech tagging task subnet, the chunking analysis task subnet, and the causal relationship extraction task subnet satisfies:

[0109]

[0110] Where p represents the pruning rate for each iteration, and α represents the percentage of parameters retained in the part-of-speech tagging task subnet, the chunking analysis task subnet, and the causal relationship extraction task subnet after iterative training.

[0111] The IMP technique prunes weights that fall below a predetermined threshold, effectively reducing the number of parameters and computational requirements while improving the efficiency of neural network training without compromising accuracy. This embodiment achieves a compact representation of the network by selectively pruning network parameters, facilitating the construction of task-specific subnetworks. This, in turn, enables the model to learn more effectively while maintaining a high level of accuracy on the task at hand.

[0112] The performance of the MPC-CE model provided in this application will be verified and analyzed using specific experimental examples below.

[0113] To label parts-of-speech tags and chunks based on existing causal relationship extraction datasets, this embodiment uses three types of tags to label a given sentence, based on POS tagging and Chunk Analysis tagging rules. This embodiment utilizes NLTK to automatically label parts-of-speech tags and semantic chunks; the tagging format is shown in Table 1. Based on this tagging, the MPC-CE model can simultaneously complete the three tasks of part-of-speech tagging, chunk analysis, and causal relationship extraction.

[0114] Table 1 shows multi-label data samples from the SemEval2010-Task 8 dataset.

[0115] Sentence Muscle fatigue is the number one POS NNP NN VBZ DT NN CD Chunk O B-NP O B-NP I-NP O CE O B-Cause O O O O

[0116]

[0117]

[0118] The SemEval-2010 Task 8 dataset is the most commonly used dataset for causal relationship extraction. In addition, considering that it can contain more types of causal relationships, this experiment combines Event StoryLine v1.0 and Causal-TimeBank to form a new causal relationship extraction dataset called MTL-CE, so as to verify the model's ability to extract different types of causal relationships.

[0119] Specifically, Task 8 of SemEval-2010 focuses on the semantic relationship between two nouns. The SemEval-2010 Task 8 dataset contains a total of 10,717 samples and includes 9 types of relationships, such as cause-effect relationships and information-topic relationships. It is one of the most commonly used datasets in the field of causal relationship extraction.

[0120] This embodiment merges two different datasets into a newly created dataset. The CausalTimeBank dataset is an annotated dataset about causal relationships, derived from the original Tempeval-3TimeBank, and its causal relationships are temporal. The StoryLine v1.0 dataset includes 566 data samples, mostly from news reports. Finally, our self-made MTL-CE dataset consists of 3482 data samples, with an equal number of causal and non-causal samples (1:1 ratio). The statistical data of the datasets are shown in Table 2. This data contains multi-domain causal relationships, enabling verification of the model's performance in extracting different types of causal relationships in multi-domain scenarios.

[0121] To meet the dataset requirements for multi-task learning, NLTK is used for part-of-speech tagging and chunking on all CE datasets. Furthermore, this embodiment evaluates the MPC-CE model (hereinafter referred to as our model) provided in this embodiment on the SemEval-2010 Task 8 dataset and the MTL-CE dataset. The training, validation, and test sets are divided in a 7:1.5:1.5 ratio.

[0122] Table 2 Statistical results of the two datasets

[0123] Dataset train verify test total SemEval-2010Task 8 7501(906) 1608(219) 1608(206) 10717(1331) MTL-CE 2438(1219) 522(261) 522(261) 3482(1741)

[0124] It should be understood that the numbers in parentheses represent the number of samples with a causal relationship.

[0125] For part-of-speech tagging tasks, since discourse tagging labels each word and does not involve word boundary issues, accuracy is used for evaluation. Accuracy (ACC) is a simple, intuitive evaluation technique that assesses the proportion of correct predictions a model produces. For chunking and causal relation extraction (CE) tasks, precision (P), recall (R), and F1 score (F1) are used as evaluation metrics.

[0126] In this experiment, different baseline models were selected for single-task learning and multi-task learning to extract causal relationships. The following is an introduction to each single-task baseline model and multi-task baseline model.

[0127] Single-task baseline model:

[0128] 1. BiLSTM+CRF: Combining BiLSTM and CRF for sequence labeling.

[0129] 2. Flair: A toolkit that uses BiLSTM and CRF for various NLP tasks, including sequence labeling.

[0130] 3. BiLSTM+LAN utilizes BiLSTM and label attention mechanisms to improve sequence labeling performance.

[0131] 4. CNN+BiLSTM: Introducing CNN and BiLSTM modules to extract semantic features from text.

[0132] 5. BERT: It utilizes stacked Transformer encoders to pre-train on a large amount of text, which can be fine-tuned for various downstream NLP tasks.

[0133] 6. BERT+BiLSTM+CRF: Combines BERT with BiLSTM and CRF for sequence labeling.

[0134] Multi-task baseline model:

[0135] 1. CNN+BiLSTM: This multi-task learning method combines CNN and BiLSTM models to perform multiple NLP tasks simultaneously.

[0136] 2. BERT+BiLSTM+MTL: Based on CNN+BiLSTM, our model uses BERT instead of the CNN module to perform multiple tasks.

[0137] 3. BERT+BiLSTM+CRF+MTL: Our model is further enhanced with CRF after BERT+BiLSTM+MTL, making the model more complete.

[0138] The experimental setup is as follows:

[0139] Experiments were conducted on SemEval-2010 Task 8 and the MTL-CE dataset, with the following hyperparameter settings: iteration pruning rate of 10, final parameter reserve rate of 0.2, learning rate of 1e-5 on SemEval-2010 Task 8, and learning rate of 5e-5 on MTL-CE, with Adam selected as the optimizer. For BiLSTM, the hidden vector dimension was set to 256, and Dropout was set to 0.5 to mitigate overfitting. The batch size was 32, and the number of iterations was 50.

[0140] Parameters were selected on the validation set, and then the optimal parameters were selected to evaluate the model performance on the test set. The experimental results are shown in Tables 3 and 4.

[0141] Table 3. Experimental results of the single-task learning model on the SemEval2010-Task 8 dataset.

[0142] Model accuracy Recall rate F1 BiLSTM+CRF 67.21 65.53 66.36 BiLSTM+CRF+Flair 75.97 66.75 71.06 BiLSTM+LAN 73.99 71.12 72.52 CNN+BiLSTM 74.79 63.35 68.59 CNN+BiLSTM+CRF 73.46 66.50 69.81 BERT 74.00 75.68 74.83 BERT+CRF 79.60 72.72 76.01 BERT+BiLSTM 75.49 75.49 74.49 BERT+BiLSTM+CRF 78.64 75.00 76.77

[0143] Table 4. Experimental results of the single-task learning model on the MTL-CE dataset.

[0144]

[0145]

[0146] Among the four BERT-based models we evaluated, the BERT+BiLSTM+CRF model achieved F1 scores of 76.77% and 73.34% on the two datasets, respectively, making it the best performing baseline method. The results in the table show that BiLSTM+CRF+Flair outperforms BiLSTM+CRF, and CNN+BiLSTM outperforms CNN+BiLSTM. This is because CNN and Flair employ character-level vector representation learning, enabling them to learn finer-grained semantic information, thus significantly improving extraction performance. Furthermore, we found that introducing BERT significantly improves the performance of all four models on both datasets. This is not surprising, as BERT excels across a wide range of NLP tasks due to its ability to capture global information. By comparing model performance on the two datasets, we further discovered that these single-task methods perform worse on the MTL-CE dataset than on the SemEval2010-Task 8 dataset. This indicates that these single-task methods cannot effectively extract accurate causal relationships when faced with different types of causal relationships across multiple domains.

[0147] Table 5 shows the multi-task learning performance of the model on the SemEval2010-Task 8 dataset.

[0148]

[0149] Table 6 shows the multi-task learning results of the model on the MTL-CE dataset.

[0150]

[0151] Experimental results for the multi-task learning-based model are shown in Tables 5 and 6. We can see that our proposed MPC-CE outperforms other existing multi-task learning models on both datasets. The multi-task learning model shows significant improvements in performance on POS, Chunk, and CE, highlighting the effectiveness of MTL in improving task performance. The CE task achieves an F1 score of 78.96% on the SemEval-2010 Task 8 dataset, while MTL-CE achieves an F1 score of 76.46%. We can also observe that MPC-CE improves the F1 score of CE by 2.19%, the best performance among all models. Notably, compared to its STL version (BERT+BiLSTM+CRF), MPC-CE achieves a 3.12% improvement on MTL-CE, the most significant improvement compared to other baseline methods, demonstrating the effectiveness of MPC-CE in handling causal noisy data.

[0152] This embodiment also employs a sparse parameter sharing strategy. To verify the effectiveness of this strategy in improving training efficiency, we conducted a statistical analysis of the model's training time. MPC-CE achieved convergence in approximately 10 hours on a 2-RTX3090 server. The model's convergence is as follows: Figure 3 and 4 As shown, there is a significant improvement in training time compared to methods without sparse sharing, confirming the substantial advantage of our MPC-CE model in accelerating training data convergence. This significant reduction in training time can be attributed to the strategic integration of parameter sharing mechanisms in MPC-CE. MPC-CE not only surpasses various baseline methods in training speed but also paves the way for simplified and time-saving model learning, laying the foundation for improved model practicality.

[0153] We analyze iterative multiple pruning in sparse sharing. Figure 5 and Figure 6 The relationship between the performance of our proposed model and the number of pruning iterations is described. Based on... Figure 5 and Figure 6 The results show that the optimal number of interactive pruning iterations varies across different datasets, as can be seen from these two figures. On the SemEval-2010 Task 8 dataset, the optimal number of iterations for POS, Chunk, and CE are 1, 3, and 4, respectively. On the other hand, on the MTL-CE dataset, the optimal number of iterations for the three tasks are 3, 8, and 3, respectively.

[0154] As demonstrated by the experiments above, the MPC-CE model provided in this application can leverage more information from other tasks related to causal relationship extraction to improve the problem of noisy data in causal relationship extraction. By simultaneously modeling multiple tasks, the model can better understand language structure and semantic roles, thereby effectively filtering noisy data. Furthermore, in the MPC-CE model, parameter correction of the task-shared layer—that is, pruning redundant parameters during training and extracting only the most widely useful parameters—enhances the model's generalization ability. Empirical results on two datasets show that, compared to the baseline, the F1 score is improved by 2.19% and 3.12%, respectively, indicating that the MPC-CE model can effectively enhance causal relationship extraction through in-depth semantic mining, improving the model's ability to mitigate noisy data.

[0155] Please see Figure 7 This invention also provides a noise reduction and causal relationship extraction device 700 based on multi-task collaborative learning, comprising:

[0156] The feature extraction module 701 is used to extract features from the task-sharing layer of the denoising causal relationship extraction model based on multi-task collaborative learning of the text to be processed, and to obtain the extraction result.

[0157] The specific task module 702 is used to input the extraction result into the specific task layer of the denoising causal relationship extraction model based on multi-task collaborative learning to perform part-of-speech tagging, chunking analysis and causal extraction tasks, respectively to obtain the feature labels corresponding to the part-of-speech tagging task, the feature labels corresponding to the chunking analysis task and the feature labels corresponding to the causal extraction task.

[0158] The denoising causal relationship extraction model based on multi-task collaborative learning is obtained by training the part-of-speech tagging task, the chunking analysis task, and the causal extraction task in parallel.

[0159] Optionally, the denoising causal relationship extraction model based on multi-task collaborative learning includes:

[0160] The part-of-speech tagging task subnet includes a first task-shared sublayer and a first specific task sublayer. The first task-shared sublayer is used to extract features from the text to be processed to obtain a first extraction result. The first specific task layer is used to perform part-of-speech tagging based on the first extraction result to obtain the feature label corresponding to the part-of-speech tagging task.

[0161] The chunking analysis task subnet includes a second task sharing sublayer and a second specific task sublayer. The second task sharing sublayer is used to extract features from the text to be processed to obtain a second extraction result. The second specific task sublayer is used to perform chunking analysis based on the second extraction result to obtain the feature label corresponding to the chunking analysis task.

[0162] The causal relationship extraction task subnet includes a third task sharing sublayer and a third specific task sublayer. The third task sharing sublayer is used to extract features from the text to be processed to obtain a third extraction result. The third feature task layer is used to extract causal relationships based on the third extraction result to obtain the feature labels corresponding to the causal relationship extraction task.

[0163] The task sharing layer includes a first task sharing sublayer, a second task sharing sublayer, and a third task sharing sublayer, and the specific task layer includes a first specific task sublayer, a second specific task sublayer, and a third specific task sublayer.

[0164] Optionally, the first task-sharing sublayer includes a first BERT semantic extraction layer and a first BiLSTM layer connected in sequence, the second task-sharing sublayer includes a second BERT semantic extraction layer and a second BiLSTM layer connected in sequence, and the third task-sharing sublayer includes a third BERT semantic extraction layer and a third BiLSTM layer connected in sequence.

[0165] Among them, the first BERT semantic extraction layer, the second BERT semantic extraction layer and the third BERT semantic extraction layer have some of the same parameters, and the first BiLSTM layer, the second BiLSTM layer and the third BiLSTM layer have some of the same parameters.

[0166] Optionally, the first specific task sub-layer includes a first feature transformation layer and a first label prediction layer connected in sequence, the second specific task sub-layer includes a second feature transformation layer and a second label prediction layer connected in sequence, and the third specific task sub-layer includes a third feature transformation layer and a third label prediction layer connected in sequence.

[0167] Optionally, the denoising causal relationship extraction device 700 based on multi-task collaborative learning further includes:

[0168] The iterative training module is used to iteratively train the denoised causal relationship extraction model based on multi-task collaborative learning to obtain the denoised causal relationship extraction model based on multi-task collaborative learning.

[0169] During the iterative training process, some parameters of the part-of-speech tagging task subnet, the chunking analysis task subnet, and the causal relationship extraction task subnet are iteratively pruned.

[0170] Optionally, the loss function for iteratively training the denoising causal relationship extraction model based on multi-task collaborative learning is:

[0171] loss task ={loss CE loss postagging loss chunk analysis};

[0172] loss task =-logp(Y true |X);

[0173]

[0174] Among them, T true Y is used to represent the true labeled sequence. X Let s(·) represent all possible label sequences, and let s(·) represent the score of the label sequence. The loss function is used to represent the loss function. CE The loss function used to represent the loss generated by the causal extraction task. pos tagging The loss function used to represent the part-of-speech tagging task is loss. chunk analysis This is used to represent the loss function generated by the chunking analysis task.

[0175] Optionally, during the nth iteration of training, the pruning rate of some parameters of the part-of-speech tagging task subnet, the chunking analysis task subnet, and the causal relationship extraction task subnet satisfies the following:

[0176]

[0177] Where p represents the pruning rate for each iteration, and α represents the percentage of parameters retained in the part-of-speech tagging task subnet, the chunking analysis task subnet, and the causal relationship extraction task subnet after iterative training.

[0178] The noise reduction and causal relationship extraction device 700 based on multi-task collaborative learning provided in this application embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0179] It should be noted that the division of units in the embodiments of this application is illustrative and only represents one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units.

[0180] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0181] like Figure 8 As shown, this application provides an electronic device 800, including: a memory 802, a processor 801, and a program stored in the memory 802 and executable on the processor 801; the processor 801 is used to read the program in the memory 802 to implement the steps in the denoising causal relationship extraction method based on multi-task collaborative learning as described above.

[0182] This application also provides a readable storage medium storing a program. When the program is executed by a processor, it implements the various processes of the above-described embodiments of the denoising causal relationship extraction method based on multi-task collaborative learning, and achieves the same technical effect. To avoid repetition, it will not be described again here. The readable storage medium can be any available medium or data storage device that the processor can access, including but not limited to magnetic storage (e.g., floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (e.g., compact disks (CDs), digital video discs (DVDs), Blu-ray discs (BD), high-definition universal discs (HVD), etc.), and semiconductor storage (e.g., read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), non-volatile memory (NAND FLASH), solid-state disks (SSDs), etc.).

[0183] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0184] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0185] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A multi-task cooperative learning-based denoising causal relation extraction method, characterized in that, include: The text to be processed is input into the task-sharing layer of a multi-task collaborative learning denoising causal relationship extraction model for feature extraction, and the extraction result is obtained. The extraction results are input into a specific task layer of the denoising causal relationship extraction model based on multi-task collaborative learning to perform part-of-speech tagging, chunking analysis, and causal extraction tasks, respectively, to obtain the feature labels corresponding to the part-of-speech tagging task, the feature labels corresponding to the chunking analysis task, and the feature labels corresponding to the causal extraction task. The denoising causal relationship extraction model based on multi-task collaborative learning is obtained by training the part-of-speech tagging task, the chunking analysis task, and the causal extraction task in parallel. The denoising causal relationship extraction model based on multi-task collaborative learning includes: The part-of-speech tagging task subnet includes a first task-shared sublayer and a first specific task sublayer. The first task-shared sublayer is used to extract features from the text to be processed to obtain a first extraction result. The first specific task sublayer is used to perform part-of-speech tagging based on the first extraction result to obtain the feature label corresponding to the part-of-speech tagging task. The chunking analysis task subnet includes a second task sharing sublayer and a second specific task sublayer. The second task sharing sublayer is used to extract features from the text to be processed to obtain a second extraction result. The second specific task sublayer is used to perform chunking analysis based on the second extraction result to obtain the feature label corresponding to the chunking analysis task. The causal relationship extraction task subnet includes a third task shared sublayer and a third specific task sublayer. The third task shared sublayer is used to extract features from the text to be processed to obtain a third extraction result. The third specific task sublayer is used to extract causal relationships based on the third extraction result to obtain the feature labels corresponding to the causal relationship extraction task. The task sharing layer includes a first task sharing sublayer, a second task sharing sublayer, and a third task sharing sublayer, and the specific task layer includes a first specific task sublayer, a second specific task sublayer, and a third specific task sublayer; The first task-sharing sublayer includes a first BERT semantic extraction layer and a first BiLSTM layer connected in sequence; the second task-sharing sublayer includes a second BERT semantic extraction layer and a second BiLSTM layer connected in sequence; and the third task-sharing sublayer includes a third BERT semantic extraction layer and a third BiLSTM layer connected in sequence. Among them, some parameters of the first BERT semantic extraction layer, the second BERT semantic extraction layer and the third BERT semantic extraction layer are the same, and some parameters of the first BiLSTM layer, the second BiLSTM layer and the third BiLSTM layer are the same. The first specific task sub-layer includes a first feature conversion layer and a first label prediction layer connected in sequence; the second specific task sub-layer includes a second feature conversion layer and a second label prediction layer connected in sequence; and the third specific task sub-layer includes a third feature conversion layer and a third label prediction layer connected in sequence.

2. The method as described in claim 1, characterized in that, The method further includes: The denoised causal relationship extraction model based on multi-task collaborative learning is iteratively trained to obtain the denoised causal relationship extraction model based on multi-task collaborative learning. During the iterative training process, some parameters of the part-of-speech tagging task subnet, the chunking analysis task subnet, and the causal relationship extraction task subnet are iteratively pruned.

3. The method as described in claim 2, characterized in that, The loss function for iteratively training the denoising causal relationship extraction model based on multi-task collaborative learning is: ； ； ； in, Used to represent the true labeled sequence Used to represent all possible label sequences Used to represent the score of the label sequence, This is used to represent the loss function generated by the causal extraction task. This is used to represent the loss function generated by the part-of-speech tagging task. This is used to represent the loss function generated by the chunking analysis task. Used to represent the input text to be processed.

4. The method as described in claim 2, characterized in that, During the nth iteration of training, the pruning rate of some parameters of the part-of-speech tagging task subnet, the chunking analysis task subnet, and the causal relationship extraction task subnet satisfies the following: ； in, Used to represent the pruning rate in each iteration. This represents the percentage of parameters retained in the part-of-speech tagging task subnet, the chunking analysis task subnet, and the causal relationship extraction task subnet after iterative training.

5. A denoising causal relationship extraction device based on multi-task collaborative learning, characterized in that, include: The feature extraction module is used to extract features from the task-sharing layer of the denoising causal relationship extraction model based on multi-task collaborative learning, and obtain the extraction results. The specific task module is used to input the extraction results into the specific task layer of the denoising causal relationship extraction model based on multi-task collaborative learning to perform part-of-speech tagging, chunking analysis and causal extraction tasks, respectively to obtain the feature labels corresponding to the part-of-speech tagging task, the feature labels corresponding to the chunking analysis task and the feature labels corresponding to the causal extraction task. The denoising causal relationship extraction model based on multi-task collaborative learning is obtained by training the part-of-speech tagging task, the chunking analysis task, and the causal extraction task in parallel. The denoising causal relationship extraction model based on multi-task collaborative learning includes: The part-of-speech tagging task subnet includes a first task-shared sublayer and a first specific task sublayer. The first task-shared sublayer is used to extract features from the text to be processed to obtain a first extraction result. The first specific task sublayer is used to perform part-of-speech tagging based on the first extraction result to obtain the feature label corresponding to the part-of-speech tagging task. The chunking analysis task subnet includes a second task sharing sublayer and a second specific task sublayer. The second task sharing sublayer is used to extract features from the text to be processed to obtain a second extraction result. The second specific task sublayer is used to perform chunking analysis based on the second extraction result to obtain the feature label corresponding to the chunking analysis task. The causal relationship extraction task subnet includes a third task shared sublayer and a third specific task sublayer. The third task shared sublayer is used to extract features from the text to be processed to obtain a third extraction result. The third specific task sublayer is used to extract causal relationships based on the third extraction result to obtain the feature labels corresponding to the causal relationship extraction task. The task sharing layer includes a first task sharing sublayer, a second task sharing sublayer, and a third task sharing sublayer, and the specific task layer includes a first specific task sublayer, a second specific task sublayer, and a third specific task sublayer; The first task-sharing sublayer includes a first BERT semantic extraction layer and a first BiLSTM layer connected in sequence; the second task-sharing sublayer includes a second BERT semantic extraction layer and a second BiLSTM layer connected in sequence; and the third task-sharing sublayer includes a third BERT semantic extraction layer and a third BiLSTM layer connected in sequence. Among them, some parameters of the first BERT semantic extraction layer, the second BERT semantic extraction layer and the third BERT semantic extraction layer are the same, and some parameters of the first BiLSTM layer, the second BiLSTM layer and the third BiLSTM layer are the same. The first specific task sub-layer includes a first feature conversion layer and a first label prediction layer connected in sequence; the second specific task sub-layer includes a second feature conversion layer and a second label prediction layer connected in sequence; and the third specific task sub-layer includes a third feature conversion layer and a third label prediction layer connected in sequence.

6. An electronic device, comprising: A memory, a processor, and a program stored in the memory and executable on the processor; characterized in that the processor is configured to read the program from the memory to implement the steps in the denoising causal relationship extraction method based on multi-task collaborative learning as described in any one of claims 1 to 4.

7. A readable storage medium for storing a program, characterized in that, When the program is executed by the processor, it implements the steps in the denoising causal relationship extraction method based on multi-task collaborative learning as described in any one of claims 1 to 4.

Images