Denoising causal relationship extraction method and device based on multi-task collaborative learning

Abstract

The invention discloses a causal relationship extraction method, and particularly relates to a denoising causal relationship extraction method and device based on multi-task collaborative learning. The method comprises the following steps: inputting a to-be-processed text into a task sharing layer of a denoising causal relationship extraction model based on multi-task collaborative learning for feature extraction to obtain an extraction result; and inputting the extraction result into a specific task layer of the de-noising causal relationship extraction model based on multi-task collaborative learning to carry out a part-of-speech tagging task, a chunk analysis task and a causal extraction task. Respectively obtaining a feature tag corresponding to the part-of-speech tagging task, a feature tag corresponding to the chunk analysis task and a feature tag corresponding to the causal extraction task; wherein the de-noising causal relationship extraction model based on multi-task collaborative learning is obtained by parallel training of the part-of-speech tagging task, the chunk analysis task and the causal extraction task. According to the invention, the causal relationship extraction accuracy can be improved.

Description

Technical Field

[0001] The present invention relates to a causal relationship extraction method, and specifically to a denoising causal relationship extraction method and device based on multi-task collaborative learning. Background Art

[0002] Event Causality Extraction (CE), as an important task in Natural Language Processing (NLP), plays an indispensable role in various downstream applications, such as knowledge graphs, event logic graphs, and question answering systems. Extracting causal relationships is quite challenging because it needs to be able to capture rich semantic information and complex language phenomena. Traditional methods use sequence labeling tasks to complete CE, and these methods have achieved much better performance than traditional methods. Subsequent introduction of pre-trained language models such as ElMo and BERT can also improve the model's understanding ability, thereby improving the model's performance.

[0003] However, these methods often face the problem of noisy data. This is mainly because some words in the text are unlikely to be labeled as "cause" or "result". Semantic components such as adjectives and prepositions cannot point to a specific entity, which introduces some noise for labeling causal entities. Traditional sequence labeling frameworks often make wrong predictions, and some adjectives and prepositions are indiscriminately predicted as "cause" or "result". For example, experiments are conducted on the SemEval-2010Task 8 dataset, Event StoryLine corpus, and Causal TimeBank, using the BiLSTM+CRF model for sequence labeling to study the impact of noisy data such as adjectives and prepositions on model performance. Experimental results show that in the first two samples of the SemEval-2010 Task8 dataset, BERT marks "blind" as "B-Cause" without considering the part of speech of the word. A similar problem occurs in the Causal TimeBank case. In addition, in the Event StoryLine sample, "off" is labeled as "B-Cause". Therefore, traditional sequence labeling frameworks often make wrong predictions, and some adjectives and prepositions are indiscriminately predicted as "cause" or "result".

[0004] It can be seen from this that the causal extraction methods commonly used in the prior art are mainly limited to the extraction of causal relationships in a single domain and cannot effectively handle causal relationships in multiple domains, resulting in low extraction accuracy. Summary of the invention

[0005] The technical problem to be solved by the present invention is that the causal extraction methods commonly used in the prior art are mainly limited to the causal relationship extraction of a single domain, and cannot effectively handle the causal relationship of multiple domains, resulting in low extraction accuracy. In order to solve the above problem, the present invention provides a denoising causal relationship extraction method and device based on multi-task collaborative learning.

[0006] The content of the present invention includes:

[0007] In a first aspect, an embodiment of the present invention provides a denoising causal relationship extraction method based on multi-task collaborative learning, comprising:

[0008] Input the text to be processed into the task sharing layer of the denoising causal relationship extraction model based on multi-task collaborative learning to extract features and obtain the extraction result;

[0009] Inputting the extraction result into the specific task layer of the denoising causal relationship extraction model based on multi-task collaborative learning to perform part-of-speech tagging task, chunk analysis task and causal extraction task, and obtaining feature labels corresponding to the part-of-speech tagging task, the chunk analysis task and the causal extraction task respectively;

[0010] Among them, the denoising causal relationship extraction model based on multi-task collaborative learning is obtained by parallel training of the part-of-speech tagging task, the chunk analysis task and the causal extraction task.

[0011] Optionally, the denoising causal relationship extraction model based on multi-task collaborative learning includes:

[0012] A part-of-speech tagging task subnet, the part-of-speech tagging task subnet comprising a first task sharing sublayer and a first specific task sublayer, the first task sharing sublayer being used to perform feature extraction on the to-be-processed text to obtain a first extraction result, and the first feature task layer being used to perform part-of-speech tagging based on the first extraction result to obtain a feature label corresponding to the part-of-speech tagging task;

[0013] A chunk analysis task subnet, the chunk analysis task subnet comprising a second task sharing sublayer and a second specific task sublayer, the second task sharing sublayer being used to perform feature extraction on the to-be-processed text to obtain a second extraction result, and the second feature task layer being used to perform chunk analysis based on the second extraction result to obtain a feature label corresponding to the chunk analysis task;

[0014] A causal relationship extraction task subnet, the causal relationship extraction task subnet comprising a third task sharing sublayer and a third specific task sublayer, the third task sharing sublayer is used to perform feature extraction on the to-be-processed text to obtain a third extraction result, and the third feature task layer is used to perform causal extraction based on the third extraction result to obtain a feature label corresponding to the causal extraction task;

[0015] The task sharing layer includes the first task sharing sublayer, the second task sharing sublayer and the third task sharing sublayer, and the specific task layer includes the first specific task sublayer, the second specific task sublayer and the third specific task sublayer.

[0016] Optionally, the first task sharing sublayer includes a first BERT semantic extraction layer and a first BiLSTM layer connected in sequence, the second task sharing sublayer includes a second BERT semantic extraction layer and a second BiLSTM layer connected in sequence, and the third task sharing sublayer includes a third BERT semantic extraction layer and a third BiLSTM layer connected in sequence;

[0017] Among them, some parameters of the first BERT semantic extraction layer, the second BERT semantic extraction layer and the third BERT semantic extraction layer are the same, and some parameters of the first BiLSTM layer, the second BiLSTM layer and the third BiLSTM layer are the same.

[0018] Optionally, the first specific task sublayer includes a first feature conversion layer and a first label prediction layer connected in sequence, the second specific task sublayer includes a second feature conversion layer and a second label prediction layer connected in sequence, and the third specific task sublayer includes a third feature conversion layer and a third label prediction layer connected in sequence.

[0019] Optionally, the method further comprises:

[0020] Iteratively training the denoising causal relationship extraction model based on multi-task collaborative learning to be trained to obtain the denoising causal relationship extraction model based on multi-task collaborative learning;

[0021] In the iterative training process, some parameters of the part-of-speech tagging task subnet, the chunk analysis task subnet and the causal relationship extraction task subnet are iteratively pruned.

[0022] Optionally, the loss function for iterative training of the denoising causal relationship extraction model to be trained based on multi-task collaborative learning is:

[0023] loss task ={loss CE , loss pos tagging , losschunk analysis};

[0024] loss task = -logp(Y true |X);

[0025]

[0026] Among them, Y true Used to represent the true label sequence, Y X is used to represent all possible label sequences, s(·) is used to represent the score of the label sequence, and loss CE Used to represent the loss function generated by the causal extraction task, loss pos tagging Used to represent the loss function generated by the part-of-speech tagging task, loss chunk analysis Used to represent the loss function generated by the chunk analysis task.

[0027] Optionally, during the nth iteration training, the pruning rate of some parameters of the iterative pruning of the part-of-speech tagging task subnet, the chunk analysis task subnet, and the causal relationship extraction task subnet satisfies:

[0028]

[0029] Wherein, p is used to represent the pruning rate of each iteration, and α is used to represent the percentage of parameters retained in the part-of-speech tagging task subnet, the chunk analysis task subnet, and the causal relationship extraction task subnet after the iterative training is completed.

[0030] In a second aspect, an embodiment of the present invention provides a denoising causal relationship extraction device based on multi-task collaborative learning, comprising:

[0031] A feature extraction module is used to input the text to be processed into the task sharing layer of the denoising causal relationship extraction model based on multi-task collaborative learning to extract features and obtain extraction results;

[0032] A specific task module, used for inputting the extraction result into the specific task layer of the denoising causal relationship extraction model based on multi-task collaborative learning to perform a part-of-speech tagging task, a chunk analysis task and a causal extraction task, and obtaining a feature label corresponding to the part-of-speech tagging task, a feature label corresponding to the chunk analysis task and a feature label corresponding to the causal extraction task respectively;

[0033] Among them, the denoising causal relationship extraction model based on multi-task collaborative learning is obtained by parallel training of the part-of-speech tagging task, the chunk analysis task and the causal extraction task.

[0034] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a program stored in the memory and executable on the processor; the processor is used to read the program in the memory to implement the steps in the denoising causal relationship extraction method based on multi-task collaborative learning as described in the first aspect.

[0035] In a fourth aspect, an embodiment of the present invention provides a readable storage medium for storing a program, which, when executed by a processor, implements the steps in the denoising causal relationship extraction method based on multi-task collaborative learning as described in the first aspect.

[0036] In an embodiment of the present invention, the text to be processed is input into the task sharing layer of the denoising causal relationship extraction model based on multi-task collaborative learning for feature extraction to obtain an extraction result; the extraction result is input into the specific task layer of the denoising causal relationship extraction model based on multi-task collaborative learning to perform part-of-speech tagging task, chunk analysis task and causal extraction task, and feature labels corresponding to the part-of-speech tagging task, chunk analysis task and causal extraction task are obtained respectively; wherein, the denoising causal relationship extraction model based on multi-task collaborative learning is obtained by parallel training of the part-of-speech tagging task, chunk analysis task and causal extraction task.

[0037] The beneficial effect of the present invention is that the denoising causal relationship extraction model of multi-task collaborative learning simultaneously performs part-of-speech tagging tasks, chunk analysis tasks and causal extraction tasks, and through parallel training of the three tasks, the model can learn information about the part-of-speech tagging tasks and chunk analysis tasks related to the causal extraction task. The trained denoising causal relationship extraction model of multi-task collaborative learning can utilize the shared knowledge and complementary information between different but related tasks to alleviate the noise information in causal relationship extraction, while obtaining additional relevant semantic information, reducing unreasonable causal relationships, and improving the accuracy of causal relationship extraction. At the same time, multi-task learning can capture more complex semantic interactions, grasp fine-grained semantic information, and enable the model to effectively adapt to different data domains and have better adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Attached Figure 1 A flowchart of a denoising causal relationship extraction method based on multi-task collaborative learning provided by an embodiment of the present invention;

[0039] Attached Figure 2 A schematic diagram of the structure of a denoising causal relationship extraction model for multi-task collaborative learning provided by an embodiment of the present invention;

[0040] Attached Figure 3 The performance of the model on the SemEval-2010 Task 8 dataset changes with epoch;

[0041] Attached Figure 4 The performance of the MTL-CE dataset model changes with epoch;

[0042] Attached Figure 5 The model performance under different iterative branch reduction times on the emEval-2010 Task 8 dataset;

[0043] Attached Figure 6 The model performance under different iterative branch reduction times on the MTL-CE dataset;

[0044] Attached Figure 7 A schematic diagram of a denoising causal relationship extraction device based on multi-task collaborative learning provided by an embodiment of the present invention;

[0045] Attached Figure 8 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In the embodiments of the present application, the term "and / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. In the embodiments of the present application, the term "multiple" refers to two or more, and other quantifiers are similar. The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first" and "second" are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0047] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0048] See also Figure 1 , Figure 1 A flowchart of a denoising causal relationship extraction method based on multi-task collaborative learning provided by an embodiment of the present invention is provided. The method specifically includes the following steps:

[0049] Step 101, input the text to be processed into the task sharing layer of the denoising causal relationship extraction model based on multi-task collaborative learning to perform feature extraction to obtain an extraction result.

[0050] Step 102: input the extraction result into the specific task layer of the denoising causal extraction model based on multi-task collaborative learning to perform part-of-speech tagging task, chunk analysis task and causal extraction task, and obtain the feature label corresponding to the part-of-speech tagging task, the feature label corresponding to the chunk analysis task and the feature label corresponding to the causal extraction task respectively.

[0051] Among them, the denoising causal relationship extraction model based on multi-task collaborative learning is obtained by parallel training of the part-of-speech tagging task, the chunk analysis task and the causal extraction task.

[0052] The method provided in the embodiment of the present application is mainly used for causal relationship extraction, and the output result corresponding to the causal extraction task is the result obtained by causal relationship extraction. Part-of-speech tagging and chunk analysis are trained in parallel with the causal extraction task as collaborative training tasks, which can denoise interference factors in a large number of semantic components, and improve the performance of the model by learning other task information related to the causal extraction task, thereby helping the model locate causal entities and realize causal relationship extraction.

[0053] Specifically, Part-Of-Speech Tagging (POS) is proposed to assign grammatical tags to words in a sentence. Most causal relationships are nouns (NN in POS tags). Part-Of-Speech tagging can help the model exclude semantically inconsistent tags when locating "cause" entities and "result" entities. Chunk Analysis is to group words in a sentence into meaningful chunks and divide different semantic components of several spans. This is also conducive to minimizing unreasonable causal relationships based on the results of part-of-speech tagging. At the same time, multi-task learning can capture more complex semantic interactions and grasp fine-grained semantic information, enabling the model to effectively adapt to different data domains and have better adaptability.

[0054] See also Figure 2 , the denoising causal relationship extraction (Multi-task with POS and Chunk for Causal Relation, MPC-CE) model based on multi-task collaborative learning in the embodiment of the present application includes a task sharing layer and a specific task layer. First, after the text to be processed is input into the task sharing layer, this layer is used to extract the common features for predicting all tasks, and then enter the specific task layer to extract the features required for specific task label prediction, that is, the label features corresponding to the part-of-speech tagging, chunk analysis and causal extraction tasks respectively, and then complete the part-of-speech tagging, chunk analysis and causal extraction tasks to obtain the output results. In this embodiment, the output results of each task are usually in the form of feature labels.

[0055] Optionally, in some embodiments, the denoising causal relationship extraction model based on multi-task collaborative learning includes:

[0056] A part-of-speech tagging task subnet, the part-of-speech tagging task subnet comprising a first task sharing sublayer and a first specific task sublayer, the first task sharing sublayer being used to perform feature extraction on the to-be-processed text to obtain a first extraction result, and the first feature task layer being used to perform part-of-speech tagging based on the first extraction result to obtain a feature label corresponding to the part-of-speech tagging task;

[0057] A chunk analysis task subnet, the chunk analysis task subnet comprising a second task sharing sublayer and a second specific task sublayer, the second task sharing sublayer being used to perform feature extraction on the to-be-processed text to obtain a second extraction result, and the second feature task layer being used to perform chunk analysis based on the second extraction result to obtain a feature label corresponding to the chunk analysis task;

[0058] A causal relationship extraction task subnet, the causal relationship extraction task subnet comprising a third task sharing sublayer and a third specific task sublayer, the third task sharing sublayer is used to perform feature extraction on the to-be-processed text to obtain a third extraction result, and the third feature task layer is used to perform causal extraction based on the third extraction result to obtain a feature label corresponding to the causal extraction task;

[0059] The task sharing layer includes the first task sharing sublayer, the second task sharing sublayer and the third task sharing sublayer, and the specific task layer includes the first specific task sublayer, the second specific task sublayer and the third specific task sublayer.

[0060] It should be understood that in the embodiment of the present application, a part-of-speech tagging task subnet is provided for the part-of-speech tagging task, a chunk analysis task subnet is provided for the chunk analysis task, and a causal relationship extraction task subnet is provided for the causal extraction task. Each subnet corresponds to its own task sharing sublayer and specific task sublayer, the task sharing sublayers of the three subnets constitute the task sharing layer, and the specific task sublayers of the three subnets constitute the specific task layer.

[0061] Optionally, in some embodiments, the first task sharing sublayer includes a first BERT semantic extraction layer and a first BiLSTM layer connected in sequence, the second task sharing sublayer includes a second BERT semantic extraction layer and a second BiLSTM layer connected in sequence, and the third task sharing sublayer includes a third BERT semantic extraction layer and a third BiLSTM layer connected in sequence;

[0062] Among them, some parameters of the first BERT semantic extraction layer, the second BERT semantic extraction layer and the third BERT semantic extraction layer are the same, and some parameters of the first BiLSTM layer, the second BiLSTM layer and the third BiLSTM layer are the same.

[0063] The following introduces the BERT semantic extraction layers (the first BERT semantic extraction layer, the second BERT semantic extraction layer, and the third BERT semantic extraction layer) and the BiLSTM layers (the first BiLSTM layer, the second BiLSTM layer, and the third BiLSTM layer).

[0064] BERT is a language representation model, which stands for Bidirectional Encoder Representations from Transformers. Its specific structure can be found in the description of the relevant technology, which will not be described here. In this embodiment, the BERT semantic extraction layer dynamically maps each word in the sentence to a feature vector based on the context information according to the context.

[0065] For example, the text to be processed includes m words, and the text to be processed is recorded as a sequence x=(x 1 , x 2 , …, x m ), the BERT semantic extraction layer takes the input sequence x = (x 1 , x 2 , …, x m ) is mapped to:

[0066] t=(t 1 , t 2 ,…,t 1 );

[0067] In the specific implementation, the BERT mapping process involves taking into account various factors, such as polysemous words, syntactic features of sentences, etc., which will not be elaborated here.

[0068] Bidirectional Long Short-Term Memory (BiLSTM) is an improved Recurrent Neural Network (RNN) designed specifically for processing sequence data. For each input sequence of the BiLSTM layer, both the forward Long Short-Term Memory (LSTM) and the backward LSTM need to encode it into a hidden vector. In this embodiment, the two directional hidden states are connected to produce a complete sequence, which helps capture the position information of each word in the sentence.

[0069] The output sequence of the forward LSTM hidden state is:

[0070]

[0071] The output sequence of the backward LSTM hidden state is:

[0072]

[0073] Combine the output sequence of the forward LSTM hidden state with the output sequence of the backward LSTM hidden state to get the complete output sequence h of the BiLSTM hidden state:

[0074]

[0075] h=(h 1 ,h 2 ,…,h m )∈R m×n ;

[0076] Among them, t represents the position of the word in the sentence, m represents the number of words in the sentence, and n represents the dimension of the vector.

[0077] It should be understood that the task-sharing sublayers of the three subnetworks all include a BERT semantic extraction layer and a BiLSTM layer connected in sequence, but the specific parameters of the BERT semantic extraction layer and the BiLSTM layer included in the three subnetworks are not exactly the same.

[0078] Optionally, in some embodiments, the first specific task sublayer includes a first feature conversion layer and a first label prediction layer connected in sequence, the second specific task sublayer includes a second feature conversion layer and a second label prediction layer connected in sequence, and the third specific task sublayer includes a third feature conversion layer and a third label prediction layer connected in sequence.

[0079] The feature conversion layer (first feature conversion layer, second feature conversion layer and third feature conversion layer) and the label prediction layer (first label prediction layer, second label prediction layer and third label prediction layer) are introduced below.

[0080] After the BiLSTM layer, this embodiment uses a feature conversion layer to map the hidden state vector h including n dimensions to k dimensions. Specifically, the feature conversion layer extracts sentence features by using a matrix, represented as P:

[0081] P=(p 1 , p 2 ,…,p m )∈R m×k ;

[0082] Among them, k is the number of labels, and different tasks correspond to different numbers of labels. For example, the CE task involves 5 different labels, including "B-cause, I-cause, B-result, I-result" and "O".

[0083] The goal of the feature conversion layer is to transform the hidden state vector of n dimensions into a low-dimensional vector of k dimensions, so as to effectively extract high-level features and achieve simultaneous training of multiple tasks by minimizing information loss in this process.

[0084] The label prediction layer uses the Conditional Random Field (CRF) algorithm for label prediction, that is, to mark the label of each word in a given sequence based on the overall maximum probability calculation, which is suitable for sequence labeling tasks with a large number of annotations.

[0085] For an input sentence X=(x 1 , x 2 , x 3 , …, x n ) and the target label sequence Y = (y 1 ,y 2 ,y 3 , …, y n ), where x i is the i-th word of X, y i is x i The label is n, and n is the length of the input sentence and the target label sequence. This embodiment defines the score of the entire label as follows:

[0086]

[0087] Among them, s(·) calculates the score of a given label sequence, is the emission matrix, which is used to represent the label vector generated from the BiLSTM layer, representing which label should be the current x i Mapping. is the transfer matrix, which stores the score of transferring from one label to another. The higher the score, the more likely the transfer is to occur. k is the number of labels, y 0 and n is the beginning and end of the sentence. The predicted label sequence can be obtained according to the score of the entire label sequence.

[0088] In this embodiment, the part-of-speech tagging task subnet includes a first BERT semantic extraction layer, a first BiLSTM layer, a first feature conversion layer, and a first label prediction layer. The chunk analysis task subnet includes a second BERT semantic extraction layer, a second BiLSTM layer, a second feature conversion layer, and a second label prediction layer. The causal relationship extraction task subnet includes a third BERT semantic extraction layer, a third BiLSTM layer, a third feature conversion layer, and a third label prediction layer. Among them, the specific contents of the BERT semantic extraction layer, BiLSTM layer, feature conversion layer, and label prediction layer of each subnet can be found in the above content. For different tasks, the relevant parameters of each network layer included in different subnets may be the same or different.

[0089] Optionally, in some embodiments, the method further comprises:

[0090] Iteratively training the denoising causal relationship extraction model based on multi-task collaborative learning to be trained to obtain the denoising causal relationship extraction model based on multi-task collaborative learning;

[0091] In the iterative training process, some parameters of the part-of-speech tagging task subnet, the chunk analysis task subnet and the causal relationship extraction task subnet are iteratively pruned.

[0092] In this embodiment, in the task sharing layer, the part-of-speech tagging task subnet, the chunk analysis task subnet, and the causal relationship extraction task subnet first share the basic network architecture and basic parameters of the BERT and BiLSTM layers, and then use iterative magnitude pruning (IMP) technology to generate specific parameters for each subnet, that is, for each of the three tasks of part-of-speech tagging, chunk tagging, and causal relationship extraction, the Attention-related parameters in the BERT semantic extraction layer and the LSTM-related parameters in the BiLSTM layer are iteratively pruned to obtain the subnet parameters for each task.

[0093] Therefore, in this embodiment, the Attention-related parameters in the BERT semantic extraction layer and the LSTM-related parameters in the BiLSTM layer in each subnet are unique to each subnet, and the remaining parameters are shared (i.e., the same). In this way, forward transfer learning can be promoted while alleviating potential over-parameterization problems. In addition, in this embodiment, the feature conversion layer and label prediction layer of each subnet are separate.

[0094] Through the above implementation, during model training, a multi-task parallel training strategy is adopted, and sparse parameter sharing is performed on the task sharing layer, so that semantic complementary information can be obtained between different tasks. The MPC-CE model can learn to denoise the semantic components that are irrelevant to the causal extraction task, so it can achieve better results in complex causal extraction scenarios, and can also process noisy single-source data and improve the performance of the model.

[0095] Optionally, for each task, the goal is to maximize the log-likelihood probability of the correct label sequence. In this embodiment, the Softmax function is used to calculate the scores of the label results from two directions, and the conditional probability of the correct label sequence is obtained as:

[0096]

[0097] Among them, Y true Used to represent the true label sequence, Y X is used to represent all possible label sequences, and s(·) is used to represent the score of the label sequence.

[0098] In order to reduce the computational cost, the logarithm of the sequence label predicted for each task is used as follows:

[0099]

[0100] The loss functions of the three tasks designed in this application are negative log-likelihood functions, so that the loss function for iterative training of the denoising causal relationship extraction model to be trained based on multi-task collaborative learning is obtained as follows:

[0101] loss task ={loss CE , losses pos tagging , loss chunk analysis};

[0102] loss task = -logp(Y true |X);

[0103] loss CE Used to represent the loss function generated by the causal extraction task, loss pos tagging Used to represent the loss function generated by the part-of-speech tagging task, loss chunk analysis Used to represent the loss function generated by the chunk analysis task.

[0104] During the model training process, the iterative multiple branch reduction technique is used to generate the task subnetwork. The basic network of the model is represented as ξ, and the corresponding model parameters are represented as θ. ξ The masking matrix is ​​represented by M t, assumes a binary value of 0 or 1, where 0 indicates parameter masking and 1 indicates parameter retention.

[0105] During the training process, we iteratively pruned the attention-related parameters in the BERT semantic extraction layer and the LSTM-related parameters in the BiLSTM layer for each of the three tasks: part-of-speech tagging, chunk tagging, and causal extraction. This enabled us to obtain the mask matrix for each task, as well as the ξ and mask matrices of the base network. Then, we used M t and θ ξ The elements of are multiplied to obtain the parameters of the subnet corresponding to task t, thereby obtaining the subnet structure, which is expressed as:

[0106] ξ t =(M t ☉θ ξ );

[0107] In order to effectively reduce the parameter scale, this example uses an iterative order of magnitude pruning technique, which requires repeatedly pruning the model parameters related to LSTM and Attention during the training process. Specifically, we perform n iterative pruning, perform a pruning operation after each training epoch, and then generate a subnet for the corresponding task.

[0108] Optionally, in some embodiments, during the nth iteration training, the pruning rate of some parameters of the part-of-speech tagging task subnet, the chunk analysis task subnet, and the causal relationship extraction task subnet satisfies:

[0109]

[0110] Wherein, p is used to represent the pruning rate of each iteration, and α is used to represent the percentage of parameters retained in the part-of-speech tagging task subnet, the chunk analysis task subnet, and the causal relationship extraction task subnet after the iterative training is completed.

[0111] IMP technology is able to prune those weight values ​​that are below a predetermined threshold. This process effectively reduces the number of parameters and computational requirements, while improving the efficiency of neural network training without affecting accuracy. This embodiment obtains a compact representation of the network by selectively pruning network parameters, facilitating the construction of subnetworks customized for specific tasks. This in turn enables the model to learn in a more efficient manner while still maintaining a high level of accuracy on the task at hand.

[0112] The performance of the MPC-CE model provided in this application will be verified and analyzed using specific experiments as examples.

[0113] In order to label parts of speech and chunks based on the existing causal relationship extraction data set, the embodiment of the present application formulates rules based on the tags of POS tagging and Chunk Analysis, and uses three types of tags to label a given sentence. This embodiment uses NLTK to automatically label parts of speech and semantic chunks. The labeling format is shown in Table 1. Based on the labeling, the MPC-CE model can be used to simultaneously complete the three tasks of part of speech tagging, chunk analysis, and causal extraction.

[0114] Table 1 Multi-label data samples from the SemEval2010-Task 8 dataset

[0115] Sentence Muscle fatigue is the number one POS NNP NN VBZ DT NN CD Chunk O B-NP O B-NP I-NP O CE O B-Cause O O O O

[0116]

[0117]

[0118] The SemEval-2010 Task 8 dataset is the most commonly used causal relationship extraction dataset. In addition, considering that it can contain more types of causal relationships, this experiment combines the event StoryLine v1.0 and Causal-TimeBank to form a new causal relationship extraction dataset called MTL-CE, so as to verify the model's ability to extract different types of causal relationships.

[0119] Specifically, Task 8 of SemEval-2010 focuses on the semantic relationship between two nouns. The SemEval-2010Task8 dataset includes a total of 10,717 samples, including 9 types of relationships, such as cause-effect relationship, information-topic relationship, etc. It is one of the most commonly used datasets in the field of causal relationship extraction.

[0120] This embodiment merges two different datasets into a newly created dataset, where CausalTimeBank is an annotated dataset about causal relationships, which comes from the original Tempeval-3TimeBank, and its causal relationships are temporal causal relationships; the event StoryLine v1.0 dataset includes 566 data samples, most of which are causal relationships from news. Finally, our self-made MTL-CE dataset consists of 3482 data samples, with an equal number of causal and non-causal samples, with a ratio of 1:1. The statistics of the dataset are shown in Table 2. The data contains multi-domain causal relationships, which can verify the performance extraction ability of the model for different types of causal relationships in multi-domain scenarios.

[0121] In order to meet the dataset requirements of multi-task learning, NLTK is used to perform part-of-speech tagging and chunk tagging on all CE datasets. In addition, this embodiment evaluates the MPC-CE model provided in this embodiment (referred to as our model in the experiment) on the SemEval-2010 Task 8 dataset and the MTL-CE dataset. The training set, validation set, and test set are divided according to the ratio of 7:1.5:1.5.

[0122] Table 2 Statistical results of two data sets

[0123] Dataset train verify test total SemEval-2010Task 8 7501(906) 1608(219) 1608(206) 10717(1331) MTL-CE 2438(1219) 522(261) 522(261) 3482(1741)

[0124] It should be understood that the numbers in brackets represent the number of samples with a causal relationship.

[0125] For the part-of-speech tagging task, since the discourse tag is a tag for each word, it does not involve the word boundary problem, so the accuracy is used for evaluation. Accuracy (ACC) is a simple, perceptual evaluation technique that evaluates the proportion of correct predictions produced by a model. For the chunk analysis (Chunk) task and the causal extraction (CE) task, precision (P), recall (R) and F1 score (F1 Score, F1) are used as evaluation indicators.

[0126] In this experiment, different baseline models were selected for single-task learning and multi-task learning to extract causal relationships. The following is an introduction to each single-task baseline model and multi-task baseline model.

[0127] Single-task baseline model:

[0128] 1. BiLSTM+CRF: Combine BiLSTM and CRF for sequence labeling.

[0129] 2. Flair: It is a toolkit that uses BiLSTM and CRF for various NLP tasks and can perform sequence labeling.

[0130] 3. BiLSTM+LAN uses BiLSTM and label attention mechanism to improve the performance of sequence labeling.

[0131] 4. CNN+BiLSTM: Introduce CNN and BiLSTM modules to extract semantic features from text.

[0132] 5. BERT: Utilizes stacked Transformer encoders to pre-train on large amounts of text that can be fine-tuned for various downstream NLP tasks.

[0133] 6. BERT+BiLSTM+CRF: Combine BERT with BiLSTM and CRF for sequence labeling.

[0134] Multi-task baseline model:

[0135] 1. CNN+BiLSTM: This multi-task learning method is proposed, which combines CNN and BiLSTM models to perform multiple NLP tasks simultaneously.

[0136] 2. BERT+BiLSTM+MTL: Based on CNN+BiLSTM, our model uses BERT instead of CNN modules to perform multiple tasks.

[0137] 3. BERT+BiLSTM+CRF+MTL: Our model BERT+BiLSTM+MTL adds CRF after deepening to make the model more complete.

[0138] The experimental setup is as follows:

[0139] Experiments were conducted on the SemEval-2010 Task 8 and MTL-CE datasets, with the following hyperparameter settings: the iteration pruning rate was set to 10, the final parameter reserve rate was 0.2, the learning rate was 1e-5 on SemEval-2010 Task 8, the learning rate was 5e-5 on MTL-CE, and Adam was selected as the optimizer. For BiLSTM, its hidden vector dimension was set to 256, and Dropout was 0.5 to reduce overfitting. The batch_size was 32, and the number of iterations was 50.

[0140] Parameter selection was performed on the validation set, and then the optimal parameters were selected to evaluate the model performance on the test set. The experimental results are shown in Tables 3 and 4.

[0141] Table 3 Experimental results of single-task learning model on SemEval2010-Task 8 dataset

[0142] Model Accuracy Recall F1 BiLSTM+CRF 67.21 65.53 66.36 BiLSTM+CRF+Flair 75.97 66.75 71.06 BiLSTM+LAN 73.99 71.12 72.52 CNN+BiLSTM 74.79 63.35 68.59 CNN+BiLSTM+CRF 73.46 66.50 69.81 BERT 74.00 75.68 74.83 BERT+CRF 79.60 72.72 76.01 BERT+BiLSTM 75.49 75.49 74.49 BERT+BiLSTM+CRF 78.64 75.00 76.77

[0143] Table 4 Experimental results of single-task learning model in MTL-CE dataset

[0144]

[0145]

[0146] Among the four BERT-based models we evaluated, the F1 score of the BERT+BiLSTM+CRF model reached 76.77% and 73.34% on the two datasets, respectively, which is the best among all baseline methods. From the results in the table, we can see that BiLSTM+CRF+Flair has better performance than BiLSTM+CRF, and CNN+BiLSTM CNN also has better performance than CNN+BiLSTM. This is because CNN and Flair use character-level vector representation learning to learn more fine-grained semantic information, thereby greatly improving the extraction performance. In addition, we found that the introduction of BERT can greatly improve the performance of all four models on both datasets, which is not surprising because BERT has shown advantages in a wide range of NLP tasks because it has the ability to capture global information. By comparing the model performance in the two datasets, we further found that these single-task methods perform worse in the MTL-CE dataset than in the SemEval2010-Task 8 dataset, which shows that these single-task methods cannot effectively perform accurate causal relationship extraction when faced with different types of causal relationships in multiple domains.

[0147] Table 5 Multi-task learning performance of the model on the SemEval2010-Task 8 dataset

[0148]

[0149] Table 6 Multi-task learning results of the model on the MTL-CE dataset

[0150]

[0151] The experimental results of the multi-task learning based models are shown in Tables 5 and 6. It can be found that the performance of our proposed MPC-CE on both datasets is better than other existing multi-task learning models. The performance of the multi-task learning models on POS, Chunk and CE has been greatly improved, which emphasizes the effectiveness of MTL in improving task performance. The CE task obtained an F1 score of 78.96% on the SemEval-2010 Task 8 dataset, while it obtained an F1 score of 76.46% on MTL-CE. We can also observe that MPC-CE improves the F1 of CE by 2.19%, which has the best performance among all models. It is worth noting that MPC-CE obtains a 3.12% improvement on MTL-CE compared with its STL version (BERT+BiLSTM+CRF), which is the most obvious improvement compared with other baseline methods, proving the effectiveness of MPC-CE in processing causal noise data.

[0152] This example also uses a sparse shared parameter strategy. To verify the ability of this strategy to improve training efficiency, we conducted a statistical analysis of the model training time. MPC-CE achieved convergence in about 10 hours on a 2-RTX3090 server. Figure 3 and 4 As shown in the figure, the training time is significantly improved compared to that without sparse sharing, confirming the significant advantage of our MPC-CE model in accelerating the convergence of training data. The significant reduction in training time can be attributed to the strategic integration of parameter sharing mechanism in MPC-CE. MPC-CE not only surpasses various baseline methods in terms of training speed, but also paves the way for simplified and time-saving model learning. It lays the foundation for improving the practicality of the model.

[0153] We analyze the iterative multiple branch reduction in sparse sharing. Figure 5 and Figure 6 Describes the relationship between the performance of our proposed model and the number of branch reductions. Figure 5 and Figure 6 The results show that the optimal number of interactive pruning is different for different datasets, which can be seen from the two figures. On the SemEval-2010 Task 8 dataset, the optimal number of iterations for POS, Chunk, and CE are 1, 3, and 4, respectively. On the other hand, the optimal number of iterations for the three tasks on the MTL-CE dataset is 3, 8, and 3, respectively.

[0154] According to the above experiments, the MPC-CE model provided in this application can use more other task information related to the causal extraction task to improve the noise data problem in causal extraction. By modeling multiple tasks at the same time, the model can better understand the language structure and semantic roles, thereby effectively filtering out noisy data. In addition, in the MPC-CE model, the generalization ability of the model is enhanced by correcting the parameters of the task sharing layer, that is, by pruning redundant parameters during the training process and extracting only the most extensive beneficial parameters. The empirical results on the two datasets show that compared with the baseline, F1 is improved by 2.19% and 3.12%, respectively, indicating that the MPC-CE model can effectively enhance causal extraction through in-depth semantic mining and improve the model's ability to alleviate noise data.

[0155] See also Figure 7 The embodiment of the present invention further provides a denoising causal relationship extraction device 700 based on multi-task collaborative learning, comprising:

[0156] A feature extraction module 701 is used to input the text to be processed into the task sharing layer of the denoising causal relationship extraction model based on multi-task collaborative learning to perform feature extraction and obtain an extraction result;

[0157] A specific task module 702 is used to input the extraction result into the specific task layer of the denoising causal relationship extraction model based on multi-task collaborative learning to perform a part-of-speech tagging task, a chunk analysis task and a causal extraction task, and obtain a feature label corresponding to the part-of-speech tagging task, a feature label corresponding to the chunk analysis task and a feature label corresponding to the causal extraction task respectively;

[0158] Among them, the denoising causal relationship extraction model based on multi-task collaborative learning is obtained by parallel training of the part-of-speech tagging task, the chunk analysis task and the causal extraction task.

[0159] Optionally, the denoising causal relationship extraction model based on multi-task collaborative learning includes:

[0160] A part-of-speech tagging task subnet, the part-of-speech tagging task subnet comprising a first task sharing sublayer and a first specific task sublayer, the first task sharing sublayer being used to perform feature extraction on the to-be-processed text to obtain a first extraction result, and the first feature task layer being used to perform part-of-speech tagging based on the first extraction result to obtain a feature label corresponding to the part-of-speech tagging task;

[0161] A chunk analysis task subnet, the chunk analysis task subnet comprising a second task sharing sublayer and a second specific task sublayer, the second task sharing sublayer being used to perform feature extraction on the to-be-processed text to obtain a second extraction result, and the second feature task layer being used to perform chunk analysis based on the second extraction result to obtain a feature label corresponding to the chunk analysis task;

[0162] A causal relationship extraction task subnet, the causal relationship extraction task subnet comprising a third task sharing sublayer and a third specific task sublayer, the third task sharing sublayer is used to perform feature extraction on the to-be-processed text to obtain a third extraction result, and the third feature task layer is used to perform causal extraction based on the third extraction result to obtain a feature label corresponding to the causal extraction task;

[0163] The task sharing layer includes the first task sharing sublayer, the second task sharing sublayer and the third task sharing sublayer, and the specific task layer includes the first specific task sublayer, the second specific task sublayer and the third specific task sublayer.

[0164] Optionally, the first task sharing sublayer includes a first BERT semantic extraction layer and a first BiLSTM layer connected in sequence, the second task sharing sublayer includes a second BERT semantic extraction layer and a second BiLSTM layer connected in sequence, and the third task sharing sublayer includes a third BERT semantic extraction layer and a third BiLSTM layer connected in sequence;

[0165] Among them, some parameters of the first BERT semantic extraction layer, the second BERT semantic extraction layer and the third BERT semantic extraction layer are the same, and some parameters of the first BiLSTM layer, the second BiLSTM layer and the third BiLSTM layer are the same.

[0166] Optionally, the first specific task sublayer includes a first feature conversion layer and a first label prediction layer connected in sequence, the second specific task sublayer includes a second feature conversion layer and a second label prediction layer connected in sequence, and the third specific task sublayer includes a third feature conversion layer and a third label prediction layer connected in sequence.

[0167] Optionally, the denoising causal relationship extraction device 700 based on multi-task collaborative learning further includes:

[0168] An iterative training module, used for iteratively training the denoising causal relationship extraction model based on multi-task collaborative learning to be trained, so as to obtain the denoising causal relationship extraction model based on multi-task collaborative learning;

[0169] In the iterative training process, some parameters of the part-of-speech tagging task subnet, the chunk analysis task subnet and the causal relationship extraction task subnet are iteratively pruned.

[0170] Optionally, the loss function for iterative training of the denoising causal relationship extraction model to be trained based on multi-task collaborative learning is:

[0171] loss task ={loss CE , losses postagging , loss chunk analysis};

[0172] loss task = -logp(Y true |X);

[0173]

[0174] Among them, T true Used to represent the true label sequence, Y X is used to represent all possible label sequences, s(·) is used to represent the score of the label sequence, and loss CE Used to represent the loss function generated by the causal extraction task, loss pos tagging Used to represent the loss function generated by the part-of-speech tagging task, loss chunk analysis Used to represent the loss function generated by the chunk analysis task.

[0175] Optionally, during the nth iteration training, the pruning rate of some parameters of the iterative pruning of the part-of-speech tagging task subnet, the chunk analysis task subnet, and the causal relationship extraction task subnet satisfies:

[0176]

[0177] Wherein, p is used to represent the pruning rate of each iteration, and α is used to represent the percentage of parameters retained in the part-of-speech tagging task subnet, the chunk analysis task subnet, and the causal relationship extraction task subnet after the iterative training is completed.

[0178] The denoising causal relationship extraction device 700 based on multi-task collaborative learning provided in the embodiment of the present application can execute the above method embodiment, and its implementation principle and technical effects are similar, which will not be repeated in this embodiment.

[0179] It should be noted that the division of units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0180] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0181] like Figure 8 As shown, an embodiment of the present application provides an electronic device 800, comprising: a memory 802, a processor 801, and a program stored in the memory 802 and executable on the processor 801; the processor 801 is used to read the program in the memory 802 to implement the steps in the denoising causal relationship extraction method based on multi-task collaborative learning as described above.

[0182] An embodiment of the present application also provides a readable storage medium, on which a program is stored. When the program is executed by a processor, the various processes of the above-mentioned denoising causal relationship extraction method embodiment based on multi-task collaborative learning are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here. Among them, the readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disk, hard disk, magnetic tape, magneto-optical disk (MO), etc.), optical storage (such as compact disk (CD), digital video disc (DVD), Blu-ray Disc (BD), high-definition versatile disc (HVD), etc.), and semiconductor memory (such as read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read only memory (EEPROM), non-volatile memory (NAND FLASH), solid-state drive (Solid State Disk or Solid State Drive, SSD)), etc.

[0183] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0184] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes a number of instructions for a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0185] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A denoising causal relationship extraction method based on multi-task collaborative learning, characterized in that: include: Input the text to be processed into the task sharing layer of the denoising causal relationship extraction model based on multi-task collaborative learning to extract features and obtain the extraction result; Inputting the extraction result into the specific task layer of the denoising causal relationship extraction model based on multi-task collaborative learning to perform part-of-speech tagging task, chunk analysis task and causal extraction task, and obtaining feature labels corresponding to the part-of-speech tagging task, the chunk analysis task and the causal extraction task respectively; Among them, the denoising causal relationship extraction model based on multi-task collaborative learning is obtained by parallel training of the part-of-speech tagging task, the chunk analysis task and the causal extraction task.

2. The method according to claim 1, characterized in that: The denoising causal relationship extraction model based on multi-task collaborative learning includes: A part-of-speech tagging task subnet, the part-of-speech tagging task subnet comprising a first task sharing sublayer and a first specific task sublayer, the first task sharing sublayer being used to perform feature extraction on the to-be-processed text to obtain a first extraction result, and the first feature task layer being used to perform part-of-speech tagging based on the first extraction result to obtain a feature label corresponding to the part-of-speech tagging task; A chunk analysis task subnet, the chunk analysis task subnet comprising a second task sharing sublayer and a second specific task sublayer, the second task sharing sublayer being used to perform feature extraction on the to-be-processed text to obtain a second extraction result, and the second feature task layer being used to perform chunk analysis based on the second extraction result to obtain a feature label corresponding to the chunk analysis task; A causal relationship extraction task subnet, the causal relationship extraction task subnet comprising a third task sharing sublayer and a third specific task sublayer, the third task sharing sublayer is used to perform feature extraction on the to-be-processed text to obtain a third extraction result, and the third feature task layer is used to perform causal extraction based on the third extraction result to obtain a feature label corresponding to the causal extraction task; The task sharing layer includes the first task sharing sublayer, the second task sharing sublayer and the third task sharing sublayer, and the specific task layer includes the first specific task sublayer, the second specific task sublayer and the third specific task sublayer.

3. The method according to claim 2, characterized in that: The first task sharing sublayer includes a first BERT semantic extraction layer and a first BiLSTM layer connected in sequence, the second task sharing sublayer includes a second BERT semantic extraction layer and a second BiLSTM layer connected in sequence, and the third task sharing sublayer includes a third BERT semantic extraction layer and a third BiLSTM layer connected in sequence; Among them, some parameters of the first BERT semantic extraction layer, the second BERT semantic extraction layer and the third BERT semantic extraction layer are the same, and some parameters of the first BiLSTM layer, the second BiLSTM layer and the third BiLSTM layer are the same.

4. The method according to claim 2, characterized in that: The first specific task sublayer includes a first feature conversion layer and a first label prediction layer connected in sequence, the second specific task sublayer includes a second feature conversion layer and a second label prediction layer connected in sequence, and the third specific task sublayer includes a third feature conversion layer and a third label prediction layer connected in sequence.

5. The method according to claim 3, characterized in that: The method further comprises: Iteratively training the denoising causal relationship extraction model based on multi-task collaborative learning to be trained to obtain the denoising causal relationship extraction model based on multi-task collaborative learning; In the iterative training process, some parameters of the part-of-speech tagging task subnet, the chunk analysis task subnet and the causal relationship extraction task subnet are iteratively pruned.

6. The method according to claim 5, characterized in that: The loss function for iterative training of the denoising causal relationship extraction model to be trained based on multi-task collaborative learning is: loss task ={loss CE ,loss postagging ,loss chunkanalysis }; loss task =-logp(Y true |X); Among them, Y true Used to represent the true label sequence, Y X is used to represent all possible label sequences, s(·) is used to represent the score of the label sequence, and loss CE Used to represent the loss function generated by the causal extraction task, loss postagging Used to represent the loss function generated by the part-of-speech tagging task, loss chunkanalysis Used to represent the loss function generated by the chunk analysis task.

7. The method according to claim 5, characterized in that: During the nth iteration training, the pruning rate of some parameters of the part-of-speech tagging task subnet, the chunk analysis task subnet, and the causal relationship extraction task subnet is iteratively pruned to meet the following requirements: Wherein, p is used to represent the pruning rate of each iteration, and α is used to represent the percentage of parameters retained in the part-of-speech tagging task subnet, the chunk analysis task subnet, and the causal relationship extraction task subnet after the iterative training is completed.

8. A denoising causal relationship extraction device based on multi-task collaborative learning, characterized in that: include: A feature extraction module is used to input the text to be processed into the task sharing layer of the denoising causal relationship extraction model based on multi-task collaborative learning to extract features and obtain extraction results; A specific task module, used for inputting the extraction result into the specific task layer of the denoising causal relationship extraction model based on multi-task collaborative learning to perform a part-of-speech tagging task, a chunk analysis task and a causal extraction task, and obtaining a feature label corresponding to the part-of-speech tagging task, a feature label corresponding to the chunk analysis task and a feature label corresponding to the causal extraction task respectively; Among them, the denoising causal relationship extraction model based on multi-task collaborative learning is obtained by parallel training of the part-of-speech tagging task, the chunk analysis task and the causal extraction task.

9. An electronic device, comprising: A memory, a processor, and a program stored in the memory and executable on the processor; wherein the processor is used to read the program in the memory to implement the steps in the denoising causal relationship extraction method based on multi-task collaborative learning as described in any one of claims 1 to 7.

10. A readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps in the denoising causal relationship extraction method based on multi-task collaborative learning as described in any one of claims 1 to 7 are implemented.

Images