Emotion-reason pair extraction method based on multi-task joint learning
By employing multi-task joint learning and a dual affine scoring mechanism, the semantic interreference between emotion and cause and the sample imbalance problem were solved, achieving more accurate binary tuple extraction and improving the modeling ability and stability of the relationship between emotion and cause.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- DALIAN MARITIME UNIVERSITY
- Filing Date
- 2026-01-28
- Publication Date
- 2026-06-05
AI Technical Summary
Existing technologies cannot fully utilize the strong semantic interreference between emotion recognition and cause recognition, resulting in limited ability to model the relationship between emotion and cause, and face the problem of imbalanced positive and negative sample distribution, which affects the prediction accuracy and stability of the model.
We employ a multi-task joint learning approach, which introduces a learnable label embedding matrix and a dual affine scoring mechanism, combined with supervised focus loss and pair consistency loss, to achieve explicit semantic interaction and complex causal relationship modeling between emotion and cause, thereby optimizing the model training process.
It significantly improves the accuracy and stability of emotion-cause pair extraction, overcomes the sample imbalance problem, and improves the model's accuracy and recall in emotion-cause pairing.
Smart Images
Figure CN122154687A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of sentiment analysis technology, and more particularly to a method for extracting sentiment-cause binary pairs based on multi-task joint learning. Background Technology
[0002] Emotion-cause binary extraction is an important task in the field of dialogue sentiment analysis. Its goal is to simultaneously identify emotion clauses expressing emotions and cause clauses eliciting those emotions from dialogue text, and pair them to form <emotion, cause> binary pairs. This task typically comprises two core subtasks: emotion recognition and cause recognition. In multimodal dialogue scenarios, this task aims to comprehensively utilize multi-source information such as text and speech to achieve more accurate emotion and causal inference.
[0003] In existing research, emotion recognition and cause identification are usually modeled as two relatively independent tasks. Specifically, some studies predict the emotion and cause of statements in a dialogue separately; others, while using a shared feature extraction layer in the model structure, still predict the emotion label and cause label independently in the output layer. The drawback is that there is only a weak or implicit association between emotion recognition and cause identification. The model cannot fully utilize the semantic association between the two during inference. Existing research has failed to explicitly introduce this two-way constraint mechanism in the modeling stage, resulting in emotion prediction results not effectively guiding cause identification, and cause prediction results being difficult to correct emotion judgments. In existing research, the modeling of the emotion-cause pairing relationship after completing emotion and cause identification remains relatively simple. A common approach is to generate all possible emotion-cause combinations using a Cartesian product after obtaining the candidate sets of emotion and cause, and then use simple similarity calculation methods (such as dot product, cosine similarity, or linear scoring functions) to sort or filter the combinations. This type of method is relatively straightforward in implementation, but its modeling capability is limited, making it difficult to effectively characterize the complex causal triggering relationship between emotion and cause. The relationship between emotion and cause is often significantly asymmetric, meaning that the cause triggers the emotion, but the emotion does not trigger the cause. Simple similarity calculations fail to capture this directional characteristic. In emotion-cause pair extraction tasks, the extremely imbalanced distribution of positive and negative samples is another critical problem. In real-world dialogue data, the number of positive samples that constitute genuine emotion-cause pairings is far less than the number of negative samples that do not. This imbalance is particularly pronounced when using Cartesian products to generate candidate pairs. Existing techniques typically do not specifically model this imbalance; instead, they directly train the model against a backdrop of a large number of negative samples. This approach easily leads to the model biasing towards predicting negative samples during training, resulting in low recall or severe prediction bias. Furthermore, simple loss function designs are insufficient to effectively guide the model to focus on the few but crucial positive samples, further reducing the accuracy of emotion-cause pairing results.
[0004] The shortcomings of existing technologies are as follows: First, existing research usually treats emotion recognition and cause recognition as independent sub-tasks, or only achieves weak association modeling by sharing underlying features. It lacks an explicit characterization of the relationship between emotion and cause at the semantic level, making it difficult to fully utilize the strong semantic interreference that objectively exists between the two, thus limiting the effectiveness of joint reasoning. Second, after obtaining emotion and cause prediction results, existing research often uses Cartesian product combined with simple similarity calculation for pairing. This type of method has limited modeling capabilities and cannot characterize the complex directional and asymmetric causal relationship between emotion and cause, resulting in insufficient accuracy of the pairing results. Finally, because the number of positive samples that can constitute a true emotion-cause correspondence is far less than the number of negative samples, existing technologies generally face the problem of extremely unbalanced distribution of positive and negative samples, leading to high model training difficulty, significant prediction bias, and limitations on overall prediction accuracy and stability. Summary of the Invention
[0005] In view of this, the purpose of this invention is to propose an emotion-cause binary extraction method based on multi-task joint learning, so as to solve the technical problem that the existing technology cannot make full use of the strong semantic interreference between emotion recognition and cause recognition.
[0006] The technical means employed in this invention are as follows:
[0007] A method for extracting emotion-cause binary pairs based on multi-task joint learning includes the following steps: S1. Obtain multimodal dialogue data and extract sentence-level representations for each sentence; S2. Input the sentence-level representation as a shared feature into the initial prediction fully connected layer of the multi-task joint learning network to perform initial predictions for emotion recognition and cause recognition tasks, and obtain the initial emotion prediction probability distribution and the initial cause prediction probability distribution. S3. Based on the initial emotion prediction probability distribution and the initial cause prediction probability distribution, the learnable parameters are weighted and summed to calculate the emotion label semantic vector and the cause label semantic vector respectively. S4. The emotion label semantic vector and the cause label semantic vector are fused with the shared dialogue representation to generate a first input representation specific to the emotion recognition task and a second input representation specific to the cause recognition task. S5. Input the first input representation and the second input representation into the final classification fully connected layer of the multi-task joint learning network for classification, and obtain the final sentiment prediction result and cause prediction result. S6. Based on the first input representation and the second input representation, calculate the pairing score between each pair of emotion clauses and cause clauses through a dual affine scoring mechanism, and determine the emotion-cause binary based on the pairing score.
[0008] Furthermore, in S3, the emotion label semantic vector and cause label semantic vector The formula is as follows:
[0009]
[0010] in, This represents the initial probability distribution for sentiment prediction. Predict the probability distribution for the initial cause. Embed a matrix for emotion tags. The reason label embedding matrix.
[0011] Furthermore, the specific steps of S4 are as follows: Acquire multimodal dialogue data and extract sentence-level representations for each sentence, then convert the sentiment tag semantic vectors... and cause label semantic vector Multimodal sentence-level representation The inputs are concatenated to construct task-specific inputs; the input for the emotion task incorporates causal label semantic vectors, enabling the model to simultaneously reference potential causal semantics when predicting emotions; the input for the causal task incorporates emotion label semantic vectors, allowing the model to identify causal clauses based on emotion semantics.
[0012]
[0013] in, The first input represents, This represents the second input.
[0014] Furthermore, in S6, the dual affine scoring mechanism is specifically as follows: The first input representation and the second input representation are normalized respectively to obtain the normalized emotion representation and cause representation; The normalized sentiment representation is subjected to an affine transformation using a learnable biaffine matrix, and then matrix multiplication is performed with the normalized causal representation to obtain a pairing score. The pairing scores are converted into pairing probabilities using a Sigmoid activation function, and the emotion-cause binary pairs are determined based on the pairing probabilities.
[0015] Furthermore, the training process of the multi-task joint learning network is optimized using a joint loss function, which includes at least: First classification loss used for emotion recognition tasks; Secondary classification loss used for cause identification tasks; Pair loss used for emotion-cause pairing; The first classification loss and the second classification loss are weighted cross-entropy losses.
[0016] Furthermore, the pairing loss is a supervised focus loss used to weight hard samples, as shown in the following formula:
[0017] in, These are the weighting coefficients for positive and negative samples. It is a focal regulation factor. These are real matching tags.
[0018] Furthermore, the joint loss function also includes pair consistency loss, which is used to model the co-occurrence relationship between sentiment clauses and cause clauses in the vicinity of the dialogue; The pairing consistency loss includes direct correlation loss and window correlation loss; The direct correlation loss is calculated based on the sentiment prediction probability and cause prediction probability at the same sentence position; The window correlation loss is calculated by sliding within a preset window and weighting the probability of sentiment prediction and the probability of cause prediction at different offset positions. The formula for the direct correlation loss is as follows:
[0019] The formula for the window correlation loss is as follows:
[0020] in, offset This represents the offset within the window. It is the weight of each offset. Indicates the offset of the cause clause in different positions.
[0021] Furthermore, the pairing consistency loss function is as follows:
[0022] The emotion recognition and cause recognition subtasks each use weighted cross-entropy as the main loss. The cross-entropy losses for emotion recognition and cause recognition are as follows:
[0023]
[0024] Where N is the number of sentences in the dialogue, and C is the category. Let i be the true label of sentence i in category k. This represents the predicted probability that sentence i belongs to category k.
[0025] The present invention also provides a storage medium comprising a stored program, wherein, when the program is executed, it performs any of the above-described emotion-cause binary extraction methods based on multi-task joint learning.
[0026] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes any of the above-described emotion-cause binary extraction methods based on multi-task joint learning through the computer program.
[0027] Compared with the prior art, the present invention has the following advantages: This invention introduces a learnable label embedding matrix and uses initial prediction probabilities to generate label semantic vectors of emotion and cause. This vector is then concatenated with shared features to form a task-specific input, thereby realizing explicit, bidirectional information interaction between emotion recognition and cause recognition tasks at the semantic level. This overcomes the problems of weak task correlation and lack of explicit semantic characterization in existing technologies, and makes full use of the strong semantic interreferentiality between the two, significantly improving the joint reasoning effect.
[0028] This invention replaces the simple Cartesian product and similarity calculation with a dual affine scoring mechanism. It uses a learnable dual affine matrix to perform affine transformation and interactive calculation on the task-specific representations of emotion and cause, which can effectively model the complex and asymmetric causal relationship between emotion and cause. This overcomes the shortcomings of existing pairing methods, such as limited modeling ability and insufficient capture of directional features, thereby significantly improving the accuracy of pairing results.
[0029] This invention optimizes the model by designing a joint loss function that includes supervised focus loss and pair consistency loss. The focus loss dynamically weights hard samples to address the problem of extreme imbalance between positive and negative samples, while the pair consistency loss uses the co-occurrence prior of emotion and cause in neighboring locations to enhance local correlation modeling. Together, they overcome the difficulties in model training and prediction bias caused by sample imbalance, thereby improving the overall prediction accuracy and stability of the model. Attached Figure Description
[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0031] Figure 1This is a schematic diagram of the method of the present invention. Detailed Implementation
[0032] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0034] like Figure 1 As shown, this invention provides a method for extracting emotion-cause binary pairs based on multi-task joint learning, comprising the following steps: S1. Obtain multimodal dialogue data and extract sentence-level representations for each sentence; Sentence-level representation after multimodal fusion As shared features in multi-task joint learning, they are input into a fully connected layer to predict the initial probabilities of emotion and cause, respectively. , The initial prediction phase provides a basic probability distribution for subsequent label semantic guidance, enabling the model to dynamically adjust the information interaction between tasks based on the preliminary prediction results.
[0035] in, The raw score for sentiment prediction. Predict the original score based on the cause.
[0036] S2. Input the sentence-level representation as a shared feature into the initial prediction fully connected layer of the multi-task joint learning network to perform initial predictions for emotion recognition and cause recognition tasks, and obtain the initial emotion prediction probability distribution and the initial cause prediction probability distribution. To further improve the joint modeling between the two tasks, this study uses a learnable label embedding matrix. and Using the predicted sentiment and causal probabilities and By weighted summing of the learnable parameters, we obtain the corresponding semantic vectors of emotion and reason labels. .
[0037]
[0038] S3. Based on the initial emotion prediction probability distribution and the initial cause prediction probability distribution, the learnable parameters are weighted and summed to calculate the emotion label semantic vector and the cause label semantic vector respectively. Label semantic vector and With shared dialogue The inputs are concatenated to construct task-specific inputs. Specifically, the input for the emotion task incorporates causal label semantic vectors, enabling the model to simultaneously reference potential causal semantics when predicting emotions; while the input for the causal task incorporates emotion label semantic vectors, allowing the model to identify causal clauses based on emotion semantics.
[0039] S4. The emotion label semantic vector and the cause label semantic vector are fused with the shared dialogue representation to generate a first input representation specific to the emotion recognition task and a second input representation specific to the cause recognition task. The task-specific inputs are fed into the fully connected layer to obtain the final logits used for classification:
[0040] S5. Input the first input representation and the second input representation into the final classification fully connected layer of the multi-task joint learning network for classification, and obtain the final sentiment prediction result and cause prediction result. Further, the final sentiment prediction probability and cause prediction probability are obtained through softmax:
[0041] S6. Based on the first input representation and the second input representation, calculate the pairing score between each pair of emotion clauses and cause clauses through a dual affine scoring mechanism, and determine the emotion-cause binary based on the pairing score.
[0042] To address the challenge of modeling causal relationships between emotions and causes, this study proposes a pairing method based on biaffine scoring, building upon multi-task joint learning. Traditional methods typically pair emotions by combining independent predictions or using simple dot product similarity, but these methods fail to effectively capture complex pairing patterns or model asymmetric causal relationships. To overcome this limitation, this study employs a biaffine scoring mechanism. It projects specific representations of both emotion and cause tasks onto an equal-dimensional space and normalizes them. A learnable biaffine matrix is used to perform an affine transformation on the emotion representation, followed by matrix multiplication with the cause representation to obtain the pairing logits. The pairing probability is then derived using a sigmoid activation function. Compared to simple dot product methods, the biaffine mechanism can learn the asymmetric mapping from emotion to cause, making it more suitable for modeling emotion-cause relationships.
[0043] The input to this module is based on task-specific input. , After normalization We use a learnable biaffine matrix to model the pairing scores between sentiment and causal clauses:
[0044] in, It is the pairing score between emotion and cause clauses. It is a learnable biaffine matrix. The scoring is converted into pairwise probabilities using the sigmoid activation function:
[0045] in, It is the Sigmoid activation function, which converts the biaffine score into a pairing probability in the [0,1] interval.
[0046] To address the imbalance between positive and negative samples during training, this study employs a supervised focus loss function to weight hard samples. Furthermore, to fully leverage the high co-occurrence of emotion and cause in nearby positions within dialogues, a pairing consistency loss is introduced. This loss considers the direct correlation between emotion and cause clauses at the same position, as well as the window correlation within a fixed window, further improving the accuracy of emotion-cause pairing. This consistency loss weights the probability of cause at different offset positions through window sliding, thereby establishing a closer semantic association between different positions.
[0047] Using supervised focus loss to weight hard samples:
[0048] in, These are the weighting coefficients for positive and negative samples. The focus adjustment factor is used to adjust the weights of hard samples. These are real matching tags.
[0049] Direct correlation:
[0050] Window relevance: Introducing a window sliding mechanism to consider the relevance of sentiment and cause clause pairs in neighboring locations:
[0051] Here, offset represents the offset within the window. It is the weight of each offset. Indicates the offset of the reason clause in different positions. Therefore, the consistency loss function is as follows:
[0052] Meanwhile, the emotion recognition and cause recognition subtasks respectively use weighted cross-entropy as the main loss, and the cross-entropy losses for emotion recognition and cause recognition tasks are as follows:
[0053] Where N is the number of sentences in the dialogue, and C is the classification category, using binary classification. The true label of sentence i in category k. This represents the predicted probability that sentence i belongs to category k.
[0054] The emotion recognition and cause recognition subtasks each employ weighted cross-entropy as the main loss function, while pair consistency loss further enhances the extraction performance of emotion-cause pairs. Through joint optimization of these three types of losses, the model achieves a good balance between sentence-level recognition and pair relationship modeling, thereby significantly improving the accuracy of multimodal emotion-cause pair extraction.
[0055] The method of the present invention is implemented as follows: Implementation environment: The experimental platform adopts a cloud server computing environment, and the hardware configuration includes: one NVIDIA RTX 3090 graphics processing unit (24GB of video memory), a 14-core Intel Xeon Gold 6330 processor, and 90GB of system memory, which can meet the computing needs of multimodal feature processing and deep neural network training.
[0056] In terms of software environment, the experiments were conducted on the Linux operating system. The deep learning models were built and trained using the TensorFlow framework, and CUDA and acceleration libraries compatible with the GPU environment were configured to improve model training and inference efficiency. Throughout the experiments, the same data preprocessing procedures, parameter initialization strategies, and random seed settings were consistently used to ensure the reproducibility of experimental results and the fairness of comparisons.
[0057] Table 1. Comparative Experiment Results Analysis Table
[0058] Table 1 presents the comparison results between our algorithm and several representative baseline methods on three tasks. It can be seen that in the two sub-tasks of sentiment clause recognition and cause clause extraction, the proposed algorithm achieves optimal or near-optimal performance on all three metrics (weighted F1 close to human evaluation). In the Pair Extraction task, our model's F1 score is significantly higher than all other methods, improving the metric by 4% compared to the two-stage ECPE method. This indicates that our algorithm has a more prominent advantage in sentiment-cause pairing, significantly improving the overall effect of sentiment-cause pair extraction while maintaining single-sentence recognition performance.
[0059] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for extracting emotion-cause binary pairs based on multi-task joint learning, characterized in that, Includes the following steps: S1. Obtain multimodal dialogue data and extract sentence-level representations for each sentence; S2. Input the sentence-level representation as a shared feature into the initial prediction fully connected layer of the multi-task joint learning network to perform initial predictions for emotion recognition and cause recognition tasks, and obtain the initial emotion prediction probability distribution and the initial cause prediction probability distribution. S3. Based on the initial emotion prediction probability distribution and the initial cause prediction probability distribution, the learnable parameters are weighted and summed to calculate the emotion label semantic vector and the cause label semantic vector respectively. S4. The emotion label semantic vector and the cause label semantic vector are fused with the shared dialogue representation to generate a first input representation specific to the emotion recognition task and a second input representation specific to the cause recognition task. S5. Input the first input representation and the second input representation into the final classification fully connected layer of the multi-task joint learning network for classification, and obtain the final sentiment prediction result and cause prediction result. S6. Based on the first input representation and the second input representation, calculate the pairing score between each pair of emotion clauses and cause clauses through a dual affine scoring mechanism, and determine the emotion-cause binary based on the pairing score.
2. The emotion-cause binary extraction method based on multi-task joint learning according to claim 1, characterized in that, In S3, the semantic vector of emotion labels and cause label semantic vector The formula is as follows: in, This represents the initial probability distribution for sentiment prediction. Predict the probability distribution for the initial cause. Embed a matrix for emotion tags. The reason label embedding matrix.
3. The emotion-cause binary extraction method based on multi-task joint learning according to claim 2, characterized in that, The specific steps for S4 are as follows: Acquire multimodal dialogue data and extract sentence-level representations for each sentence, then convert the sentiment tag semantic vectors... and cause label semantic vector Sentence-level representation of multimodal The inputs are concatenated to construct task-specific inputs; the input for the emotion task incorporates causal label semantic vectors, enabling the model to simultaneously reference potential causal semantics when predicting emotions; the input for the causal task incorporates emotion label semantic vectors, allowing the model to identify causal clauses based on emotion semantics. in, The first input represents, This represents the second input.
4. The emotion-cause binary extraction method based on multi-task joint learning according to claim 1, characterized in that, In S6, the dual affine scoring mechanism is specifically as follows: The first input representation and the second input representation are normalized respectively to obtain the normalized emotion representation and cause representation; The normalized sentiment representation is subjected to an affine transformation using a learnable biaffine matrix, and then matrix multiplication is performed with the normalized causal representation to obtain a pairing score. The pairing scores are converted into pairing probabilities using a Sigmoid activation function, and the emotion-cause binary pairs are determined based on the pairing probabilities.
5. The emotion-cause binary extraction method based on multi-task joint learning according to claim 1, characterized in that, The training process of the multi-task joint learning network is optimized using a joint loss function, which includes at least the following: First classification loss used for emotion recognition tasks; Secondary classification loss used for cause identification tasks; Pair loss used for emotion-cause pairing; The first classification loss and the second classification loss are weighted cross-entropy losses.
6. The emotion-cause binary extraction method based on multi-task joint learning according to claim 5, characterized in that, The pairing loss is a supervised focus loss used to weight hard samples, and the formula is as follows: in, These are the weighting coefficients for positive and negative samples. It is a focal regulation factor. These are real matching tags.
7. The emotion-cause binary extraction method based on multi-task joint learning according to claim 5, characterized in that, The joint loss function also includes pair consistency loss, which is used to model the co-occurrence relationship between sentiment clauses and cause clauses in the vicinity of the dialogue; The pairing consistency loss includes direct correlation loss and window correlation loss; The direct correlation loss is calculated based on the sentiment prediction probability and cause prediction probability at the same sentence position; The window correlation loss is calculated by sliding within a preset window and weighting the probability of sentiment prediction and the probability of cause prediction at different offset positions. The formula for the direct correlation loss is as follows: The formula for the window correlation loss is as follows: in, offset This represents the offset within the window. It is the weight of each offset. Indicates the offset of the cause clause in different positions.
8. The emotion-cause binary extraction method based on multi-task joint learning according to claim 5, characterized in that, The pairing consistency loss function is as follows: The emotion recognition and cause recognition subtasks each use weighted cross-entropy as the main loss. The cross-entropy losses for emotion recognition and cause recognition are as follows: Where N is the number of sentences in the dialogue, and C is the category. Let i be the true label of sentence i in category k. This represents the predicted probability that sentence i belongs to category k.
9. A storage medium, characterized in that, The storage medium includes a stored program, wherein, when the program is executed, it performs the emotion-cause binary extraction method based on multi-task joint learning as described in any one of claims 1 to 8.
10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the emotion-cause binary extraction method based on multi-task joint learning as described in any one of claims 1 to 8 through the computer program.