Multimodal emotion fusion method based on inverse reinforcement learning

Abstract

The embodiment of the invention relates to the field of multi-modal emotion analysis, and discloses a multi-modal emotion fusion method based on inverse reinforcement learning. Dynamic weighted fusion is carried out through modal features such as voice, facial expressions and texts, an optimal modal fusion strategy is derived from known emotion label data by adopting inverse reinforcement learning, and the contribution weight of each modal in emotion analysis is automatically adjusted, so that efficient, accurate and stable emotion recognition is realized. Through the technology of the invention, the sentiment analysis system can adaptively optimize modal fusion in various sentiment situations, the generalization ability of the sentiment analysis system in different scenes is significantly improved, the dependence on a large amount of annotated data is reduced, and the sentiment recognition precision is improved at the same time. The method has a wide application prospect and can be widely applied to the fields of intelligent customer service, emotion calculation and the like. The method and the device can be at least used for solving the problems of fixed modal weight and low emotion recognition accuracy in the prior art.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and affective computing, and in particular to a multimodal affective fusion technology based on inverse reinforcement learning, and in particular to a technology that improves the accuracy and adaptability of affective analysis by dynamically adjusting the fusion strategy of different modal data. Background Art

[0002] Sentiment analysis is a core task in affective computing, widely used in fields such as intelligent customer service, affective computing, emotion recognition, and voice assistants. Traditional sentiment analysis methods typically rely on single-modal data for emotion recognition and on training deep learning models with large amounts of labeled data. While these methods achieve good results when processing single-modal data, the accuracy of single-modal analysis is insufficient to meet the complex emotion recognition requirements of multimodal sentiment analysis.

[0003] With the development of artificial intelligence (AI) technology, multimodal sentiment analysis has gradually become an effective method for emotion recognition. By integrating information from multiple modalities, such as speech, facial expressions, and text, multimodal sentiment analysis can improve the accuracy and generalization of sentiment analysis. However, in practical applications, how to effectively integrate information from different modalities, how to dynamically adjust the weights of different modalities, and how to deal with noise and heterogeneity between modalities remain challenges.

[0004] Currently, most multimodal emotion fusion methods use weighted averaging, support vector machines, or deep neural networks to fuse multimodal data. However, these methods rely on hand-crafted rules, are unable to adaptively adjust modal weights, and are susceptible to noisy data, limiting the accuracy of emotion recognition. Furthermore, traditional deep learning methods rely heavily on labeled data, requiring a large number of labeled samples to achieve good results. This complicates implementation in scenarios where data is scarce or labeling is expensive.

[0005] In recent years, inverse reinforcement learning (IRL) has been gradually introduced as an effective learning strategy for multimodal sentiment fusion. By deriving the optimal fusion strategy from existing sentiment-labeled data, IRL avoids the need for manual rule design and can adaptively adjust the fusion method between modalities in different emotional contexts. However, most current IRL methods are limited to single-modal sentiment analysis and have not fully considered how to apply IRL to optimize modal fusion strategies in multimodal sentiment analysis.

[0006] Furthermore, existing sentiment analysis systems lack effective human-computer interaction feedback mechanisms, making it impossible to leverage expert experience to correct results and optimize models. Expert feedback can help sentiment analysis systems achieve better adaptability and generalization in practical applications. However, existing multimodal sentiment analysis methods are still relatively weak in leveraging expert feedback mechanisms, which limits the accuracy and reliability of sentiment analysis.

[0007] Therefore, there is an urgent need for a multimodal sentiment fusion method based on inverse reinforcement learning that can automatically learn the optimal fusion strategy for different modal data and dynamically adjust the weights of each modality in sentiment analysis tasks to achieve more accurate and robust sentiment recognition. At the same time, this method should incorporate expert feedback to optimize the performance of the sentiment analysis system, thereby improving the system's adaptability and accuracy, and enhancing the effectiveness of sentiment analysis technology in practical applications. Summary of the Invention

[0008] The purpose of this invention is to address the shortcomings of existing technologies and propose a multimodal emotion fusion method based on inverse reinforcement learning. This method automatically learns the optimal fusion strategy for emotion data and combines it with expert feedback to achieve efficient and accurate emotion recognition. Traditional multimodal emotion analysis methods typically rely on manually designed rules or fixed weights when handling modal fusion and emotion recognition. This invention overcomes this shortcoming by using inverse reinforcement learning technology to dynamically adjust the fusion weights of different modalities.

[0009] The present invention improves the accuracy of sentiment analysis by deriving the optimal modal fusion strategy from labeled data. Specifically, first, feature extraction is performed on multimodal sentiment data from speech, facial expressions, text, etc. to form a multimodal feature set; then, the inverse reinforcement learning algorithm is used to reverse-infer the optimal modal fusion strategy from the labeled sentiment label data to optimize the weighted combination of modal features; then, the weighted features are processed through the sentiment analysis model to automatically generate preliminary sentiment prediction results. Next, experts use the interactive interface to correct the preliminary sentiment prediction results and generate enhanced modal fusion prompts to optimize the fusion strategy. Finally, based on expert feedback, the sentiment analysis model will be iteratively updated to continuously optimize the sentiment prediction results until the preset accuracy standard is reached.

[0010] The advantages of this invention lie in its ability to automatically optimize multimodal sentiment fusion strategies by integrating the power of inverse reinforcement learning. This technology also further improves prediction accuracy through expert feedback, significantly reducing manual intervention and enhancing the efficiency and accuracy of sentiment analysis. Compared to existing multimodal sentiment analysis methods, this invention no longer relies on fixed modal fusion rules. Instead, it dynamically adjusts the weights of each modality through intelligent learning, thereby improving the system's adaptability and generalization capabilities.

[0011] The specific implementation of the present invention is as follows: a multimodal emotion fusion method based on inverse reinforcement learning, which is characterized in that the method first extracts features from speech, facial expressions and text data to form a multimodal emotion feature set; then, based on the inverse reinforcement learning algorithm, the optimal modal fusion strategy is derived from the labeled emotion data, and the contribution weight of each modality is automatically adjusted; then, the strategy is used to perform weighted fusion of each modal feature, and input into the emotion analysis model to generate preliminary emotion prediction results; then, the expert corrects the preliminary results through the interactive interface to generate enhanced modal fusion prompts; finally, the enhanced fusion prompts are input into the emotion analysis model again, and the emotion prediction results are optimized through inverse reinforcement learning and feedback mechanism, ultimately achieving accurate and adaptive emotion recognition. Specifically, the method proposed by the present invention includes the following specific steps:

[0012] 1) Extract features from other modal emotional data such as speech, facial expressions, and text to generate a modal feature set;

[0013] 1.1: Use convolutional neural networks to extract audio features from speech data, facial expression features from facial expression data through facial expression recognition technology, and emotional features from text data through natural language processing technology;

[0014] 1.2: Standardize the extracted modal features to ensure that the features of different modalities are on the same scale for subsequent fusion;

[0015] 1.3: Use word embedding technology to convert text data into high-dimensional feature vectors, and use sentiment dictionaries to analyze the sentiment tendency of the text;

[0016] 1.4: Use temporal modeling to process the temporal features of speech and facial expression data to enhance the dynamics and temporal dependencies of emotional features.

[0017] 2) Based on the inverse reinforcement learning algorithm, the optimal modality fusion strategy is inferred from the existing sentiment label data;

[0018] 2.1: Using the inverse reinforcement learning algorithm, we derive the reward function for multimodal fusion from the label information in the annotated emotion dataset;

[0019] 2.2: The reward function is designed based on the accuracy of the emotion label. If the fused emotion prediction is consistent with the true label, the reward value is positive, otherwise it is negative;

[0020] 2.3: In each round of learning, the algorithm optimizes the modal fusion strategy by calculating the contribution of each modal feature to sentiment prediction;

[0021] 2.4: Update the parameters in the fusion strategy through the back-propagation algorithm to ensure that the model adaptively adjusts the modal weights in a variety of emotional contexts to improve the accuracy of sentiment analysis.

[0022] 3) Dynamically weight different modal features using the derived fusion strategy and input the weighted features into the sentiment analysis model to automatically generate preliminary sentiment prediction results;

[0023] 3.1: Use the derived fusion strategy to perform weighted fusion of features from different modalities to obtain weighted sentiment feature representation;

[0024] 3.2: Input the weighted emotional feature representation into the sentiment analysis model, use the model to predict the emotional label, and generate preliminary emotional prediction results;

[0025] 3.2.1: The sentiment analysis model uses the deep Q-learning algorithm to optimize the strategy based on the reward value of sentiment prediction;

[0026] 3.2.2: During the optimization process, use the experience replay mechanism to store and utilize historical data to accelerate convergence and avoid overfitting;

[0027] 3.2.3: The fusion strategy dynamically adjusts weights and updates strategies to ultimately obtain the optimal modal weighted combination;

[0028] 3.2.4: Based on backpropagation and policy gradient optimization, the model can automatically learn the optimal modal weight distribution in different emotional contexts.

[0029] 3.3: Use activation functions in the sentiment analysis model to classify weighted fusion features and output sentiment category probabilities;

[0030] 3.4: After the initial prediction results are generated, the quality of the results is evaluated using evaluation indicators to provide feedback for subsequent steps.

[0031] 4) Feedback and correction of preliminary emotion prediction results through expert interaction, and generation of enhanced modality fusion prompts;

[0032] 4.1: Experts use the interactive interface to mark and correct the emotional regions that are incorrect or inaccurate in the preliminary emotion prediction results;

[0033] 4.2: Expert feedback can be corrected by clicking on the error part in the interface or directly entering the emotion label;

[0034] 4.3: During the interaction process, the system records the expert's correction operations and generates revised fusion strategy prompts to optimize the sentiment prediction results;

[0035] 4.4: The human-computer interaction system is able to process expert feedback in real time, form enhanced modality fusion prompts, and pass them to the sentiment analysis model for updating.

[0036] 5) Using the enhanced fusion prompts of expert feedback, the sentiment analysis model is driven again to update the sentiment prediction results;

[0037] 5.1: The modal fusion prompts enhanced by expert feedback are fed back into the sentiment analysis model;

[0038] 5.2: Use the sentiment analysis model to update the sentiment prediction results based on the enhanced fusion prompt propagation and perform iterative optimization of sentiment prediction;

[0039] 5.3: The updated sentiment prediction results will be used in the next round of learning to further optimize the model with new sentiment label data;

[0040] 5.4: After each expert feedback, the system is able to recalculate the weights of each modality and adapt to the new emotional pattern.

[0041] 6) Repeat steps 3) and 4) until the emotion prediction accuracy reaches a preset threshold or meets actual application requirements.

[0042] Compared with the prior art, the present invention has the following beneficial technical effects and significant technical progress:

[0043] 1) This paper proposes a multimodal emotion fusion method based on inverse reinforcement learning. By automatically deriving the optimal fusion strategy, it can adaptively adjust the weights of each modality in different emotional contexts, thereby significantly improving the accuracy of emotion analysis and system adaptability.

[0044] 2) The present invention combines an expert feedback mechanism to allow experts to revise sentiment prediction results and generate enhanced fusion prompts, which effectively improves the accuracy of the sentiment analysis system, reduces dependence on large amounts of labeled data, and enhances the robustness of the system in complex situations. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Flowchart of the present invention;

[0046] Figure 2 Schematic diagram of the specific operation of Example 1. DETAILED DESCRIPTION

[0047] This invention aims to address the shortcomings of existing multimodal emotion fusion methods, particularly the challenges of automatically optimizing modal fusion strategies and integrating expert knowledge. To this end, the present invention proposes a multimodal emotion fusion method based on inverse reinforcement learning (IRL), combining an IRL algorithm with an expert feedback mechanism to improve the accuracy of sentiment analysis and the adaptability of the system. First, the present invention extracts features from emotion data in different modalities, such as speech, facial expressions, and text, to generate emotion feature sets for each modality. These feature sets serve as input data for subsequent modal fusion and emotion prediction. Next, an inverse reinforcement learning algorithm is used to train on labeled emotion data to derive an optimal modal fusion strategy. This strategy learns a weighted fusion strategy between modalities from emotion labels by inversely deriving a reward function to achieve optimal modal fusion results. During each round of training, the IRL algorithm automatically adjusts the weights of modal features, enabling the model to adaptively optimize modal fusion in different emotional contexts. Then, using the optimal fusion strategy obtained through training, the features of each modality are weightedly fused to form a weighted emotion feature representation. The weighted emotional features are input into the sentiment analysis model to generate preliminary sentiment predictions. This prediction assigns a probability to each emotion label, which is then used for subsequent feedback optimization. Experts use a human-computer interaction interface to correct the preliminary sentiment predictions, marking incorrect or ambiguous emotion regions. The expert feedback is automatically recorded and used to generate enhanced modality fusion prompts. This enhanced fusion prompt is then passed back into the sentiment analysis model as new input. Finally, the enhanced modality fusion prompt is re-input into the sentiment analysis model for iterative optimization. This process further propagates and optimizes the sentiment prediction results through the built-in video segmentation module or memory-attention mechanism until the system reaches the preset prediction accuracy and meets the application requirements. In this way, the system can gradually improve the accuracy and robustness of emotion recognition through the combined effects of expert feedback and automatic learning.

[0048] See attached Figure 1 The present invention provides a music emotion intervention method based on human-in-the-loop, which specifically includes the following steps:

[0049] 1) Extract features from other modal emotional data such as speech, facial expressions, and text to generate a modal feature set;

[0050] 1.1: Use convolutional neural networks to extract audio features from speech data, facial expression features from facial expression data through facial expression recognition technology, and emotional features from text data through natural language processing technology;

[0051] 1.2: Standardize the extracted modal features to ensure that the features of different modalities are on the same scale for subsequent fusion;

[0052] 1.3: Use word embedding technology to convert text data into high-dimensional feature vectors, and use sentiment dictionaries to analyze the sentiment tendency of the text;

[0053] 1.4: Use temporal modeling to process the temporal features of speech and facial expression data to enhance the dynamics and temporal dependencies of emotional features.

[0054] 2) Based on the inverse reinforcement learning algorithm, the optimal modality fusion strategy is inferred from the existing sentiment label data;

[0055] 2.1: Using the inverse reinforcement learning algorithm, we derive the reward function for multimodal fusion from the label information in the annotated emotion dataset;

[0056] 2.2: The reward function is designed based on the accuracy of the emotion label. If the fused emotion prediction is consistent with the true label, the reward value is positive, otherwise it is negative;

[0057] 2.3: In each round of learning, the algorithm optimizes the modal fusion strategy by calculating the contribution of each modal feature to sentiment prediction;

[0058] 2.4: Update the parameters in the fusion strategy through the back-propagation algorithm to ensure that the model adaptively adjusts the modal weights in a variety of emotional contexts to improve the accuracy of sentiment analysis.

[0059] 3) Dynamically weight different modal features using the derived fusion strategy and input the weighted features into the sentiment analysis model to automatically generate preliminary sentiment prediction results;

[0060] 3.1: Use the derived fusion strategy to perform weighted fusion of features from different modalities to obtain weighted sentiment feature representation;

[0061] 3.2: Input the weighted emotional feature representation into the sentiment analysis model, use the model to predict the emotional label, and generate preliminary emotional prediction results;

[0062] 3.2.1: The sentiment analysis model uses the deep Q-learning algorithm to optimize the strategy based on the reward value of sentiment prediction;

[0063] 3.2.2: During the optimization process, use the experience replay mechanism to store and utilize historical data to accelerate convergence and avoid overfitting;

[0064] 3.2.3: The fusion strategy dynamically adjusts weights and updates strategies to ultimately obtain the optimal modal weighted combination;

[0065] 3.2.4: Based on backpropagation and policy gradient optimization, the model can automatically learn the optimal modal weight distribution in different emotional contexts.

[0066] 3.3: Use activation functions in the sentiment analysis model to classify weighted fusion features and output sentiment category probabilities;

[0067] 3.4: After the initial prediction results are generated, the quality of the results is evaluated using evaluation indicators to provide feedback for subsequent steps.

[0068] 4) Feedback and correction of preliminary emotion prediction results through expert interaction, and generation of enhanced modality fusion prompts;

[0069] 4.1: Experts use the interactive interface to mark and correct the emotional regions that are incorrect or inaccurate in the preliminary emotional prediction results;

[0070] 4.2: Expert feedback can be corrected by clicking on the error part in the interface or directly entering the emotion label;

[0071] 4.3: During the interaction process, the system records the expert's correction operations and generates revised fusion strategy prompts to optimize the sentiment prediction results;

[0072] 4.4: The human-computer interaction system is able to process expert feedback in real time, form enhanced modality fusion prompts, and pass them to the sentiment analysis model for updating.

[0073] 5) Using the enhanced fusion prompts of expert feedback, the sentiment analysis model is driven again to update the sentiment prediction results;

[0074] 5.1: The modal fusion prompts enhanced by expert feedback are fed back into the sentiment analysis model;

[0075] 5.2: Use the sentiment analysis model to update the sentiment prediction results based on the enhanced fusion prompt propagation and perform iterative optimization of sentiment prediction;

[0076] 5.3: The updated sentiment prediction results will be used in the next round of learning to further optimize the model with new sentiment label data;

[0077] 5.4: After each expert feedback, the system is able to recalculate the weights of each modality and adapt to the new emotional pattern.

[0078] 6) Repeat steps 3) and 4) until the emotion prediction accuracy reaches a preset threshold or meets actual application requirements.

[0079] The present invention will be further described below with reference to specific examples and accompanying drawings.

[0080] Example 1

[0081] See Figure 2This embodiment describes the emotion recognition task in multimodal sentiment analysis. First, a labeled emotion dataset is selected, including three modalities: speech, facial expression, and text. For each modal data, data preprocessing and feature extraction are performed. For example, a convolutional neural network is used to extract speech features, a facial expression recognition algorithm is used to extract facial expression features, and a BERT model is used to extract text features. Next, an inverse reinforcement learning algorithm is used for training to derive an optimal modal fusion strategy from the labeled emotion data. This strategy learns how to adjust the fusion weights of each modality based on different emotion labels by inversely deriving the reward function. Specifically, the IRL algorithm automatically optimizes the fusion method between modalities based on the emotion label information in each round of training, thereby achieving the best emotion prediction results. Next, the optimal fusion strategy obtained through training is used to perform a weighted fusion of the features of each modality to generate a weighted emotion feature representation. These weighted features are then input into the sentiment analysis model to generate preliminary emotion prediction results. The sentiment analysis model makes predictions based on the weighted fused features, generating corresponding prediction probabilities for each emotion label. Experts then provide feedback and corrections to the initial sentiment prediction results through a human-computer interaction interface, quickly marking out incorrect or ambiguous sentiment areas. The expert's corrections are automatically recorded and used to generate enhanced modal fusion prompts. This enhanced prompt is then passed back to the sentiment analysis model as a new input to optimize the model's sentiment prediction results. Finally, the enhanced modal fusion prompt is re-input into the sentiment analysis model, where it is further propagated and optimized through the built-in memory attention mechanism or other optimization modules. This process continues until the sentiment prediction accuracy reaches a preset threshold or meets the application requirements. Through continuous expert feedback and automatic optimization, the sentiment analysis system can achieve iterative performance improvement, ultimately achieving high sentiment recognition accuracy.

[0082] The above description is only a preferred embodiment of the present invention, and certain modifications may be made thereto within the scope defined by the claims of the present invention, but all modifications will fall within the scope of protection of the present invention.

Claims

1. A multimodal emotion fusion method based on inverse reinforcement learning, characterized in that: The method comprises the following steps: Extract features from multimodal emotional data such as speech, facial expressions, and text to generate a modal feature set; Based on the inverse reinforcement learning algorithm, the optimal modality fusion strategy is inferred from the existing sentiment label data; The derived modal fusion strategy is used to dynamically weight features of different modalities, and the weighted features are input into the sentiment analysis model to automatically generate preliminary sentiment prediction results. Feedback and correction of preliminary emotion prediction results are performed through expert interaction, and enhanced modality fusion prompts are generated; The enhanced modality fusion prompts generated by feedback correction of preliminary emotion prediction results through expert interaction drive the visual base model and propagate the updated segmentation results through the video segmentation mechanism; Repeat the steps of dynamically weighting different modal features using the derived modal fusion strategy and generating fusion prompts through expert interaction correction until the emotion prediction accuracy reaches a preset threshold or meets actual application requirements.

2. A multimodal emotion fusion method based on inverse reinforcement learning according to claim 1, characterized in that: The feature extraction of multimodal emotion data from speech, facial expression, and text to generate a modal feature set specifically includes: Convolutional neural networks are used to extract audio features from speech data, facial expression features are extracted from facial expression data through facial expression recognition technology, and emotional features are extracted from text data through natural language processing technology; Standardize the extracted features of each modality to ensure that the features of different modalities are on the same scale for subsequent fusion; Use word embedding technology to convert text data into high-dimensional feature vectors, and use sentiment dictionaries to analyze the sentiment tendency of the text; Temporal modeling is used to process the temporal features of speech and facial expression data to enhance the dynamics and temporal dependencies of emotional features.

3. The multimodal emotion fusion method based on inverse reinforcement learning according to claim 1 is characterized in that: The inverse reinforcement learning algorithm is used to infer the optimal modality fusion strategy from the existing emotion label data, specifically including: Using the inverse reinforcement learning algorithm, we derive the reward function for multimodal fusion through the label information in the labeled emotion dataset. The reward function is designed based on the accuracy of the emotion label. If the fused emotion prediction is consistent with the true label, the reward value is positive, otherwise it is negative. In each round of learning, the algorithm optimizes the modal fusion strategy by calculating the contribution of each modal feature to sentiment prediction; The parameters in the fusion strategy are updated through the back-propagation algorithm to ensure that the model adaptively adjusts the modal weights in a variety of emotional situations to improve the accuracy of sentiment analysis.

4. A multimodal emotion fusion method based on inverse reinforcement learning according to claim 1, characterized in that: The derived modality fusion strategy is used to dynamically weight different modality features, and the weighted features are input into the sentiment analysis model to automatically generate preliminary sentiment prediction results, specifically including: The derived fusion strategy is used to perform weighted fusion on the features of different modalities to obtain the weighted emotional feature representation; The weighted emotional feature representation is input into the sentiment analysis model, and the sentiment label is predicted by the model to generate preliminary sentiment prediction results; In the sentiment analysis model, activation functions are used to classify weighted fusion features and output sentiment category probabilities; After the initial prediction results are generated, the quality of the results is assessed using evaluation metrics to provide feedback for subsequent steps.

5. The multimodal emotion fusion method based on inverse reinforcement learning according to claim 1, characterized in that: The feedback correction of the preliminary emotion prediction results through expert interaction and the generation of enhanced modality fusion prompts specifically include: Experts use the interactive interface to mark and correct erroneous or inaccurate emotion regions in the preliminary emotion prediction results; Expert feedback can be corrected by clicking on the error part in the interface or directly entering the emotional label; During the interaction process, the system records the expert's correction operations and generates revised fusion strategy prompts to optimize the sentiment prediction results; The human-computer interaction system is able to process expert feedback in real time, form enhanced modality fusion prompts, and pass them to the sentiment analysis model for updating.

6. The multimodal emotion fusion method based on inverse reinforcement learning according to claim 1, characterized in that: The method of using expert interaction to feedback and correct the preliminary emotion prediction results and generate enhanced modality fusion prompts to drive the visual basic model and propagate the updated segmentation results through the video segmentation mechanism specifically includes: The modality fusion prompts enhanced by expert feedback are input into the sentiment analysis model again; The sentiment analysis model is used to propagate and update the sentiment prediction results according to the enhanced fusion prompts, and the sentiment prediction is iteratively optimized. The updated sentiment prediction results will be used in the next round of learning to further optimize the model with new sentiment label data; After each expert feedback, the system is able to recalculate the weights of each modality and adapt to the new emotional pattern.

7. The multimodal emotion fusion method based on inverse reinforcement learning according to claim 4 is characterized in that: The step of inputting the weighted emotional feature representation into the emotional analysis model, predicting the emotional label through the model, and generating a preliminary emotional prediction result specifically includes: The sentiment analysis model uses a deep Q-learning algorithm to optimize strategies based on the reward value of sentiment prediction; During the optimization process, an experience replay mechanism is used to store and utilize historical data to accelerate convergence and avoid overfitting; The fusion strategy dynamically adjusts weights and updates strategies to ultimately obtain the optimal modal weighted combination; Based on back-propagation and policy gradient optimization, the model can automatically learn the optimal modal weight distribution in different emotional contexts.

Images