Graph Neural Network for Dialogue Emotion Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current dialogue emotion recognition systems rely solely on single sentence-based models, neglecting time series and interaction information, making them inadequate for dialogue scenes.
Innovation Solution
A dialogue emotion correction method using a graph neural network that extracts multi-modal features, fuses interaction information, and incorporates time-series context to improve emotion recognition accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If single sentence-based emotion recognition models are used, then the model complexity is low and easy to implement, but the emotion recognition accuracy in dialogue scenes is insufficient because time series information and interaction information are not considered
Solution Approach 1:
The model is segmented into three distinct modules: a base emotion recognition model for individual sentence analysis, an interaction modeling module for capturing talker relationships, and a time-series modeling module for contextual evolution. This segmentation allows each module to specialize in specific aspects while maintaining manageable complexity through modular architecture.
Solution Approach 2:
The patent merges multiple information sources (acoustic features, text features, image features) into multi-modal features, and combines the outputs of the base emotion recognition model, interaction modeling, and time-series modeling through fusion layers. This merging integrates diverse data sources and processing results to achieve comprehensive emotion recognition.
2Measurement precision
If multi-modal features and interaction information are integrated, then the emotion recognition accuracy is improved, but the computational complexity and processing time increase
Solution Approach 1:
The model performs preliminary feature extraction and multi-modal feature fusion at the beginning, preparing processed representations of acoustic, text, and image data before they are needed for emotion recognition. This preliminary processing reduces the computational burden during the actual emotion prediction phase.
Solution Approach 2:
The interaction modeling module and time-series modeling module serve as intermediaries that process and transform raw multi-modal features into refined representations. These intermediary modules filter and organize information before passing it to the final emotion recognition layer, making the overall process more efficient.
3Measurement precision
If interaction information between talkers is modeled, then the understanding of empathetic emotions is improved, but the device complexity increases due to additional modeling requirements
Solution Approach 1:
The interaction modeling module uses a universal graph neural network architecture that can handle multiple types of relationships (empathy, turn-taking, speaker identity) through a single unified framework. This multi-functional approach reduces the need for separate specialized models for different interaction types.
4Measurement precision
If time-series context information is incorporated, then the understanding of emotional evolution in dialogue is improved, but the computational resources and model complexity increase
Solution Approach 1:
The time-series modeling module dynamically processes emotional contexts by capturing the evolution of emotions across dialogue turns. The model adapts to temporal patterns and contextual changes, allowing it to handle dynamic emotional states while maintaining a structured approach through recurrent neural network architectures.
Data Source
AI summary
Disclosed is a dialogue emotion correction method based on a graph neural network, including: extracting acoustic features, text features, and image features from a video file to fuse them into multi-modal features; obtaining an emotion prediction result of each sentence of a dialogue in the video file by using the multi-modal features; fusing the emotion prediction result of each sentence with interaction information between talkers in the video file to obtain interaction information fused emotion features; combining, on the basis of the interaction information fused emotion features, with context-dependence relationship in the dialogue to obtain time-series information fused emotion features; correcting, by using the time-series information fused emotion features, the emotion prediction result of each sentence that is obtained previously as to obtain a more accurate emotion recognition result.