Context-Aware Emotion Recognition With Long- and Short-Term Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional automated emotion recognition methods fail to accurately consider spatiotemporal context features, leading to incomplete understanding of a person's emotions.
Innovation Solution
An emotion recognition method utilizing long-term and short-term memory to store and integrate context information, including facial expressions, full-body regions, and voice, through a context-aware approach using a vision model, language model, and emotion recognizer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional automated emotion recognition methods focus only on facial expressions and outward expressions, then the system complexity is reduced, but the emotion recognition accuracy is insufficient due to lack of spatiotemporal context
Solution Approach 1:
The patent segments the emotion recognition system into multiple independent modules: facial expression analysis module, body language analysis module, voice analysis module, and spatiotemporal context integration module. Each module processes specific features independently and their results are integrated through a neural network, resolving the contradiction by organizing complexity into manageable segments while achieving comprehensive emotion recognition
Solution Approach 2:
The patent employs a composite approach by integrating multiple types of input data (facial expressions, body language, voice characteristics) and multiple types of context information (spatial context, temporal context) into a unified emotion recognition framework. This composite structure allows the system to leverage diverse information sources simultaneously, improving accuracy while managing complexity through structured integration
2Measurement precision
If the system stores and processes all context information for every frame, then the emotion recognition accuracy improves, but the computational load and processing time increase significantly
Solution Approach 1:
The patent extracts only the essential spatiotemporal context features needed for emotion recognition from the full input data stream. The system selectively extracts spatial context (environmental cues, object relationships) and temporal context (emotion transitions, duration patterns) rather than processing all possible features, thereby reducing processing time while maintaining recognition accuracy
Solution Approach 2:
The system performs preliminary processing of context information by pre-defining spatial context templates and temporal context windows before main emotion recognition occurs. This preliminary organization of context data allows for faster integration during actual emotion analysis, reducing real-time processing requirements while preserving comprehensive context utilization
3Speed
If the system considers only short-term facial expressions, then the processing speed is maintained, but the understanding of long-term emotional states and periodic patterns is lost
Solution Approach 1:
The patent adds a temporal dimension to the emotion recognition system by integrating time-based context analysis alongside spatial facial expression analysis. The system processes emotions across multiple time scales - immediate facial expressions and longer-term emotional patterns - by incorporating temporal context windows that track emotion evolution over time, thus preserving temporal information while maintaining processing efficiency through multi-scale analysis
Data Source
AI summary
Disclosed herein are an emotion recognition method and method based on context information. The emotion recognition method includes detecting information corresponding to an emotion recognition subject from an input image, extracting a recognition subject feature based on the information corresponding to the emotion recognition subject, extracting a context feature based on the input image, storing the recognition subject feature and the context feature in a short-term memory, and storing the context feature in a long-term memory.