Dynamic Facial Expression Recognition via Hierarchical Emotion Grouping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Dynamic facial expression recognition in videos faces challenges due to irrelevant frames, occluded facial features, and non-frontal poses, which degrade the performance of existing systems.
Innovation Solution
A hierarchical emotion grouping approach using an affectivity extraction network (AEN) that combines features from different semantic levels through a 2D convolutional neural network, temporal transformers, and semantic-to-affective converters, with attention-based converters and frame-level emotion-guided loss functions to maintain emotional representation and reduce information loss.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If video frames are used for dynamic facial expression recognition, then temporal information and practical reliability are improved, but performance degrades due to irrelevant frames, occlusions, and non-frontal poses
Solution Approach 1:
The patent extracts and removes harmful frames from video sequences using detection mechanisms that identify occluded, non-frontal, or irrelevant frames. By extracting only the useful frames that contain clear facial expressions, the system eliminates the degradation caused by irrelevant frames while maintaining temporal information from video data.
Solution Approach 2:
The patent introduces an intermediary filtering layer between video input and facial expression recognition. This intermediary component detects and selects appropriate frames based on quality criteria, mediating between the raw video data and the recognition algorithm to prevent harmful factors from degrading performance.
2Measurement precision
If hierarchical emotion grouping with multiple affective layers is implemented, then discriminative feature representation and recognition accuracy are improved, but system complexity increases
Solution Approach 1:
The patent segments the facial expression recognition task into multiple hierarchical levels: basic emotion recognition, intensity assessment, and complex emotion classification. By dividing the recognition process into distinct hierarchical layers, the system achieves improved accuracy through progressive feature analysis while managing complexity through modular organization of recognition stages.
Solution Approach 2:
The patent adds a hierarchical dimension to the feature representation space, organizing emotions across multiple affective layers from basic to complex. This dimensional organization allows the system to capture nuanced emotional information at different levels of abstraction, improving measurement precision while structuring complexity in a manageable hierarchical framework.
Data Source
AI summary
Embodiments described herein provide systems and methods for facial expression recognition (FER). Embodiments herein combine features of different semantic levels and classifies both sentiment and specific emotion categories with emotion grouping. Embodiments herein include a model with a bottom-up branch that learns facial expressions representation at different semantic levels and output pseudo labels of facial expressions for each frame using a 2D FER model, and a top-down branch that learns discriminative representations by combining feature vectors of each semantic level for recognizing facial expressions at the corresponding emotion group.