Dynamic Facial Expression Recognition with 3D Segment Aggregation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video-based dynamic facial expression recognition methods are susceptible to inaccuracies due to facial movements, particularly when individuals speak, leading to reduced recognition accuracy.
Innovation Solution
A method involving segmentation of videos into segments, using a 3D convolutional network to extract and aggregate features of face image frames within each segment, and then aggregating features across segments to determine expression labels, while accounting for time series relationships.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If video-based dynamic facial expression recognition is performed on complete videos, then comprehensive expression information can be captured, but recognition accuracy deteriorates due to facial movements such as speaking
Solution Approach 1:
The video is divided into multiple video segments, and each video segment is further divided into multiple image frames. This segmentation allows the system to process smaller units of facial expressions independently, reducing the impact of facial movements such as speaking that occur across the entire video. By analyzing expressions in segmented frames rather than the complete video, the system achieves more accurate recognition while isolating harmful movement artifacts.
2Productivity
If image-based facial expression recognition is used, then processing speed is improved, but recognition accuracy deteriorates due to limited expression information
Solution Approach 1:
The method segments videos into multiple video segments and further segments each video segment into multiple image frames. This segmentation strategy enables parallel processing of multiple frames while maintaining temporal relationships, thus improving processing speed compared to analyzing complete videos, while capturing more comprehensive expression information than single-frame analysis.
Solution Approach 2:
The patent introduces a temporal dimension by analyzing multiple image frames within video segments and aggregating results across segments. This multi-dimensional approach (spatial frames + temporal sequence) enriches expression information without sacrificing processing efficiency, as the segmented structure allows for optimized computation across different time scales.
3Loss of information
If complete video analysis is performed for expression recognition, then comprehensive temporal information is obtained, but computational complexity increases
Solution Approach 1:
The video is segmented into multiple video segments, and each video segment is divided into multiple image frames. This hierarchical segmentation reduces computational complexity by breaking down the large-scale video analysis problem into smaller, more manageable sub-problems. The system processes each frame and segment independently or in small batches, avoiding the need to compute across the entire video at once, thus reducing memory requirements and computational burden while preserving temporal information through the segmented structure.
Data Source
AI summary
Embodiments of the present disclosure disclose a dynamic facial expression recognition method, a device, and product. The method includes: segmenting a video to be recognized into video segments; extracting and aggregating, for each video segment, features of face image frames in the video segment by using a 3D convolutional network, to obtain target feature data of the video segment; aggregating target feature data of the plurality of video segments, to obtain target feature data of the video to be recognized; and determining an expression label of the video to be recognized based on the target feature data of the video to be recognized, to obtain an expression label recognition result of the video to be recognized.