Facial Expression Recognition Using Multi-Frame Feature Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for simulating human facial expressions in virtual characters suffer from a 'jitter phenomenon' due to inconsistent facial expression predictions across consecutive frames, leading to inaccuracies and discontinuities in the animation.
Innovation Solution
A method that fuses facial image features from a current frame with historical and subsequent frames using a preset fusion ratio, incorporating a feature extraction and fusion network to determine a target facial image feature, which is then used to predict expression information, thereby improving the continuity and accuracy of facial expressions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If facial expression prediction is performed independently for each frame, then the processing speed is fast, but the expression continuity deteriorates causing jitter phenomenon
Solution Approach 1:
The patent merges facial expression predictions from multiple frames (current frame, historical frames, and future frames) into a single integrated prediction for the current frame. This is achieved by extracting features from multiple frames, fusing these features through a neural network, and generating a unified expression prediction that ensures temporal consistency and eliminates jitter while maintaining processing efficiency.
2Device complexity
If facial expression prediction uses only current frame image, then the computational complexity is low, but the measurement precision deteriorates
Solution Approach 1:
The patent performs preliminary action by pre-extracting facial features from historical and future frames alongside the current frame. These pre-extracted features are then fused through a neural network to produce a more accurate expression prediction. This approach improves measurement precision by incorporating temporal context from multiple frames while managing computational complexity through efficient feature fusion.
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
The embodiment of the disclosure provides an expression information recognition method, apparatus and device, a readable storage medium and a product. The method includes: acquiring a current frame image, a historical image and a subsequent image of the current frame image, wherein the image comprises a facial image of a target object; determining target facial image feature in the current frame image based on the current frame image, the historical image and the subsequent image; and determining expression information of the target object in the current frame image based on the target facial image feature.