Facial Expression Recognition via Multi-Feature Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing facial expression recognition systems face challenges in natural scenes due to interference from light irradiation, head posture changes, and face occlusion, leading to low accuracy and insufficient feature extraction, particularly in dynamic video streams where time-series context information is crucial.
Innovation Solution
The method preprocesses video frames to eliminate irrelevant factors like light irradiation and head posture, extracts facial features using histogram of oriented gradients (HOG), second-order features, and deep neural network features, and fuses these to create comprehensive facial feature vectors for improved expression recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional facial expression recognition methods are used in natural scenes, then the system is simple to implement, but the recognition accuracy is low due to interference from light irradiation, head posture changes, and face occlusion
Solution Approach 1:
The patent segments the facial recognition process into multiple independent modules: light irradiation elimination module, head posture elimination module, occlusion handling module, and feature extraction module. Each module addresses a specific interference factor separately, allowing the system to handle complex natural scene challenges through coordinated operation of specialized sub-systems.
Solution Approach 2:
The patent applies preliminary preprocessing actions to eliminate light irradiation and head posture effects before the main feature extraction process. By removing these interfering factors in advance, the subsequent expression recognition operates on cleaned data, significantly improving accuracy without requiring the main recognition algorithm to be overly complex.
2Measurement precision
If only spatial feature information from single images is extracted, then the processing speed is fast, but the recognition accuracy is insufficient in dynamic video streams where time-series context is crucial
Solution Approach 1:
The patent implements continuous temporal feature extraction and fusion across multiple video frames. By maintaining continuous analysis of time-series data and fusing temporal features with spatial features, the system captures dynamic expression evolution without interrupting the video stream processing, achieving both accuracy and efficiency.
Solution Approach 2:
The patent transitions from analyzing only single static images to processing video frames in the temporal dimension. By extracting features across multiple time points and fusing them with spatial features, the system adds a temporal dimension to the analysis, enabling recognition of dynamic expressions while maintaining processing efficiency through optimized feature fusion.
3Reliability
If comprehensive feature extraction including HOG, second-order features, and deep neural network features is performed, then the recognition robustness is improved, but the computational complexity increases
Solution Approach 1:
The patent merges multiple feature extraction methods (HOG, second-order features, and deep neural network features) into a unified feature fusion framework. By combining these complementary features and processing them through an integrated system, the patent achieves robust multi-dimensional representation of facial expressions while optimizing computational efficiency through shared processing components.
Data Source
AI summary
An expression recognition method under a natural scene comprises: converting an input video into a video frame sequence in terms of a specified frame rate, and performing facial expression labeling on the video frame sequence to obtain a video frame labeled sequence; removing natural light impact, non-face areas, and head posture impact elimination on facial expression from the video frame labeled sequence to obtain an expression video frame sequence; augmenting the expression video frame sequence to obtain a video preprocessed frame sequence; from the video preprocessed frame sequence, extracting HOG features that characterize facial appearance and shape features, extracting second-order features that describe a face creasing degree, and extracting facial pixel-level deep neural network features by using a deep neural network; then, performing vector fusion on these three obtain facial feature fusion vectors for training; and inputting the facial feature fusion vectors into a support vector machine for expression classification.