Facial Expression Recognition Using Bilinear Network Second-Order Features
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing expression recognition technologies face low accuracy due to the reliance on first-order information, which is not robust enough for real-world scenarios, especially under varying gestures and illuminations, leading to poor recognition of facial expressions, particularly at low intensity levels.
Innovation Solution
The method employs a combination of convolutional neural network, fully connected network, and bilinear network models to extract and process second-order information from facial images, improving recognition accuracy by utilizing key point coordinate vectors and enhancing data with random transformations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single model convolutional neural network is used to recognize expressions based on extracted features, then the recognition process is simple, but the accuracy is low due to poor robustness against gestures and illuminations
Solution Approach 1:
The patent segments the expression recognition process into multiple specialized components: a convolutional neural network for feature extraction, a fully connected network for processing first-order information, and a bilinear network for extracting second-order information. Each component focuses on a specific aspect of expression analysis, improving overall accuracy while maintaining manageable complexity through modular design.
Solution Approach 2:
The patent transitions from analyzing only first-order feature information to incorporating second-order information through the bilinear network. This dimensional expansion in the feature space enables the system to capture more nuanced expression patterns and interactions between features, significantly improving recognition accuracy especially for low-intensity expressions.
2Use of energy by moving object
If only first-order information is used for expression recognition, then the processing is computationally efficient, but the robustness against gestures and illuminations is poor
Solution Approach 1:
The patent introduces second-order information processing through the bilinear network, adding a new dimension to the feature analysis. This enables the system to capture interaction effects between features that are invisible in first-order analysis, improving robustness against varying gestures and illumination conditions while maintaining reasonable computational efficiency through efficient bilinear operations.
Solution Approach 2:
The patent creates a composite information processing system that combines first-order features from the fully connected network with second-order features from the bilinear network. This composite approach leverages the strengths of both processing stages, creating a more robust representation that is resilient to gestures and illumination variations.
3Device complexity
If a single model neural network is used for expression recognition, then the model structure is simple, but the accuracy is low especially in low expression intensity cases
Solution Approach 1:
The patent divides the expression recognition task across three specialized network models, each optimized for specific processing functions. The convolutional network extracts spatial features, the fully connected network processes first-order statistics, and the bilinear network captures second-order interactions. This segmentation allows each component to be relatively simple while the combination achieves high accuracy.
Solution Approach 2:
The patent enhances the model's capability to detect low-intensity expressions by incorporating second-order information through the bilinear network. This additional dimensional analysis allows the system to detect subtle expression patterns that would be missed by first-order analysis alone, significantly improving accuracy for faint or low-intensity expressions.
Data Source
AI summary
Embodiments of the present disclosure provide an expression recognition method, apparatus, electronic device and storage medium. An expression recognition model includes a convolutional neural network model, a fully connected network model and a bilinear network model. During an expression recognition process, after an image to be recognized is pre-processed to obtain a facial image and a key point coordinate vector, the facial image is computed by the convolutional neural network model to output a first feature vector, the key point coordinate vector is computed by the fully connected network model to output a second feature vector, the first feature vector and the second feature vector are computed by the bilinear network model to obtain second-order information, and an expression recognition result in turn is obtained according to the second-order information. During this process, robustness of gestures and illuminations is better, and accuracy of expression recognition is improved.