Multi-stage Neural Network for Vehicle Facial Expression Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing facial expression recognition technologies struggle to accurately recognize partial and fine facial expressions, which are crucial for improving emotion classification performance, especially in vehicle occupants where variations in pose and blockage occur.
Innovation Solution
A multi-stage neural network approach is employed, involving a first neural network with residual blocks, followed by sequential application of neural networks to extract first-level, second-level, and third-level features. These features are then selectively combined and concatenated using attention modules and graph convolutional networks to enhance discriminative power and reduce edge information loss.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If facial landmark extraction method is used, then facial expression recognition can be performed, but detailed facial expressions that are difficult to represent as landmarks cannot be recognized
Solution Approach 1:
The patent segments the facial region into multiple sub-regions (e.g., left eye, right eye, nose, mouth, chin) and extracts features from each sub-region separately using neural networks. This allows detailed expressions in specific areas to be captured even when overall facial landmarks are insufficient or obscured.
Solution Approach 2:
The patent transitions from traditional 2D landmark-based representation to a multi-dimensional feature extraction approach using neural networks that operate on segmented facial sub-regions. This enables capture of subtle expression variations that cannot be represented by discrete landmarks alone.
2Measurement precision
If multi-stage neural network approach is used, then feature extraction capability is improved, but computational complexity increases
Solution Approach 1:
The neural network is divided into multiple stages that process different levels of facial features: first stage extracts global facial features, second stage extracts local sub-region features, and third stage extracts detailed expression features. Each stage focuses on specific aspect of feature extraction, improving overall accuracy while managing computational complexity through hierarchical processing.
Data Source
AI summary
A device prepares an input image including a facial region to perform facial expression recognition of a vehicle occupant, inputs the input image to a first neural network including basic modules including a residual block, applies a second neural network to the output of the first neural network to extract first-level features and segment the output of the first neural network into local regions, applies a third neural network to each of local regions to extract second-level features, segments the output of the first neural network into patch regions greater than the number of local regions, and applies a fourth neural network to each of the patch regions to extract third-level features, and combines and concatenates at least some of the first-level, second-level, and third-level features, input the concatenated and combined features to a classifier, and classify an emotion through the classifier.