Deep Facial Feature Fusion for Recognition Under Appearance Variations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing visual recognition systems in video surveillance face challenges in accurately classifying facial features due to large appearance variations caused by light conditions, view angles, and occlusions, making it difficult to distinguish between similar targets.
Innovation Solution
A deep learning system utilizing a global sub-network, local sub-network, and fusion network with a symmetric triplet loss function to extract and discriminate facial features, employing gradient back-propagation and adaptive weight updates to optimize feature representations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional feature extraction methods (color histogram, Haar features, Gabor features) are used, then the system can process images with basic recognition capability, but the recognition accuracy deteriorates under large appearance variations caused by light conditions, view angles, and occlusions
Solution Approach 1:
The patent divides the face image into multiple local regions (eyes, nose, mouth, cheeks) and extracts features from each region separately using dedicated convolutional neural networks. This segmentation allows the system to capture discriminative local features that remain stable under various appearance variations, thereby improving recognition accuracy despite changes in lighting, pose, or occlusion.
Solution Approach 2:
The patent transitions from traditional 2D image processing to 3D deep feature space by employing multiple convolutional layers that generate hierarchical feature representations. The deep neural networks transform input images into high-dimensional feature vectors that capture complex patterns and relationships, enabling the system to distinguish between different identities even when appearance variations occur in the original image space.
2Measurement precision
If deep learning methods with multiple convolutional layers are used, then the feature extraction capability is improved, but the computational complexity and training difficulty increase
Solution Approach 1:
The patent divides the deep learning system into multiple independent convolutional neural networks, each responsible for extracting features from specific local face regions. This segmentation of the network architecture reduces the complexity of training a single large network while maintaining the benefits of deep feature extraction. Each sub-network can be trained independently on region-specific features, simplifying the overall training process.
Solution Approach 2:
The patent combines the feature vectors extracted from multiple local regions into a unified feature representation for identity recognition. By merging the outputs of several specialized convolutional networks, the system achieves comprehensive feature discrimination capability while keeping each individual network relatively simple and manageable during training.
3Loss of information
If local feature extraction is performed on divided patches, then the spatial correspondence information is preserved, but the overall feature representation becomes more complex
Solution Approach 1:
The patent divides the input image into multiple local patches corresponding to different face regions and processes each patch through separate convolutional layers. This segmentation preserves spatial correspondence information by maintaining the relationship between local features and their positions in the original image, while the structured arrangement of patch summaries keeps the overall representation manageable.
Solution Approach 2:
The patent applies convolutional operations selectively to specific local regions rather than processing the entire image uniformly. By focusing computational resources on extracting features from key face regions (eyes, nose, mouth), the system preserves important spatial correspondence information while avoiding the complexity of processing all image regions with equal detail.
Data Source
Figure 1
Figure 2
Figure 3(a)~4(b)
AI summary
A visual recognition system to process images includes a global sub-network including a convolutional layer and a first max pooling layer. A local sub-network is connected to receive data from the global sub-network, and includes at least two convolutional layers, each connected to a max pooling layer. A fusion network is connected to receive data from the local sub-network, and includes a plurality of fully connected layers that respectively determine local feature maps derived from images. A loss layer is connected to receive data from the fusion network, set filter parameters, and minimize ranking error.