Pose Invariant Face Recognition via 3D Spatial Transformer Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing face recognition systems struggle with large pose variations between enrollment and probe images, particularly in real-world applications like law enforcement, where frontal mugshot images are matched with non-frontal or profile images, leading to decreased accuracy due to self-occluded regions and nuisance transformations.
Innovation Solution
The method employs 3D Spatial Transformer Networks to generate a 3D face model from input images at any pose, normalizing off-angle faces to a frontal viewpoint by masking non-visible regions and using only the stable half-face for recognition, transforming out-of-plane rotation into a 2D alignment problem suitable for convolutional neural networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If pose synthesis or pose correction methods are used to handle pose variations, then face recognition can be performed on off-angle images, but self-occluded regions cause degradation in recognition accuracy as pose angle increases
Solution Approach 1:
The face image is segmented into two independent half-face images based on the pose angle. For positive pose angles, the left half-face is extracted; for negative pose angles, the right half-face is extracted. This segmentation eliminates self-occluded regions by selecting only the visible portion of the face, thereby maintaining recognition accuracy across various pose angles while improving pose tolerance.
2Loss of information
If whole face models are used for recognition, then complete face information is available, but recognition accuracy degrades at extreme pose angles due to self-occlusion
Solution Approach 1:
The method extracts only the visible half-face region from the input image based on the estimated pose angle. By taking out and using only the relevant visible portion (left or right half-face) rather than the entire face image, the system avoids the degradation caused by self-occluded regions while still capturing sufficient discriminative information for accurate recognition.
3Measurement precision
If traditional deep learning approaches are used, then state-of-the-art results are achieved on frontal images, but performance drops in challenging settings with pose variation and degradation
Solution Approach 1:
The method performs preliminary pose angle estimation and half-face selection before the actual recognition process. By determining the pose angle and selecting the appropriate half-face region in advance, the system prepares the input data in a pose-invariant manner, enabling traditional deep learning approaches to maintain high accuracy across varying poses without requiring complex pose-invariant architectures.
Data Source
AI summary
The disclosed method generates a pose invariant feature by normalizing off-angle faces to generate a pose invariant input image. Any face recognition mode can be used with this pre processing step. In this method, method, the 3D Spatial Transformer Networks is used to extract a 3D model of the face from an input at any pose.