3D Facial Attribute Generation via Stacked GANs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current facial recognition systems face challenges in handling pose-variance in 3D facial attributes, leading to difficulties in generating accurate and complete facial representations from single input images, especially with self-occlusions and varying facial attributes like beards or sunglasses.
Innovation Solution
A system utilizing a hardware processor-based 3D regression network for generating a space position map and camera perspective matrix, followed by a two-part stacked Generative Adversarial Network (GAN) comprising a Texture Completion GAN (TC-GAN) and a 3D Attribute generation GAN (3DA-GAN) to complete and generate pose-variant 3D facial attributes, addressing self-occlusions and varying attributes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a single input image is used for facial attribute generation, then the input complexity is reduced, but the accuracy of 3D facial attributes deteriorates due to pose-variance and self-occlusions
Solution Approach 1:
The system segments the facial attribute generation task into multiple stages: first generating a space position map and camera perspective matrix from the single input image, then using these to guide the generation of complete 3D facial attributes. This segmentation allows the system to handle pose-variance and self-occlusions systematically while maintaining input simplicity.
Solution Approach 2:
The system introduces intermediate representations (space position map and camera perspective matrix) that mediate between the single input image and the final 3D facial attributes. These intermediaries encode pose information and guide the attribute generation process, resolving the contradiction between simple input and accurate output.
2Device complexity
If pose-variance is handled using traditional methods, then the system complexity is low, but the ability to generate complete facial attributes deteriorates due to self-occlusions
Solution Approach 1:
The system transitions from 2D image processing to 3D attribute generation by introducing a space position map that encodes three-dimensional spatial information. This dimensional transformation enables the system to reason about self-occlusions and pose-variance in 3D space, improving attribute completeness while managing complexity through structured representations.
Solution Approach 2:
The system performs preliminary generation of the space position map and camera perspective matrix before generating the final facial attributes. This preliminary action captures pose information early in the process, enabling subsequent steps to compensate for self-occlusions and ensure complete attribute generation.
Data Source
AI summary
A system is provided for pose-variant 3D facial attribute generation. A first stage has a hardware processor based 3D regression network for directly generating a space position map for a 3D shape and a camera perspective matrix from a single input image of a face and further having a rendering layer for rendering a partial texture map of the single input image based on the space position map and the camera perspective matrix. A second stage has a hardware processor based two-part stacked Generative Adversarial Network (GAN) including a Texture Completion GAN (TC-GAN) stacked with a 3D Attribute generation GAN (3DA-GAN). The TC-GAN completes the partial texture map to form a complete texture map based on the partial texture map and the space position map. The 3DA-GAN generates a target facial attribute for the single input image based on the complete texture map and the space position map.