Deep Neural Network Face Normalization for Unconstrained Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods struggle to normalize unconstrained human face images, particularly in removing perspective distortions, relighting, and neutralizing expressions, which are essential for high-fidelity three-dimensional avatar creation and portrait manipulation, often requiring accurate scene and face parameters and failing to produce plausible results under harsh lighting conditions.
Innovation Solution
A deep learning framework using generative adversarial networks (GANs) for perspective undistortion, lighting normalization, and expression neutralization, which sequentially removes perspective distortions, re-illuminates images with evenly lit diffuse illumination, and neutralizes facial expressions, preserving high-resolution details and likeness, without relying on three-dimensional face models or known camera parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional image-based relighting and facial alteration methods are used, then accurate prior knowledge about the person's face or scene parameters is required, but this makes the system unable to process unconstrained photos with harsh lighting conditions and facial expressions
Solution Approach 1:
The patent transforms the input image through multiple parameter changes: perspective transformation to remove distortion, lighting parameter adjustment to achieve even illumination, and expression parameter modification to neutralize facial expressions. These parameter transformations enable the system to process unconstrained photos while maintaining reliability.
Solution Approach 2:
The normalization process is segmented into three independent modules: perspective undistortion, lighting normalization, and expression neutralization. Each module handles a specific aspect of the transformation, allowing the system to process complex unconstrained images by breaking down the task into manageable segments that can be applied sequentially.
2Ease of operation
If end-to-end face normalization systems are used, then the processing is simplified, but the output resolution is low and not suitable for high-fidelity three-dimensional avatar creation
Solution Approach 1:
Instead of using a single end-to-end system that produces low-resolution output, the patent segments the normalization process into three separate modules that can be applied independently. This segmentation allows each module to focus on its specific task while preserving high-resolution details, and the processed images can be used for high-fidelity three-dimensional avatar creation.
3Manufacturing precision
If individual normalization tasks for distortion, lighting, pose, and expressions are performed separately, then high quality results are achieved, but the system complexity increases
Solution Approach 1:
The patent divides the face normalization task into three distinct modules: perspective undistortion module, lighting normalization module, and expression neutralization module. Each module is optimized for its specific function, achieving high quality results. The modular architecture, while increasing component count, allows for independent optimization and reuse of each module in different applications.
Solution Approach 2:
The three normalization modules are designed to be universal and can be applied individually or in combination depending on the specific requirements. Each module can process various types of input images and produce standardized output, making them multi-functional tools that can be reused across different applications such as facial recognition, avatar creation, and portrait manipulation.
4Measurement precision
If three-dimensional face models or known camera parameters are used for normalization, then accurate results are achieved, but this requirement limits the system's ability to process unconstrained photos
Solution Approach 1:
The system performs self-service by automatically estimating camera parameters and face geometry from the input image itself, without requiring external prior knowledge. The perspective undistortion module infers the necessary transformation parameters directly from the image content, enabling the system to process completely unconstrained photos while maintaining normalization accuracy.
Data Source
AI summary
A system, method, and apparatus for generating a normalization of a single two-dimensional image of an unconstrained human face. The system receives the single two-dimensional image of the unconstrained human face, generates an undistorted face based on the unconstrained human face by removing perspective distortion from the unconstrained human face via a perspective undistortion network, generates an evenly lit face based on the undistorted face by normalizing lighting of the undistorted face via a lighting translation network, and generates a frontalized and neutralized expression face based on the evenly lit face via an expression neutralization network.