Digital Face Appearance Synthesis via Style-Based Generators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current techniques for generating digital faces lack fine-grained control over attributes like gender, age, identity, and ethnicity, and fail to align appearance maps with 3D geometry, resulting in unrealistic digital faces.
Innovation Solution
A computer-implemented method using two machine learning models: a style-based generator to create low-resolution appearance maps based on user-selected styles, and a super-resolution generator to upscaled these maps and 3D geometry, ensuring alignment and realistic rendering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If typical neural generative models are used to generate photorealistic images of faces, then photorealism is improved, but fine-grained control of facial characteristics (gender, age, identity, ethnicity) is lost
Solution Approach 1:
The patent segments facial characteristics into distinct semantic attributes (gender, age, identity, ethnicity) that can be independently controlled. The style-based generator decomposes the face generation process into separate style vectors corresponding to different attributes, allowing users to control each attribute independently while maintaining photorealism.
2Productivity
If appearance maps are generated without alignment to 3D geometry, then generation speed is improved, but alignment between appearance maps and 3D geometry deteriorates
Solution Approach 1:
The patent performs preliminary alignment by generating appearance maps that are inherently aligned with 3D geometry through the style-based generator. The generator incorporates 3D geometry information during the generation process, ensuring that texture details, lighting, and shading are pre-aligned with the corresponding 3D facial structures before rendering.
3Manufacturing precision
If artists manually create realistic-looking digital faces, then quality and realism are improved, but time consumption increases
Solution Approach 1:
The patent replaces the manual mechanical process of artistic face creation with an automated neural network-based system. The style-based generator automatically generates photorealistic digital faces with proper 3D geometry alignment, eliminating the need for manual artist intervention while maintaining or improving quality and reducing time consumption from hours/months to seconds.
Data Source
AI summary
Techniques are disclosed for generating digital faces. In some examples, a style-based generator receives as inputs initial tensor(s) and style vector(s) corresponding to user-selected semantic attribute styles, such as the desired expression, gender, age, identity, and/or ethnicity of a digital face. The style-based generator is trained to process such inputs and output low-resolution appearance map(s) for the digital face, such as a texture map, a normal map, and/or a specular roughness map. The low-resolution appearance map(s) are further processed using a super-resolution generator that is trained to take the low-resolution appearance map(s) and low-resolution 3D geometry of the digital face as inputs and output high-resolution appearance map(s) that align with high-resolution 3D geometry of the digital face. Such high-resolution appearance map(s) and high-resolution 3D geometry can then be used to render standalone images or the frames of a video that include the digital face.