Semantic Deep Face Models for Realistic 3D Facial Geometry
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current multi-linear morphable models for generating 3D facial geometry lack realism and user control over facial identities and expressions, as they rely on linear combinations that result in unrealistic expressions and artifacts.
Innovation Solution
A computer-implemented method using separate encoders for facial identities and expressions, based on deep neural networks, to generate realistic faces by decoding encoded representations of identities and expressions, allowing for semantic control and improved expressiveness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If linear combinations of dataset shapes are used to generate faces, then manufacturing simplicity is maintained, but manufacturing precision deteriorates due to unrealistic expressions and artifacts
Solution Approach 1:
The model segments facial features into discrete landmarks that can be independently manipulated. Each landmark is detected and controlled separately, allowing precise control over facial expressions and identities without requiring complex full-face 3D modeling. This segmentation enables realistic face generation by treating facial features as controllable individual elements rather than monolithic structures.
Solution Approach 2:
The invention changes the parameter representation from continuous 3D mesh coordinates to discrete landmark positions with associated expression parameters. By representing faces as a set of key landmarks (eyes, nose, mouth, etc.) with controllable positional and expressive parameters, the system achieves higher manufacturing precision while maintaining manageable model complexity through parameterized control.
2Measurement precision
If separate encoders for identity and expression are used, then measurement precision of facial dimensions is improved, but device complexity increases
Solution Approach 1:
The dual-encoder architecture segments the face generation task into two distinct functional components: identity encoding and expression encoding. The first encoder processes identity-related features while the second encoder handles expression-related features. This segmentation allows precise measurement and control of different facial dimensions independently, achieving high measurement precision for both identity and expression without requiring a single overly complex model.
Solution Approach 2:
Each encoder is designed to be universal in its own domain - the identity encoder handles all identity variations across different faces, while the expression encoder handles all expression variations. This multi-functionality within specialized components allows the system to achieve comprehensive control precision for both identity and expression dimensions while keeping each individual encoder's structure manageable and interpretable.
Data Source
AI summary
Techniques are disclosed for training and applying nonlinear face models. In embodiments, a nonlinear face model includes an identity encoder, an expression encoder, and a decoder. The identity encoder takes as input a representation of a facial identity, such as a neutral face mesh minus a reference mesh, and outputs a code associated with the facial identity. The expression encoder takes as input a representation of a target expression, such as a set of blendweight values, and outputs a code associated with the target expression. The codes associated with the facial identity and the facial expression can be concatenated and input into the decoder, which outputs a representation of a face having the facial identity and expression. The representation of the face can include vertex displacements for deforming the reference mesh.