3D Face Reconstruction Using Regularized Expression Coefficients
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional three-dimensional face image reconstruction methods based on a single picture often lead to erroneous results due to ambiguity in the depth direction, particularly when faces are rotated, resulting in incorrect sizes of facial features.
Innovation Solution
A method that involves acquiring real and predicted two-dimensional face key points, solving a loss function with an additional regular constraint term to optimize expression coefficients, and reconstructing a three-dimensional face image using these coefficients, while considering expression basis vectors to accurately represent face states.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If conventional three-dimensional face image reconstruction methods based on single picture are used, then the reconstruction process is simple and fast, but the accuracy is low due to depth ambiguity causing erroneous optimization of expression coefficients
Solution Approach 1:
The patent applies preliminary action by pre-establishing a three-dimensional deformable model with identity and expression components before the reconstruction process. The model is prepared in advance with PCA decomposition, allowing the optimization process to focus on finding coefficients rather than building the model structure during reconstruction, thus improving accuracy while maintaining simplicity
Solution Approach 2:
The patent implements feedback by using a loss function that compares predicted two-dimensional key points with real two-dimensional key points. This feedback mechanism guides the optimization of expression coefficients, allowing the system to iteratively improve reconstruction accuracy by adjusting coefficients to minimize the difference between predicted and actual key point positions
2Manufacturing precision
If expression coefficients are optimized without additional constraints, then the optimization process is fast, but the results are inaccurate due to depth ambiguity causing unrealistic face states
Solution Approach 1:
The patent applies parameter changes by modifying the loss function to include an additional constraint term that penalizes unrealistic expression coefficients. This constraint term changes the optimization landscape to guide the solver toward physically plausible expression states, improving accuracy without requiring excessive optimization iterations
Solution Approach 2:
The patent introduces an intermediary constraint term in the loss function that acts as a mediator between the key point matching objective and the expression realism requirement. This intermediary term ensures that expression coefficients remain within realistic bounds while still allowing the optimization to converge efficiently
3Manufacturing precision
If high precision laser radars or multi-view three-dimensional reconstruction are used, then the reconstruction accuracy is high, but the equipment requirements are high and the process is time consuming
Solution Approach 1:
The patent applies copying by creating a three-dimensional deformable model that replicates the essential geometric and expressive characteristics of human faces. This virtual model serves as a copy of real face structures, allowing accurate reconstruction from single two-dimensional images without requiring complex physical scanning equipment like laser radars or multiple camera views
Solution Approach 2:
The patent transforms the reconstruction problem from requiring complex physical measurements to solving for parameters (identity coefficients, expression coefficients, rotation, translation, focal length) in a pre-established deformable model. This parameter-based approach simplifies the equipment requirements while maintaining reconstruction accuracy
Data Source
AI summary
Techniques for reconstructing three-dimensional face images are described herein. The disclosed techniques include acquiring a real two-dimensional face key point and a predicted two-dimensional face key point; solving a first loss function consisting of the real two-dimensional face key point, the predicted two-dimensional face key point and a preset additional regular constraint term to iteratively optimize an expression coefficient, where the additional regular constraint term is used for constraining the expression coefficient such that the expression coefficient represents a real state of a face; and reconstructing a three-dimensional face image based on the expression coefficient.