3D Face Shape Prediction Using Synthetic Training Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3D facial reconstruction methods are highly sensitive to lighting conditions and require extensive real data for training, making them inefficient and costly for commercial applications.
Innovation Solution
A deep learning model using Densenet 121 architecture predicts 3D face shape weights directly from images without intermediate representations, trained on a synthetic dataset of diverse lighting conditions and high-quality synthetic data, refined with landmark-based deformation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If current 3D facial reconstruction methods are used, then reconstruction can be performed, but the results are sensitive to lighting models and materials
Solution Approach 1:
The patent transforms the reconstruction approach by changing the parameter representation from direct geometric modeling to statistical shape space representation. By representing faces as linear combinations of basis shapes from a 3DMM, the method becomes invariant to lighting conditions and material properties, directly resolving the sensitivity issue while maintaining reconstruction accuracy
Solution Approach 2:
The patent creates a synthetic copy of real facial data through the 3DMM framework. Instead of directly processing real 3D scans which are sensitive to lighting, the method synthesizes infinite variations of facial shapes and appearances from a compact statistical model, achieving robustness while preserving the essential geometric features
2Quantity of substance
If extensive datasets of real human faces are collected, then more training data is available, but data acquisition becomes difficult and time-consuming
Solution Approach 1:
The patent synthesizes training data by copying and transforming the compact 3DMM basis shapes through parametric variations. This generates unlimited synthetic facial images with ground truth annotations without requiring actual 3D scanning of real faces, dramatically reducing data acquisition time while providing sufficient training quantity
Solution Approach 2:
The patent performs preliminary construction of a 3DMM from a small set of real 3D scans before synthesis. This pre-established statistical model serves as a foundation for generating all subsequent training data, eliminating the need to collect extensive real facial data for each training scenario
3Device complexity
If face meshes with insufficient vertex counts are used, then processing is simpler, but subtle facial features cannot be accurately captured
Solution Approach 1:
The patent resolves the vertex count limitation by moving from 2D mesh surface representation to 3D volumetric shape representation through the 3DMM framework. This dimensional transformation allows accurate capture of subtle facial features in three-dimensional space without being constrained by the vertex density of 2D face meshes
Solution Approach 2:
The patent changes the representation parameters from fixed mesh vertex coordinates to continuous shape coefficients in the 3DMM space. This parameter transformation enables high-precision feature representation through linear combinations of basis shapes, achieving fine feature capture without increasing mesh complexity
Data Source
AI summary
Exemplary embodiments include a computer-implemented method of training a neural network for facial reconstruction including collecting a set of 3D head scans, combining each feature of each 3D head scan with a weight to create a modified set of 3D head scans, training the neural network using the modified set of head scans, and inputting a real digital facial image into the neural network for facial reconstruction. Further exemplary embodiments include the set of 3D head scans comprising approximately a tenth or less in quantity in comparison to a quantity of the modified set of 3D head scans. The modified set of 3D head scans may comprise features found in the set of 3D head scans or the modified set of 3D head scans may consist of features found in the set of 3D head scans.