3D Facial Model Generation Using DNN Texture Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating 3D facial models from a single 2D image in games and simulations face challenges such as requiring multiple images, producing unsatisfactory results for faces differing from the example set, and failing to capture fine geometric details, while also being limited by inadequate training data and requiring prior knowledge of geometry or illumination.
Innovation Solution
A method involving landmark detection and deep neural network operations to classify facial features into predefined texture components, using a 3D face template and corresponding parameters to generate a 3D facial model, which includes both shape and texture components based on probability calculations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If 3DMM method is used for single image reconstruction, then the method is simple and efficient, but it cannot generate faces with individual characteristics and fails to reproduce fine geometric details
Solution Approach 1:
The patent segments the facial reconstruction process into multiple independent modules: landmark detection identifies key facial points, DNN classification categorizes facial features into texture components, and 3D model generation synthesizes the final result. This segmentation allows each module to specialize in specific tasks, improving both simplicity and precision simultaneously.
Solution Approach 2:
The patent introduces intermediate representations including landmark detection results and DNN classification probabilities as mediators between the input 2D image and the final 3D model. These intermediaries bridge the gap by progressively transforming 2D image data into structured 3D facial geometry with fine details.
2Manufacturing precision
If SFS-based method is used to capture fine-scale facial details, then high quality reconstruction is achieved, but prior knowledge about geometry or illumination is required which limits accuracy when assumptions are not satisfied
Solution Approach 1:
The patent extracts and removes the requirement for prior knowledge about geometry or illumination from the reconstruction process. Instead of assuming known lighting conditions or facial geometry, the method directly processes the input 2D image through landmark detection and DNN classification to generate 3D models without external priors.
Solution Approach 2:
The system performs self-service by automatically adapting to different input images without requiring external calibration or prior knowledge. The DNN model learns from training data and automatically adjusts its classification behavior, making the system self-sufficient and eliminating the need for manual setup or assumptions about illumination conditions.
3Productivity
If CNN-based 3DMM is applied for face reconstruction, then the method is efficient, but inadequate training data results in shallow residual network and limited performance
Solution Approach 1:
The patent performs preliminary actions by pre-defining texture components and their corresponding 3D face meshes before actual reconstruction. The DNN model is pre-trained on comprehensive datasets to learn robust facial feature classifications. This preliminary preparation ensures that when reconstruction is performed, the system has pre-established knowledge structures to reliably map 2D features to 3D geometry without requiring extensive training during deployment.
Data Source
AI summary
A method of generating 3D facial geometry for a computing device is disclosed. The method comprises obtaining a 2D image, performing a deep neural network, DNN, operation on the 2D image, to classify each of facial features of the 2D image as texture components and obtain probabilities that the facial feature belong to the texture components, wherein the texture components are represented by 3D face mesh and are predefined in the computing device, and generating a 3D facial model based on a 3D face template predefined in the computing device and the texture component with the highest probability.