3D Face Modeling Using Neural Networks for Single Image Input
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for generating dynamic 3D face models are time-consuming and require expensive setups with multiple cameras, and generic blend shapes often produce similar, artificial face expressions, limiting their usefulness in real-world applications.
Innovation Solution
An electronic device and method using neural networks, specifically a Generative Adversarial Network (GAN), to generate 3D face models from a single 2D color image, allowing for efficient creation of dynamic 3D face models with diverse expressions without the need for multiple cameras or manual image capture from various viewpoints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If manual creation of blend shapes from multiple images is used, then 3D face models can be generated, but the process requires expensive setup with multiple cameras and is time-consuming
Solution Approach 1:
The patent uses a single captured image as a copy or representation of the person's face, and through neural network processing, generates multiple 3D face models with different expressions from this single copy, eliminating the need for multiple cameras and manual image capture from various viewpoints
Solution Approach 2:
The patent replaces the mechanical system of multiple physical cameras and manual image capture with an intelligent system using neural networks that can generate 3D face models and expressions computationally from a single 2D image
2Productivity
If generic blend shapes are used for 3D face modeling, then models can be created quickly, but the face expressions become similar and artificial, reducing their usefulness
Solution Approach 1:
The patent applies local quality by generating expressions that are specific to the individual person being modeled, rather than using generic expressions that are the same for everyone. The neural network learns the unique facial characteristics and generates person-specific expressions that maintain both speed and diversity
Solution Approach 2:
The patent changes the parameters of expression generation by using neural networks to learn and generate diverse facial expressions from a single image, transforming the static input into multiple dynamic expressions with varied parameters such as mouth opening, eye closure, and facial muscle positioning
3Manufacturing precision
If multiple images from different viewpoints are captured manually, then accurate 3D models can be created, but the process is time-consuming and requires expensive equipment
Solution Approach 1:
The patent performs preliminary action by using the neural network to pre-process a single captured image and generate all necessary 3D face models and expressions in advance, eliminating the need for time-consuming manual capture of multiple images from different viewpoints while maintaining accuracy
Data Source
AI summary
An electronic device and method for 3D face modeling based on neural networks is provided. The electronic device receives a two-dimensional (2D) color image of a human face with a first face expression and obtains a first three-dimensional (3D) mesh of the human face with the first face expression based on the received 2D color image. The electronic device generates first texture information and a first set of displacement maps. The electronic device feeds the generated first texture information and the first set of displacement maps as an input to the neural network and receives an output of the neural network for the fed input. Thereafter, the electronic device generates a second 3D mesh of the human face with a second face expression which is different from the first face expression based on the received output.