Digital Human Facial Expression Mapping via Shape-Texture Hybrid Model
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for transferring facial expressions from performers to virtual humans are labor-intensive, computationally demanding, and result in unrealistic and stiff expressions in VR devices, leading to unsatisfactory user experiences.
Innovation Solution
A method and device for generating digital human facial expressions using a shape and texture model trained with perturbation experiments, combined with active appearance modeling, to create a mapping relationship between facial marks and 3D models, enabling accurate and efficient expression transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If manual modeling and animation refinement are used to transfer facial expressions to virtual humans, then the realism and accuracy of facial expressions are improved, but the time consumption and labor intensity increase significantly
Solution Approach 1:
The patent replaces manual mechanical modeling and animation refinement processes with an automated deep learning-based system. The system uses pre-trained neural networks to automatically transfer facial expressions from performance capture data to virtual human models, eliminating the need for time-consuming manual adjustment while maintaining high expression accuracy and realism.
Solution Approach 2:
The patent creates a digital copy of the facial expression transfer process through deep learning models. The system learns from annotated training data and automatically generates corresponding facial expressions for virtual humans, copying the essence of manual animation refinement into an automated computational process that is both accurate and time-efficient.
2Measurement precision
If deep neural networks are used for training generalized facial expression models, then the accuracy of facial expression tracking is improved, but the computational load and hardware requirements increase significantly
Solution Approach 1:
The patent performs preliminary training of deep neural networks on annotated facial expression data before actual application. By pre-training the models in advance with comprehensive training data, the system achieves high accuracy facial expression tracking without requiring complex hardware during the actual expression transfer process, as the computational heavy lifting is done beforehand.
3Manufacturing precision
If traditional VR devices are used for social and facial expression interaction games, then the basic VR functionality is maintained, but the facial expressions appear stiff and lack realism
Solution Approach 1:
The patent replaces the basic, rigid facial expression systems in traditional VR devices with an advanced deep learning-based expression transfer system. This substitution enables realistic and subtle facial expressions in VR social interaction games, dramatically improving user experience while maintaining the core VR functionality.
Data Source
AI summary
The present disclosure provides a method and device for generating digital human facial expressions and models. The method for generating digital human facial expressions comprises capturing a performer's facial expression video; selecting a plurality of frames or all frames from the facial expression video and fitting each selected frame by using an active appearance model to obtain a plurality of facial marks; calculating values of controllers for driving expressions of 3D model of a digital human, based on the plurality of facial marks obtained from each selected frame and a pre-determined mapping relationship from facial expressions to the 3D model of the digital human. The method of the present disclosure utilizes both the shape information and the statistical analysis for the texture information, constructing a hybrid model that interconnects shape and texture, and thus achieves improved fitting accuracy.