3D Virtual Character Generation with Virtual Triangles for Facial Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generating 3D virtual characters for video conferencing is time-consuming and does not accurately reflect participants' face shapes and expressions, limiting the number of choices and realism.
Innovation Solution
A 3D virtual character model is created based on a participant's facial image, using 3D Morphable Face Models (3DMM) to animate expressions and poses, with a two-stage deformation process and virtual triangles to ensure accurate facial feature representation, including a face and non-face region deformation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If traditional methods are used to generate 3D virtual characters, then the process is time-consuming and computationally intensive, but the accuracy and realism of facial feature representation is poor
Solution Approach 1:
The virtual character model is segmented into multiple components including face region, non-face region, and virtual triangles that connect these regions. This segmentation allows independent optimization of each region's deformation while maintaining overall coherence, enabling accurate facial feature representation through targeted 3DMM analysis of specific facial areas without requiring complete regeneration of the entire character model.
Solution Approach 2:
The patent introduces virtual triangles as an additional dimensional element between the face region and non-face region. These virtual triangles enable smooth deformation transfer across region boundaries by adding a intermediate geometric dimension, allowing accurate representation of facial features while reducing computation time through efficient deformation propagation rather than exhaustive model regeneration.
2Manufacturing precision
If detailed 3D modeling is performed to accurately reflect participants' face shapes and expressions, then the realism is improved, but the computing resources and time required increase significantly
Solution Approach 1:
The patent applies local quality by focusing 3DMM analysis and deformation computation specifically on the face region and its connecting virtual triangles, while keeping the non-face region relatively simplified. This localized approach maintains high realism for critical facial features while reducing overall computing resource consumption by avoiding detailed modeling of less critical areas.
Solution Approach 2:
Virtual triangles serve as an intermediary structure between the face region and non-face region, enabling efficient deformation transfer. These virtual triangles act as a computational mediator that propagates facial deformations to adjacent regions without requiring full 3D reconstruction of the entire character, thereby reducing computing resources while maintaining facial realism.
3Productivity
If a simple virtual character model is used to reduce computing resources, then the generation time is reduced, but the accuracy of facial feature matching deteriorates
Solution Approach 1:
The patent performs preliminary action by pre-defining the virtual triangle structure and establishing the deformation transfer framework before actual character generation. This preliminary setup enables rapid generation of accurate virtual characters during video conferencing by avoiding time-consuming computations and allowing quick deformation application to pre-configured regional structures.
Solution Approach 2:
The virtual triangle structure enables dynamic deformation propagation that adapts to real-time facial expressions and poses. The system dynamically adjusts the deformation parameters of virtual triangles based on captured facial movements, maintaining high accuracy in facial feature matching while generating characters efficiently through real-time parameter adjustment rather than complete model regeneration.
4Manufacturing precision
If the virtual character model includes comprehensive face and non-face region deformation, then the representation accuracy is improved, but the complexity of the deformation process increases
Solution Approach 1:
The deformation process is segmented into distinct stages: face region deformation using 3DMM, virtual triangle deformation for intermediate transition, and non-face region deformation. This segmentation simplifies the overall complex deformation process by breaking it into manageable steps with specific computational operations for each region, making the comprehensive representation achievable through systematic rather than monolithic processing.
Data Source
AI summary
Systems and methods for generating virtual characters for video conferencing are provided. In an example, a computing device accesses a source human face model, a target human face model, and a source virtual character model. The computing device deforms the source virtual character model based on the source human face model and the target human face model to generate a target virtual character model. The source virtual character model includes a face region and a non-face region. Deforming the source virtual character model includes generating, for the source virtual character model, virtual triangles connecting the face region and the non-face region of the source virtual character model, deforming the source virtual character model with minimized deformation to the virtual triangles to generate the target virtual character model, and removing the virtual triangles from the target virtual character model. The computing device renders the target virtual character model.