3D Virtual Character Deformation Transfer for Facial Feature Matching

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Generating 3D virtual characters for video conferencing is time-consuming and does not accurately reflect participants' face shapes and expressions, limiting the number of choices and realism.

Innovation Solution

A 3D virtual character generation system that uses 3D Morphable Face Models (3DMM) to create virtual characters based on participant facial images, applying deformation transfer and facial feature enhancement to match expressions and poses, with a two-stage deformation process to minimize artifacts and preserve desired features.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If traditional manual methods are used to create 3D virtual characters, then customization and realism can be achieved, but the process is time-consuming and reduces productivity

Engineering Contradiction:

Improvevirtual character realismVSAvoidgeneration speed

Core Design Contradiction:

Manufacturing precisionVSProductivity

Solution Approach 1:

The patent uses 3D Morphable Face Models (3DMM) to create a standardized base model that can be digitally replicated and deformed to match different participants' faces. This allows rapid generation of customized virtual characters without manual creation for each participant, significantly improving productivity while maintaining realism through accurate facial feature mapping

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The system applies deformation transfer by adjusting parameters of the base 3DMM model to transform it into different virtual character variations. By changing deformation parameters based on participant facial data, the system efficiently generates customized characters without recalculating the entire model from scratch, resolving the contradiction between customization precision and generation speed

Inventive Principle:

Principle #35Parameter changes

2Productivity

If 3D virtual characters are generated without accurate facial feature matching, then generation speed can be maintained, but the characters do not accurately reflect participants' face shapes and expressions

Engineering Contradiction:

Improvegeneration speedVSAvoidfacial feature accuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The patent replaces manual facial modeling with automated computer vision and machine learning algorithms that analyze participant images to extract facial features. This substitution enables rapid, accurate extraction of facial geometry and expressions without manual intervention, simultaneously achieving high generation speed and precise facial feature matching

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system introduces an intermediary deformation transfer mechanism that bridges the base 3DMM model and the target participant's facial features. This intermediary layer processes facial data and applies appropriate deformations to the base model, ensuring accurate facial feature matching while maintaining efficient generation through pre-computed deformation fields

Inventive Principle:

Principle #24Intermediary (Mediator)

3Productivity

If a single-stage deformation process is used, then the process is simpler and faster, but it generates artifacts and reduces the quality of virtual characters

Engineering Contradiction:

Improveprocessing efficiencyVSAvoidvirtual character quality

Core Design Contradiction:

ProductivityVSManufacturing precision

Solution Approach 1:

The patent divides the deformation process into multiple stages: first extracting facial features from participant images, then applying deformation transfer to the base 3DMM model, and finally refining the result through additional processing steps. This segmentation allows each stage to focus on specific aspects of quality improvement, reducing artifacts while maintaining overall processing efficiency

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system performs preliminary actions by pre-computing deformation fields and facial feature mappings before generating the final virtual character. This preparation enables the main deformation process to proceed more efficiently with fewer artifacts, as the heavy computational lifting is done in advance on simplified representations

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS20250252691A13D digital virtual character generation with facial feature enhancement

Publication Date: 2025.08.07 ZOOM VIDEO COMM INC

AI summary

Systems and methods for generating virtual characters for video conferencing with feature enhancement are provided. In an example, a computing device access a source human face model, a target human face model, and a source virtual character face model. The device further accesses a first virtual feature triangle marked on the source human face model and a second virtual feature triangle marked on the source virtual character face model. The second virtual feature triangle corresponds to the first virtual feature triangle. The device deforms the source virtual character face model based on the source human face model and the target human face model to generate a target virtual character face model. The deforming includes minimizing a loss function comprising a term defined based on a difference between the second virtual feature triangle and the first virtual feature triangle. The device renders the target virtual character face model.