Local Identity-Aware Facial Rig Generation via Patch-Based Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for facial rig generation are inefficient, requiring manual creation of numerous blendshapes and struggling to produce accurate depictions of facial expressions, especially in animation production environments.
Innovation Solution
The technique involves generating a blendshape model with vertices, meshes, and patches, and modifying blendweight values based on facial depictions in a database and sample depictions of a target character, to create an output facial rig model that generates realistic expressions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If blendshape models with dozens or hundreds of blendshapes are used, then facial expression accuracy is improved, but manual creation time and complexity increase significantly
Solution Approach 1:
The face is divided into multiple patches that can be independently optimized. Each patch represents a local facial region that can be adjusted separately, allowing the system to achieve high overall accuracy without requiring a large number of global blendshapes. This segmentation enables localized control over facial deformations.
Solution Approach 2:
Different patches are optimized with different priorities and constraints based on their local characteristics. The system applies local quality principles by allowing certain facial regions to have more flexibility while maintaining stricter control over others, achieving accurate facial expressions without uniformly high complexity across the entire model.
2Manufacturing precision
If nonlinear methods are used for depicting facial expressions, then accuracy is improved, but ease of operation and modifiability deteriorate
Solution Approach 1:
By segmenting the face into patches, the system transforms the complex nonlinear optimization problem into multiple smaller, more manageable local optimization problems. Each patch can be independently adjusted and optimized, making the overall system more intuitive and easier to operate while maintaining high accuracy through the collective effect of all patches.
3Manufacturing precision
If 3D scans or digitally-sculpted 3D depictions are required, then manufacturing precision is improved, but ease of manufacture and accessibility worsen
Solution Approach 1:
The system uses 2D images as copies or projections of the target character's face instead of requiring actual 3D scans. These 2D images are then processed and transformed into a 3D blendshape model, making the process more accessible while maintaining sufficient accuracy for animation purposes.
Solution Approach 2:
2D images serve as an intermediary representation between the target character and the final 3D facial rig. This intermediary format is easier to acquire and work with than direct 3D scans, while still containing sufficient information to generate an accurate facial model through the patch-based optimization process.
4Adaptability or versatility
If a large number of blendshapes are manually created, then facial expression coverage is improved, but productivity decreases
Solution Approach 1:
The patch-based approach segments the facial model into independent regions that can be optimized separately. This segmentation allows the system to achieve comprehensive facial expression coverage by adjusting individual patches rather than relying on a large number of pre-defined blendshapes, significantly improving productivity while maintaining versatility.
Solution Approach 2:
The system automatically optimizes patch weights and parameters based on the input images, eliminating the need for manual creation of numerous blendshapes. This self-service capability maintains comprehensive facial expression coverage while dramatically reducing the time and effort required for facial rig creation.
Data Source
AI summary
The present invention sets forth a technique for performing facial rig generation. The technique includes generating a blendshape model including a plurality of vertices, a plurality of meshes, and a plurality of patches. The technique also includes modifying one or more blendweight values associated with each of the plurality of patches based on a plurality of facial depictions included in a facial database and one or more sample depictions of a target character and generating an output facial rig model based on the blendshape model and the one or more modified blendweight values. The technique further includes generating one or more expressive depictions of the target character based at least on the output facial rig.