Facial Animation via Segmented Action Unit Weights
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for generating realistic and believable facial animation using performance data face challenges in creating natural-looking models, as they often result in unnatural expressions and are difficult to modify, especially when trying to adjust specific facial movements independently.
Innovation Solution
The system defines action units, calibrates them using performance data, determines weights for each unit, generates weighted activations, and applies these to a digital facial model, allowing for user adjustments to recalculate and refine the animation, utilizing a Facial Action Coding System (FACS) to manage and retarget facial motion capture data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If principal component analysis is used to generate facial models from performance data, then the model dimensionality is reduced, but the facial expressions become unnatural and the model becomes difficult to modify
Solution Approach 1:
The patent segments the facial model into independent action units (AU) that correspond to specific facial muscle movements. Each AU represents a discrete facial expression component that can be independently controlled and adjusted, replacing the holistic but unnatural principal component approach with granular, interpretable units.
Solution Approach 2:
The patent changes the parameter representation from mathematical principal components to physiologically-based action unit weights. This allows the model to maintain low dimensionality while achieving natural expressions through weights that correspond to real facial muscle activations rather than abstract mathematical components.
2Measurement precision
If principal component analysis is used to generate facial models, then the model is mathematically optimized, but post-development modification becomes difficult and non-intuitive
Solution Approach 1:
By segmenting the facial model into independent action units, the patent enables artists to modify specific facial expressions by adjusting individual AU weights without affecting other components. This segmentation makes the model intuitive to control while preserving mathematical optimization through weighted combinations.
Solution Approach 2:
The patent introduces action units as an intermediary layer between the mathematical performance data and the final facial model. This intermediary allows for intuitive artistic control and modification while maintaining the mathematical rigor needed for realistic animation.
3Ease of operation
If traditional keyframe techniques are used for facial animation, then artistic control is maintained, but the process becomes time-consuming and complex
Solution Approach 1:
The patent enables the system to automatically solve for action unit weights from performance data, reducing the manual keyframing workload. Artists can capture performance data once and have the system automatically generate the corresponding facial animations, significantly reducing development time while maintaining artistic control through the action unit framework.
Solution Approach 2:
The patent changes the workflow from manual keyframe animation to performance-driven parameter extraction. By capturing real performance data and translating it into action unit weights, the system maintains artistic authenticity while dramatically reducing the time required to create realistic facial animations.
4Manufacturing precision
If action units are calibrated using performance data, then realistic facial expressions are achieved, but the system complexity increases
Solution Approach 1:
The patent segments the complex calibration process into individual action unit calibrations, each corresponding to a specific facial muscle group. This segmentation makes the calibration process more manageable and interpretable while achieving realistic expressions through the cumulative effect of calibrated AUs.
Data Source
AI summary
A method of animating a digital facial model, the method including: defining a plurality of action units; calibrating each action unit of the plurality of action units via an actor's performance; capturing first facial pose data; determining a plurality of weights, each weight of the plurality of weights uniquely corresponding to the each action unit, the plurality of weights characterizing a weighted combination of the plurality of action units, the weighted combination approximating the first facial pose data; generating a weighted activation by combining the results of applying the each weight to the each action unit; applying the weighted activation to the digital facial model; and recalibrating at least one action unit of the plurality of action units using input user adjustments to the weighted activation.