Blendshape Regularization for Facial Animation Retargeting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for creating realistic 3D animations of human faces are limited in accurately modeling complex geometries and subtle expressions, leading to unrealistic animations due to sensitivity in recognizing emotional shifts and noise in tracking data.
Innovation Solution
A computer-implemented method for image retargeting that involves acquiring motion capture data from markers on a subject, calculating blendshapes, and evaluating an error function to determine suitability for retargeting, using a weighted combination of shape primitives based on physiological characteristics, with regularization techniques to minimize error and optimize animation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If performance-driven facial animation with motion capture is used, then realistic facial animation can be achieved, but noise in tracking data and inaccuracies in modeling complex geometries reduce the realism
Solution Approach 1:
The patent introduces blendshapes as an intermediary representation between motion capture data and the final facial animation. Instead of directly applying noisy tracking data to the facial model, the system uses blendshapes (pre-defined facial expressions and geometries) as a mediator to smooth and regularize the animation, thereby reducing the impact of tracking noise while maintaining realism
Solution Approach 2:
The system changes parameters by applying regularization techniques to the motion capture data before retargeting. This involves modifying the tracking data through mathematical regularization processes that filter out noise and inconsistencies, transforming the raw data into a cleaner form suitable for accurate facial animation
2Measurement precision
If a large number of markers are used to improve motion capture accuracy, then tracking precision improves, but device complexity and processing requirements increase
Solution Approach 1:
The patent applies local quality by using a focused set of markers placed at specific anatomical locations on the face rather than covering the entire face uniformly. This localized approach places markers only where they provide maximum information for facial expression capture, reducing the total number of markers needed while maintaining high tracking accuracy for critical facial regions
Solution Approach 2:
The system transforms the marker data through parameter changes by applying regularization and blendshape-based retargeting. This mathematical transformation converts the raw marker positions into smoothed animation parameters, effectively reducing the impact of any individual marker noise and lowering the overall complexity requirements
3Reliability
If animation cleanup with smoothing filters is applied, then noise is reduced, but the process becomes time-consuming and iterative
Solution Approach 1:
The patent implements preliminary action by performing regularization of the motion capture data during the initial retargeting process rather than as a separate cleanup step. The blendshape-based system inherently smooths the animation curves during the transformation from source to target character, eliminating the need for subsequent iterative smoothing operations
Solution Approach 2:
The system applies self-service by having the retargeting process automatically regularize and smooth the animation data through the blendshape calculation. The mathematical formulation of the blendshape retargeting inherently performs the noise-reduction function that would otherwise require separate manual cleanup operations, making the system self-sufficient
Data Source
AI summary
Systems and methods for image retargeting are provided. Image data may be acquired that includes motion capture data indicative of motion of a plurality of markers disposed on a surface of a first subject. Each of the markers may be associated with a respective location on the first subject. A plurality of blendshapes may be calculated for the motion capture data based on a configuration of the markers. An error function may be identified for the plurality of blendshapes, and it may be determined that the plurality of blendshapes can be used to retarget a second subject based on the error function. The plurality of blendshapes may then be applied to a second subject to generate a new animation.