Facial Animation Using 2D Depth Maps and Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for creating and animating digital actors are complex, costly, and prone to failure due to noise, requiring long turn-around times and substantial manual post-processing, lacking robustness in capturing human geometry, appearance, and motion.
Innovation Solution
A method that uses 2D images and 3D depth maps to determine expression parameters for a user-specific model, combined with animation priors to animate a digital character, decoupling rigid and non-rigid motion, and employing optimization calculations to generate animation parameters that mimic the user's facial expressions, using blendshapes and blendshape weights for representation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If 3D scanning and motion capture are used to create and animate digital actors, then the quality of digital actor creation is improved, but the device complexity and production costs increase substantially
Solution Approach 1:
The patent uses 2D images and depth maps as simplified copies of the user's face instead of requiring full 3D scanning. These 2D representations are processed to extract expression parameters that drive the digital character's animation, eliminating the need for complex 3D acquisition hardware while maintaining animation quality
Solution Approach 2:
The patent extracts only the essential expression parameters from 2D images and depth maps rather than capturing complete 3D geometry and motion data. By taking out only the necessary facial expression information, the system avoids the complexity of full 3D scanning and motion capture systems
2Reliability
If existing animation methods are used, then digital actors can be animated, but the methods lack robustness and fail in the presence of noise
Solution Approach 1:
The patent employs an optimization process that iteratively adjusts expression parameters to best match the input 2D images and depth maps. This feedback mechanism continuously refines the animation parameters, making the system robust against noise and variations in input quality
Solution Approach 2:
The patent pre-processes the 2D images and depth maps to extract expression parameters before animation. By preparing and validating the input data in advance through optimization calculations, the system establishes a robust foundation that prevents noise-related failures during the animation process
3Manufacturing precision
If complex acquisition systems and manual post-processing are used, then digital actors can be created, but the turn-around time and production costs increase
Solution Approach 1:
The patent enables the system to automatically process 2D images and depth maps to generate animation parameters without requiring manual post-processing. The optimization process self-adjusts the expression parameters, eliminating the need for time-consuming manual intervention while maintaining high quality results
Solution Approach 2:
The patent replaces manual post-processing operations with automated optimization calculations. By substituting mechanical/manual processes with computational algorithms, the system dramatically reduces turn-around time while preserving animation quality
Data Source
AI summary
A method of animating a digital character according to facial expressions of a user, comprising the steps of, (a) obtaining a 2D image and 3D depth map of the face of the user, (b) determining expression parameters for a user expression model so that a facial expression of the user-specific expression model represents the face of the user shown in the 2D image and 3D depth map (c) using the expression parameters and an animation prior to determine animation parameters usable to animate a digital character, wherein the animation prior is a sequence of animation parameters which represent predefined animations of a digital character (d) using the animation parameters to animate a digital character so that the digital character mimics the face of the user.