Facial Secondary Dynamics Prediction via Machine Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer animation techniques struggle to accurately remove secondary dynamics from facial expressions, leading to unrealistic representations due to the inaccuracy of kinetic models used to simulate human facial movements.
Innovation Solution
A data-driven approach using a modeling engine that generates a prediction model to quantify and predict secondary dynamics by analyzing geometric representations of facial expressions under different loading conditions, allowing for the reduction or elimination of secondary dynamics in computer animations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If kinetic models are used to simulate secondary dynamics in facial expressions, then secondary dynamics can be removed from computer animations, but the accuracy of the simulation is insufficient due to the complexity of facial tissues
Solution Approach 1:
The patent replaces traditional kinetic models with a data-driven machine learning approach. Instead of using complex mechanical simulations of facial tissues, the system captures real facial performance data and uses machine learning models to predict and remove secondary dynamics, achieving higher accuracy without the computational complexity of kinetic modeling.
Solution Approach 2:
The patent creates digital copies of real facial expressions through motion capture and geometric modeling. By capturing actual human facial performance and creating accurate geometric models, the system copies real facial behavior patterns, which are then used to train machine learning models that can predict and remove secondary dynamics more accurately than kinetic models.
2Ease of manufacture
If traditional motion capture and rendering techniques are used, then computer animations can be generated, but secondary dynamics disrupt the realism of facial expressions
Solution Approach 1:
The patent replaces traditional kinetic modeling approaches with machine learning-based prediction. The system uses captured performance data to train models that predict secondary dynamics, then removes these predictions from the animation. This approach maintains ease of animation generation while significantly improving realism by accurately capturing and removing unwanted secondary movements.
Solution Approach 2:
The patent implements a feedback mechanism where the machine learning model continuously predicts secondary dynamics based on captured performance data and adjusts the animation accordingly. The system captures real facial movements, predicts secondary dynamics, removes them, and refines the prediction model based on the difference between captured and corrected animations, improving realism iteratively.
Data Source
AI summary
A modeling engine generates a prediction model that quantifies and predicts secondary dynamics associated with the face of a performer enacting a performance. The modeling engine generates a set of geometric representations that represents the face of the performer enacting different facial expressions under a range of loading conditions. For a given facial expression and specific loading condition, the modeling engine trains a Machine Learning model to predict how soft tissue regions of the face of the performer change in response to external forces applied to the performer during the performance. The modeling engine combines different expression models associated with different facial expressions to generate a prediction model. The prediction model can be used to predict and remove secondary dynamics from a given geometric representation of a performance or to generate and add secondary dynamics to a given geometric representation of a performance.