Masked Motion Inpainting for Physically Plausible Character Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating computer-generated character animations lack the ability to create physically plausible motions without specifying all joint positions and orientations, and machine learning models trained for specific tasks cannot be easily controlled by animators to produce diverse animations.
Innovation Solution
A two-stage training technique is employed to develop a partially-constrained controller using reinforcement learning and supervised imitation learning, allowing characters to perform actions based on sparse goals specified in various modalities, such as joint constraints or text descriptions, by generating actions through a trained machine learning model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If kinematic modeling is used to compute joint positions and orientations between key frames, then the animation can be generated without specifying all joint positions, but the resulting animation is not physically realistic because forces are not considered
Solution Approach 1:
The patent replaces traditional kinematic modeling with a physics-based neural network system. Instead of using mechanical kinematics to compute joint positions, the system uses a neural network trained on physics simulations to predict joint positions and orientations based on forces, masses, and other physical parameters, thereby achieving physical realism while maintaining ease of use.
Solution Approach 2:
The patent changes the fundamental parameters used for animation generation from kinematic constraints to physical forces and masses. The neural network takes force vectors, mass values, and other physical parameters as input to compute joint positions and orientations, transforming the animation generation process from a kinematic problem to a physics-based problem.
2Reliability
If machine learning models are trained for specific tasks (e.g., walking or sitting), then the model can generate physically plausible motions for that task, but the model cannot be used to generate animations where the character performs different motions (e.g., running or climbing stairs)
Solution Approach 1:
The patent creates a universal neural network model that can generate physically plausible animations for multiple different motions and tasks. The model is trained to handle various motions such as walking, running, sitting, and climbing stairs using a unified architecture that processes force vectors, mass values, and other physical parameters, allowing it to adapt to different motion types without requiring task-specific models.
Solution Approach 2:
The patent implements a dynamic neural network that can adapt its behavior based on the input physical parameters and desired motion type. The model dynamically adjusts its predictions based on the specific task requirements, allowing it to generate appropriate animations for different motions while maintaining physical plausibility across all tasks.
3Adaptability or versatility
If a machine learning model receives a latent vector of numbers as input to output character joint positions and orientations, then the model can generate diverse motions not limited to specific tasks, but the numbers in the latent vector are not easily interpretable by animators who have difficulty selecting specific values
Solution Approach 1:
The patent introduces an intermediary interface between the animator and the neural network model. Instead of requiring animators to directly manipulate abstract latent vectors, the system provides a user-friendly interface that allows animators to specify high-level motion goals or constraints, and the neural network automatically translates these into appropriate joint positions and orientations, making the system both versatile and easy to control.
Data Source
AI summary
One embodiment of a method for animating characters includes receiving one or more goals specified in one or more modalities, generating, via a trained machine learning model and based on the one or more goals, a first action for a character to perform, where the trained machine learning model is trained to process inputs in multiple modalities, and causing the character to perform the first action within a computer-based or physical environment.