Physics-Based Character Control Through Masked Motion Inpainting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating computer-generated character animations lack the ability to create physically plausible motions without manual specification of joint positions and orientations, and machine learning models trained for specific tasks are not easily controllable by animators.
Innovation Solution
A two-stage training technique for a partially-constrained controller using reinforcement learning and supervised imitation learning to generate animations based on sparse goals, such as joint constraints or text descriptions, allowing animators to control character movements more effectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If kinematic modeling is used to compute joint positions and orientations, then the animation can be generated without manual specification of all joints, but the resulting animation is not physically realistic because forces are not considered
Solution Approach 1:
The patent replaces traditional kinematic modeling with a physics-based approach that uses rigid body dynamics simulations. Instead of computing joint positions through mathematical kinematics, the system simulates physical forces, masses, and collisions to determine motion, thereby achieving both automation and physical realism simultaneously
Solution Approach 2:
The system changes the fundamental parameters used for motion generation from kinematic constraints to physical force parameters. By modeling forces, masses, and collision dynamics, the system generates motions that are physically plausible while maintaining automated generation capabilities
2Reliability
If machine learning models are trained for specific tasks (e.g., walking or sitting), then the model can generate physically plausible motions, but the model cannot be used to generate animations where the character performs different motions (e.g., running or climbing stairs)
Solution Approach 1:
The patent creates a universal physics-based character controller that can generate multiple types of motions (walking, running, sitting, climbing) using a single unified physics simulation framework. The system uses goal-conditioned reinforcement learning to adapt the same physical model to different motion tasks by conditioning on task-specific goals, eliminating the need for separate models for each motion type
3Adaptability or versatility
If machine learning models use latent vectors as input, then the model can generate diverse character motions, but the numbers in the latent vector are not easily interpretable by animators who have difficulty selecting specific values corresponding to particular desired motions
Solution Approach 1:
The patent introduces goal-conditioned control as an intermediary layer between the animator's high-level intentions and the low-level physics simulation. Animators specify goals in intuitive terms (e.g., 'character walks to the door'), and the system translates these into physics-based control signals, making the system both versatile and easy to control
Solution Approach 2:
The system segments the control into two independent parts: high-level goal specification (easy to understand and control) and low-level physics simulation (handles the complex latent space). This segmentation allows animators to control the system at a high level without needing to understand or manipulate complex latent vector representations
Data Source
AI summary
One embodiment of a method for animating characters includes receiving one or more goals specified in one or more modalities, generating, via a trained machine learning model and based on the one or more goals, a first action for a character to perform, where the trained machine learning model is trained to process inputs in multiple modalities, and causing the character to perform the first action within a computer-based or physical environment.