MVAE Character Pose Prediction Without Motion Capture Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies struggle to generate realistic and goal-directed character poses in real-time within three-dimensional virtual environments without relying on stored motion capture data or inefficient memory access, particularly in tasks beyond locomotion.
Innovation Solution
A two-step approach using autoregressive conditional variational autoencoders (MVAEs) for kinematic motion generation, combined with reinforcement learning, to predict future poses by sampling from a latent variable distribution and applying a decoder network, which operates without requiring stored motion capture data during training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If motion capture data is stored and retrieved to generate character poses, then pose generation can be performed, but memory requirements and data storage become problematic
Solution Approach 1:
The patent extracts only the essential latent representation from motion capture data during training, discarding the raw motion capture data itself. The decoder learns to generate poses directly from this compressed latent space, eliminating the need to store and retrieve large amounts of original motion capture data during operation.
Solution Approach 2:
The patent creates a computational model (decoder) that copies the essential pose generation capability from trained data without requiring the original data to be stored. The decoder is trained to map latent variables to poses, creating a self-contained system that can generate poses without accessing the original motion capture database.
2Productivity
If predetermined poses are manually created and stored, then pose retrieval is possible, but real-time generation capability is lost
Solution Approach 1:
The patent performs preliminary training of the decoder model during an offline phase, where the system learns the mapping from latent variables to poses using training data. Once trained, the decoder can generate poses in real-time during gameplay without requiring manual intervention or pre-stored poses, as the learning has already been completed beforehand.
Solution Approach 2:
The patent replaces the manual mechanical process of creating and retrieving predetermined poses with an automated neural network-based system. The decoder uses learned patterns from training data to automatically generate poses in real-time, substituting manual creator intervention with an automated computational model.
3Measurement precision
If motion capture data is used during training, then decoder accuracy improves, but memory requirements increase
Solution Approach 1:
The patent extracts only the essential latent representation from motion capture data during training, discarding the raw motion capture data itself. The decoder learns to generate poses directly from this compressed latent space, eliminating the need to store and retrieve large amounts of original motion capture data during operation.
Data Source
AI summary
Some embodiments herein can include methods and systems for predicting next poses of a character within a virtual gaming environment. The pose prediction system can identify a current pose of a character, generate a gaussian distribution representing a sample of likely poses based on the current pose, and apply the gaussian distribution to the decoder. The decoder can be trained to generate a predicted pose based on a gaussian distribution of likely poses. The system can then render the predicted next pose of the character within the three-dimensional virtual gaming environment. Advantageously, the pose prediction system can apply a decoder that does not include or use input motion capture data that was used to train the decoder.