Physics-Guided 3D Motion Generation With Keyframe Validation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing motion generation technologies, whether deep generative models or physics engines, face limitations: deep generative models generate unrealistic motions violating physics laws, while physics engines are computationally expensive and prone to cumulative errors.
Innovation Solution
Combining a deep generative model with a physics engine, using a physics simulator and inverse dynamics network, to downsample motion datasets for keyframes, generate initial motions, and interpolate between frames to ensure physical accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If deep generative models are used for motion generation, then productivity and efficiency are improved, but manufacturing precision and reliability deteriorate due to violations of physical laws
Solution Approach 1:
The patent introduces a physics engine as an intermediary component between the deep generative model and the final motion output. The deep generative model generates candidate motions efficiently, then the physics engine simulates and validates these motions to ensure they obey physical laws. This mediator approach allows the system to maintain high productivity from the generative model while achieving physical accuracy through the physics-based validation layer.
2Manufacturing precision
If physics engines are used for motion generation, then manufacturing precision and reliability are improved, but productivity and use of energy deteriorate due to high computational costs
Solution Approach 1:
The patent applies partial action by using the physics engine selectively rather than for complete motion generation. The deep generative model handles the bulk of motion generation work, and the physics engine is applied partially to validate and refine specific aspects of the generated motions. This partial application of physics-based methods maintains physical accuracy while avoiding the full computational burden of using physics engines for entire motion sequences.
3Reliability
If physics engines operate in autoregressive fashion, then reliability is improved by avoiding cumulative errors, but productivity deteriorates due to sequential generation process
Solution Approach 1:
The patent uses preliminary action by having the deep generative model generate complete motion sequences in advance, before physics-based validation. This allows the system to prepare candidate motions that can then be efficiently validated by the physics engine in parallel or batch mode, rather than processing frame-by-frame sequentially. The preliminary generation of full motion sequences enables faster overall processing while maintaining reliability through subsequent physics-based checking.
Data Source
AI summary
A device may convert a motion dataset to a compatible representation for use in a physics engine, wherein the physics engine includes a physics simulator and inverse dynamics network. The device may further downsample the motion dataset to obtain keyframes for motion generation and forming a downsampled motion dataset. The device may further execute a deep generative model based on the downsampled motion dataset to generate a first generated motion. The device may further execute the physics engine by feeding pairs of consecutive keyframes into the physics simulator and the inverse dynamics network to generate a second generated motion. The device may further combine the first generated motion and the second generated motion to form a combined generated motion, wherein the combined generated motion is generated by executing the physics engine with the first generated motion. The device may further generate a simulated motion video from the combined generated motion.