Collision-Free Garment Deformation Learning for Real-Time Virtual Try-On
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual try-on technologies lack the ability to realistically model garment deformations on diverse body shapes and poses in real-time without requiring post-processing to remove collisions between the garment and body models.
Innovation Solution
A learning-based method that models garment deformations in a collision-free space using a Diffused Human Model (DHM) to extend standard statistical human body models, allowing for the projection of physics-based simulations to a canonical space where garments are collision-free and maintain realistic wrinkles and dynamics, utilizing a generative neural network to predict deformations based on body shape and motion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If physics-based simulation is used to model garment deformation, then realism and accuracy of cloth behavior are improved, but computational cost and runtime performance deteriorate
Solution Approach 1:
The patent pre-computes a collision-free generative space during an offline training phase using physics-based simulations. This pre-computed space encodes realistic garment deformations without collisions, which can then be rapidly applied during runtime without requiring expensive real-time physics calculations, thus resolving the contradiction between accuracy and performance
Solution Approach 2:
The patent replaces the traditional mechanical physics-based simulation system with a data-driven neural network model. The neural network learns the mapping from body pose to garment deformation from pre-computed physics simulations, substituting complex mechanical calculations with efficient neural network inference that maintains realism while achieving real-time performance
2Productivity
If data-driven methods are used to compute cloth deformation, then runtime performance is improved, but realism and accuracy of nonlinear cloth behavior deteriorate
Solution Approach 1:
The patent performs preliminary physics-based simulations to generate training data that captures realistic nonlinear cloth behavior. This pre-computed data is then used to train a neural network that can efficiently reproduce these realistic deformations during runtime without requiring complex real-time physics calculations
Solution Approach 2:
The patent transforms the problem from directly simulating complex nonlinear cloth physics to learning a parameterized mapping from body pose to garment deformation. By changing the approach from mechanical simulation to data-driven parameter prediction, the system achieves both runtime performance and realism
3Manufacturing precision
If conventional physics-based simulation with collision detection is used, then accuracy of garment-body interaction is improved, but computational cost and complexity increase
Solution Approach 1:
The patent extracts and removes the collision detection and resolution components from the simulation system. Instead of computing collisions between garment and body, the method learns a collision-free generative space that directly predicts garment deformation without interpenetration, eliminating the need for complex collision handling while maintaining accuracy
Solution Approach 2:
The patent replaces the mechanical collision detection and resolution system with a data-driven approach. The neural network learns to predict garment deformation that is inherently collision-free by training on physics-based simulation data, substituting complex mechanical interaction calculations with efficient neural network inference
4Manufacturing precision
If post-processing is applied to remove collisions, then realism of garment appearance is improved, but additional computational time and processing steps are required
Solution Approach 1:
The patent performs preliminary action by pre-computing a collision-free generative space during offline training. This pre-computed space ensures that garment deformations are predicted without collisions from the start, eliminating the need for post-processing collision removal and saving computational time while maintaining realism
Data Source
AI summary
Systems and methods are provided that learn garment deformations such that they are essentially collision free. A diffused, volumetric body model representation of the underlying body together with the construction of a subspace for the garment model that yields a differentiable, canonical space configuration. This subspace is used for the regression of the garment model deformation and its dynamics. In this way, garment model deformations are predicted avoiding collisions, and the complexity for inference is reduced, such that a learned representation yields higher quality than previously achievable. The generated garments exhibit a large amount of spatial and temporal detail, and can be produced extremely quickly via the pre-trained networks.