3D Clothing Point-Cloud Draping for Realistic Virtual Try-On
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods struggle to model realistic clothing geometry and appearance on humans due to wide variations in geometry and appearance, as well as the complexity of clothing interaction with the body, especially for loose clothing.
Innovation Solution
A method using point clouds and neural rendering to model clothing geometry and appearance, where a draping network is trained with video sequences to predict point clouds and outfit codes, enabling adaptation to body poses and shapes, and a system for real-time outfit retargeting on users.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional mesh-based or volumetric methods are used for clothing simulation, then fabric continuity and coverage are maintained, but computational complexity and processing time increase significantly
Solution Approach 1:
The clothing garment is segmented into multiple panels that are independently represented as 2D quadrilateral surfaces. Each panel can be simulated and rendered separately, reducing the overall computational complexity compared to mesh-based approaches while maintaining visual continuity through proper panel接缝 handling.
Solution Approach 2:
The patent transitions from 3D mesh/volumetric representations to 2D quadrilateral panel representations. This dimensional reduction simplifies the computational model while maintaining the essential visual and physical properties of the clothing garment through point-based rendering techniques.
2Measurement precision
If detailed mesh models are used to represent clothing, then visual accuracy and detail are improved, but memory usage and processing requirements increase
Solution Approach 1:
The patent uses point-based representations that can be rendered to create visual copies of the clothing surface. Multiple points can represent complex surface geometry without requiring actual mesh data, reducing memory usage while maintaining visual fidelity through point distribution and rendering techniques.
Solution Approach 2:
By representing 3D clothing surfaces as 2D quadrilateral panels with distributed points, the patent reduces the data structure complexity and memory requirements while preserving visual accuracy through appropriate point density and rendering methods.
3Reliability
If physics-based simulation is applied to entire clothing garments, then movement realism is improved, but computational cost and processing time increase
Solution Approach 1:
The clothing garment is divided into multiple independent panels that can be simulated separately. This segmentation allows physics-based simulation to be applied to individual panels with fewer computational requirements, while maintaining overall movement realism through proper panel connection and interaction modeling.
Solution Approach 2:
Instead of applying full physics simulation to the entire garment, the patent applies physics-based simulation selectively to key panels or regions that most influence movement realism. This partial application reduces computational cost while maintaining acceptable movement fidelity for the overall garment.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
Provided are virtual try-on applications, telepresence applications, relating to modeling realistic clothing worn by humans and realistic modeling of humans in three-dimension (3D). Proposed is a hardware comprising software products that perform method for imaging clothes on a person, that is adapted to the body pose and the body shape, based on point cloud draping model, the method including using of point cloud and a neural network that synthesizes such point clouds to capture/model the geometry of clothing outfits, and using of point based differentiable neural rendering to capture the appearance of clothing outfits.