3D Garment Reconstruction With Dense Mesh Gap Filling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for reconstructing 3D garment models from 2D images face challenges such as the inability to handle arbitrary garment styles, loss of texture information, reliance on predefined templates, and the need for manual intervention, leading to inaccurate and costly processes that are not scalable for fast-fashion scenarios.
Innovation Solution
A method and system that reconstructs a 2D image of a garment into a 3D image by generating a dense mesh using a point cloud, splitting 3D points into meshes based on an image grid structure, filling missing regions with additional meshes, and applying Poisson surface reconstruction, while utilizing SMPL for encoding and decoding to predict depth, segmentation, normal, and RGB maps, and employing UV parameterization and inpainting to enhance texture.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If template-based methods are used for garment reconstruction, then the process is simpler and faster, but the system cannot handle arbitrary garment styles and loses texture information
Solution Approach 1:
The method segments the garment reconstruction process into distinct stages: point cloud generation from 2D images, mesh generation from point clouds, and texture mapping. This segmentation allows each stage to be optimized independently, enabling both speed and flexibility. The mesh generation stage uses grid-based sampling to create geometry, while texture mapping preserves appearance details, resolving the contradiction between speed and style adaptability.
Solution Approach 2:
The invention transitions from 2D image input to 3D point cloud representation, then to 3D mesh structure. This dimensional transformation enables the system to capture arbitrary garment styles in three-dimensional space while maintaining the efficiency of 2D image processing. The point cloud serves as an intermediate representation that preserves geometric flexibility before mesh generation.
2Adaptability or versatility
If template-free methods are used for garment reconstruction, then arbitrary garment styles can be modeled, but the process becomes more complex and expensive
Solution Approach 1:
The invention replaces complex manual mesh generation and texture mapping processes with automated algorithms. Point cloud-to-mesh conversion uses computational geometry algorithms to automatically generate mesh structures from point cloud data. Texture mapping uses image processing algorithms to automatically transfer 2D textures onto 3D surfaces, eliminating the need for manual intervention and reducing system complexity while maintaining arbitrary style support.
Solution Approach 2:
The method uses parameterized mesh generation where grid resolution, point sampling density, and texture mapping parameters can be adjusted to control complexity. By optimizing these parameters, the system achieves arbitrary garment style modeling without requiring excessively complex processing, balancing flexibility with computational efficiency.
3Extent of automation
If existing learning-based methods are used, then reconstruction can be automated, but predefined templates restrict application to small sets of garment classes
Solution Approach 1:
The invention creates a universal reconstruction pipeline that can handle any garment style through a single unified process. The point cloud generation, mesh generation, and texture mapping stages form a multi-functional system that adapts to different garment types without requiring style-specific templates or models, achieving both automation and broad garment class coverage.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables accurate and detailed digitization of garments, allowing for virtual try-on experiences and eliminating the need for predefined templates, with improved texture representation and seamless estimation, suitable for arbitrary garment styles.
Implementation Method 1
generating at least one additional mesh for the missing regions by sampling the set of 3D points on the point cloud using a Poisson surface reconstruction method
Data Source
AI summary
A system and method of reconstructing a three-dimensional (3D) model of a garment using an image are provided. The method includes (i) receiving a two-dimensional (2D) image from a user device associated with a user, (ii) representing the garment into a point cloud of the garment by extracting the garment from the 2D image, and (iii) reconstructing a three-dimensional (3D) image of the 2D image by generating a dense mesh of the garment by (a) splitting the set of 3D points into a mesh based on an image grid structure to identify the missing regions in the point cloud, (b) generating an additional mesh for the missing regions by sampling the set of 3D points on the point cloud using a Poisson surface reconstruction method, and (c) generating a dense mesh by combining the mesh with the additional mesh to fill the missing regions in the point cloud of the garment.