Virtual Try-On Neural Network Framework Using Coarse-to-Fine Warping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional digital image systems for virtual try-on experiences face issues with accuracy, efficiency, and flexibility due to inaccuracies in generating digital images, resource-intensive processing, and reliance on three-dimensional information, which limits their application and increases costs.
Innovation Solution
A two-stage neural network framework is employed for accurate virtual try-on image generation, utilizing a coarse-to-fine warping process and texture transfer to align and modify product images with model images, reducing the need for three-dimensional data and enhancing processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If conventional digital image systems use three-dimensional information to render virtual try-on images, then the realism and accuracy of garment depiction is improved, but the computational resource requirements and processing time increase significantly
Solution Approach 1:
The patent uses two-dimensional image copying and transformation techniques instead of three-dimensional rendering. The source garment image is copied and warped to match the target pose while preserving texture details through image processing operations rather than computationally intensive 3D rendering
Solution Approach 2:
The patent replaces the mechanical 3D rendering system with a neural network-based image processing system. The neural network learns to directly transform 2D garment images to match target poses, substituting the complex mechanical 3D model processing with a more efficient learned transformation
2Manufacturing precision
If conventional systems require three-dimensional information for virtual try-on, then the accuracy of garment fitting is improved, but the flexibility and ease of application are reduced due to difficulty in obtaining 3D data
Solution Approach 1:
The system copies and transforms two-dimensional garment images instead of requiring three-dimensional models. This copying approach allows any garment with a 2D image to be used in virtual try-on, greatly increasing system flexibility and adaptability
Solution Approach 2:
The patent changes the fundamental parameter from requiring 3D information to working with 2D images. This parameter change enables the system to process any garment that has a 2D representation, significantly expanding the range of applicable garments and models
3Quantity of substance
If conventional systems synthesize entire digital images to depict garment textures, then the completeness of the virtual try-on image is improved, but the accuracy of texture boundaries is reduced causing blurring and bleeding
Solution Approach 1:
The patent segments the garment image processing into distinct components: pose estimation, image warping, and texture transfer. This segmentation allows precise control over texture boundaries during the warping process, preventing blurring and bleeding while maintaining complete garment depiction
Solution Approach 2:
The system performs preliminary pose estimation and image warping before final texture transfer. This preliminary action prepares the source garment image with correct geometric transformation, ensuring that texture boundaries are properly aligned before the final composition, thus preventing boundary artifacts
Data Source
AI summary
The present disclosure relates to systems, methods, and non-transitory computer readable media for generating a virtual try-on digital image utilizing a unified neural network framework. For example, the disclosed systems can utilize a coarse-to-fine warping process to generate a warped version of a product digital image to fit a model digital image. In addition, the disclosed systems can utilize a texture transfer process to generate a corrected segmentation mask indicating portions of a model digital image to replace with a warped product digital image. The disclosed systems can further generate a virtual try-on digital image based on a warped product digital image, a model digital image, and a corrected segmentation mask. In some embodiments, the disclosed systems can train one or more neural networks to generate accurate outputs for various stages of generating a virtual try-on digital image.