Virtual Clothing Try-On Using Pose-Guided Garment Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training generative AI networks for virtual clothing try-ons is challenging due to the difficulty in obtaining ground truth images of the same person wearing different garments, particularly in the same pose.
Innovation Solution
The training process utilizes pose information, inpaint areas, and text prompts to generate images of a person wearing a target garment without requiring paired input-output images, preserving garment details and adjusting the garment to match the person's pose and characteristics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional training methods using paired input-output images are used, then the neural network can be trained with ground truth data, but it becomes extremely difficult to obtain paired images of the same person wearing different garments in the same pose
Solution Approach 1:
The patent uses pose estimation to create a digital copy/template of the person's body shape and posture from the input image. This template is then used to guide the generative model in synthesizing the target garment on the person, effectively copying the structural information without requiring actual paired photographs
Solution Approach 2:
The patent introduces pose information and body templates as intermediary elements between the input image and the generated output. These intermediaries bridge the gap by providing structural guidance without requiring direct paired training data, thus solving the data collection difficulty while maintaining training reliability
2Ease of manufacture
If generative models are trained without paired ground truth images, then data collection becomes easier, but it becomes difficult to ensure accurate pose matching and garment detail preservation
Solution Approach 1:
The patent applies local quality by focusing the generative model's attention on specific regions of the image where garment details should be preserved.通过使用注意力机制和局部区域生成，确保关键部位（如衣物褶皱、图案、纹理）的高精度还原，而不是平均分配计算资源到整个图像
Solution Approach 2:
The patent replaces the mechanical system of actual garment try-on photography with a computational image generation system. Instead of physically dressing a person and taking photographs (mechanical process), the system uses neural networks to synthesize the garment appearance digitally, achieving comparable or superior results without physical constraints
3Reliability
If more reference images are used for training, then the model can learn better garment details and pose variations, but the training complexity and computational resources increase
Solution Approach 1:
The patent creates a universal body template that can be applied across different garments, poses, and clothing styles. This single template structure serves multiple functions: it provides pose guidance, body shape adaptation, and garment positioning for various try-on scenarios, eliminating the need for separate training models for different conditions
Data Source
AI summary
An artificial intelligence (AI) network or neural network is trained, using a relatively small number of reference images of a target garment, to enable virtual clothing try-ons of the target garment. Example methods include determining a pose for a person depicted in an input image, determining an area of the input image to replace with a target garment, changing values of pixels within the area, and inputting the pose, the area, and a text prompt describing the target garment, into a neural network, to generate an output image, wherein the neural network is trained to generate the target garment. Example methods include training the neural network with images of clothing in a same class or category as the target garment to teach the neural network to shape the target garment in accordance with a pose of the person and to preserve other clothing and the background.