Virtual Try-On Generator Using Hierarchical Appearance Flow Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional virtual try-on systems fail to accurately generate photorealistic digital images of a person in complex poses wearing garments that require significant geometric deformation, often resulting in textural artefacts and missing or additional body parts due to limited ability to predict three-dimensional geometries from two-dimensional images.
Innovation Solution
A generator system that computes candidate appearance flow maps using convolutional networks to warp garments based on poses, combined with a convolutional gated recurrent network to prevent over-warping, and predicts conditional segmentation masks to accurately depict the person wearing the garment, incorporating prior data for structural and geometric integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If conventional systems warp garments based on pose using simple deformation models, then the processing speed is fast, but the manufacturing precision deteriorates due to textural artifacts and incorrect body part geometry
Solution Approach 1:
The system segments the garment deformation process into multiple hierarchical levels using pixel-block sizes (e.g., 16x16, 32x32, 64x64 pixels). Each level processes displacement at a different resolution, allowing coarse adjustments at lower resolutions and fine adjustments at higher resolutions, thereby improving manufacturing precision without overwhelming computational complexity
Solution Approach 2:
The system applies local quality by computing displacement maps for different pixel-block sizes, where each block size processes specific spatial frequencies. This allows different regions of the garment to be deformed with appropriate levels of detail, improving local deformation accuracy while maintaining overall system efficiency
2Measurement precision
If conventional systems use basic warping techniques, then the device complexity is low, but the measurement precision deteriorates due to inability to accurately predict three-dimensional geometries from two-dimensional images
Solution Approach 1:
The system introduces an intermediary conditional segmentation mask that bridges the gap between 2D input images and 3D geometry prediction. This mask provides structural guidance to the warping process, enabling accurate body part geometry prediction by segmenting the image into meaningful regions before applying deformation
Solution Approach 2:
The system performs preliminary action by generating the conditional segmentation mask before the warping process. This pre-computed structural information guides subsequent deformation steps, ensuring that body parts are correctly identified and positioned before garment warping is applied
3Adaptability or versatility
If conventional systems apply aggressive warping to handle complex poses, then the adaptability to different poses is improved, but object-generated harmful factors worsen due to textural artifacts from over-deformation
Solution Approach 1:
The system segments the warping process into hierarchical levels with different pixel-block sizes. This segmentation prevents over-deformation by applying warping at appropriate scales, where coarse levels handle major pose changes and fine levels preserve local texture details, thereby reducing textural artifacts while maintaining complex pose adaptability
Solution Approach 2:
The system applies partial warping at each hierarchical level rather than full warping at a single level. By distributing the deformation across multiple partial steps with increasing resolution, the system achieves the necessary adaptability for complex poses while avoiding the excessive deformation that causes textural artifacts
Data Source
AI summary
In implementations of systems for generating images for virtual try-on and pose transfer, a computing device implements a generator system to receive input data describing a first digital image that depicts a person in a pose and a second digital image that depicts a garment. Candidate appearance flow maps are computed that warp the garment based on the pose at different pixel-block sizes using a first machine learning model. The generator system generates a warped garment image by combining the candidate appearance flow maps as an aggregate per-pixel displacement map using a convolutional gated recurrent network. A conditional segment mask is predicted that segments portions of a geometry of the person using a second machine learning model. The generator system outputs a digital image that depicts the person in the pose wearing the garment based on the warped garment image and the conditional segmentation mask using a third machine learning model.