Clothing Pose Correction for Non-Frontal Virtual Try-On
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Customers shopping online for clothes lack the experience of trying on garments as they would in a physical store, leading to a need for systems and methods to virtually try on clothes and other items.
Innovation Solution
A system utilizing deep-learning blocks and a generative pose transfer model to transform non-frontal images of clothing into frontal images, filling in missing areas with simulated cloth, and tuning parameters using a cloth feature loss function to create a final frontal image for virtual try-on.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If non-frontal images of clothing are used for online shopping, then the variety and accessibility of clothing options are improved, but the ability to accurately visualize how the clothing looks on the customer is degraded
Solution Approach 1:
The system creates a virtual copy of the customer's body using depth map data and generates virtual try-on images that replicate how the clothing would appear on the customer. This copying approach allows accurate visualization from non-frontal product images by transferring the clothing appearance to a virtual model of the customer.
Solution Approach 2:
The system transforms the viewing angle parameter of clothing images by using pose transfer technology to generate frontal views from non-frontal images. It also adjusts parameters such as body shape, size, and pose to match the customer's characteristics, enabling accurate visualization despite the original image being non-frontal.
2Device complexity
If conventional image processing methods are used to transform non-frontal images into frontal images, then the complexity of the system is reduced, but the accuracy and detail of the transformed images are degraded
Solution Approach 1:
The system replaces traditional mechanical image processing methods with deep learning-based neural networks. The pose transfer model and image generation model use artificial intelligence to automatically transform non-frontal images into accurate frontal images, substituting complex mechanical processing with intelligent algorithms that achieve higher precision.
Solution Approach 2:
The system combines multiple technological components including depth map processing, pose transfer models, image generation models, and loss functions to create a composite AI system. This composite approach integrates various processing techniques to achieve accurate image transformation while managing overall system complexity.
3Speed
If simple image transformation techniques are used, then the processing speed is improved, but the quality and realism of the virtual try-on images are degraded
Solution Approach 1:
The system performs preliminary processing by extracting depth map data and body shape information before the actual image transformation. The pose transfer model is pre-trained with large datasets, and the image generation model prepares virtual models in advance. These preliminary actions enable faster real-time processing while maintaining high image quality during the actual virtual try-on.
Solution Approach 2:
The system maintains continuous processing through the pipeline of depth map extraction, pose transfer, and image generation. The models are designed to process images continuously without interruption, ensuring both speed and quality by avoiding repeated preprocessing steps and maintaining optimal processing states throughout the transformation.
Data Source
AI summary
A method can include: obtaining a non-frontal image of an item of clothing from a catalog as a candidate for being transformed into a frontal image; extracting, using multiple deep-learning blocks, pixel data of a cloth point of interest of the non-frontal image; re-aligning, using a generative pose transfer model, the non-frontal image by altering an angle alignment of a non-frontal pose and filling in missing areas with simulated cloth matching the cloth point of interest into the frontal image; and tuning, using a cloth feature loss function, multiple parameters of the frontal image for a final version of the frontal image. Other embodiments are disclosed.