Segmentation-Guided Image Processing for Full-Body Virtual Try-On
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for 3D human body virtual try-on are limited to transferring only small accessories or makeup, require advance modeling of clothing, consume excessive computing resources, and result in unnatural fitting, thus failing to meet user demands for realistic experiences.
Innovation Solution
A method and device for image processing that obtains target and reference segmentation maps, extracts image features, and transfers reference dress-up to the target object without prior modeling, using segmentation encoders and transferrers to achieve full-body 3D virtual try-on, ensuring accurate fitting and environmental condition transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If advance modeling of clothing is required for virtual try-on, then the fitting accuracy is improved, but the device complexity and computing resources increase significantly
Solution Approach 1:
The system performs preliminary segmentation of the target image to obtain the target object segmentation map before the transfer process. This preliminary action prepares the target object region in advance, enabling the subsequent dress-up transfer to be performed more efficiently without requiring complex advance clothing modeling, thus resolving the contradiction between fitting accuracy and device complexity
Solution Approach 2:
The patent introduces segmentation maps (target object segmentation map and reference dress-up segmentation map) as intermediary elements that facilitate the transfer process. These segmentation maps act as mediators between the source and target images, enabling accurate fitting through feature-based transfer rather than complex geometric modeling, thereby reducing device complexity while maintaining fitting accuracy
2Manufacturing precision
If advance modeling of clothing is performed, then the fitting quality is improved, but the computing resources and time consumption increase excessively
Solution Approach 1:
The system extracts only the necessary features from the source and target images through encoding, rather than performing complete advance modeling of clothing. By extracting key features (target image feature and reference image feature) and using segmentation maps, the system achieves fitting quality while significantly reducing computing resources and processing time
Solution Approach 2:
Instead of performing complex advance modeling, the system creates a simplified representation by copying and transferring dress-up features from the reference image to the target object using segmentation maps. This copying approach maintains fitting quality by preserving essential visual characteristics while avoiding the computational burden of complete modeling
3Device complexity
If only small accessories or makeup are transferred, then the device complexity is reduced, but the adaptability and user satisfaction decrease
Solution Approach 1:
The patent implements a universal transfer framework that can handle various types of dress-up items (clothing, accessories, makeup, etc.) through the same segmentation and feature transfer mechanism. The segmentation maps and feature extraction process are general-purpose and can accommodate different categories of transferable items, thereby increasing adaptability without significantly increasing device complexity
Data Source
AI summary
Embodiments of the present disclosure relate to a method, a device, and a computer program product for image processing. The method includes: obtaining a target object segmentation map for a target image and a reference dress-up segmentation map for a reference image, wherein the target image presents a target object, and the reference image presents reference dress-up; extracting a target image feature of the target image and a reference image feature of the reference image by means of image encoding of the target image and the reference image; and transferring the reference dress-up in the reference image to the target object based on the target object segmentation map, the reference dress-up segmentation map, the target image feature, and the reference image feature.