Pixel-Based Garment Transfer Using Flow Fields for Realistic AR Try-On
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models for augmented reality (AR) systems struggle to generate realistic images of users wearing fashion items, requiring significant user effort and resources, leading to unrealistic presentations and resource waste.
Innovation Solution
A pixel-based deformation system using machine learning models processes images to generate a flow field indicating pixel likelihood and location, allowing for efficient overlay of fashion items on users, reducing time and expense.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing machine learning models are used for AR garment transfer, then the system can process images, but the output images are unrealistic and require significant user effort and resources
Solution Approach 1:
The patent segments the garment transfer process into distinct pixel-level operations. By processing images at the pixel level and using flow fields to map pixel correspondences between source and target images, the system breaks down the complex garment transfer task into manageable units that can be processed efficiently and realistically
Solution Approach 2:
The patent uses pixel copying from source images to generate target images. By copying and transforming pixels according to learned flow fields rather than generating images from scratch, the system preserves realistic details while reducing computational resource requirements
2Productivity
If existing machine learning models are used for AR garment transfer, then the system can generate images, but the process requires significant user effort and time
Solution Approach 1:
The patent performs preliminary actions by pre-processing source images to extract pixel correspondences and generate flow fields in advance. This preliminary processing enables faster real-time garment transfer operations, reducing the time and user effort required during actual AR try-on operations
Solution Approach 2:
The system uses self-service mechanisms by automatically learning and applying pixel-level transformations without requiring manual user intervention. The neural network models automatically handle image alignment, warping, and blending, eliminating the need for users to manually adjust parameters or perform tedious operations
Data Source
AI summary
Methods and systems are disclosed for using machine learning models to perform pixel-based deformation of fashion items. The methods and systems receive one or more images depicting a person in an individual pose and receive a first source image depicting a first view of a target fashion item and a second source image depicting a second view of the target fashion item. The methods and systems process, using one or more machine learning models, the one or more images that depict the person in the individual pose together with the first and second source images to generate a flow field, the flow field indicating a likelihood of existence and location of each pixel of the one or more images relative to the first and second source images. The methods and systems modify a portion of the one or more images to overlay the target fashion item on the person.