Virtual Shoe Try-On with Foot Movement Smoothing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge of purchasing shoes online is that users cannot test the fit before buying, leading to difficulties in finding shoes that fit properly, resulting in potential dissatisfaction and increased returns.
Innovation Solution
A virtual shoe try-on feature using machine learning to predict how a shoe will fit on a user's foot by generating and superimposing 2D models of the shoe onto a video of the foot, with a smoothing process to ensure consistent predictions across frames, improving the accuracy of the simulation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a virtual shoe try-on system is implemented, then user experience and fit accuracy are improved, but computational complexity and processing time increase
Solution Approach 1:
The system performs preliminary actions by pre-processing video frames to extract foot contours and keypoint locations before the actual shoe rendering. Bounding boxes are predicted and stored in advance, and the affine transformation is pre-computed from these bounding boxes, allowing the shoe model to be rapidly rendered and superimposed without real-time computational burden during the try-on display phase
Solution Approach 2:
The system creates a simplified 2D representation (copy) of the foot from the video frames by detecting contours and keypoints. This 2D foot model is then used as a template to which shoe models are superimposed, avoiding the need for complex 3D foot scanning and modeling while achieving sufficient accuracy for virtual try-on purposes
2Productivity
If independent predictions are generated for each video frame, then processing speed is improved, but prediction consistency and simulation accuracy deteriorate
Solution Approach 1:
The system applies feedback by using the predicted bounding box from one frame to influence the prediction in the next frame. The affine transformation computed from the bounding box provides a reference frame that helps stabilize subsequent predictions, ensuring that the foot representation remains consistent across frames while still allowing for natural movement and position changes
Data Source
AI summary
According to an embodiment, a method includes estimating, based at least in part on a first location of a bounding box for a foot in a first frame of a sequence of frames and a second location of the bounding box in a second frame of the sequence of frames, a smoothing transformation that characterizes a movement of the foot in the sequence of frames and smoothing, based at least in part on the smoothing transformation, a movement of a structure of the foot from a first location of the structure in the first frame to a second location of the structure in the second frame. The method also includes superimposing a model of a shoe onto the foot in the sequence of frames based at least in part on the smoothed movement.