Body Landmark Virtual Try-On Without Depth Sensors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AR systems require depth sensors to modify images, increasing device cost and complexity, and struggle to recognize and apply visual effects to a user's whole body, especially at varying distances and with multiple users, while also failing to account for physical properties like density and weight.
Innovation Solution
Utilizing multiple machine learning models to extract body landmark features and apply visual effects in real-time without generating a rig, allowing seamless addition of AR graphics on mobile devices by computing deviations between body landmarks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If depth sensors are used to modify images in AR systems, then image modification capability is improved, but device cost and complexity increase
Solution Approach 1:
The patent extracts the essential function of depth sensing by identifying and utilizing key body landmark points (shoulders, elbows, wrists, hips, knees, ankles) from standard camera images. This allows the system to achieve depth-aware AR effects without physically extracting or requiring depth sensors, thereby maintaining image modification capability while reducing device complexity
Solution Approach 2:
The patent replaces the mechanical/optical depth sensing system with a computational approach using machine learning models and pose estimation algorithms. Instead of using physical depth sensors to capture depth information, the system substitutes this with 2D image processing and mathematical modeling to infer 3D body positions and apply AR effects
2Measurement precision
If depth sensors are used to recognize whole body, then body recognition accuracy is improved, but processing complexity increases
Solution Approach 1:
The patent segments the whole body into distinct anatomical regions and landmark points (head, shoulders, arms, torso, legs, feet). By dividing the complex task of whole-body recognition into smaller, manageable segments with specific key points, the system achieves accurate body recognition using standard cameras while reducing the computational burden compared to processing entire depth maps
3Manufacturing precision
If rigid body transformation is used to adjust AR graphics, then adjustment precision is improved, but adaptability to physical properties decreases
Solution Approach 1:
The patent extends rigid body transformation by introducing additional parameters related to physical properties such as density and weight. This allows the AR graphics to be adjusted not only based on geometric transformations (position, orientation) but also to account for physical characteristics, thereby maintaining precision while increasing adaptability to different objects and conditions
Data Source
AI summary
Methods and systems are disclosed for transferring garments from one real-world object to another in real time using body landmarks. The system receives a first image that includes a depiction of a first person wearing a fashion item in a first pose. The system obtains a second image that includes a depiction of a second person in a second pose and generates a first set of body landmarks corresponding the first person in the first pose and a second set of body landmarks corresponding the second person wearing in the first pose. The system computes a deviation between the first set of body landmarks and the second set of body landmarks. The system generates a new image that depicts the second person wearing the fashion item worn by the first person based on the deviation between the first set of body landmarks and the second set of body landmarks.