Interactive Fashion Clothing Segmentation for Depth-Sensor-Free AR
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems require depth sensors to modify images, increasing device cost and complexity, and struggle to recognize and apply visual effects to a user's whole body, especially when multiple users are present or at varying distances from the camera.
Innovation Solution
A system that segments articles of clothing or garments using machine learning techniques, allowing for the application of augmented reality elements based on facial expressions without depth sensors, and tracks the position of clothing separately from body parts to enable intuitive interaction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If depth sensors are used to modify images in augmented reality systems, then image modification capability is improved, but device cost and complexity increase
Solution Approach 1:
The patent extracts and removes the depth sensor requirement from the augmented reality system. By using machine learning-based segmentation models that can process standard 2D images, the system eliminates the need for additional depth sensing hardware while maintaining the ability to identify and modify specific regions of interest in images.
Solution Approach 2:
The patent replaces the mechanical/optical depth sensing system with a computational approach using machine learning segmentation models. The system uses trained neural networks to infer depth and object boundaries from 2D images, substituting physical depth sensors with algorithmic processing.
2Adaptability or versatility
If augmented reality systems use traditional image processing, then they can process images, but they struggle to recognize and apply visual effects to the user's whole body, especially when multiple users are present or at varying distances
Solution Approach 1:
The patent applies segmentation technology to divide the image into distinct regions corresponding to different users and their clothing. The machine learning model segments each user's body and clothing separately, enabling independent processing and visual effect application to each user regardless of their position or the presence of other users in the scene.
Solution Approach 2:
The patent applies different processing qualities to different regions of the image. By identifying and segmenting clothing regions versus body regions, the system applies appropriate visual effects and modifications locally to each segment, maintaining image quality and enabling versatile whole-body recognition.
Data Source
AI summary
Methods and systems are disclosed for performing operations comprising: receiving an image that includes a depiction of a person wearing a fashion item; generating a segmentation of the fashion item worn by the person depicted in the image; identifying a facial expression of the user depicted in the image; and in response to identifying the facial expression, applying one or more augmented reality elements to the fashion item worn by the person based on the segmentation of the fashion item worn by the person.