3D Avatar Fashion Transfer Without Depth Sensors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AR systems require depth sensors to modify images, increasing device cost and complexity, and struggle to recognize and apply visual effects to whole bodies or fashion items, leading to inefficient processing and image quality issues, especially on mobile devices.
Innovation Solution
Utilizing machine learning models to extract fashion items from real-world images and apply them to 3D avatar models without generating rigs, reducing processing complexity and enabling seamless AR graphics addition on mobile devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If depth sensors are used to modify images in AR systems, then image modification capability is improved, but device cost and complexity increase
Solution Approach 1:
The patent extracts and removes the depth sensor component from the AR system while retaining the image modification capability through alternative means (machine learning-based fashion item detection and application). This eliminates the need for complex depth sensing hardware while achieving the desired visual effects.
Solution Approach 2:
The patent replaces the mechanical/optical depth sensing system with a computational approach using machine learning models. Instead of using physical depth sensors to capture and process depth information, the system uses 2D image analysis with AI algorithms to detect fashion items and apply visual effects, substituting hardware-based depth measurement with software-based image processing.
2Reliability
If depth sensors are used to modify images, then visual effect application is improved, but processing complexity increases
Solution Approach 1:
The patent replaces complex depth-based processing pipelines with simplified machine learning-based 2D image analysis. The system uses trained neural networks to directly detect fashion items from standard 2D images and apply corresponding visual effects, eliminating the need for complex depth map generation, 3D reconstruction, and multi-step processing required by depth sensor systems.
Solution Approach 2:
The patent creates a simplified computational model that copies the essential functionality of depth-based systems through 2D image analysis. By training machine learning models on 2D fashion item images, the system replicates the visual effect application capability without requiring the complex depth sensing and processing infrastructure.
3Reliability
If depth sensors and complex processing are used, then AR visual effects are achieved, but resource requirements increase making it unsuitable for mobile devices
Solution Approach 1:
The patent replaces energy-intensive depth sensing hardware and complex processing pipelines with lightweight machine learning models optimized for mobile devices. The system uses efficient 2D image analysis algorithms that can run on mobile processors without requiring dedicated depth sensor hardware, significantly reducing power consumption and computational resource requirements.
Solution Approach 2:
The patent employs lightweight, optimized machine learning models that can be deployed on mobile devices with limited resources. These models are designed to be computationally efficient and energy-saving, enabling AR visual effects on mobile platforms without requiring expensive, resource-intensive hardware configurations.
Data Source
AI summary
Methods and systems are disclosed for transferring garments from a real-world object to a virtual object. The system receives, by a client device, an image that includes a depiction of a real-world object having a fashion item in a real-world environment. The system accesses a three-dimensional (3D) avatar model of a human and generates a graphic item corresponding to the fashion item being worn by the real-world object depicted in the image. The system modifies the 3D avatar model of the human based on the graphic item and presents the 3D avatar model that has been modified based on the graphic item within a view of the real-world environment on the client device.