Virtual Eyewear Fitting With AI Occlusion Removal
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional augmented reality systems require users to remove their glasses for accurately displaying virtual eyewear, which can hinder users with vision impairments.
Innovation Solution
An augmented reality system that allows users to keep their glasses on while virtually trying on eyewear by using an AI-powered diffusion model to remove occlusions and render eyewear on a 3D avatar.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional augmented reality systems require users to remove their glasses for accurately displaying virtual eyewear, then the accuracy of virtual eyewear display is improved, but the accessibility for users with vision impairments deteriorates
Solution Approach 1:
The system extracts and removes the occluding object (existing glasses) from the facial region in the input image using AI-powered diffusion models, generating a clean image without occlusions. This allows the virtual try-on system to work with a clear view of the face while the user keeps their corrective glasses on during the actual photo capture.
Solution Approach 2:
The system introduces an intermediary processing step that generates a synthetic clean image as a mediator between the user's actual appearance with glasses and the virtual try-on rendering. This intermediary representation enables accurate virtual eyewear placement without requiring the user to physically remove their corrective glasses.
2Adaptability or versatility
If the system uses AI-powered diffusion models to remove occlusions, then the ability to display virtual eyewear over existing glasses is improved, but the system complexity increases
Solution Approach 1:
The system replaces traditional mechanical or manual methods of achieving clean facial images (such as requiring users to physically remove glasses or use complex multi-camera setups) with AI-powered diffusion models that automatically remove occlusions through computational processing.
Solution Approach 2:
The system changes the approach from capturing physically clean images to generating computationally clean images by transforming the input image through diffusion model processing, altering the fundamental parameter of image acquisition from physical to computational.
3Manufacturing precision
If the system generates clean images by removing occlusions, then the quality of virtual try-on rendering is improved, but the processing time increases
Solution Approach 1:
The system performs the computationally intensive occlusion removal and clean image generation as a preliminary step before the actual virtual try-on rendering. By completing this processing upfront, the system enables faster subsequent rendering operations and provides users with high-quality results.
Solution Approach 2:
The system creates a synthetic copy of the user's face without occlusions through AI generation, rather than requiring the user to physically recreate a glass-free state. This copied clean image can then be used for rendering without requiring repeated processing.
Data Source
AI summary
A computing device obtains a first image depicting a facial region of a user and an object occluding a portion of the facial region of the user. The computing device generates a second image of the user without the object occluding the portion of the facial region of the user based on the first image. A selection comprising desired eyeglasses is obtained, and the computing device generates a third image comprising the desired eyeglasses rendered on the second image to perform virtual try-on of the desired eyeglasses for the user to evaluate.