AR Object Concealment for Realistic Virtual Try-On
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for trying on new glasses, especially for individuals wearing corrective glasses, are impractical due to difficulties in assessing frame suitability and visibility issues caused by tinted lenses, and existing augmented reality systems cannot realistically remove real objects like glasses from images or videos.
Innovation Solution
A method for generating images that detects and conceals real objects, such as glasses, by creating a mask that modifies the object's appearance and texture, allowing for realistic virtual fitting without the need for the object to be physically removed, enabling users to see themselves on a screen as if the object were not present, with the option to change colors and textures, and allowing for real-time interaction and movement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Illumination intensity
If traditional mirror-based trying on method is used, then the user can see their reflection, but users wearing corrective lenses cannot see themselves clearly due to visual impairment and tinted lenses
Solution Approach 1:
The system creates a digital copy (photograph) of the user wearing the glasses and processes this copy to remove the glasses visually, rather than requiring the user to physically remove the glasses to see themselves. This allows users with visual impairments to see their reflection clearly through the camera capture and processing system.
Solution Approach 2:
The system introduces a camera and image processing system as an intermediary between the user and their reflection. Instead of directly viewing their reflection in a mirror (which is blocked by visual impairment), the user sees their image through the camera capture and digital processing pipeline, which removes the visual obstacles.
2Measurement precision
If facial recognition techniques are used to remove glasses from images, then the face can be reconstructed without glasses, but the technique only works in 2D and fails when glasses are wider than the face or the face is not facing forward
Solution Approach 1:
The system transitions from 2D facial recognition to 3D facial modeling. By creating a three-dimensional model of the user's face and head, the system can accurately represent facial features from any viewing angle, not just frontal 2D views. This 3D model allows the glasses to be removed visually while maintaining accuracy across multiple perspectives.
3Measurement precision
If facial recognition techniques are used to remove glasses, then the face can be reconstructed, but the technique only works with very thin frames and excludes all pairs of glasses with thick frames
Solution Approach 1:
The system uses 3D facial modeling to capture the full spatial extent of the face and glasses. This three-dimensional approach allows the system to accurately detect and remove glasses of any frame thickness, as the 3D model can represent the physical space occupied by thick frames without losing facial detail or accuracy.
4Adaptability or versatility
If virtual try-on systems are used where the user sees themselves on a screen, then the user can try on virtual glasses, but the technique does not allow for a realistic experience where the user can see their image on the screen as in a mirror
Solution Approach 1:
The system captures a photograph of the user and creates a realistic digital copy that can be displayed on a screen. By processing this copy to remove the glasses and overlaying virtual glasses, the system maintains the realism of seeing oneself as in a mirror while enabling virtual try-on functionality.
Data Source
Figure 1~2
Figure 3
Figure 4(a)~4(e)
AI summary
The invention relates to a method for generating a final image from an initial image comprising an object suitable for being worn by an individual. Said method comprises the following steps: a) detecting the presence of said object in the initial image; b) superimposing a first overlay on the initial image, the first overlay comprising a mask at least partially covering the object on the initial image; and c) modifying the appearance of at least a part of the mask. The invention makes it possible, in particular, to conceal all or part of an object in an image or a video. The invention also relates to an augmented reality method intended to be used by an individual wearing a vision device on the face, and a device for fitting a virtual object.