Real-Time Virtual Glasses Try-On Using Face Morphology and Template Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual and augmented reality solutions for real-time virtual try-on of glasses lack realism and interactivity, and are computationally intensive, making them insufficient for real-time integration into photographs or videos, especially on devices with limited capabilities.
Innovation Solution
A unified process for real-time realistic rendering of virtual glasses on a user's face involves face analysis, glasses positioning, and real-time tracking, using techniques like Viola-Jones classifiers, Luxand features alignment, and a statistical geometric model for face morphology estimation, along with physically-based rendering for realistic appearance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If current virtual or augmented reality solutions are used for glasses try-on, then virtual fitting can be achieved, but realism and interactivity are insufficient and computational requirements are high
Solution Approach 1:
The patent replaces complex augmented reality systems with a simplified approach using 2D image processing and template matching. Instead of using full 3D augmented reality rendering which requires high computational power, the system uses detected face features and template images to composite the glasses onto the user's face in real-time, significantly reducing computational requirements while maintaining realism.
Solution Approach 2:
The system creates realistic glasses try-on by copying and overlaying template images of glasses onto detected face regions. Rather than generating 3D models or using complex rendering, the patent uses 2D template matching and image compositing techniques to place realistic glasses images onto the user's face, achieving high realism with lower computational cost.
2Measurement precision
If real-time tracking and face analysis are performed, then stability and accuracy are improved, but processing time and computational load increase
Solution Approach 1:
The patent segments the face detection and tracking process into distinct stages: initial face detection using Viola-Jones, feature point extraction using Luxand, and template matching for glasses positioning. This segmentation allows each stage to be optimized independently, maintaining high accuracy while enabling real-time processing through efficient algorithm selection at each stage.
Solution Approach 2:
The system performs preliminary face detection and feature extraction before glasses template matching. By pre-detecting face boundaries, landmarks, and orientation using optimized algorithms like Viola-Jones and Luxand, the system prepares the face region in advance, which accelerates the subsequent glasses compositing process and enables real-time performance.
3Speed
If a unified process working on real-time camera streams is implemented, then real-time interaction is achieved, but computational resources on client devices are strained
Solution Approach 1:
The patent extracts and processes only the essential elements needed for glasses try-on: face detection, key feature points, and template matching. Rather than processing the entire camera stream or performing full 3D rendering, the system extracts only the relevant face region and performs 2D compositing, significantly reducing computational load and power consumption on client devices.
Solution Approach 2:
The system dynamically adjusts processing based on detected face characteristics and camera conditions. The template matching and compositing operations are adapted in real-time based on face orientation, distance, and lighting conditions, allowing the system to maintain real-time performance while optimizing resource usage on power-constrained devices.
Data Source
Figure 1
Figure 2~3
AI summary
The invention aims at a method for creating a real-time realistic- looking image of a pair of virtual glasses positioned on a real user face image issued from a single image, a bundle of pictures, a video clip or a real-time camera stream, characterized in that it comprises the following steps: • - 100. A face analysis process to estimate the 3D rotation matrix, the 3D translation vector, morphology parameters of the user's face; • - 200. A glasses positioning process associated with a try-on mode for look or real-size simulation that estimate the 3D rotation matrix, the 3D translation vector, and global and local deformation parameters of the virtual glasses, and arms articulation, adapted to the morphology parameters computed in step 100; • - 400. A realistic rendering process to mix the real user face image with the virtual glasses positioned in step 200.