Real-Time Eyewear Scaling with Facial Landmarks and Depth Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems require calibration using reference objects to determine the scale of a user's face, which is burdensome and inefficient, affecting user experience and resource utilization.
Innovation Solution
The system computes a real-world scale of a user's face by combining facial landmarks with a depth map, allowing augmented reality elements to be accurately positioned and sized without calibration, and continuously updates this scale as the user moves.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If calibration using reference objects is used to determine face scale, then measurement precision is improved, but device complexity and ease of operation worsen due to burden on user
Solution Approach 1:
The system automatically detects facial landmarks and computes face scale without requiring user action to place reference objects. The computer vision system performs self-service by identifying facial features (eyes, nose, mouth, jawline) and calculating scale parameters autonomously, eliminating the calibration burden from the user while maintaining measurement precision
Solution Approach 2:
The patent replaces the mechanical calibration process (physically placing reference objects on the face) with a computational approach using computer vision and machine learning. The system uses algorithms to detect facial landmarks and compute scale through image processing, substituting physical mechanical calibration with digital computational methods
2Measurement precision
If calibration using reference objects is used to determine face scale, then measurement precision is improved, but loss of time increases due to calibration requirement
Solution Approach 1:
The system performs preliminary action by pre-training machine learning models on large datasets of facial images to recognize facial landmarks and compute scale parameters. This preliminary training enables the system to quickly determine face scale during actual use without requiring time-consuming calibration procedures, as the computational models are already prepared to identify facial features and calculate measurements
Solution Approach 2:
The patent replaces the time-consuming mechanical calibration process with rapid computational methods. Computer vision algorithms process facial images in real-time or near-real-time to determine scale parameters, eliminating the need for users to spend time physically calibrating the system with reference objects while maintaining measurement accuracy
3Measurement precision
If calibration using reference objects is used to determine face scale, then measurement precision is improved, but use of energy increases due to additional processing
Solution Approach 1:
The patent replaces the energy-intensive mechanical calibration process with more efficient computational methods. Instead of requiring users to physically position reference objects and the system to process multiple calibration images, the machine learning models directly compute face scale from standard facial images, reducing the number of processing steps and associated energy consumption while maintaining measurement precision
Solution Approach 2:
The system performs self-service by automatically detecting facial landmarks and computing scale parameters without requiring additional calibration resources. The computer vision system efficiently processes standard facial images to determine measurements, eliminating the need for separate calibration procedures that would consume additional energy, while maintaining accurate face scale determination
Data Source
AI summary
Methods and systems are disclosed for performing operations comprising: receiving an image that includes a depiction of a face of a user; generating a plurality of landmarks of the face based on the received image; removing a set of interfering landmarks from the plurality of landmarks resulting in a remaining set of landmarks of the plurality of landmarks; obtaining a depth map for the face of the user; and computing a real-world scale of the face of the user based on the depth map and the remaining set of landmarks.