True-Size Eyewear Using Facial Landmarks and Depth Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems require calibration using reference objects to determine the scale of a user's face, which is burdensome and inefficient, affecting user experience and resource utilization.
Innovation Solution
The system computes a true or real-world scale of a user's face by combining facial landmarks with a depth map, allowing augmented reality elements to be applied without calibration, and adjusts their size and position based on facial topology, updating as the user moves.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If calibration using reference objects is used to determine face scale, then measurement precision is improved, but device complexity and ease of operation worsen due to burdensome calibration procedures
Solution Approach 1:
The system automatically determines face scale by detecting facial landmarks and using depth map data without requiring user intervention for calibration. The device performs self-calibration by leveraging its own sensors and algorithms to compute the scaling factor needed for accurate AR element placement.
Solution Approach 2:
The patent replaces the mechanical calibration process (physically placing reference objects on the face) with an automated computational approach using facial landmark detection and depth map analysis. This substitution eliminates the need for physical reference objects and manual calibration steps.
2Measurement precision
If calibration using reference objects is used to determine face scale, then measurement precision is improved, but loss of time worsens due to additional calibration steps
Solution Approach 1:
The system performs facial landmark detection and depth map acquisition as preliminary actions that occur automatically when the application is launched or when the camera is activated. These preliminary actions prepare the necessary data for immediate face scale computation without requiring separate calibration steps later.
Solution Approach 2:
The system automatically computes face scale using pre-acquired depth map and landmark data without requiring user-initiated calibration actions. This self-service approach eliminates the time users would otherwise spend performing calibration procedures.
3Measurement precision
If calibration using reference objects is used to determine face scale, then measurement precision is improved, but use of energy worsens due to additional processing requirements
Solution Approach 1:
The depth map acquisition and facial landmark detection processes serve multiple functions: they enable both accurate face scale determination and the placement of augmented reality elements. By making these computational components multi-functional, the system avoids the need for separate calibration-specific processing that would consume additional energy.
Data Source
AI summary
Methods and systems are disclosed for performing operations comprising: receiving an image that includes a depiction of a face of a user; generating a plurality of landmarks of the face based on the received image; removing a set of interfering landmarks from the plurality of landmarks resulting in a remaining set of landmarks of the plurality of landmarks; obtaining a depth map for the face of the user; and computing a real-world scale of the face of the user based on the depth map and the remaining set of landmarks.