Head-Mounted Wearable Fit Detection Using Image-Based 3D Pose
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for procuring wearable devices, such as smart glasses, do not provide accurate fitting and customization without access to a retail establishment, particularly for incorporating prescription lenses and ensuring proper display and ophthalmic fit.
Innovation Solution
A method for detecting keypoints and facial landmarks in images captured by a user's device to determine a three-dimensional pose of a fitting frame, allowing for configuration of a head-mounted wearable computing device's display and prescription lenses based on these measurements, enabling accurate fit and customization without in-person assistance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If image-based detection is used to determine fit parameters, then accessibility and convenience are improved, but measurement precision may deteriorate compared to professional in-person fitting
Solution Approach 1:
The system creates a three-dimensional digital copy of the fitting frame based on two-dimensional image data. By detecting keypoints in captured images and mapping them to a 3D model, the system reconstructs the frame's geometry and spatial relationships, enabling accurate fit parameter extraction without requiring physical measurement tools or professional equipment.
Solution Approach 2:
The system transforms two-dimensional image data into three-dimensional fit parameters by detecting keypoints in multiple images and performing correspondence between 2D keypoint positions and 3D model coordinates. This dimensional transformation enables the extraction of depth, orientation, and spatial relationships that cannot be obtained from single 2D views.
2Measurement precision
If keypoint detection and 3D pose determination are implemented, then display fit and ophthalmic fit accuracy are improved, but device complexity increases
Solution Approach 1:
The system divides the fitting frame into multiple identifiable keypoint locations (bridge portion, hinge points, peripheral edges, saddle portions). By detecting these segmented keypoint positions independently and mapping them to corresponding locations on a 3D model, the system can accurately determine the frame's pose and derive fit parameters without requiring complex overall frame analysis.
Solution Approach 2:
The system introduces a pre-defined three-dimensional model of the fitting frame as an intermediary between the captured images and the fit parameter extraction. This 3D model serves as a reference framework that guides keypoint detection and enables the translation of 2D image coordinates into meaningful 3D pose information and fit measurements.
3Measurement precision
If multiple keypoints and facial landmarks are detected, then ophthalmic fit measurements are improved, but processing time increases
Solution Approach 1:
The system performs preliminary detection of multiple keypoints on the fitting frame and facial landmarks before deriving fit parameters. By identifying all relevant keypoint locations (bridge, hinges, edges, saddles) and facial features (pupils, eye corners) in advance, the system can efficiently compute fit measurements without requiring iterative or repeated detection processes.
Data Source
AI summary
A system and method of detecting display fit measurements and/or ophthalmic measurements for a head mounted wearable computing device including a display device is provided. An image of a fitting frame worn by a user of the computing device is captured by the user, through an application running on the computing device. One or more keypoints and/or features and/or landmarks are detected in the image including the fitting frame. A three-dimensional pose of the fitting frame is determined based on the detected keypoints and/or features and/or landmarks, and configuration information associated with the fitting frame. The display device of the head mounted wearable computing device can then be configured based on the three-dimensional pose of the fitting frame as captured in the image.