Localizing Map Quality Evaluation for Mixed Reality Headsets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current optical systems in mixed reality (MR) devices struggle to accurately localize users within their environment, leading to challenges in correctly positioning virtual objects relative to the real world.
Innovation Solution
The system creates a localizing map of the environment using a camera system on the image display device, processes images to remove undesirable data, and employs a metric to assess the quality of the map based on co-visibility of reference points, ensuring accurate user localization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a camera system is used to create a localizing map for user localization, then the accuracy of user localization is improved, but the processing time and computational complexity increase
Solution Approach 1:
The system performs preliminary actions by capturing multiple images of the environment before user localization is needed. These images are processed to identify reference points and create a localizing map in advance, so that when localization is required, the system can quickly match current camera views against the pre-processed map without performing heavy computational tasks in real-time.
Solution Approach 2:
The localizing map is segmented into multiple regions or cells, each containing pre-identified reference points. This segmentation allows the system to divide the large-scale environmental data into smaller, more manageable portions that can be processed and stored efficiently, reducing the overall computational burden during localization operations.
2Reliability
If multiple reference points are used to assess map quality based on co-visibility, then the reliability of localization is improved, but the device complexity increases
Solution Approach 1:
The system introduces an intermediary metric called 'co-visibility' that quantifies the reliability of reference points based on how many different camera positions can observe each point. This intermediary measure simplifies the assessment process by providing a single numerical value that correlates with localization reliability, avoiding the need for complex multi-dimensional analyses of reference point distributions.
Solution Approach 2:
The system changes the parameter used to assess map quality from a complex spatial distribution analysis to a simpler co-visibility count parameter. By transforming the quality assessment into a parameter based on the number of visible reference points from different viewpoints, the system reduces computational complexity while maintaining reliability assessment accuracy.
3Manufacturing precision
If undesirable data is removed from images during map creation, then the manufacturing precision of the localizing map is improved, but the loss of information increases
Solution Approach 1:
The system applies local quality control by selectively removing only the undesirable portions of images (such as noisy regions, saturated pixels, or artifacts) while preserving the useful environmental information. This localized data cleaning approach improves the precision of the localizing map by eliminating specific problematic data points without discarding the overall structural and semantic information about the environment.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An apparatus configured to be worn on a head of a user, includes: a screen configured to present graphics to the user; a camera system configured to view an environment in which the user is located; and a processing unit configured to determine a map based at least in part on output(s) from the camera system, wherein the map is configured for use by the processing unit to localize the user with respect to the environment; wherein the processing unit of the apparatus is also configured to obtain a metric indicating a likelihood of success to localize the user using the map, and wherein the processing unit is configured to obtain the metric by computing the metric or by receiving the metric.