Cross-Reality Map Localization Using Rough-to-Refined Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cross reality (XR) systems face challenges in efficiently and accurately localizing XR devices within large environments, requiring extensive computational resources and latency, which hampers immersive and seamless user experiences.
Innovation Solution
A system that utilizes persistent maps stored in a database to enable efficient localization of XR devices by performing rough and refined localization techniques, leveraging feature descriptors and consensus-based alignment to reduce computational load and latency, allowing for quick and accurate positioning of virtual content relative to the physical world.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional localization methods are used in XR systems, then localization accuracy can be maintained, but computational resources and latency increase significantly
Solution Approach 1:
The localization process is segmented into multiple stages: rough localization using persisted maps to identify candidate regions, followed by refined localization using sensor data and feature matching within those regions. This segmentation allows the system to maintain accuracy while reducing overall computational load by limiting intensive processing to smaller candidate regions rather than entire environments.
Solution Approach 2:
Persisted maps are pre-computed and stored containing localization data from previous XR sessions. When a new session begins, the system performs rough localization by comparing current sensor data against these pre-computed maps to quickly identify candidate regions, avoiding the need to process entire environment maps from scratch and significantly reducing initial localization time and computational resources.
2Measurement precision
If traditional localization methods are used in XR systems, then localization accuracy can be maintained, but latency increases which hampers user experience
Solution Approach 1:
The localization pipeline is divided into fast rough localization using persisted maps to narrow down candidate regions, followed by more accurate but computationally intensive refined localization only within those restricted regions. This segmentation dramatically reduces overall latency by limiting the scope of intensive processing to small candidate regions rather than entire environments.
Solution Approach 2:
Persisted maps are pre-computed during previous XR sessions and stored for rapid retrieval. During new sessions, these pre-computed maps enable fast rough localization by allowing the system to quickly compare current sensor data against stored representations of known environments, identifying candidate regions without performing full environment scanning and processing.
3Stability of the object's composition
If rough localization is performed for all collections of features, then consistency can be maintained, but computational load increases
Solution Approach 1:
The system performs rough localization selectively only for candidate regions identified from persisted maps, rather than uniformly processing all collections of features. This partial action approach maintains localization consistency in critical areas while improving overall computational efficiency by skipping rough localization in regions already well-constrained by other data sources.
Solution Approach 2:
The system uses feedback from multiple localization sources (persisted maps, sensor data, feature matching results) to determine whether rough localization is necessary for each collection of features. When other localization methods already provide sufficient accuracy and consistency, the system skips rough localization, optimizing computational efficiency while maintaining overall localization consistency through the consensus mechanism.
Data Source
AI summary
A cross reality system enables any of multiple devices to efficiently and accurately access previously persisted maps, even maps of very large environments, and render virtual content specified in relation to those maps. The cross reality system may quickly process a batch of images acquired with a portable device to determine whether there is sufficient consistency across the batch in the computed localization. Processing on at least one image from the batch may determine a rough localization of the device to the map. This rough localization result may be used in a refined localization process for the image for which it was generated. The rough localization result may also be selectively propagated to a refined localization process for other images in the batch, enabling rough localization processing to be skipped for the other images.