Scene Summary Maps Using High-Utility Objects for Mobile Localization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine vision techniques for augmented and virtual reality applications face challenges with resource-constrained mobile devices due to the accumulation of large 3D visual representations that exceed computational budgets, necessitating a more efficient method to manage and localize within environments.
Innovation Solution
A method and device that generate summary maps based on the estimated utility of objects within a scene, utilizing machine learning to identify high utility weight objects, filter data, and maintain scene reference maps of constrained size, allowing for efficient localization and pose estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the device accumulates captured imagery data over time to improve localization accuracy, then the localization precision is improved, but the computational load and memory requirements exceed the device's computational budget
Solution Approach 1:
The patent extracts and retains only the most useful visual features and objects from the accumulated imagery data, discarding redundant information. This selective extraction maintains localization accuracy while significantly reducing the volume of stored data to fit within device constraints.
Solution Approach 2:
The patent applies different quality levels to different parts of the data structure. High-utility objects and features are maintained with high fidelity, while less critical data is compressed or discarded, creating a hierarchical data structure that optimizes both accuracy and storage efficiency.
2Measurement precision
If the device maintains a detailed 3D visual representation of the environment, then the scene recognition accuracy is improved, but the memory consumption exceeds available resources
Solution Approach 1:
The patent segments the environment into discrete objects and features, maintaining detailed representations only for high-utility elements while using simplified representations for less critical areas. This segmentation allows selective memory allocation based on importance.
Solution Approach 2:
The patent maintains partial representations of the complete environment, focusing computational resources on capturing essential features and objects that provide the most value for localization and scene recognition, rather than attempting to store complete high-fidelity data of all areas.
3Loss of information
If the device processes and stores all captured imagery data, then the completeness of the reference map is improved, but the processing time and computational power required exceed available budgets
Solution Approach 1:
The patent performs preliminary filtering and feature extraction during the data capture phase, identifying and tagging high-utility objects and features before full processing. This preliminary action reduces the burden of subsequent processing while maintaining reference map completeness.
Solution Approach 2:
The patent dynamically adjusts processing parameters such as feature detection sensitivity, data compression ratios, and storage resolution based on the utility assessment of different scene elements. This parameter adjustment optimizes the balance between completeness and processing time for different types of data.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An electronic device [100] generates a summary map [265] of a scene based on data representative of objects having a high utility [245] for identifying the scene when estimating a current pose [275] of the electronic device and localizes the estimated current pose with respect to the summary map. The electronic device identifies scenes based on groups of objects [215] appearing together in consistent configurations over time, and identifies utility weights for objects appearing in scenes, wherein the utility weights indicate a predicted likelihood that the corresponding object will be persistently identifiable by the electronic device in the environment over time and are based at least in part on verification by one or more mobile devices. The electronic device generates a summary map of each scene based on data representative of objects having utility weights above a threshold.