Mobile AR Localization via Semantic Object Constellations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality technologies face challenges in accurately localizing mobile devices within a 3D space, particularly due to sensitivity to environmental changes and computational intensity of point-cloud-based localization methods.
Innovation Solution
A method that records frames and motion data to detect 3D constellations of objects, generates classification vectors, and aligns these with a 3D localization map, allowing for real-time rendering of augmented reality assets by leveraging local object detection models and remote processing for improved accuracy and reduced latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If point-cloud-based localization methods are used, then localization accuracy is improved, but computational intensity and processing time increase
Solution Approach 1:
The patent extracts only the essential features needed for localization (object classifications and relative positions) from the full point cloud data, transmitting only these extracted features to the remote system rather than the complete point cloud, thereby reducing computational intensity while maintaining localization accuracy
Solution Approach 2:
The localization process is segmented into two parts: local feature extraction and remote matching. The mobile device performs only feature extraction and transmission, while the remote system performs the computationally intensive matching against the 3D localization map, dividing the computational workload to reduce local processing time and intensity
2Measurement precision
If point-cloud-based localization methods are used, then localization accuracy is improved, but processing time increases
Solution Approach 1:
The 3D localization map is pre-computed and stored with object classifications and relative positions before runtime. During actual localization, the system only needs to match extracted features against this pre-prepared map, significantly reducing processing time while maintaining accuracy
Solution Approach 2:
Only essential localization features (object classifications and relative positions) are extracted and transmitted, eliminating the need to process and transmit entire point cloud datasets, thereby reducing processing time while preserving the accuracy needed for successful localization
3Reliability
If traditional localization methods are used, then environmental changes affect accuracy, but the system remains simple
Solution Approach 1:
The system changes from using raw geometric parameters (point cloud coordinates) to using semantic parameters (object classifications and relative positions). This parameter transformation makes the localization more robust to environmental changes such as lighting variations and occlusions, as semantic features are more stable across different conditions
Solution Approach 2:
Object classifications serve as an intermediary layer between the raw sensor data and the localization map matching process. This intermediary representation is more invariant to environmental changes, allowing the system to maintain reliability without significantly increasing complexity
Data Source
AI summary
A method includes: recording a series of frames; recording a set of motion data representing motion of the mobile device; detecting relative positions of a 3D constellation of objects based on the series of frames and the set of motion data; generating classifications of the 3D constellation of objects by calculating a classification of each object in a set of object classes; calculating a transform aligning the 3D constellation of objects with a 3D localization map; accessing a set of augmented reality assets defined by the 3D localization map; calculating a position of the mobile device relative to the 3D localization map based on the transform and the set of motion data; and rendering the set of augmented reality assets based on positions of the set of augmented reality assets in the 3D localization map and based on the position of the mobile device in the 3D localization map.