Real-Time Panoramic Tracking on Mobile Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Outdoor Augmented Reality tracking systems face challenges with accuracy and stability due to reliance on inertial sensors, particularly gyroscope measurements, and existing panoramic tracker methods require offline panorama creation or suffer from high computational costs unsuitable for mobile platforms.
Innovation Solution
A method for real-time panoramic mapping and tracking on mobile platforms, where camera images are projected onto a cylindrical map, keypoints are extracted and used to determine orientation, and the map is extended and corrected to remove errors and improve tracking accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If real-time panoramic mapping is implemented on mobile platforms, then tracking accuracy and robustness are improved, but computational costs become prohibitively high
Solution Approach 1:
The patent segments the panoramic mapping process into incremental updates. Instead of processing the entire panorama at once, it updates only the portions of the cylindrical map that correspond to the current camera view and rotation, dividing the computational workload into smaller manageable segments that can be processed in real-time on mobile devices
Solution Approach 2:
The patent applies partial action by processing only the necessary portions of the map update. It calculates and projects camera images only for the angular range covered by the current view, rather than processing the entire 360-degree panorama, thereby reducing computational cost while maintaining sufficient mapping accuracy
2Power
If offline panorama creation is used, then computational costs are reduced, but tracking responsiveness and real-time performance deteriorate
Solution Approach 1:
The patent transforms the static offline panorama creation process into a dynamic real-time system. The cylindrical map is continuously updated as the mobile device rotates, with new camera images being projected and integrated into the map structure in real-time, enabling adaptive tracking that responds to changing environmental conditions
Solution Approach 2:
The patent performs preliminary actions by pre-defining the cylindrical map structure and projection parameters before actual tracking begins. This setup allows the system to quickly process and integrate new camera images without performing complex calculations from scratch during real-time operation
3Measurement precision
If gyroscope sensors are used for tracking, then rotational motion measurement is improved, but tracking stability deteriorates due to sensor noise and drift
Solution Approach 1:
The patent implements feedback by continuously comparing the current camera image against the existing cylindrical map and adjusting the orientation estimation accordingly. This closed-loop approach allows the system to detect and correct drift and noise accumulation by referencing the visual features in the map, thereby stabilizing tracking despite gyroscope sensor imperfections
Solution Approach 2:
The cylindrical map serves as an intermediary that mediates between the raw gyroscope measurements and the final tracking results. Instead of directly using noisy gyroscope data for tracking, the system projects camera images onto the map structure, which smooths and stabilizes the orientation estimates by leveraging the geometric constraints of the map
4Manufacturing precision
If camera images are projected onto a cylindrical map, then mapping accuracy is improved, but processing time increases
Solution Approach 1:
The patent applies local quality by adjusting the projection and processing parameters based on the specific region of the cylindrical map being updated. Different areas of the map may receive different levels of processing detail depending on their importance for tracking, allowing high accuracy where needed while reducing processing time in less critical regions
Data Source
AI summary
A mobile platform maps and tracks the orientation of the mobile platform in real-time as the mobile platform rotates. The mobile platform captures a plurality of camera images as the camera rotates and projects the plurality of camera images consecutively to a panoramic cylindrical map. The map is extended by projecting areas of any camera image that correspond to unmapped portions of the panoramic cylindrical map. Keypoints are extracted from mapped portions of the map and using the keypoints the orientation of the camera is determined. Accumulated error is removed when closing the panoramic cylindrical map by extending the map horizontally beyond 360 degrees to produce overlapping regions and extracting and matching keypoints from the overlapping regions. Additionally, an incorrect estimate of the initial orientation of the map may be corrected by determining the correct orientation and re-projecting the map onto a map with the correct orientation.