3D Map Consistency Checks for Accurate AR Camera Re-Localization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing camera re-localization methods in augmented reality applications often output inaccurate pose estimates, leading to incorrect virtual element placement and user frustration due to low localization accuracy.
Innovation Solution
Perform simulated consistency checks on 3D maps by dividing scan data into mapping and validation sets, using a game server to test accuracy through pose estimation and sensor data comparison, and generating localizability scores for points of interest.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing re-localization methods use computer vision algorithms to estimate pose from 3D models, then the system can provide location-based AR services, but the pose estimates are inaccurate and do not match the actual device position and orientation
Solution Approach 1:
The patent implements a feedback mechanism where sensor data (accelerometer, gyroscope, magnetometer) is continuously compared with pose estimates from the 3D model to detect and correct localization errors. The system uses sensor fusion to provide corrective feedback that adjusts the pose estimates, ensuring they match the actual device position and orientation in the physical environment.
Solution Approach 2:
The patent introduces sensor data as an intermediary between the 3D model and the final pose estimate. Instead of relying solely on computer vision algorithms, the system uses sensor measurements as a mediating reference frame to verify and correct the positional and orientational data derived from image processing, thereby improving overall accuracy.
2Productivity
If the system directs users to the closest point of interest without verifying localization accuracy, then the application responds quickly, but users may be guided to locations with low successful localization likelihood
Solution Approach 1:
The patent applies preliminary action by pre-evaluating the reliability of localization at each point of interest before directing users there. The system performs advance verification of localization accuracy using sensor data comparison and only then recommends points of interest, ensuring that users are directed to locations where successful localization is highly likely while maintaining efficient response times.
3Measurement precision
If the system uses sensor data to verify pose estimates, then localization accuracy improves, but the processing complexity and computational requirements increase
Solution Approach 1:
The patent segments the localization process into distinct modules: computer vision-based pose estimation, sensor data acquisition, sensor fusion processing, and error correction. This segmentation allows each component to be optimized independently and processed in a systematic pipeline, reducing overall processing complexity while maintaining high localization accuracy through coordinated operation of specialized sub-systems.
Data Source
AI summary
The present disclosure describes approaches to camera re-localization that improve the accuracy of re-localization determinations by performing simulated consistency checks for three-dimensional maps. Client devices associated with users of a location-based application transmit image scans to a game server, which divides the received scan data into mapping sets used to generate 3D maps of environments and validation sets used to test the accuracy of the maps. To perform the testing, the game server identifies query scans in the validation set having GPS coordinates within a threshold distance of the mapped location and uses the 3D map of the environment to generate a pose estimate for each frame. The results of the localization queries are analyzed by comparing differences between the localization pose estimates and differences between the poses of independent pairs of frames in the query scan to evaluate the accuracy of the 3D map.