LiDAR-IMU Sensor Fusion for Stable AR Pose Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems face challenges in maintaining precise camera tracking, especially in dynamic or featureless environments and where traditional positioning technologies like GPS are unavailable or unreliable, leading to misalignment between virtual and real-world content.
Innovation Solution
A system combining a LiDAR sensor and an IMU for high-frequency pose estimation, utilizing simultaneous localization and mapping (SLAM) and sensor fusion techniques, including extended Kalman and neural network Kalman filters, to generate accurate and smooth pose estimates at video frame rates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional camera tracking methods using visual features or markers are used, then the system is simple to implement, but tracking precision deteriorates in dynamic or featureless environments
Solution Approach 1:
The patent combines multiple sensor types (LiDAR, IMU, and optionally camera) into a unified tracking system. The LiDAR sensor provides 3D spatial information, the IMU provides motion data, and the camera provides visual features. By merging these sensors and fusing their data through sensor fusion algorithms, the system achieves high tracking precision in diverse environments while managing complexity through integrated processing.
Solution Approach 2:
The patent introduces an intermediary processing layer that includes sensor fusion algorithms (such as Kalman filters or graph optimization) to combine data from multiple sensors. This intermediary layer mediates between the raw sensor inputs and the final tracking output, enabling the system to leverage complementary information from different sensors to achieve robust tracking in challenging environments.
2Measurement precision
If additional sensors like IMUs or GPS receivers are incorporated to improve tracking performance, then tracking accuracy is improved, but reliability deteriorates due to accumulating errors or signal unavailability
Solution Approach 1:
The patent merges LiDAR-based SLAM tracking with IMU-based motion estimation in a complementary manner. The LiDAR provides absolute position references that prevent IMU drift, while the IMU fills in the temporal gaps between LiDAR scans. This combination achieves both high accuracy and reliability by using each sensor's strengths to compensate for the other's weaknesses.
Solution Approach 2:
The patent implements feedback mechanisms through sensor fusion algorithms that continuously adjust tracking estimates based on consistent information from multiple sensors. The system uses LiDAR point cloud matches to correct IMU accumulation errors and uses IMU data to predict motion between LiDAR frames, creating a feedback loop that maintains both accuracy and reliability over extended periods.
3Stability of the object's composition
If LiDAR data is processed at high frequency to achieve smooth pose estimates, then tracking smoothness is improved, but processing time increases
Solution Approach 1:
The patent segments the sensor fusion processing into distinct stages: (1) LiDAR data processing and SLAM pose estimation, (2) IMU data processing and motion prediction, and (3) sensor fusion combining both streams. This segmentation allows each stage to operate independently and optimize for its specific requirements, reducing overall processing time while maintaining smooth pose estimates.
Solution Approach 2:
The patent performs preliminary actions by pre-processing LiDAR and IMU data separately before fusion, including filtering, feature extraction, and motion prediction. The IMU data is used to predict expected LiDAR motion between frames, and this prediction is performed in advance to reduce real-time computational burden and achieve smoother pose estimates with lower processing latency.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables seamless augmented reality experiences by providing stable and precise camera tracking in diverse environments, including GPS-denied spaces, with the ability to handle dynamic scenes and maintain consistent performance over extended periods.
Implementation Method 1
a light detection and ranging (LiDAR) sensor physically attached to a target device and configured to generate LiDAR data of an environment
Implementation Method 2
an inertial measurement unit (IMU) configured to generate IMU data at a second frequency
Data Source
AI summary
The present disclosure provides a system for tracking a target device in three-dimensional space. The system includes a light detection and ranging (LiDAR) sensor physically attached to a target device and configured to generate LiDAR data of an environment at a first frequency, and an inertial measurement unit (IMU) configured to generate IMU data at a second frequency higher than the first frequency. At least one processor is configured to perform simultaneous localization and mapping (SLAM) using the LiDAR data to generate SLAM pose data at at least the first frequency, perform sensor fusion of the SLAM pose data and the IMU data to generate fused pose data at a third frequency higher than the first frequency and lower than the second frequency, and output the fused pose data to provide position and orientation of the target device.