Auto-Harmonization for Vehicular AR Registration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality systems, particularly those using optical see-through head-mounted displays, face challenges in achieving accurate spatio-temporal registration in unprepared real-world environments, especially in moving vehicles, due to high demands on absolute 6-DOF pose accuracy and relative calibration, leading to mis-registration and 'swim' issues.
Innovation Solution
An optical/inertial hybrid tracking system with novel solutions for optics, algorithms, synchronization, and alignment is developed to achieve precise registration, using fiducial stickers, retro-reflective fiducials, and advanced calibration techniques, along with a Kalman filter-based auto-harmonization algorithm to align the tracking system with the vehicle's inertial navigation system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Illumination intensity
If optical see-through HMD is used for augmented reality, then the user can see the physical world in real-time, but accurate spatio-temporal registration becomes extremely difficult due to latency and alignment requirements
Solution Approach 1:
The patent introduces an inertial measurement unit (IMU) as an intermediary device that measures head motion independently of the optical path. The IMU data serves as a mediator to compensate for optical latency and achieve accurate registration between the virtual and physical worlds in see-through HMDs.
Solution Approach 2:
The patent replaces purely optical tracking mechanisms with a hybrid system that incorporates inertial sensors (accelerometers and gyroscopes). This substitution allows the system to measure head motion through mechanical sensing rather than relying solely on optical path delay, thereby achieving better spatio-temporal registration.
2Measurement precision
If high tracking accuracy is achieved through multiple sensors and calibration, then registration precision improves, but system complexity and computational load increase significantly
Solution Approach 1:
The patent combines optical tracking data with inertial measurement unit (IMU) data into a unified tracking system. By merging these two data sources, the system achieves higher tracking accuracy than either system could provide alone, while the integration is managed through established sensor fusion algorithms.
Solution Approach 2:
The system performs self-calibration using the auto-harmonization algorithm, which automatically adjusts for misalignments between the camera and IMU reference frames. This eliminates the need for manual calibration procedures and reduces the operational complexity of maintaining high tracking accuracy.
3Measurement precision
If manual calibration procedures are used to align tracking system with vehicle INS, then registration accuracy improves, but setup time and operational complexity increase
Solution Approach 1:
The patent implements an auto-harmonization algorithm that automatically calibrates the tracking system by comparing camera-derived orientation with INS-derived orientation. The system performs this calibration without human intervention, eliminating manual setup time while maintaining high accuracy through iterative optimization.
Solution Approach 2:
The system performs calibration computations in advance during system initialization or idle periods, rather than requiring calibration at the moment of use. This preliminary action ensures the system is ready for immediate operation with pre-computed calibration parameters, reducing operational downtime.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system provides unprecedented spatio-temporal registration accuracy, eliminating noticeable mis-registration and 'swim' in dynamic environments, ensuring robust and precise augmented reality experiences in vehicles.
Implementation Method 1
retro-reflective fiducials
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method, a system, and a computer program product for tracking an object moving relative to a moving platform, the moving platform moving relative to an external frame of reference. The system includes An inertial sensor for tracking the object relative to the external reference frame, a non-inertial sensor for tracking the object relative to the moving platform, and a processor to perform sensor fusion of the inertial and non-inertial measurements in order to accurately track the object and concurrently estimate the misalignment of the non-inertial sensor's reference frame relative to the moving platform.