AR Device Pose Updates Using Interpolated IMU Poses for High-Motion Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems face challenges in accurately and stably positioning virtual objects in real-world scenes due to issues with initial device pose determination and tracking, particularly in high-motion environments where image features are difficult to match and sensor drift occurs.
Innovation Solution
A hybrid approach combining visual odometry and motion sensor information is used to track camera motion, refining the device pose by comparing image frames and 3D model features, and applying a transformation based on inertial measurement unit poses, with low-pass filtering to stabilize the pose updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If visual odometry is used to track camera motion, then virtual object placement accuracy is improved, but tracking fails in high-motion environments where image features are difficult to match
Solution Approach 1:
The patent combines visual odometry with inertial measurement unit (IMU) data to create a hybrid tracking system. The IMU provides reliable motion information during high-motion scenarios where visual features become difficult to track, while visual odometry provides accurate pose estimation during stable periods. This merging of complementary sensing modalities resolves the contradiction between measurement precision and tracking reliability.
Solution Approach 2:
The system uses an intermediary fusion algorithm that combines visual odometry results with IMU measurements. The IMU acts as an intermediary that bridges gaps in visual tracking during high-motion periods, providing continuous pose estimation even when image feature matching fails. This intermediary approach maintains tracking reliability while preserving visual odometry's measurement precision.
2Speed
If sensor data is used for pose determination, then tracking speed is improved, but sensor drift causes accuracy degradation over time
Solution Approach 1:
The system implements feedback by continuously comparing IMU-based pose estimates with visual odometry results. When visual features are reliably detectable, the visual system provides feedback to correct accumulated IMU drift. This feedback loop maintains long-term pose accuracy while preserving the high tracking speed provided by the IMU during periods when visual matching is difficult.
3Reliability
If a hybrid approach combining visual odometry and IMU is used, then robustness to tracking failures is improved, but device complexity increases
Solution Approach 1:
The system dynamically adjusts the weighting and fusion strategy between visual odometry and IMU data based on current tracking conditions. During high-motion periods when visual features are difficult to match, the system dynamically shifts reliance toward IMU data. During stable periods with good feature visibility, it dynamically emphasizes visual odometry. This dynamic adaptation improves robustness without requiring permanently complex fusion mechanisms for all scenarios.
Data Source
AI summary
An augmented reality device generates an updated position and orientation (pose) value by initially determining, using image-based processing, a pose estimate from a current image frame, a previous image frame, and a previous pose. First and second inertial measurement unit (IMU) poses, having timestamps on either side of a timestamp of the current image frame are determined. An interpolated IMU pose is determined from the two IMU poses and the three timestamps. A transformation between the current-image pose estimate and the interpolated IMU pose is determined and applied to the current-image pose estimate to generate a pose update, for use in operating the augmented reality device.