Mixed Reality Headset Tracking in Non-Inertial Vehicle Frames
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing headset tracking technologies fail to provide high-quality 6DoF tracking in moving vehicles due to the fusion of inertial and visual information methods that assume a stationary inertial frame, leading to rendering issues in moving platforms like cars and boats.
Innovation Solution
A prediction model is used to reconcile discrepancies between inertial measurement unit (IMU) data and camera data by defining a non-inertial reference frame fixed to the moving platform, incorporating motion parameters and filtering interior tracking points to enhance headset and controller tracking in vehicles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If fusion of inertial information from IMU and visual information from cameras is used for 6DoF tracking, then tracking quality is improved in stationary environments, but rendering accuracy deteriorates in moving vehicles
Solution Approach 1:
The system dynamically adapts the reference frame based on detected motion. When vehicle motion is detected through IMU data analysis, the system transitions from assuming a stationary inertial frame to implementing a moving reference frame that accounts for platform acceleration, enabling accurate tracking both inside and outside vehicles
Solution Approach 2:
The system changes the fundamental parameter of the reference frame assumption based on operating conditions. By monitoring IMU data for vehicle motion patterns and adjusting whether to treat the reference frame as inertial or non-inertial, the system optimizes rendering accuracy for different environments while maintaining tracking quality
2Adaptability or versatility
If camera observes features not moving with respect to inertial reference frame, then visual tracking works well outside vehicles, but tracking fails inside moving platforms
Solution Approach 1:
The system dynamically determines whether the camera is observing stationary or moving features by analyzing IMU data for vehicle motion. When motion is detected, the system adapts by implementing a moving reference frame that compensates for platform movement, enabling reliable tracking of interior features that would otherwise appear to move due to vehicle acceleration
Solution Approach 2:
The system uses IMU data as an intermediary to bridge the gap between visual features and the reference frame. By fusing IMU motion information with camera observations, the system can correctly interpret whether observed feature movement is due to headset motion or platform motion, enabling reliable tracking in both stationary and moving environments
Data Source
AI summary
Various aspects of the subject technology relate to systems, methods, and machine-readable media for rendering media in a mixed reality device moving in a non-inertial reference frame. Various aspects include identifying tracking points associated with a moving platform defined by a non-inertial reference frame; receiving a first and a second set of monitoring data from a mixed reality headset comprising: a camera device, a headset IMU, and a controller IMU; receiving a second set of tracking data from the mixed reality headset; predicting by a prediction model a movement of the tracking points; determining a discrepancy between the predicted movement of at least one of the tracking points and the second set of tracking data, wherein the discrepancy exceeds an error threshold; and in response to determining the discrepancy, generating display data for rendering a display image in the mixed reality headset.