AR VR Input Device Tracking with IMU and Bayesian Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AR/VR systems face challenges in tracking HMDs and hand controllers due to occlusions, echoes, and signal disruptions, which affect system performance and require more robust tracking solutions.
Innovation Solution
An input device equipped with an IMU and sensors that detect emissions from remote emitters, using Bayesian estimation and probabilistic techniques like Kalman filters to continuously update the position and orientation in real-time, and employing time-division-multiplexing, frequency-division-multiplexing, or code-division-multiplexing schemes to enhance tracking accuracy and robustness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If outside-in tracking systems with emitters and sensors are used, then tracking capability is provided, but occlusions and signal disruptions negatively impact tracking reliability
Solution Approach 1:
The patent combines outside-in tracking (emitters and sensors) with inside-out tracking (IMU) systems into a unified tracking architecture. The processor fuses data from both external emitters/sensors and internal IMU measurements to determine position and orientation, ensuring continuous reliable tracking even when external signals are occluded or disrupted.
Solution Approach 2:
The IMU acts as an intermediary that provides continuous tracking data independent of external emitters. When external signals are blocked or disrupted, the IMU maintains tracking capability by providing inertial measurement data that the processor uses to calculate position and orientation without relying on line-of-sight to external markers.
2Measurement precision
If multiple sensors and emitters are deployed for robust tracking, then tracking accuracy improves, but device complexity increases
Solution Approach 1:
The input device is designed with multi-functionality, incorporating both external sensors for outside-in tracking and an internal IMU for inside-out tracking. This universal design allows the same device to operate with either or both tracking methods depending on environmental conditions, reducing the need for separate specialized devices while maintaining high tracking accuracy.
Solution Approach 2:
The input device carries its own IMU that provides self-contained tracking capability independent of external infrastructure. This self-service approach reduces reliance on complex external emitter arrays and allows the device to maintain accurate tracking using its own onboard sensors, simplifying the overall system architecture.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The solution enables precise and robust tracking of input devices in AR/VR environments with improved accuracy and reliability, even in conditions of occlusions and signal disruptions, by integrating IMU data and sensor emissions for real-time position and orientation updates.
Implementation Method 1
an internal measurement unit (IMU) controlled by the one or more processors and configured to measure an acceleration and velocity (e.g., linear or angular velocity) of the input device
Implementation Method 2
determine a time-of-flight (TOF) of the detected emissions
Data Source
AI summary
An AR/VR input device include a processor(s), an internal measurement unit (IMU), and a plurality of sensors configured to detect emissions received from a plurality of remote emitters. The processor(s) can be configured to: determine a time-of-flight (TOF) of the detected emissions, determine a first estimate of a position and orientation of the input device based on the TOF of a subset of the detected emissions and the particular locations of each of the plurality of sensors on the input device that are detecting the detected emissions, determine a second estimate of the position and orientation of the input device based on the measured acceleration and velocity from the IMU, and continuously update a calculated position and orientation of the input device within the AR/VR environment in real-time based on a Beyesian estimation (e.g., Extended Kalman filter) that utilizes the first estimate and second estimate.