Multi-Filter Pose Tracking for Handheld VR Objects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing pose tracking systems for handheld objects in virtual reality experiences face issues with out-of-order, delayed, or missing data, leading to increased latency and jitter, which negatively impact user experiences.
Innovation Solution
The implementation of a method using multiple filters to manage and fuse data, where a second opportunistic filter is instantiated when one queue is empty, allowing for continuous pose updates and publication while maintaining temporal accuracy, and adjusting the timeframe for publishing updated poses based on data transport reliability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single filter is used to process optical pose data and IMU data, then the system structure is simple, but latency and jitter increase when data arrives out of order or is delayed
Solution Approach 1:
The patent divides the single filter into multiple filters (first filter and second filter) that operate in parallel. The first filter processes data using both optical pose data and IMU data, while the second filter processes data using only optical pose data when IMU data is delayed or missing. This segmentation allows the system to handle out-of-order data without increasing overall structural complexity significantly, while maintaining pose tracking accuracy.
Solution Approach 2:
The patent implements dynamic filter selection based on data arrival timing. The system dynamically determines whether to use the first filter (when both data types are available) or the second filter (when optical pose data is available but IMU data is delayed). This dynamic adaptation reduces jitter and latency by ensuring continuous pose updates without waiting for delayed IMU data.
2Measurement precision
If data is waited for in strict temporal order, then measurement precision is maintained, but productivity and responsiveness decrease
Solution Approach 1:
The patent performs preliminary processing of optical pose data through the second filter before IMU data arrives. This allows the system to generate an initial pose estimate using available optical data, maintaining measurement precision through proper filtering, while avoiding idle waiting time. When IMU data subsequently arrives, the system can integrate it to refine the pose estimate without losing productivity.
Solution Approach 2:
The patent ensures continuous pose updates by maintaining two parallel filtering paths. The second filter continuously processes optical pose data even when IMU data is delayed, ensuring the system never stops producing pose estimates. This continuity maintains both measurement precision (through proper filtering) and productivity (through uninterrupted pose updates).
Data Source
AI summary
One disclosed example provides a method for determining a pose of a handheld object in a surrounding environment. Optical pose data is stored in an image queue of a first filter. IMU data is received from an IMU of the handheld object and stored in an IMU queue of the first filter. Using at least a portion of the optical pose data and the IMU data, an initial pose of the handheld object is determined and outputted. The method determines that either the image queue or the IMU queue is empty. A second filter comprising the one empty queue and the other non-empty queue is instantiated as a copy of the first filter. Using the data from the non-empty queue in the second filter, the initial pose of the handheld object is updated to an updated pose, and the updated pose is outputted.