HMD Head Pose Estimation With Motion-Synchronized IMU Frame Skipping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current SLAM methods in HMD devices face challenges with throughput and accuracy due to noisy IMU data, leading to erroneous pose estimation, and the computation-intensive bundle adjustment process is resource-heavy, affecting seamless AR navigation and interaction.
Innovation Solution
A method and device for motion synchronization-based head pose estimation that selectively processes IMU data using a feature-aware skipping strategy, integrating only selected data points and reducing computation through selective pre-integration and bundle adjustment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If IMU data is used to increase pose throughput, then the data processing frequency increases, but the noise in IMU data introduces drifts and errors in predicted pose
Solution Approach 1:
The patent extracts only the useful components of IMU data by selectively processing data points based on motion thresholds. It separates high-frequency motion data (above threshold) from low-frequency motion data (below threshold), processing only the necessary portions to maintain accuracy while improving throughput.
Solution Approach 2:
The patent applies partial action by not processing all IMU data points uniformly. Instead, it selectively processes data points that exceed motion thresholds, avoiding unnecessary processing of redundant low-frequency data while maintaining pose estimation accuracy.
2Measurement precision
If bundle adjustment is performed to refine pose estimation, then the accuracy of head pose improves, but the computation-intensive process increases runtime operations and power consumption
Solution Approach 1:
The patent performs bundle adjustment selectively rather than on every frame. It skips bundle adjustment for frames where motion thresholds are not exceeded, reducing computational load and power consumption while maintaining accuracy for significant motion events.
Solution Approach 2:
The patent implements periodic bundle adjustment based on motion triggers rather than continuous execution. Bundle adjustment is performed periodically when motion thresholds indicate significant head movement, reducing overall computational frequency and power consumption.
3Measurement precision
If all IMU data points are processed for pre-integration, then the predicted pose is more accurate, but the computational load increases significantly
Solution Approach 1:
The patent extracts only relevant IMU data points for pre-integration by filtering based on motion thresholds. It removes unnecessary data points that do not contribute significantly to pose accuracy, reducing computation intensity while maintaining predicted pose accuracy.
Solution Approach 2:
The patent applies different processing qualities to different portions of IMU data. High-frequency motion data receives full processing attention for accuracy, while low-frequency data is either skipped or processed with reduced intensity, optimizing the balance between accuracy and computational load.
Data Source
AI summary
A method for motion synchronization-based head pose estimation, by a head mounted display (HMD) device and an HMD device for performing the same are provided. The method includes receiving, by the HMD device, motion data from a plurality of motion sensors of the HMD device, receiving, by the HMD device, a plurality of image frames from at least one simultaneous localization and mapping SLAM camera of the HMD device, estimating, by the HMD device, a plurality of motion parameters of head movements of a user from the plurality of image frames received from memory, generating, by the HMD device, a filtered subset of the motion data received from the plurality of motion sensors based on the plurality of motion parameters of the head movements, synchronizing, by the HMD device, the plurality of image frames received from the memory and filtered subset of motion data, and estimating, by the HMD device, the head pose based on the synchronized plurality of image frames and motion data.