Body-Mounted IMU Sensing for Low-Latency XR Pose Forecasting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing XR devices face challenges with latency and inaccuracy in pose forecasting due to delays in processing hand movements, leading to suboptimal rendering of virtual content.
Innovation Solution
Employing an external sensor, such as an IMU, connected to a user's extremity to provide external tracking data that complements on-board sensor data for improved pose forecasting, reducing processing latency and enhancing accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If hand tracking is performed using on-board sensors only, then device complexity is reduced, but pose forecasting accuracy and responsiveness deteriorate due to processing latency
Solution Approach 1:
The patent introduces an external IMU sensor as an intermediary device that complements the on-board sensors. This external sensor acts as a mediator to provide additional motion data, enabling more accurate pose forecasting without requiring the XR device itself to become significantly more complex. The external sensor processes motion data independently and feeds results back to the XR device.
2Measurement precision
If external IMU sensor is added to provide tracking data, then pose forecasting accuracy improves, but device complexity increases
Solution Approach 1:
The external IMU sensor serves multiple functions: it tracks motion of the extremity, provides data for pose forecasting, and can be used to determine field of view status. This multi-functionality justifies the added complexity by delivering multiple benefits from a single external device addition.
3Productivity
If hand tracking processes all captured images, then tracking completeness is maintained, but processing time increases causing latency
Solution Approach 1:
Instead of processing all captured images for hand tracking, the system uses partial action by leveraging the external IMU sensor data to identify and prioritize only the most relevant images or time windows for processing. This selective approach reduces processing time while maintaining adequate tracking accuracy.
Data Source
AI summary
In examples described herein, a sensor external to an extended reality (XR) device is connected to an extremity of a user of the XR device. The external sensor is communicatively coupled to the XR device. The XR device captures image data comprising one or more images of the extremity of the user. The XR device accesses external tracking data generated by the external sensor. A forecast of a pose of the extremity is generated based on the image data and the external tracking data. The forecast may be used for tracking of the extremity or to render virtual content for presentation to the user.