Body-Mounted Hand Sensor for Low-Latency XR Pose Forecasting

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing XR devices face delays in rendering virtual content due to processing lag, leading to inaccurate positioning of virtual objects relative to moving body parts, such as hands, which detracts from the user experience.

Innovation Solution

Employing an external sensor, like an IMU, connected to a user's extremity to provide external tracking data that complements on-board sensor data for more accurate pose forecasting, reducing latency and improving rendering accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If processing time is reduced to improve responsiveness, then rendering speed improves, but positioning accuracy of virtual objects deteriorates

Engineering Contradiction:

Improverendering speedVSAvoidpositioning accuracy

Core Design Contradiction:

SpeedVSMeasurement precision

Solution Approach 1:

The system performs pose forecasting in advance to predict future positions of body parts. By calculating predicted poses before rendering occurs, the system compensates for processing delays, ensuring that virtual objects are positioned accurately relative to the user's actual hand position despite the time required for processing.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces an intermediary forecasting mechanism that bridges the gap between captured hand pose data and rendered virtual content. This intermediary process predicts intermediate future positions, allowing the system to maintain both speed and accuracy by pre-calculating where hand gestures will be when rendering occurs.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Measurement precision

If external sensors are added to improve tracking accuracy, then pose forecasting accuracy improves, but device complexity increases

Engineering Contradiction:

Improvepose forecasting accuracyVSAvoidsensor system complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The system uses a multi-functional approach where the same external sensor (e.g., IMU) serves multiple purposes: tracking hand pose, forecasting future positions, and providing feedback for rendering adjustments. This universal use of the sensor maximizes its utility while minimizing the need for additional specialized components.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The external sensor system is designed to be self-contained and self-calibrating, where the sensor provides its own reference frame and the system automatically synchronizes it with the XR device's coordinate system. This self-service capability reduces the complexity of integration and calibration procedures.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS12499633B1Body-mounted sensor for improved pose forecasting

Publication Date: 2025.12.16 SNAP INC

AI summary

In examples described herein, a sensor external to an extended reality (XR) device is connected to an extremity of a user of the XR device. The external sensor is communicatively coupled to the XR device. The XR device captures image data comprising one or more images of the extremity of the user. The XR device accesses external tracking data generated by the external sensor. A forecast of a pose of the extremity is generated based on the image data and the external tracking data. The forecast may be used for tracking of the extremity or to render virtual content for presentation to the user.