XR Hand Pose Reprojection for Low-Latency Power Saving

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

XR systems face challenges with high power consumption and latency due to the use of powerful processors and split rendering between companion devices and head-mounted displays, leading to uncomfortable weight and short battery life in portable devices.

Innovation Solution

Implementing techniques for hand tracking in XR systems that involve generating an initial hand pose, scheduling a reprojection based on a hand pose transmission time, and transmitting the reprojected hand pose to a host device for rendering, thereby optimizing power consumption and reducing latency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If powerful processors are used to perform feature analysis and complex functions quickly, then processing speed and rendering performance are improved, but power consumption increases

Engineering Contradiction:

Improveprocessing speedVSAvoidpower consumption

Core Design Contradiction:

SpeedVSUse of energy by moving object

Solution Approach 1:

The system divides processing tasks between the HMD and companion device. The HMD performs lightweight hand tracking and feature extraction, while the companion device handles complex rendering and processing. This segmentation allows the HMD to use lower power processors while maintaining processing speed through distributed computation.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system performs feature extraction and hand pose generation in advance on the HMD before transmitting to the companion device for rendering. This preliminary action on the HMD reduces the computational burden during real-time rendering, improving overall processing speed while allowing the HMD to use less power.

Inventive Principle:

Principle #10Preliminary action

2Duration of action of moving object

If large batteries are included to extend battery life, then duration of action is improved, but weight increases making the device uncomfortably heavy

Engineering Contradiction:

Improvebattery lifeVSAvoiddevice weight

Core Design Contradiction:

Duration of action of moving objectVSWeight of moving object

Solution Approach 1:

The system segments computational tasks between HMD and companion device, allowing the HMD to use a smaller, lighter battery since it only needs to power lightweight hand tracking and feature extraction, not full rendering. The companion device handles the power-intensive rendering tasks.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The HMD creates lightweight copies of essential tracking data (hand poses, feature points) and transmits them to the companion device for rendering. This allows the HMD to use minimal processing power and smaller battery while maintaining full functionality through data copying rather than full computation.

Inventive Principle:

Principle #26Copying

3Use of energy by moving object

If processing tasks are split between companion device and HMD to reduce power consumption, then power consumption is reduced, but latency increases between frame rendering and display

Engineering Contradiction:

Improvepower consumptionVSAvoidlatency

Core Design Contradiction:

Use of energy by moving objectVSLoss of time

Solution Approach 1:

The HMD performs feature extraction and hand pose generation in advance and transmits this data to the companion device. The companion device uses this pre-processed data for rendering, which reduces the computational time needed and minimizes latency despite the split architecture.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements a feedback loop where the HMD continuously tracks hand poses and feeds this information to the companion device for real-time rendering adjustments. This continuous feedback ensures that latency is minimized by constantly updating the rendering based on current hand positions rather than processing everything in batches.

Inventive Principle:

Principle #23Feedback

4Ease of operation

If the HMD is made lightweight and small to improve portability, then ease of operation is improved, but computational power is reduced

Engineering Contradiction:

ImproveportabilityVSAvoidcomputational power

Core Design Contradiction:

Ease of operationVSPower

Solution Approach 1:

The system segments computational responsibilities so the lightweight HMD handles only essential tracking functions (hand pose estimation, feature extraction) while the companion device performs computationally intensive rendering. This allows the HMD to be portable while maintaining adequate computational power through task division.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The HMD acts as an intermediary that captures hand movements and transmits this data to the companion device for rendering. This intermediary role allows the HMD to remain lightweight while still enabling full computational functionality through the companion device partnership.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS12572197B2Hand tracking and power optimization for extended reality (XR) systems

Publication Date: 2026.03.10 QUALCOMM INC

AI summary

Techniques and systems are provided for processing sensor data. For instance a process can include obtaining an image, the image including a hand; generating an initial hand pose of the hand based on the image; scheduling a time to perform a reprojection of the initial hand pose based on a hand pose transmission time; reprojecting the initial hand pose to a reprojected hand pose at the scheduled time; and transmitting, at the hand pose transmission time, the reprojected hand pose to a host device for rendering a frame based on the reprojected hand pose.