XR Hand Pose Scale Estimation for Accurate Depth Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
XR devices face challenges in accurately tracking hands due to inconsistent hand scale estimates resulting from varying hand poses, which degrade the quality of XR experiences by causing errors in depth estimation and virtual content placement.
Innovation Solution
The XR device maintains a data structure with multiple hand scale estimates corresponding to different hand poses, dynamically switching between multi-camera and single-camera modes for calibration and tracking, using machine learning models to adapt to changing hand poses and improve tracking accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single hand scale estimate is used for all hand poses, then the device complexity is reduced, but the measurement precision of hand tracking deteriorates
Solution Approach 1:
The patent segments the hand tracking problem by maintaining multiple hand scale estimates, each corresponding to a specific hand pose. Instead of using a single generic scale estimate, the system divides the solution space into pose-specific segments, allowing accurate scale estimation for each pose category while keeping the overall system manageable through organized data structures.
Solution Approach 2:
The patent changes the parameter of hand scale estimation by adapting the scale value based on detected hand pose. The system selects or adjusts the hand scale estimate parameter according to the current pose, transforming a static parameter approach into a dynamic one that responds to changing conditions, thereby improving measurement precision without excessive complexity.
2Measurement precision
If multi-camera mode is used for hand scale calibration, then the measurement precision of hand scale estimate is improved, but the use of energy increases
Solution Approach 1:
The patent implements dynamic switching between multi-camera and single-camera modes based on the current operational context. The system dynamically adjusts its calibration approach, using multi-camera mode only when high precision is required or when pose changes demand updated scale estimates, while relying on single-camera mode during stable conditions to conserve energy.
Solution Approach 2:
The patent changes the operational parameter of camera mode selection based on hand pose detection and tracking requirements. By monitoring pose stability and accuracy requirements, the system transitions between high-power multi-camera calibration mode and low-power single-camera tracking mode, optimizing the balance between measurement precision and energy consumption.
3Loss of time
If hand scale estimate is not updated for pose changes, then the loss of time is reduced, but the reliability of depth estimation deteriorates
Solution Approach 1:
The patent implements a feedback mechanism that monitors hand pose changes and triggers hand scale estimate updates only when necessary. The system continuously feedbacks on pose detection results and compares them against stored pose representations, initiating recalibration only when significant pose changes are detected, thus balancing time efficiency with estimation reliability.
Solution Approach 2:
The patent prepares multiple hand scale estimates in advance for different hand pose categories. By pre-computing and storing pose-specific scale estimates during initial calibration, the system avoids time-consuming real-time calculations when poses change, quickly selecting the appropriate pre-prepared estimate while maintaining depth estimation reliability.
Data Source
AI summary
Examples in the present disclosure relate to scale estimation for facilitating extended reality (XR) experiences. An image of a hand of a user is captured via one or more optical sensors of an XR device. The image is processed to detect a hand pose relative to the XR device. A hand scale estimate corresponding to the detected hand pose is accessed. The hand scale estimate is one of a plurality of hand scale estimates each uniquely associated with a respective hand pose. The hand scale estimate is applied to generate positional data for one or more features of the hand of the user. The XR device tracks the hand of the user based on the positional data while the user uses the XR device.