XR Hand Scale Estimation Using Mobile Device Calibration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional hand-tracking technologies in XR systems are cumbersome and lack accuracy due to scale ambiguity, particularly in mobile settings, where the same visual cues can represent different actual hand sizes depending on distance and angle, affecting interaction quality.
Innovation Solution
An XR system uses a user's interaction with a mobile device equipped with cameras and inertial motion sensors to facilitate initial calibration and ongoing adjustment of hand scale estimations through specific gestures, leveraging existing hardware for simplified calibration and enhanced portability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional hand-tracking technologies are used in XR systems, then the system can provide basic hand tracking functionality, but the measurement precision of hand scale is poor due to scale ambiguity
Solution Approach 1:
The patent performs hand scale calibration before actual XR interactions by capturing images of the user's hand interacting with a mobile device. This preliminary calibration establishes accurate scale factors that are stored and applied during subsequent XR sessions, eliminating the need for continuous complex calibration while maintaining high measurement precision
Solution Approach 2:
The patent introduces a mobile device as an intermediary calibration tool between the XR system and the user's hand. The mobile device serves as a known reference object with measurable dimensions, allowing the system to calculate accurate hand scale factors through image analysis without requiring complex dedicated calibration hardware
2Adaptability or versatility
If traditional hand-tracking technologies are used, then the system can operate in mobile settings, but the interaction quality deteriorates due to scale ambiguity from distance and angle variations
Solution Approach 1:
The system performs hand scale calibration before XR interactions by capturing images of the user's hand interacting with a mobile device. This preliminary calibration establishes accurate scale factors that are stored and applied during subsequent XR sessions, eliminating the need for continuous complex calibration while maintaining high measurement precision
Solution Approach 2:
The system uses feedback from the calibrated hand scale factors to continuously adjust and refine hand tracking accuracy. By comparing expected hand dimensions with actual captured images and applying the calibrated scale factors, the system compensates for variations in distance and angle, maintaining reliable interaction quality across different mobile settings
3Measurement precision
If complex calibration systems are implemented to improve hand scale accuracy, then measurement precision improves, but ease of operation deteriorates due to cumbersome calibration procedures
Solution Approach 1:
The patent makes the mobile device serve multiple functions: it acts as both the user's regular communication device and as the calibration reference tool. The mobile device's known physical dimensions are leveraged for calibration, eliminating the need for separate dedicated calibration hardware and simplifying the overall system while maintaining high measurement precision
Solution Approach 2:
The system enables users to perform their own hand scale calibration using their personal mobile devices without requiring operator intervention or specialized equipment. The calibration process is designed to be user-friendly, allowing users to simply position their hand over the mobile device and follow on-screen instructions, making the system accessible and easy to operate
Data Source
AI summary
An XR system is provided that enhances user interaction within extended reality environments through precise hand scale estimation. The XR system is configured to capture tracking data of a user's hand as the user interacts with a mobile device. Concurrently, the XR system captures pose data of itself and uses the tracking data and the pose data to determine a reference line segment. This segment aids in calculating three-dimensional distances between node pairs of the user's hand. By employing these measurements, the XR system effectively calculates a hand scale factor that is used for accurately integrating the user's hands into an XR user interface.