Heterogeneous Sensor Calibration for Full-Body XR Motion Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing HMD trackers lack the capability to simultaneously track full body motions of users without requiring additional wearable sensors, complicating the registration and display of user motions across different sensor systems.
Innovation Solution
A method and apparatus for calibrating an outside-in HMD tracker with an RGB-D camera without using a separate calibration tool, enabling precise tracking of head, hand, upper body, and lower body motions by initializing and optimizing the coordinate systems of both sensors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If an outside-in HMD tracker is used to track head and hand motion, then head and hand motion tracking is achieved, but full body motion tracking capability is lost
Solution Approach 1:
The patent combines the HMD tracker and RGB-D camera into a unified tracking system. The HMD tracker provides precise head and hand motion data, while the RGB-D camera captures full body motion. By merging these two heterogeneous sensors and calibrating their coordinate systems, the system achieves both precise head/hand tracking and comprehensive full body tracking capability without requiring separate wearable sensors.
2Adaptability or versatility
If a separate wearable sensor is added to track full body motion, then full body motion tracking is achieved, but device complexity increases
Solution Approach 1:
The patent makes the RGB-D camera serve multiple functions: it tracks full body motion (upper and lower body) while the HMD tracker handles head and hand tracking. Instead of adding dedicated wearable sensors for each body part, the system uses the multi-functional RGB-D camera to capture comprehensive body motion data, reducing overall device complexity while maintaining full body tracking capability.
Solution Approach 2:
The patent introduces a coordinate system calibration mechanism as an intermediary between the HMD tracker and RGB-D camera. This calibration process aligns the coordinate systems of the two heterogeneous sensors, enabling seamless integration of their data. The calibration acts as a mediator that resolves the incompatibility between different sensor coordinate systems, allowing unified full body motion tracking without additional wearable sensors.
3Adaptability or versatility
If motion data from multiple heterogeneous sensors is integrated, then full body motion tracking is achieved, but coordinate system alignment difficulty increases
Solution Approach 1:
The patent employs a coordinate system calibration mechanism as an intermediary to align the HMD tracker and RGB-D camera coordinate systems. The calibration process establishes transformation relationships between the different coordinate systems, enabling seamless integration of motion data from both sensors. This intermediary calibration step resolves the complexity of directly integrating heterogeneous sensor data with different coordinate references.
Data Source
AI summary
A method and an apparatus for registering motion of a user tracked by heterogeneous sensors are disclosed. The method includes initializing the first sensor so that an origin of a first sensor coordinate system is adjacent to an origin of a device coordinate system, wherein the device coordinate system is a three-dimensional coordinate system with a current position of the device as the origin; and determining a calibration parameter for transforming coordinates of the plurality of feature points according to the first sensor coordinate system to coordinates according to a second sensor coordinate system, based on the position and orientation of the device in the second sensor coordinate system.