Collaborative XR Hand Tracking Using External Data Streams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
XR devices face challenges in accurately tracking objects, such as hands, when they exit the field of view or become occluded, leading to computational inefficiencies and rapid battery drain.
Innovation Solution
An imaging device utilizes data streams from external devices to merge with captured images, allowing it to determine the location of objects even when they are not in view, and generates outputs based on this merged data to maintain accurate tracking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If hand tracking is performed using only the XR device's image sensors, then tracking accuracy within the field of view is maintained, but the device experiences rapid battery drain and computational overload when hands exit the FOV or become occluded
Solution Approach 1:
The patent combines data from multiple sources including the XR device's image sensors and external device data streams to perform collaborative tracking. This merging allows the system to maintain tracking accuracy when hands exit the field of view or become occluded, while distributing computational load to reduce battery consumption on the XR device.
Solution Approach 2:
The patent introduces external devices as intermediaries that capture and transmit data streams containing hand tracking information. These external devices act as mediators that relieve the XR device from continuous high-power image processing, enabling the XR device to maintain tracking reliability without rapid battery drain.
2Reliability
If the XR device continuously processes image data to maintain accurate hand tracking, then tracking reliability is improved, but device complexity and computational requirements increase
Solution Approach 1:
The patent segments the tracking system into multiple independent components: the XR device's image sensors, external devices with their own sensors, and a coordination mechanism that integrates data from both. This segmentation distributes computational complexity across multiple devices, allowing each to perform simpler tasks while maintaining overall tracking continuity.
Solution Approach 2:
The patent creates a universal tracking system that can operate in multiple modes: using only XR device sensors, using only external device data streams, or combining both sources. This multi-functionality allows the system to adapt to different computational resource availability while maintaining tracking reliability.
3Use of energy by moving object
If the XR device uses external device data streams for tracking, then battery life is conserved, but tracking precision may be reduced when the object is within the XR device's field of view
Solution Approach 1:
The patent implements dynamic switching between different data sources based on real-time conditions. When the hand is within the XR device's field of view and detection conditions are good, the system prioritizes XR device sensor data for higher precision. When the hand exits the FOV or detection quality degrades, the system dynamically transitions to using external device data streams, conserving battery while maintaining acceptable tracking precision.
Data Source
AI summary
An imaging system can receive an image of a portion of an environment. The environment can include an object, such as a hand or a display. The imaging device can identify a data stream from an external device, for instance by detecting the data stream in the image or by receiving the data stream wirelessly from the external device. The imaging device can detect a condition based on the image and/or the data stream, for instance by detecting that the object is missing from the image, by detecting that a low resource at the imaging device, and/or by detecting visual media content displayed by a display in the image. Upon detecting the condition, imaging device automatically determines a location of the object (or a portion thereof) using the data stream and/or the image. The imaging device generates and/or outputs content that is based on the location of the object.