Multi-modal Input Device Tracking via Magnetic and Visual Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for tracking input devices in computer vision technologies face challenges when the input device is occluded, leading to operational issues such as latency and failure in accurately determining the device's movement.
Innovation Solution
The implementation of a system that uses image data from an electronic device to identify and track an input device via a tracking device, which can be stationary, utilizing magnetic or optical tracking to determine the relative positioning between the tracking device and the input device, allowing for continued tracking even when one device is occluded.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If visual object tracking is used to track the input device, then the tracking system can identify the device in visible conditions, but the system fails or experiences latency when the device is occluded
Solution Approach 1:
The patent combines multiple tracking modalities (visual tracking from the electronic device and magnetic/optical tracking from the stationary tracking device) into a unified tracking system. This merging allows the system to maintain continuous tracking of the input device by switching between or fusing data from different tracking methods, ensuring reliability even when one modality fails due to occlusion.
Solution Approach 2:
The stationary tracking device acts as an intermediary that provides alternative tracking data when the primary visual tracking path is blocked. This intermediary tracking device uses magnetic or optical fields to track the input device, serving as a mediator that bridges the gap when direct visual tracking is unavailable, thereby maintaining system reliability without requiring complex active countermeasures.
2Reliability
If magnetic or optical tracking is used via a stationary tracking device, then tracking can continue during occlusion, but the device complexity increases
Solution Approach 1:
The tracking system is segmented into two independent components: a mobile electronic device with visual sensors and a stationary tracking device with magnetic/optical sensors. This segmentation allows each component to specialize in one tracking modality, reducing the complexity burden on each individual device while collectively providing robust multi-modal tracking. The stationary device handles occlusion scenarios without requiring the mobile device to carry additional complex sensors.
3Reliability
If multi-modal tracking is implemented to maintain tracking during occlusion, then reliability improves, but the computational requirements and processing complexity increase
Solution Approach 1:
The patent extracts and separates the computational processing tasks between different devices. The stationary tracking device performs local processing of magnetic/optical tracking data and transmits only essential positional information to the electronic device. This extraction of processing responsibilities reduces the computational burden on the mobile electronic device, allowing it to focus on visual tracking and fusion without handling all raw sensor data, thereby managing complexity while maintaining reliability.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution enhances the accuracy of input device tracking by combining image and depth data to maintain tracking functionality even when the input device is not directly visible, improving precision and reliability in extended reality environments.
Implementation Method 1
utilizing magnetic or optical tracking to determine the relative positioning between the tracking device and the input device
Implementation Method 2
utilizing magnetic or optical tracking to determine the relative positioning between the tracking device and the input device
Implementation Method 3
determining a pose of a tracking device in a physical environment based on first sensor data from an image sensor
Data Source
AI summary
Various implementations disclosed herein include devices, systems, and methods that track a movement of an input device. For example, an example process may include determine a pose of a tracking device in a physical environment based on first sensor data from an image sensor. The process then may receive, from the tracking device, first positional data corresponding to a first relative positioning between the tracking device and an input device in the physical environment, where the first positional data is determined based on second sensor data obtained via a sensor on the tracking device. The process then may track movement of the input device in the physical environment based at least in part on the first positional data and the pose of the tracking device. The process then may determine an input for the electronic device based at least in part on tracking the movement of the input device.