Handheld Device Visual Tracking with Fiducials and IMU
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems face challenges in achieving high-accuracy localization of peripheral devices, particularly handheld devices, due to limitations in current electromagnetic tracking systems.
Innovation Solution
A visual tracking system using imaging devices to capture fiducial markers on wearable and handheld devices, combined with inertial measurement units, to determine the position and orientation of the handheld device relative to the wearable device, employing different data processing schemes based on the number of fiducials detected.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If electromagnetic tracking systems are used for localization, then the localization function is provided, but the localization accuracy is insufficient
Solution Approach 1:
The patent replaces electromagnetic tracking systems with a visual tracking system using cameras to capture fiducial markers. This substitution achieves higher localization accuracy (improving measurement precision) while maintaining tracking reliability through robust computer vision algorithms and multiple fiducial detection
Solution Approach 2:
The patent uses fiducial markers (visual copies/reference objects) attached to handheld devices that can be captured and processed by cameras. These fiducials serve as optical replicas that enable precise localization without requiring electromagnetic fields, resolving the contradiction between accuracy and reliability
2Measurement precision
If visual tracking with fiducials is used, then localization accuracy is enhanced, but the device complexity increases
Solution Approach 1:
The patent extracts the tracking functionality into separate components: fiducial markers on handheld devices and camera-based detection on wearable devices. This extraction achieves high localization accuracy while managing complexity by separating the tracking elements from the core processing system
Solution Approach 2:
The patent uses existing cameras in wearable devices for dual purposes: environmental sensing and fiducial detection. This multi-functionality approach enhances localization accuracy without proportionally increasing device complexity, as the same hardware serves multiple functions
3Measurement precision
If multiple fiducials are required for accurate tracking, then localization precision improves, but the difficulty of detecting and measuring increases
Solution Approach 1:
The patent employs fiducial markers with distinct visual characteristics (colors, patterns, geometries) that make them easily distinguishable by camera-based detection systems. These visual variations improve tracking precision while simplifying detection through robust color and pattern recognition algorithms
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enhances localization accuracy of handheld devices in augmented reality systems, reducing reliance on electromagnetic tracking and improving the integration of virtual content with real-world imagery.
Implementation Method 1
obtain a first image of a first plurality of fiducials affixed to the wearable device and a second image of a second plurality of fiducials affixed to the handheld device
Implementation Method 2
detecting, by an inertial measurement unit (IMU) mounted to the handheld device, linear accelerations and rotational velocities of the handheld device
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Techniques are disclosed for performing localization of a handheld device with respect to a wearable device (302). At least one sensor mounted to the handheld device, such as an inertial measurement unit (IMU), may obtain handheld data indicative of movement of the handheld device with respect to the world. An imaging device mounted to the wearable device (302) may capture a fiducial image containing a number of fiducials affixed to the other device. The number of fiducials contained in the image are determined. Based on the number of fiducials, at least one of a position and an orientation of the handheld device with respect to the wearable device (302) are updated based on the image and the handheld data in accordance with a first operating state, a second operating state, or a third operating state.