AR Registration via Multi-Modal Marker and Affine Transform
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems face challenges in accurately registering tracking systems with their visual space, especially when the graphics need to overlay on real-world environments and are not representative of visible objects.
Innovation Solution
The method involves acquiring images from cameras with known positions and orientations, estimating the 3D position of multi-modal marker devices, and computing an affine transform to register the tracking system's coordinate system with the augmented reality device's visual space.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional registration methods are used to align tracking systems with augmented reality displays, then the system complexity is reduced, but the registration precision and accuracy of graphics overlay deteriorate
Solution Approach 1:
The patent introduces a multi-modal marker device as an intermediary element that bridges the tracking system and augmented reality display. The marker contains both visual fiducial markers detectable by cameras and tracking sensors detectable by electromagnetic tracking systems, enabling precise registration between different coordinate systems through this mediating object
Solution Approach 2:
The multi-modal marker device performs multiple functions simultaneously: it serves as a visual reference for camera-based augmented reality systems, a tracking target for electromagnetic tracking sensors, and a spatial registration tool. This multi-functionality enables a single device to facilitate precise alignment across different measurement and visualization systems
2Ease of operation
If the augmented reality headset is fixed to a user for portable use, then the ease of operation is improved, but the difficulty of detecting and measuring the position and orientation of graphics deteriorates
Solution Approach 1:
The system continuously captures images from cameras, detects the position and orientation of the multi-modal marker, and uses this feedback to dynamically update the registration transform. This closed-loop feedback mechanism compensates for head movements and maintains accurate spatial alignment despite the portable, fixed-to-user configuration
Solution Approach 2:
The patent replaces complex mechanical registration systems with an optical-electromagnetic detection system. Instead of using mechanical alignment mechanisms, the system uses camera-based visual detection and electromagnetic tracking to automatically determine spatial relationships, simplifying the mechanical structure while improving measurement capability
3Adaptability or versatility
If computer-generated graphics are overlaid on real-world environments that are not representative of visible objects, then the adaptability of the augmented reality system is improved, but the measurement precision of the registration deteriorates
Solution Approach 1:
The multi-modal marker device serves as a stable intermediary reference that provides known spatial relationships between the tracking system and the augmented reality coordinate system. This mediator enables accurate registration even when the background environment lacks recognizable features or does not represent visible objects
Data Source
AI summary
An example method may include acquiring images from cameras, each having a known position and orientation with respect to a spatial coordinate system of an augmented reality (AR) device. The acquired images may include portions of a multi-modal marker device that includes at least one tracking sensor having a three-dimensional position that is detectable in a coordinate system of a tracking system. A three-dimensional position is estimated for the portions of the multi-modal marker device with respect to the spatial coordinate system of the AR device based on each of the respective acquired images and the known position and orientation of the cameras with respect to the spatial coordinate system of the AR device. The method also includes computing an affine transform configured to register the coordinate system of the tracking system with a visual space of a display that is in the spatial coordinate system of the AR device.