6DOF AR Tracking on Weak Textures via Patch Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Augmented Reality technologies face challenges in accurately registering 3D content onto real-world surfaces, particularly with weakly textured surfaces and limited degrees of freedom, leading to instability and limited usability across various environments.
Innovation Solution
A direct method for 6DOF registration that uses coarse-to-fine image matching, allowing immediate tracking without pre-training, and capable of handling weakly textured surfaces by matching entire images or image patches, ensuring robust and accurate tracking across multiple complexity levels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the direct approach is used to match all pixels between frames, then comprehensive surface tracking is achieved, but computational intensity increases and illumination changes cannot be handled
Solution Approach 1:
The patent segments the image into a grid of patches and processes only selected patches rather than all pixels. This segmentation allows the system to handle comprehensive surface tracking while reducing computational intensity by focusing processing on informative regions only.
Solution Approach 2:
The patent applies local quality by differentiating the processing of different image regions. Selected patches that contain useful information are processed with full attention, while other regions are skipped. This creates local variations in processing quality that optimize the balance between tracking comprehensiveness and computational efficiency.
2Adaptability or versatility
If 6DOF registration is implemented, then full range of camera motions is supported, but sensitivity to local minima increases causing tracking failures
Solution Approach 1:
The patent performs preliminary 2DOF registration before attempting 6DOF registration. This preliminary action establishes a stable baseline and reduces the search space for the subsequent 6DOF optimization, thereby reducing sensitivity to local minima while maintaining full motion range support.
Solution Approach 2:
The patent implements a dynamic registration process that adaptively switches between 2DOF and 6DOF modes based on surface characteristics and tracking conditions. This dynamic approach allows the system to use simpler 2DOF registration when it provides sufficient stability, and switch to 6DOF when full motion support is needed, optimizing the balance between reliability and versatility.
3Measurement precision
If existing 6DOF methods are used, then theoretical registration is achieved, but robustness for mass market applications is insufficient
Solution Approach 1:
The patent performs preliminary 2DOF registration to establish a stable baseline before attempting 6DOF registration. This preliminary action reduces the complexity of the optimization problem and provides a reliable starting point, thereby improving both registration accuracy and product stability for mass market applications.
Solution Approach 2:
The patent introduces 2DOF registration as an intermediary step between initial camera positioning and full 6DOF tracking. This intermediary process provides a stable foundation that mediates the transition to 6DOF, making the overall system more robust and reliable while maintaining theoretical registration accuracy.
4Productivity
If local features approach is used, then real-time tracking is achieved, but usability is limited to well-textured surfaces
Solution Approach 1:
The patent extracts and processes only selected informative patches from the image rather than relying on extracting local features from the entire image. This extraction approach allows tracking on surfaces with weak or no texture by selecting patches that contain the most useful information, thereby improving surface type compatibility while maintaining real-time tracking speed.
Solution Approach 2:
The patent changes the fundamental parameter of what is being tracked from local features to patch-based intensity patterns. This parameter change enables the system to work on weakly textured surfaces by using intensity variations across patches rather than relying on distinctive local features, thereby expanding surface type compatibility while maintaining tracking efficiency.
Data Source
AI summary
The present system provides an on the fly simple to complex 6DOF registration approach using the direct method. On the fly means it does not require training time, a user points a phone/camera to a planar surface and can start tracking it instantly. Simple to complex means the system performs registration in multiple levels of complexity from 2DOF to 6DOF. By increasing the complexity model the system enables more surfaces to be tracked and for surfaces that are tracked the system can avoid local minima solution providing a more robust and accurate 6DOF tracking. Even surfaces that are very weak in features can be tracked in 6DOF and virtual content can be registered to them. The system enables playing Augmented Reality games on low-end devices such as mobile phones on almost any surface in the real world.