Hybrid Corner and Edge Tracking for AR Pose Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems face challenges in accurately detecting and tracking low feature objects, as they lack distinct features, making it difficult to establish stable keypoints and compute distinctive appearance-based descriptors for object detection and pose estimation.
Innovation Solution
The system employs a method involving template matching and edge-based registration, using normalized cross correlation to match a 2D template with the image frame, followed by iterative edge alignment to establish correspondences between 2D and 3D contour points, enabling accurate pose estimation and tracking of both rich and low textured objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If appearance-based descriptors (e.g., ORB descriptors) are used for feature matching, then tracking accuracy is improved for rich feature objects, but the method fails for low feature objects that lack distinctive appearance characteristics
Solution Approach 1:
The patent segments the feature detection process into two independent parts: corner detection (using algorithms like FAST or Harris corner detector) and edge detection (using algorithms like Canny edge detector). This segmentation allows the system to handle different object types separately - rich feature objects use appearance-based descriptors while low feature objects use geometric structure, resolving the contradiction between accuracy for rich features and versatility for low feature objects.
Solution Approach 2:
The patent creates a universal tracking system that can handle both rich feature objects and low feature objects through a unified framework. The system combines corner-based tracking (for structured objects) and edge-based tracking (for low feature objects) into a single multi-functional tracker, allowing one system to serve multiple object types that previously required separate approaches.
2Reliability
If corner-based tracking is used, then tracking is robust for structured objects, but drift occurs over time due to accumulation of errors
Solution Approach 1:
The patent merges corner-based tracking and edge-based tracking into a hybrid system. Corner tracking provides robustness for structured objects while edge tracking helps correct drift by providing additional geometric constraints. The combined system leverages the strengths of both approaches to maintain both robustness and precision over time.
Solution Approach 2:
The patent implements a feedback mechanism where edge information is used to correct and refine corner-based tracking results. The edge-based registration process provides feedback that adjusts the pose estimation, compensating for accumulated drift and maintaining accuracy over extended tracking sequences.
3Measurement precision
If template matching is used for object detection, then initial pose estimation is achieved, but the method cannot handle objects without distinctive templates
Solution Approach 1:
The patent segments the detection approach into template matching (for objects with distinctive features) and corner/edge-based detection (for objects without distinctive features). This segmentation allows the system to use appropriate methods for different object types, maintaining both initial pose estimation accuracy and versatility.
Solution Approach 2:
The patent performs preliminary corner and edge detection on the image frame before attempting template matching. This preliminary action identifies geometric structures that can serve as fallback tracking targets when template matching fails, ensuring the system is prepared to handle objects without distinctive features.
Data Source
AI summary
A method includes acquiring, from a camera, an image frame including a representation of an object, and retrieving from a memory, data containing a template of a first pose of the object. A processor compares the first template to the image frame. A plurality of candidate locations in the image frame having a correlation with the template exceeding a predetermined threshold is determined. Edge registration on at least one candidate location of the plurality of candidate locations is performed to derive a refined pose of the object. Based at least in part on the performed edge registration, an initial pose of the object is determined, and a display image is output for display on a display device. The position at which the display image is displayed and/or the content of the display image is based at least in part on the determined initial pose of the object.