Hybrid Tracking Method for Real-Time Object Pose Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current object tracking algorithms in augmented reality are computationally costly and not robust enough to handle noise, rapid movements, and changes in brightness, making them unsuitable for implementation on low-power devices like mobile phones with mediocre image sensors.
Innovation Solution
A hybrid tracking method that uses key images and selective point of interest tracking, combining KLT-like algorithms with predictive pose estimation to efficiently follow object representations in a sequence of images without requiring significant computational resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If current object tracking algorithms are used in augmented reality, then position and orientation of virtual objects can be determined, but computational cost becomes too high for low-power devices
Solution Approach 1:
The patent segments the tracking process into two distinct phases: an initialization phase that uses computationally intensive recognition algorithms to identify objects and establish initial pose estimates, and a subsequent tracking phase that uses lighter-weight algorithms to follow object motion. This segmentation allows the system to use accurate but expensive algorithms only when necessary, while relying on efficient algorithms for continuous tracking, thereby reducing overall computational energy consumption on low-power devices.
Solution Approach 2:
The patent performs preliminary object recognition and pose estimation during an initialization phase before actual tracking begins. By pre-establishing object identities, initial positions, and orientation data, the system avoids the need to perform full recognition algorithms during real-time tracking. This preliminary action stores essential information that can be used by simpler tracking algorithms, significantly reducing computational energy requirements during the tracking phase while maintaining accurate pose estimation.
2Productivity
If traditional tracking algorithms are used, then object pose can be followed in real time, but robustness to noise and rapid movements is insufficient
Solution Approach 1:
The patent merges two complementary approaches: recognition-based tracking that uses object identity and pre-stored 3D model information, and traditional feature-based tracking that follows visual features through the video sequence. By combining these approaches, the system benefits from the robustness of recognition methods to noise and illumination changes, while maintaining the real-time performance of feature tracking. The integration allows the system to handle rapid movements and noisy conditions more reliably than either method alone.
Solution Approach 2:
The patent introduces 3D object models as an intermediary between the image data and the tracking process. These pre-acquired 3D models serve as a reference that mediates the matching process, allowing the system to interpret 2D image features in the context of known 3D object geometry. This intermediary enables more robust pose estimation under noisy conditions and rapid movements, as the 3D model provides structural constraints and expectations that guide the tracking algorithm even when visual features are temporarily obscured or distorted.
3Measurement precision
If recognition algorithms are used to identify objects, then object presence and approximate position can be determined, but processing time increases significantly
Solution Approach 1:
The patent segments the processing timeline into an offline initialization phase and an online tracking phase. During the offline phase, computationally intensive object recognition algorithms are executed to identify objects, extract their 3D models, and establish initial pose information. These results are stored for reuse. During the online tracking phase, the system uses this pre-computed information to rapidly update object positions without repeating the full recognition process, thereby minimizing processing time loss while maintaining accurate object identification.
Solution Approach 2:
The patent performs object recognition, 3D model extraction, and initial pose estimation as preliminary actions during an offline initialization phase before real-time tracking begins. By completing these computationally expensive tasks in advance, the system prepares all necessary object information (identities, 3D models, initial poses) that can be directly used during tracking. This preliminary action eliminates the need to perform recognition algorithms during time-critical real-time operation, significantly reducing processing time while preserving accurate object identification.
Data Source
Figure 1
Figure 2
Figure 3a~3b
AI summary
The invention in particular relates to the hybrid tracking of representations of objects in an image sequence using at least one key image. After acquiring a first and second image including a representation of the tracked object, a first image portion is identified in the first image, and a second image portion is retrieved from the key image. A relative position of a first image portion of said second image, similar to the first image portion of the first image, is estimated. A second image portion of the first or second image, similar to the second image portion of the key image, is sought. The relative position of the object is then estimated according to the relative positions of the first image portions and the second image portions.