Environment Map Fusion for Robust Object Pose Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems face challenges in robustly tracking objects, especially when they become small, partially occluded, or move out of the camera's field of view, leading to inaccurate pose estimation and tracking jitter.
Innovation Solution
The system utilizes a dynamically generated 3D environment map data to estimate the target pose, fusing it with the environment pose to improve tracking robustness and extend the trackable range beyond the visible limits, even for moving targets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional target-only tracking methods are used, then the system is simple to implement, but tracking reliability deteriorates when the target is small, occluded, or out of view
Solution Approach 1:
The patent combines target-based tracking with environment-based SLAM tracking into a unified system. The environment map is built by integrating target map data with dynamically captured environment features, creating a composite tracking system that leverages both target appearance and surrounding environment geometry to maintain tracking reliability under various conditions.
Solution Approach 2:
The environment map serves as an intermediary structure that connects the camera to the target. When direct target tracking fails, the system uses the pre-built environment map to estimate camera pose and infer target pose, acting as a mediator that maintains tracking continuity even when the target is occluded or out of view.
2Measurement precision
If target-based tracking is used, then the system has simple data requirements, but measurement precision deteriorates for distant or small targets
Solution Approach 1:
The patent transitions from 2D image-space target tracking to 3D environment-space tracking. By building a three-dimensional environment map from multiple views and using it for pose estimation, the system extends accurate tracking to distant targets that would be too small or feature-poor in the camera view, leveraging the additional spatial dimension of the environment structure.
3Adaptability or versatility
If pre-built SLAM maps are used, then the system has fast runtime performance, but adaptability deteriorates for dynamic environments or moving targets
Solution Approach 1:
The system performs preliminary environment mapping during an initialization phase, capturing and processing environment features before runtime tracking begins. This pre-built environment map enables fast runtime performance, while the system maintains adaptability by using the flexible SLAM framework that can accommodate dynamic changes and moving targets during operation.
Data Source
Figure 1A~1B
Figure 2
Figure 3
AI summary
A computer-implemented method of tracking a target object in an object recognition system includes acquiring a plurality of images with a camera. The method further includes simultaneously tracking the target object and dynamically building environment map data from the plurality of images. The tracking of the target object includes attempting to estimate a target pose of the target object with respect to the camera based on at least one of the plurality of images and based on target map data. Next, the method determines whether the tracking of the target object with respect to the camera is successful. If not, then the method includes inferring the target pose with respect to the camera based on the dynamically built environment map data. In one aspect the method includes fusing the inferred target pose with the actual target pose even if tracking is successful to improve robustness.