AR Camera Pose Tracking with 2D-2D and 3D-2D Key Points
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing SLAM systems face errors and tracking losses due to insufficient key points during camera pose estimation, particularly in fast movements, leading to inaccurate positioning and navigation.
Innovation Solution
A method combining 3D-2D and 2D-2D correspondences to enhance key point estimation by using previously tracked key points and triangulation, ensuring sufficient key points for accurate camera pose determination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If only 3D-2D correspondences are used for camera pose estimation, then the estimation process is simpler, but the number of key points is insufficient leading to tracking losses
Solution Approach 1:
The patent combines 3D-2D correspondences and 2D-2D correspondences into a unified camera pose estimation framework. The 2D-2D correspondence module tracks key points across consecutive image frames, supplementing the key points from 3D-2D triangulation. This merging ensures sufficient key point quantity for reliable tracking while maintaining estimation accuracy through the integrated optimization process.
2Reliability
If more key points are used for camera pose estimation, then tracking stability improves, but the computational complexity increases
Solution Approach 1:
The patent divides the camera pose estimation process into separate functional modules: a 3D-2D correspondence module for initial key point generation through triangulation, and a 2D-2D correspondence module for additional key point tracking. This segmentation allows each module to operate independently with optimized algorithms, managing computational complexity while achieving reliable tracking through combined results.
Solution Approach 2:
The system performs preliminary key point tracking using 2D-2D correspondences before final camera pose estimation. By pre-identifying and tracking key points across frames, the system prepares sufficient correspondence data in advance, reducing the computational burden during the actual pose estimation phase and improving overall processing efficiency.
3Quantity of substance
If 3D-2D and 2D-2D correspondences are combined, then key point quantity increases, but the estimation process becomes more complex
Solution Approach 1:
The patent implements a unified optimization framework that handles both 3D-2D and 2D-2D correspondences simultaneously. This multi-functional estimation process uses a single mathematical model to process different types of correspondences, eliminating the need for separate estimation pipelines and reducing overall process complexity despite handling multiple key point sources.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
Embodiments of the present application disclose a method for determining pose information of an image capturing device, including: acquiring, by an image capturing device, a first image of a target object; extracting, by the image capturing device, coordinate information of a first two-dimensional (2D) key point corresponding to the target object from the first image; acquiring, by the image capturing device, a second image of the target object; acquiring, by the image capturing device, coordinate information of a second 2D key point in the second image according to the coordinate information of the first 2D key point; acquiring, by the image capturing device, coordinate information of a three-dimensional (3D) key point corresponding to the target object; and determining, by the image capturing device, camera pose information of the second image according to the coordinate information of the first 2D key point, the coordinate information of the second 2D key point, and the coordinate information of the 3D key point. In the embodiments of the present application, more key points can be obtained during the estimation of camera pose information, so that errors in camera pose information are reduced, and a tracking loss because of an excessively small quantity of key points in tracking can be avoided.