Camera Pose Refinement Using Height Map Orthographic Projection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing photogrammetry systems face challenges in accurately determining camera poses, especially when surface textures are weak, leading to difficulties in finding correspondences between images and resulting in inaccurate 3D model alignment and potential presence of specular reflections.
Innovation Solution
The method involves receiving images associated with height maps, determining mappings between them to form orthographic views, and refining camera poses based on alignment data and mutual information scores, using techniques such as homography decomposition and displacement fields to correct for parallax and improve spatial relationships between cameras.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If PnP techniques are used to recover camera poses from image correspondences, then camera pose can be determined, but accuracy deteriorates when surface texture is weak making correspondences difficult to find
Solution Approach 1:
The patent introduces a 3D model as an intermediary object that mediates between multiple images. Instead of directly finding correspondences between images (which is difficult when texture is weak), the system projects images onto a 3D model and uses the model's geometric structure as a common reference frame. This intermediary 3D model provides stable geometric features that facilitate accurate pose determination even when image textures are weak.
Solution Approach 2:
The patent transitions from 2D image space to 3D model space by projecting images onto a three-dimensional representation. This dimensional change allows the system to exploit the additional depth information and geometric constraints of the 3D model, providing more robust features for pose estimation compared to relying solely on 2D image correspondences.
2Measurement precision
If Mutual Information is used to assess alignment between images and 3D model, then camera pose can be determined, but the process requires a pre-existing 3D model which increases system complexity
Solution Approach 1:
The patent performs preliminary action by first constructing a 3D model from the input images before using it for pose determination. This preliminary 3D model construction creates a reusable geometric reference that simplifies subsequent pose estimation operations. The Mutual Information assessment is then performed in the context of this pre-established 3D model, improving accuracy while the one-time model construction cost is amortized across multiple pose determinations.
3Manufacturing precision
If orthographic projection is used to correct parallax, then image alignment improves, but the process becomes computationally intensive requiring precise depth information
Solution Approach 1:
The patent applies orthographic projection selectively to specific regions of the image based on depth information, rather than uniformly to the entire image. By identifying regions with significant parallax effects (typically foreground regions closer to the camera) and applying correction only there, the system achieves improved alignment precision while reducing overall computational requirements compared to full-image processing.
Data Source
AI summary
A system and method determining a camera pose. The method comprises receiving a first image and a second image, the first and second images being associated with a camera pose and a height map for pixels in each corresponding image, and determining a mapping between the first image and the second image using the corresponding height maps, the camera pose and a mapping of the second image to an orthographic view. The method further comprises determining alignment data between the first image transformed using the determined mapping and the second image and determining a refined camera pose based on the determined alignment data and alignment data associated with at least one other camera pose.