Robust Camera Pose Estimation for Image Stitching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image stitching methods face challenges in accurately estimating camera poses, especially when dealing with large numbers of images or poorly matched feature points, leading to distortion and visual mismatch in stitched images.
Innovation Solution
A robust camera pose estimation method that constructs relative camera pose matrices and exploits their underlying rank-2 structure and joint sparsity, using a novel optimization approach to improve stitching performance even under abnormal camera pose errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If pairwise image stitching methods are used, then the method works well for a small number of images with explicit matching points, but when the total number of images is large or feature points are not well matched, false matching points cause abnormal camera pose error that propagates to subsequent estimates and deteriorates stitching performance
Solution Approach 1:
The patent combines multiple reference images into a bundle adjustment framework, merging information from multiple overlapping images simultaneously rather than processing pairs sequentially. This integration allows the system to leverage multiple matching points across the bundle, reducing the impact of false matches in any single pair and improving overall camera pose estimation reliability.
Solution Approach 2:
The patent implements an iterative optimization process where camera pose estimates are refined through multiple passes. The system uses feedback from the bundle adjustment process to correct initial pose estimates, progressively reducing errors and improving the accuracy of camera pose measurements through repeated refinement cycles.
2Reliability
If bundle adjustment is used to improve robustness by considering matching feature points between a target image and a bundle of overlapping reference images, then the overall robustness is improved due to the increase of matching points, but false matching points still inevitably cause camera pose errors and performance is not quite satisfactory for high precision stitching
Solution Approach 1:
The patent modifies the bundle adjustment objective function by introducing weight parameters that adaptively adjust the influence of different matching points. By changing these parameters dynamically during optimization, the system can down-weight suspicious or false matches while maintaining the robustness benefits of using multiple reference images, thereby improving precision without sacrificing reliability.
Solution Approach 2:
The patent transforms the static bundle adjustment process into a dynamic optimization procedure where the system adapts during execution. The algorithm dynamically identifies and adjusts for potential false matches through iterative refinement, allowing the system to maintain robustness while progressively improving precision by learning from the data during the optimization process.
Data Source
AI summary
An image forming system for constructing a whole large image of an object is provided. The image forming system includes an interface configured to sequentially acquire images of partial areas of the object obtained by a camera, wherein two adjacent images of the partial areas include overlap portions, wherein the sequential images correspond to a three dimensional (3D) surface of the object, wherein the geometrical information (geometrical parameters) of the object is provided, wherein an initial pose of the camera is provided. The system further includes a memory to store computer-executable programs including a pose estimation method and a stitching method, and an image stitching processor configured to perform steps of estimating camera poses with respect to the sequential images by the robust camera pose estimation method which imposes low-rankness and joint sparsity of relative pose matrices, projecting the sequential images into a 3D surface based on the camera pose estimations, and constructing a two-dimensional (2D) image of the object by interpolating projected surface images; outputting the constructed 2D image via the interface.