Image Stitching via Non-linear Optimization and Multi-constraint Cost Function
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image stitching methods face challenges with lens distortion and parallax errors due to the use of linear algorithms, leading to misalignments in stitched images, especially for non-planar scenes.
Innovation Solution
The method employs non-linear least-square optimization and multi-constraint cost function minimization to determine global homography for each image pair using a Direct Linear Transformation (DLT) method with Random Sampling Consensus (RANSAC) and local warping techniques, incorporating data, photometric, and geometric smoothness constraints to refine the stitched image.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional linear algorithms are used for homography-based transformation, then the computation is simpler and faster, but lens distortion and parallax errors cause misalignments in non-planar scenes
Solution Approach 1:
The patent transitions from linear algorithms to non-linear least-square optimization algorithms, changing the mathematical parameter space from linear to non-linear. This enables the system to model lens distortion and parallax effects accurately while maintaining computational feasibility through iterative optimization methods.
Solution Approach 2:
The patent replaces the conventional mechanical/linear transformation model with a non-linear optimization-based model. By substituting the linear homography calculation with non-linear least-square optimization, the system can handle complex optical distortions and achieve higher alignment precision in non-planar scenes.
2Measurement precision
If direct image alignment approaches are used to determine homography using all overlapped pixel information, then comprehensive alignment is achieved, but execution time increases significantly
Solution Approach 1:
The patent segments the alignment process into two stages: first using feature-based methods to obtain initial homography estimates, then refining with non-linear optimization on selected correspondences. This segmentation reduces the computational burden compared to processing all pixel information directly while maintaining alignment accuracy.
Solution Approach 2:
Instead of using all overlapped pixel information, the patent selectively uses feature point correspondences and their associated error metrics in the non-linear optimization. This partial action approach focuses computational resources on the most informative data points, reducing execution time while preserving homography accuracy.
3Productivity
If feature-based image alignment approaches are used to calculate homography for each image pair, then the process is faster and more robust, but linear algorithms ignore lens distortion and parallax in non-planar scenes
Solution Approach 1:
The patent changes the algorithmic parameter space from linear to non-linear optimization, enabling the fast feature-based approach to also account for lens distortion and parallax effects. The non-linear least-square optimization refines the feature-based homography estimates while maintaining processing efficiency.
Solution Approach 2:
The patent maintains the continuous workflow of feature-based alignment by integrating non-linear optimization as a refinement step rather than a separate process. This continuity preserves the speed advantage of feature-based methods while continuously improving alignment accuracy through iterative optimization.
Data Source
AI summary
The present disclosure provides a system and a method for stitching images using non-linear optimization and multi-constraint cost function minimization. Most of conventional homography based transformation approaches for image alignment, calculate transformations based on linear algorithms which ignore parameters such as lens distortion and unable to handle parallax for non-planar images resulting in improper image stitching with misalignments. The disclosed system and the method generates initial stitched image by estimating a global homography for each image using estimated pairwise homography matrix and feature point correspondences for each pair of images, based on a non-linear optimization. Local warping based image alignment is applied on the initial stitched image, using multi-constraint cost function minimization to mitigate aberrations caused by noises in the global homography estimation to generate the refined stitched image. The refined stitched image is accurate and free from misalignments and poor intensities.