Distorted Image Alignment Using Global Nonlinear Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image stitching methods fail to effectively align and unwarp images with excessive distortion, such as those captured with wide-angle or fisheye lenses, leading to poor quality panoramas due to lens distortions and inefficient processing.
Innovation Solution
A multi-stage image alignment and unwarping method that applies an initial unwarping function to feature points to generate substantially rectilinear coordinates, followed by global nonlinear optimization to refine unwarping functions and rotations, allowing for efficient stitching of distorted images into panoramic images without requiring intermediate rectilinear images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If conventional image stitching methods are used on distorted images, then the processing is simpler, but the alignment quality and panorama quality deteriorate due to lens distortions
Solution Approach 1:
The patent segments the image stitching process into distinct stages: distortion detection, unwarping, and stitching. By separating the unwarping operation from the stitching operation, the method can handle distorted images more effectively without overwhelming computational complexity at any single stage
Solution Approach 2:
The patent applies preliminary unwarping operations to correct lens distortions before performing the stitching process. By pre-correcting the distorted images using detected distortion parameters, the subsequent stitching operation works with corrected images, improving alignment quality without requiring complex distortion handling during stitching
2Manufacturing precision
If intermediate rectilinear images are generated during stitching, then the alignment accuracy improves, but the memory usage and computational intensity increase significantly
Solution Approach 1:
The patent extracts and applies only the necessary unwarping transformations to feature points and image coordinates without generating complete intermediate rectilinear images. By working with transformed coordinates and distortion parameters rather than full intermediate images, the method maintains alignment accuracy while reducing memory consumption
Solution Approach 2:
The patent operates in the parameter space of coordinate transformations rather than in the image space. By performing calculations on distortion parameters and transformation matrices rather than manipulating large intermediate image data, the method achieves accurate alignment with reduced computational intensity and memory usage
3Manufacturing precision
If lens distortion is not modeled, then the processing is faster and simpler, but the stitching quality deteriorates with visible artifacts
Solution Approach 1:
The patent introduces distortion parameters as additional variables in the stitching process. By modeling lens distortion through parameters such as radial distortion coefficients and applying these transformations during the stitching pipeline, the method improves stitching quality by correcting distortion-related artifacts while maintaining reasonable processing speeds through efficient parameter-based corrections
Data Source
AI summary
A method and apparatus for aligning and unwarping distorted images in which an initial unwarping function is applied to the coordinates of feature points of a set of input component images to generate substantially rectilinear feature points. The substantially rectilinear feature points are then used to estimate focal lengths, centers, and relative rotations for pairs of the input images. A global nonlinear optimization is applied to the initial unwarping function(s) and the relative rotations to generate optimized unwarping functions and rotations for the component images. The optimized unwarping functions and rotations may then be used to render a panoramic image, generally in the form of a spherical projection, from the input component images.