Seamless Image Stitching via Mesh Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image stitching techniques often result in artifacts and fail to produce seamless, smooth images, especially when merging images with significant misalignment or parallax, leading to distorted portions and poor image quality in 3D applications.
Innovation Solution
The method involves warping and defishing images onto a sphere, performing edge detection using Sobel gradients, and applying grid optimization with spring constraints to align edge maps, minimizing differences in normalized gradients, and then blending the images using seam estimation, while considering geometric and local distortions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If simple blending by superimposing images is used, then the process is simple, but the overlapping portions cannot be seamlessly merged due to perspective differences
Solution Approach 1:
The patent divides the image stitching process into multiple stages: initial rough alignment using homography, followed by local refinement using mesh optimization. This segmentation allows each stage to focus on specific aspects - global structure first, then local details - resolving the contradiction between process simplicity and alignment precision.
Solution Approach 2:
The patent employs dynamic mesh optimization where the mesh structure is iteratively adjusted to minimize distortion. The mesh deformation is computed dynamically based on feature point correspondences and homography transformations, allowing the system to adapt to perspective differences while maintaining computational efficiency.
2Manufacturing precision
If content-preserving warping is applied to refine alignment, then alignment precision improves, but image distortion in other portions increases
Solution Approach 1:
The patent applies local quality by using a mesh structure that allows different regions of the image to be transformed independently. The mesh optimization locally adjusts each vertex and edge to minimize distortion in specific areas while preserving the overall image structure, thus achieving precise alignment without excessive global distortion.
Solution Approach 2:
The patent performs preliminary homography alignment before mesh optimization. This preliminary action establishes a rough alignment framework that guides subsequent local refinements, ensuring that the mesh optimization works within a already-aligned structure and minimizes additional distortion.
3Ease of manufacture
If global 2D homography alignment is used, then the alignment process is simple, but parallax differences cause misalignment in overlapping regions
Solution Approach 1:
The patent segments the alignment process into global homography transformation and local mesh optimization. The global homography handles the coarse alignment, while the local mesh optimization specifically addresses parallax-induced misalignments in overlapping regions by independently adjusting mesh vertices based on feature point correspondences.
Solution Approach 2:
The patent transitions from 2D homography transformation to a 3D mesh structure for optimization. By representing the image as a deformable mesh with vertices, edges, and faces, the system can accommodate parallax differences through 3D spatial transformation while maintaining 2D visual appearance, thus resolving the limitation of planar homography.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
In one embodiment, a method includes accessing a first image and a second image, where at least part of the first image overlaps with at least part of the second image. The first and second images are divided into portions associated with a first set of grid points, where each grid point in the first set corresponds to a portion of the first image or the second image. Differences in the region of overlap between the first and second images are determined. One or more grid points in the first set and the corresponding portions of the first image or the second image are moved relative to one or more other grid points in the first set based on the determined differences.