Epipolar Line Pixel Displacement for Panoramic Image Stitching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Panoramic video stitching using multiple cameras faces challenges due to parallax errors caused by displacement of image sensors, leading to mismatches between images from individual cameras, which existing methods struggle to address effectively.
Innovation Solution
A computerized system with a capture device and processor that transforms and combines source images from multiple lenses, applying color correction, exposure correction, and disparity measurement to align pixels along epipolar lines, reducing parallax errors through discrete refinement and spatial smoothing operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of moving object
If multiple cameras with displaced image sensors are used to capture panoramic video, then the field of view coverage is improved, but parallax errors cause mismatch between images from individual cameras
Solution Approach 1:
The patent applies preliminary action by performing color correction and exposure correction on source images before the stitching process. The system corrects chromaticity and brightness differences between overlapping regions in advance, ensuring that images from multiple cameras with different sensors and lenses are color-matched and exposure-matched before alignment and combining, thereby preventing color and brightness mismatches in the final panoramic image
Solution Approach 2:
The patent replaces mechanical alignment methods with computational image processing. Instead of relying on precise mechanical positioning of multiple cameras, the system uses digital image transformation, disparity measurement, and pixel displacement along epipolar lines to achieve sub-pixel alignment accuracy. This computational approach substitutes for mechanical precision and effectively compensates for sensor displacement and lens distortion
2Area of moving object
If image transformation and combining operations are applied to create panoramic images, then the field of view is widened, but computational complexity increases
Solution Approach 1:
The patent applies segmentation by dividing the panoramic stitching process into distinct modular operations: color correction, exposure correction, disparity measurement, epipolar line identification, pixel displacement, and final combining. Each operation processes specific aspects of the image data independently, allowing for optimized computation and reducing overall complexity compared to attempting to solve all alignment issues in a single complex transformation
Solution Approach 2:
The patent changes parameters by working in the epipolar line coordinate system rather than standard image coordinates. By transforming the problem into epipolar space, the system reduces the 2D alignment problem to 1D displacement along epipolar lines, significantly reducing computational complexity. The system also adjusts displacement magnitude based on measured disparity values, applying minimal necessary corrections rather than uniform transformations
3Manufacturing precision
If pixels are displaced to align images from multiple cameras, then parallax errors are reduced, but image distortion may occur
Solution Approach 1:
The patent applies local quality by performing pixel displacement operations selectively along epipolar lines rather than applying uniform transformation across the entire image. The displacement magnitude is determined locally based on disparity measurements at each epipolar line position, allowing precise alignment at each location while preserving the overall geometric structure. This localized approach prevents global distortion that would result from uniform transformation
Data Source
AI summary
Multiple images may be combined to obtain a composite image. Individual images may be obtained with different camera sensors and/or at different time instances. In order to obtain the composite image source images may be aligned in order to produce a seamless stitch. Source images may be characterized by a region of overlap. A disparity measure may be determined for pixels along a border region between the source images. A warp transformation may be determined using a refinement process configured to determine displacement of pixels of the border region based on the disparity. Pixel displacement at a given location may be constrained to direction configured tangential to an epipolar line corresponding to the location. The warp transformation may be propagated to pixels of the image. Spatial and/or temporal smoothing may be applied. In order to obtain refined solution, the warp transformation may be determined at multiple spatial scales.