Multi-Sensor Image Stitching with Rolling Shutter Warp Inversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Image stitching with electronic rolling shutters introduces distortions due to unsynchronized image capture times, causing motion artifacts and increased processing complexity, particularly in systems with multiple image sensors.
Innovation Solution
Invert the order of electronic rolling shutter correction and parallax correction by first determining warp mapping at lower resolution and then applying parallax correction at higher resolution, reducing processing demands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If electronic rolling shutter correction is applied before parallax correction in image stitching, then motion artifacts are reduced, but processing complexity and computing resources increase significantly
Solution Approach 1:
The patent inverts the conventional correction sequence by applying parallax correction first and then electronic rolling shutter correction. This reversal reduces processing complexity because the parallax correction aligns images at lower resolution initially, and the subsequent rolling shutter correction operates on already-aligned images, minimizing the computational burden while still achieving high image quality with reduced motion artifacts.
2Area of stationary object
If multiple image sensors are used for capturing images simultaneously, then field of view coverage is improved, but synchronization errors and motion artifacts increase
Solution Approach 1:
The patent applies parameter changes by introducing temporal offset parameters that account for the different capture times of each image sensor. By modeling and compensating for the time-dependent motion parameters of each sensor, the system can stitch images from multiple sensors with different fields of view while correcting for the synchronization errors and motion artifacts introduced by unsynchronized capture.
3Measurement precision
If electronic rolling shutter correction is applied at high resolution, then correction accuracy is improved, but processing time and computational resources increase
Solution Approach 1:
The patent segments the correction process into two distinct stages: first applying parallax correction at lower resolution to establish basic image alignment, then applying electronic rolling shutter correction at higher resolution to refine the alignment. This segmentation allows the computationally intensive high-resolution correction to operate on pre-aligned images, reducing the overall processing time while maintaining high correction accuracy in the final output.
Data Source
AI summary
Systems and methods are disclosed for image signal processing. For example, methods may include receiving a first image from a first image sensor, receiving a second image from a second image sensor; obtaining corrected images based on the first image and the second image; obtaining stabilized images based on the corrected images; applying a parallax correction to the stabilized images to obtain a composite image; obtaining a transformed image from the composite image; and encoding an output image based on the transformed image.