Rolling Shutter Compensation Multi-Camera Stitching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image stitching technologies face challenges in aligning pixels from multiple cameras with CMOS sensors, particularly due to the rolling shutter effect, which causes distortions and mismatched borders in panoramic images.
Innovation Solution
A computerized system that corrects the rolling shutter effect by determining corresponding rows and acquisition times in component images from multiple cameras, applying orientation information to transform images into an equirectangular plane, and interpolating sensor orientations to align pixels accurately.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If frame-level synchronization is used for multi-camera imaging, then the synchronization process is simple, but pixel mismatch and border artifacts occur in panoramic images due to rolling shutter effects
Solution Approach 1:
The patent segments the image acquisition process by row, assigning unique row identifiers and tracking acquisition times for each row across multiple cameras. This row-level segmentation enables precise temporal-spatial mapping of pixels from different cameras, resolving pixel mismatch issues while maintaining manageable complexity through systematic organization of acquisition data
Solution Approach 2:
The patent performs preliminary actions by capturing and storing metadata (row identifiers, acquisition times, camera identifiers) during the image acquisition phase. This preliminary data collection enables subsequent precise alignment operations without requiring complex real-time synchronization, thus maintaining ease of operation while achieving high pixel alignment accuracy
2Manufacturing precision
If rolling shutter correction is implemented with row-level timing analysis, then pixel alignment accuracy improves, but computational complexity and processing time increase
Solution Approach 1:
The patent introduces an intermediary data structure (row acquisition time metadata) that mediates between the raw rolling shutter data and the final alignment process. This intermediary layer simplifies the computational complexity by pre-organizing timing information, enabling efficient matching of corresponding rows across cameras without requiring complex real-time calculations
Solution Approach 2:
The patent creates copies of row data with associated metadata (acquisition time, row identifier, camera identifier) that can be independently processed and matched. This copying approach enables parallel processing and simplifies the alignment algorithm by working with discrete, tagged data units rather than complex continuous signals
3Area of stationary object
If multiple cameras with CMOS sensors are used for panoramic imaging, then field of view increases, but distortions and border artifacts appear due to rolling shutter effects
Solution Approach 1:
The patent implements feedback by continuously monitoring and recording acquisition times for each row across multiple cameras, then using this feedback information to dynamically adjust the alignment process. This feedback mechanism enables the system to compensate for rolling shutter distortions in real-time, maintaining high image quality across the expanded panoramic field of view
Data Source
AI summary
Images may be obtained using a moving camera comprised of two or more rigidly mounted image sensors. Camera motion may change camera orientation when different portions of an image are captured. Pixel acquisition time may be determined based on image exposure duration and position of the pixel in the image array (pixel row index). Orientation of the sensor may at the pixel acquisition time instance may be determined. Image transformation may be performed wherein a given portion of the image may be associated with a respective transformation characterized by the corrected sensor orientation. In some implementations of panoramic image acquisition, multiple source images may be transformed to, e.g., equirectangular plane, using sensor orientation that is corrected for the time of pixel acquisition. Use of orientation correction may improve quality of stitching by, e.g., reducing contrast of border areas between portions of the transformed image obtained by different image sensors.