Rolling Shutter Image Distortion Correction via Timestamped Ego-Motion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Rolling shutter distortion in camera systems used for driver assistance and vehicle control systems complicates accurate ego-motion computation, as it causes varying camera motion across image frames, making real-time processing computationally prohibitive and affecting the performance of Structure-from-Motion algorithms.
Innovation Solution
The solution involves capturing multiple image frames with a rolling shutter camera, recording timestamps for corresponding image points, computing ego-motions based on these timestamps, and using iterative processes to correct image distortions by adjusting the ego-motion matrix to account for the rolling shutter effect, allowing for pixel-by-pixel or row-by-row corrections to achieve a global ego-motion representation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If rolling shutter is used for image acquisition, then real-time processing capability is improved, but image distortion occurs due to varying camera motion across frames
Solution Approach 1:
The patent segments the image acquisition process by row, with each row having its own timestamp. This segmentation allows the system to process and correct distortion row-by-row rather than requiring complete frame processing, enabling real-time correction while maintaining accuracy for each segment.
Solution Approach 2:
The patent applies preliminary action by computing ego-motion and applying distortion correction parameters to each row immediately after acquisition, before the next frame is captured. This preliminary correction prevents distortion accumulation and enables real-time processing without waiting for complete frame analysis.
2Measurement precision
If pixel-by-pixel ego-motion computation is performed to correct rolling shutter distortion, then measurement precision is improved, but computational complexity increases making real-time processing prohibitive
Solution Approach 1:
The patent segments the computational domain from pixel-level to row-level operations. By computing distortion correction parameters at the row level using timestamps rather than individual pixel coordinates, the system maintains measurement precision while reducing computational complexity by approximately 1000-fold compared to pixel-by-pixel computation.
Solution Approach 2:
The patent changes the computational parameters from pixel coordinates to row timestamps. This parameter transformation allows the system to compute ego-motion using temporal information (timestamps) rather than spatial information (pixel positions), significantly reducing computational complexity while preserving the ability to correct rolling shutter distortion accurately.
3Loss of information
If Structure-from-Motion algorithms are applied to rolling shutter images, then three-dimensional scene reconstruction is achieved, but algorithm performance deteriorates due to uncorrected distortion
Solution Approach 1:
The patent applies preliminary distortion correction to the rolling shutter images before they are fed into Structure-from-Motion algorithms. By pre-correcting the distortion using row-level timestamps and computed ego-motion parameters, the SfM algorithms receive undistorted input, ensuring reliable performance and accurate three-dimensional scene reconstruction.
Solution Approach 2:
The patent introduces an intermediary distortion correction step between image acquisition and Structure-from-Motion processing. This intermediary process uses timestamp-based ego-motion computation to generate correction parameters that are applied to the images, serving as a bridge that transforms distorted rolling shutter images into corrected images suitable for reliable SfM algorithm execution.
Data Source
AI summary
Correcting image distortion during camera motion using a system including a processor and a camera having a rolling shutter. Multiple image frames are captured by the camera equipped with the rolling shutter. The captured image frames include a base image frame and a previous image frame. Multiple time stamps are recorded respectively for multiple corresponding image points in the previous and base image frames. For the corresponding image points, multiple ego-motions are computed responsive to the time stamps of the corresponding image points of the base image frame and the previous image frame to correct the image distortion caused by the rolling shutter.