Rolling Shutter Image Recomposition via Swath Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Digital cameras with rolling shutters face challenges in reducing motion-induced de-registration errors when capturing multi-frame images, leading to blurred images due to the exposure of different sensor regions at different times, causing motion blur and registration issues.
Innovation Solution
A method for recompositing multi-frame images using a digital camera with a rolling shutter, where the image is divided into horizontal swaths, and motion detection is performed using the Y′CbCr color space, with key lines and stride points to identify edge-like pixels, allowing for re-registration of swaths to correct camera motion, thereby reducing motion blur and registration errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If multi-frame compositing is used to reduce noise, then noise reduction is improved, but motion blur is not reduced and registration errors occur due to rolling shutter
Solution Approach 1:
The image is divided into multiple horizontal swaths, with each swath being processed independently for motion detection and re-registration. This segmentation allows precise local alignment of each swath across multiple frames, correcting rolling shutter artifacts while maintaining noise reduction benefits through multi-frame compositing.
Solution Approach 2:
Motion detection and re-registration are performed locally on key lines and stride points within each swath rather than globally across the entire image. This local processing approach accurately corrects motion-induced de-registration in each region while preserving the noise reduction effect of averaging multiple frames.
2Manufacturing precision
If rolling shutter is used to reduce motion blur per pixel, then individual pixel motion blur is reduced, but time-dependent de-registration occurs between different sensor regions
Solution Approach 1:
Motion detection is performed on key lines and stride points before final image composition. By detecting motion in advance and calculating re-registration offsets for each swath, the system corrects de-registration issues before combining multiple frames, ensuring both pixel-level sharpness and overall frame alignment.
Solution Approach 2:
The system uses edge-like pixels identified through motion detection as feedback to determine re-registration offsets. This feedback mechanism continuously adjusts the alignment of each swath based on detected motion, compensating for rolling shutter de-registration while maintaining the benefits of reduced individual pixel motion blur.
Data Source
AI summary
A rolling shutter digital camera. Each photographic image of a given exposure duration is captured as a multi-frame burst of frames each having a shorter exposure duration to minimize motion blur and to reduce sensor noise by averaging. Each frame is quantized into swaths, captured sequentially by the rolling shutter. Swaths of the first frames are analyzed to select a set of best motion detection reference regions. Swaths of subsequent frames are analyzed versus only those regions, to reduce required computation, and are re-registered accordingly. Corresponding swaths of each frame are accumulated. The accumulator is normalized to the desired bit depth and written as the final image. Averaging of the multiple frames is improved by re-registering swaths rather than entire frames, because of the time delta caused by the rolling shutter. Computation is further reduced by selecting candidate points only along a key line of each swath of the first frame, and still further by pre-limiting the search by using a thumbnail version of the first frame to cull most candidate points. Thresholds are used to ensure adequate motion detection regions are used for each swath.