Electronic Image Stabilization for High Dynamic Range Rolling Shutter Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for electronic image stabilization (EIS) in high dynamic range (HDR) imaging with rolling shutter image sensors either require increased bandwidth and processing power when stabilizing individual exposures or result in reduced image quality when stabilizing merged exposures, due to difficulties in accurately accounting for camera motion across different exposure times.
Innovation Solution
A method that receives and merges images captured with different exposure times by rolling shutter image sensors, determines the source of each pixel block in the HDR image, and applies electronic image stabilization using motion data specific to each exposure time, with blending of motion data for mixed pixel blocks to achieve coherent stabilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If EIS is performed on each exposure individually before merging, then image stabilization quality is improved, but bandwidth requirements and processing power increase
Solution Approach 1:
The patent segments the image processing by identifying which pixels in the merged HDR image originate from the first exposure and which from the second exposure. This segmentation allows selective application of stabilization transformations based on the source exposure, enabling quality stabilization without processing entire images multiple times.
Solution Approach 2:
The patent applies local quality by using different stabilization transformations for different regions of the image. Specifically, pixels from the first exposure use a first transformation while pixels from the second exposure use a second transformation, allowing each region to be stabilized according to its specific motion characteristics.
2Quantity of substance
If EIS is performed on merged exposures after merging, then bandwidth efficiency is improved, but image stabilization quality deteriorates
Solution Approach 1:
The patent segments the merged HDR image to identify the origin of each pixel block, allowing the system to apply appropriate stabilization transformations without needing to process separate exposures through the entire pipeline. This maintains bandwidth efficiency while preserving stabilization quality.
Solution Approach 2:
The patent performs preliminary stabilization transformations on the individual exposures before merging, but only stores the transformation parameters rather than the fully processed images. This preliminary action enables quality stabilization while avoiding the bandwidth cost of storing and processing multiple complete images.
3Measurement precision
If separate exposures are stored in memory until EIS is performed, then processing time is increased, but stabilization accuracy is improved
Solution Approach 1:
The patent extracts only the necessary information (pixel origin identification and transformation parameters) from the separate exposures, rather than storing and processing the complete images. This extraction approach maintains stabilization accuracy by preserving motion information while significantly reducing memory requirements and processing time.
Solution Approach 2:
The patent uses transformation parameters that are calculated from the motion data of each exposure, allowing the stabilized image to be constructed by applying these transformations during the merge process rather than performing full EIS processing on stored images.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present invention relates generally to an algorithm for electronic image stabilization, EIS for a high dynamic range, HDR, image captured by a rolling shutter image sensor and more specifically to an algorithm configured to perform EIS on the HDR image, by, for each specific block of pixels among the blocks of pixels in the HDR image, the specific block of pixels spatially corresponding to a pixel region of the rolling shutter image sensor, determining whether the specific block of pixels is copied from pixel data of the first image or the second image, or is a blend of pixel data from both the first image and the second image, and use motion data measured when capturing the first and second images for performing EIS based on the origin of the specific block of pixels.