HDR Image Alignment Using Rolling Shutter Motion Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Electronic image capture devices suffer from limited dynamic range, leading to artifacts such as wobbling and loss of detail in high contrast scenes due to mismatched object locations in images captured at different exposure times.
Innovation Solution
Applying rolling shutter correction (RSC) and electronic image stabilization (EIS) to align and fuse image frames captured at different exposure times, reducing distortion and noise through motion data correction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Illumination intensity
If multiple image frames are captured at different exposure times to improve dynamic range, then the device can obtain images with details in both low light and high light portions, but objects may move or the device may move between captures causing misalignment and wobbling artifacts
Solution Approach 1:
The patent applies rolling shutter correction and electronic image stabilization as preliminary actions before fusing the image frames. By correcting for device motion and rolling shutter effects in advance, the system ensures that objects remain aligned across frames with different exposure times, preventing misalignment artifacts while maintaining the improved dynamic range achieved through multi-exposure capture
2Loss of information
If image frames are captured at different exposure times to compensate for limited dynamic range, then details in dark and bright regions can be preserved, but the time separation between captures causes objects to be located at different locations creating artifacts
Solution Approach 1:
The patent uses motion data from motion sensors as feedback to determine alignment corrections for image frames captured at different exposure times. By continuously monitoring device motion and applying corrective transformations based on this feedback, the system compensates for the time separation between captures, ensuring that objects remain properly aligned across frames while preserving detail information from both dark and bright regions
3Illumination intensity
If multiple representations are combined with different exposure lengths to improve dynamic range, then the device can capture details in varying light conditions, but the mismatch of object locations between representations creates wobbling artifacts in video
Solution Approach 1:
The patent introduces motion data from motion sensors as an intermediary element that mediates between the multiple image representations captured at different exposure lengths. This intermediary information about device motion is used to calculate alignment corrections that reconcile the positional mismatches between frames, enabling the system to combine representations with improved dynamic range while maintaining precise object alignment and eliminating wobbling artifacts
Data Source
Figure 1
Figure 2A~2B
Figure 3~4
AI summary
Image frames for computational photography may be corrected, such as through rolling shutter correction (RSC), prior to fusion of the image frames to reduce wobble and jitter artifacts present in a video sequence of HDR-enhanced image frames. First and second motion data regarding motion of the image capture device may be determined for times corresponding to the capturing of the first and second image frames, respectively. The rolling shutter correction (RSC) may be applied to the first and second image frames based on both the first and second motion data. The corrected first and second image frames may then be aligned and fused to obtain a single output image frame with higher dynamic range than either of the first or second image frames.