Multi-Frame HDR Fusion Using Similarity-Based Frame Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image fusion techniques fail to produce high dynamic range images when there is significant movement in the scene, resulting in blurred or corrupted views due to the use of fixed or predetermined time differences between reference and highlight recovery frames.
Innovation Solution
An image fusion engine dynamically selects image frames based on similarity levels, combining frames with minimal shutter lag by comparing history frames to a highlight recovery frame, discarding dissimilar frames to reduce motion artifacts and enhance dynamic range.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If fixed or predetermined time differences between reference and highlight recovery frames are used, then image fusion can be performed, but motion artifacts occur and dynamic range is compromised when there is significant movement in the scene
Solution Approach 1:
The patent applies dynamics by transitioning from fixed/predetermined time differences to dynamic time difference selection. The system dynamically adjusts the time difference between reference frame and highlight recovery frame based on detected scene motion, allowing the fusion process to adapt to varying motion conditions and eliminate motion artifacts while maintaining high dynamic range image quality
Solution Approach 2:
The patent changes the parameter of time difference between frames from a fixed value to a variable parameter that is adjusted based on scene motion detection. By modifying this temporal parameter dynamically, the system optimizes image fusion accuracy under different motion conditions without compromising dynamic range
2Reliability
If multiple image frames are captured and processed, then high dynamic range images can be generated, but processing time and latency increase
Solution Approach 1:
The patent applies partial action by selectively processing only the necessary number of frames based on scene conditions. Instead of always processing a fixed large number of frames, the system adjusts the quantity of frames to process based on motion detection and dynamic range requirements, reducing unnecessary processing latency while maintaining high dynamic range quality when needed
Solution Approach 2:
The patent performs preliminary action by pre-selecting and buffering candidate reference frames before the actual image fusion operation. This preliminary preparation allows the system to quickly select the most appropriate reference frame when capture is triggered, avoiding time-consuming search and selection processes during critical capture moments and reducing overall processing latency
3Loss of time
If history frames from preview mode are used for fusion, then shutter lag is reduced, but frame similarity may be insufficient leading to blurred views
Solution Approach 1:
The patent applies feedback by implementing a similarity detection mechanism that evaluates the match between history frames and highlight recovery frames. This feedback loop allows the system to assess frame similarity and use it as a criterion for selecting reference frames, ensuring that only sufficiently similar frames are used for fusion, thereby maintaining reliability while preserving minimal shutter lag
Solution Approach 2:
The patent changes the selection criterion for reference frames from a simple time-based selection to a similarity-based selection. By introducing frame similarity as a key parameter for frame selection, the system ensures that history frames with adequate similarity to the highlight recovery frame are chosen, preventing blurred views while maintaining minimal shutter lag
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods, systems, apparatuses, and computer-readable storage mediums described herein are configured to fuse image frames captured by a camera to generate an image frame having high dynamic range and reduced amount of noise. For instance, after a shutter button of the camera is activated, one or more history image frames captured during a preview mode of the camera (e.g., before the shutter button is activated) are compared to a highlight recovery frame captured after the shutter button is activated to determine a level of similarity therebetween. The history image frame(s) may be captured with a first exposure value, and the highlight recovery frame may be captured with a second exposure value that is different than the first exposure value. History image frame(s) that are determined to be relatively similar to the highlight recovery frame are combined with the highlight recovery frame to generate the image frame having high dynamic range.