Burst Image Reference Frame Selection for Low-Light Restoration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Images captured by smartphone cameras in low-light environments suffer from significant noise due to low signal-to-noise ratio, resulting in poor image quality, and existing methods like capturing multiple burst images or deep learning-based restoration techniques are inefficient in terms of processing burden, memory access, and power consumption.
Innovation Solution
A method utilizing a recursive neural network architecture that includes an image restoration network and a loss prediction network to determine a reference frame with minimal loss value, allowing for real-time image enhancement by sequentially processing burst images, reducing processing costs and improving speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If multiple burst images are captured and processed using deep learning-based restoration methods, then image quality is improved, but processing burden and power consumption increase
Solution Approach 1:
The patent extracts only the essential information needed for image restoration by selecting a single reference frame from multiple burst images based on loss values, rather than processing all images equally. This extraction approach reduces the amount of data processed by the deep learning model, thereby lowering power consumption while maintaining image quality improvement.
Solution Approach 2:
The patent segments the image processing task into two stages: first, selecting a reference frame using a lightweight loss calculation method; second, applying the computationally intensive deep learning restoration only to the selected reference frame. This segmentation reduces overall processing burden and power consumption compared to restoring all burst images.
2Manufacturing precision
If multiple burst images are captured and processed using deep learning-based restoration methods, then image quality is improved, but processing time increases
Solution Approach 1:
The patent extracts only the reference frame that requires full deep learning restoration, while other frames are processed more lightly or not at all. This selective extraction significantly reduces processing time compared to restoring all burst images through the complete deep learning pipeline.
Solution Approach 2:
The patent performs preliminary loss value calculation on all burst images to identify the best reference frame before applying the time-consuming deep learning restoration. This preliminary action enables faster overall processing by avoiding redundant restoration of suboptimal frames.
3Productivity
If a reference frame is selected based on loss values using neural networks, then processing efficiency is improved, but device complexity increases
Solution Approach 1:
The patent applies different processing qualities to different frames: a lightweight loss calculation is applied to all burst images, while full deep learning restoration is applied only to the selected reference frame. This local quality differentiation improves processing efficiency while managing device complexity by avoiding uniform high-complexity processing of all frames.
4Manufacturing precision
If deep learning-based restoration is applied to all burst images, then image quality is improved, but memory access increases
Solution Approach 1:
The patent extracts and processes only the reference frame through the full deep learning restoration pipeline, while other frames undergo minimal processing. This extraction strategy significantly reduces memory access requirements compared to restoring all burst images, as fewer large feature maps and intermediate representations need to be stored and transferred.
Data Source
Figure 1A
Figure 1B
Figure 2A
AI summary
A processor-implemented method includes: obtaining a plurality of image frames acquired for a scene within a predetermined time; determining loss values respectively corresponding to the plurality of image frames; determining a reference frame among the plurality of image frames based on the loss values; and generating a final image of the scene based on the reference frame.