Image Fusion Using Low-Resolution Frames to Cut Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing resolution of image data captured by image capture devices leads to higher processing demands, resulting in increased power consumption and latency, which can shorten battery life and impact image quality.
Innovation Solution
Perform image processing operations on reduced resolution image frames, including noise filtering and multi-frame fusion, to reduce bandwidth and power requirements while maintaining or improving image quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If image processing operations are performed on high-resolution image frames, then image quality is maintained, but power consumption and processing time increase
Solution Approach 1:
The image processing pipeline is segmented into two resolution levels: low-resolution frames undergo noise filtering and multi-frame fusion operations, while high-resolution frames are processed only for final output. This segmentation allows computationally intensive operations to be performed on smaller data sets, reducing power consumption while preserving image quality in the final high-resolution output.
Solution Approach 2:
Low-resolution intermediate frames serve as mediators in the processing pipeline. These intermediate frames undergo noise filtering and multi-frame fusion to produce cleaned low-resolution references, which are then upsampled and combined with high-resolution frames. This intermediary approach enables complex processing to be performed efficiently on reduced data while maintaining final image quality.
2Measurement precision
If image processing operations are performed on high-resolution image frames, then image quality is maintained, but processing speed decreases
Solution Approach 1:
The processing workload is segmented by resolution: noise filtering and multi-frame fusion are performed on low-resolution frames to improve processing speed, while high-resolution frames are reserved for final output generation. This segmentation reduces the computational burden on the processing pipeline, increasing overall processing speed while maintaining image quality through the upsampling and combination steps.
Solution Approach 2:
Noise filtering and multi-frame fusion are performed as preliminary actions on low-resolution frames before the final high-resolution output is generated. By pre-processing the low-resolution references and storing them in memory, the system reduces the computational load during final high-resolution processing, thereby improving processing speed without compromising image quality.
3Measurement precision
If high-resolution image frames are stored and processed, then image quality is maintained, but bandwidth requirements increase
Solution Approach 1:
The data flow is segmented by resolution: low-resolution frames are stored and processed in memory to reduce bandwidth consumption, while high-resolution frames are processed only when needed for final output. This segmentation significantly reduces the bandwidth required for storing and transferring reference frames, as low-resolution data occupies much less memory and transmission bandwidth.
Solution Approach 2:
Low-resolution copies of reference frames are created and stored in memory instead of high-resolution frames. These low-resolution copies are sufficient for noise filtering and multi-frame fusion operations, and they consume significantly less bandwidth for storage and processing. The high-resolution quality is restored only in the final output frame through upsampling and combination operations.
Data Source
AI summary
This disclosure provides systems, methods, and devices for image signal processing that support improved image quality of a high-resolution image frame with reduced bandwidth and power consumption requirements. In a first aspect, a method of image processing includes receiving a first image frame depicting a scene at a first resolution, and determining a fourth image frame by fusing a second image frame depicting the scene with a third image frame retrieved from a memory. Each of the second, third, and fourth image frames is at a second resolution less than the first resolution. The method further includes upscaling the fourth image frame from the second resolution to the first resolution, and determining an output image frame by fusing the first image frame and the fourth image frame after upscaling the fourth image frame. Other aspects and features are also claimed and described.