Line-wise Feature Map Compression for Real-Time Super-Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing CNN-based super-resolution methods are difficult to implement in real-time due to high computational complexity and memory requirements, making it challenging to convert low-resolution images to high-resolution images efficiently, especially in low-complexity hardware.
Innovation Solution
An image processing device and method that utilize line-wise operations, including a receiver, convolution operators, compressors, and quantizers, to process images efficiently, reducing memory usage and computational load by employing one-dimensional convolutions, depth-wise separable convolutions, and feature map compression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If CNN-based super-resolution methods are used to improve image quality, then image reconstruction accuracy is improved, but computational complexity and memory requirements increase excessively
Solution Approach 1:
The patent segments the convolution operation into multiple stages: initial convolution to generate feature maps, compression stage to reduce feature map size, and reconstruction stage to generate high-resolution output. This segmentation allows the system to process images in manageable chunks rather than requiring all computational resources simultaneously.
Solution Approach 2:
The patent changes the parameters of the convolution operation by using depth-wise separable convolutions instead of standard convolutions, and by dynamically adjusting feature map compression ratios. These parameter changes reduce the computational burden while maintaining acceptable image reconstruction quality.
2Measurement precision
If CNN-based super-resolution methods are used to improve image quality, then image reconstruction accuracy is improved, but memory requirements increase excessively
Solution Approach 1:
The patent extracts and processes only the most essential feature information from the input image through selective convolution operations and feature map compression. By taking out only the critical features needed for reconstruction rather than processing all image data, memory requirements are significantly reduced.
Solution Approach 2:
The patent transforms the problem from processing full 2D feature maps to processing compressed 1D feature representations. This dimensional reduction allows the system to maintain image reconstruction capability while using far less memory to store and process intermediate results.
3Measurement precision
If standard convolution operations are used to process images, then image processing quality is maintained, but processing speed decreases
Solution Approach 1:
The patent divides the image processing into segmented stages with compression and reconstruction phases. This segmentation enables parallel processing of different image regions and reduces the time each pixel spends in the processing pipeline, thereby increasing overall processing speed.
Solution Approach 2:
The patent changes convolution parameters including kernel sizes, stride values, and compression ratios to optimize for speed. By adjusting these parameters, the system achieves faster processing while maintaining acceptable image quality through the use of depth-wise separable convolutions and selective feature processing.
Data Source
AI summary
Disclosed are an image processing method and device using a line-wise operation. The image processing device, according to one embodiment, comprises: a receiver for receiving an image; a first convolution operator for generating a feature map by performing a convolution operation on the basis of the image; and a compressor for compressing the feature map into units of at least one line; and a decompressor for reconstructing the feature map compressed into units of lines.