Slice-Based CNN Upscaling for SoC Memory Constraints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional CNN-based single image super-resolution (SISR) algorithms are not suitable for system-on-chip (SoC) implementation due to increased memory and computation requirements, limiting their performance in upscaling low-resolution images to high-resolution images.
Innovation Solution
A device and method for upscaling resolution based on slice images, which divides low-resolution images into multiple slice images, uses a convolutional neural network with a cascading block for residual operations and an upscaling block to generate high-resolution images, reducing the size of the vertical receptive field of convolution filters and adjusting parameters using loss functions based on pixel similarity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional CNN-based SISR algorithms are used to generate high-resolution images from low-resolution images, then image upscaling quality is improved, but memory requirements and computation amount increase, making SoC implementation difficult
Solution Approach 1:
The patent divides the input low-resolution image into multiple slice images (e.g., 4 slices) and processes each slice independently through the CNN network. This segmentation approach reduces the memory footprint and computational load compared to processing the entire high-resolution image at once, while still achieving effective super-resolution upscaling.
Solution Approach 2:
The patent introduces a vertical receptive field dimension by using asymmetric convolution filters with different heights and widths (e.g., 3x1 or 1x3 filters). This dimensional change allows the network to capture vertical contextual information efficiently while maintaining reduced memory requirements compared to traditional square filters.
2Measurement precision
If conventional CNN-based SISR algorithms with many layers and filters are used, then image upscaling quality is improved, but device complexity increases, limiting SoC implementation
Solution Approach 1:
The network is divided into multiple cascading blocks, each containing a limited number of convolutional layers and filters. This modular segmentation reduces the complexity of each individual block while maintaining overall network depth for quality upscaling. Each block processes slice images independently with reduced filter sets.
Solution Approach 2:
The patent employs asymmetric convolution filters with non-square dimensions (e.g., 3x1, 1x3) to reduce the number of parameters and filters needed compared to traditional square filters. This dimensional change decreases device complexity while preserving the ability to capture spatial features for high-quality image reconstruction.
3Quantity of substance
If the size of vertical receptive field is reduced by adjusting convolution filter size, then memory requirements are reduced for SoC implementation, but image processing capability may be compromised
Solution Approach 1:
The patent uses asymmetric convolution filters with reduced vertical dimensions (e.g., height=3, width=1 or height=1, width=3) to minimize memory requirements while maintaining image processing capability. The asymmetric design allows efficient capture of vertical contextual information with fewer parameters, balancing memory constraints and processing reliability.
Solution Approach 2:
By dividing the image into slices and processing them with reduced-receptive-field filters, the patent achieves effective image processing with lower memory requirements. The slice-based approach compensates for the reduced filter size by providing multiple localized processing opportunities across the image.
Data Source
AI summary
Provided is a device for upscaling resolution based on a slice image, in which a low-resolution image is divided into a plurality of slice images so that a high-resolution image is generated. The device includes a convolution operation unit configured to convert a low-resolution input slice image into a high-resolution output slice image using a convolutional neural network. The convolutional neural network includes a cascading block configured to perform a convolution operation using a convolution filter having a predetermined size and a residual operation on an input feature map generated from the low-resolution input slice image to generate an output feature map, and an upscaling block configured to upscale the output feature map to generate the high-resolution output slice image.