CNN Image Upscaling for Real-Time High-Resolution Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing CNN-based image processing pipelines suffer from detail/quality loss and are not suitable for real-time processing of high-resolution images above 1080p due to complex computational methods like Trainable Guided Filters and Deep Bilateral Learning.
Innovation Solution
A method and apparatus using a convolutional neural network (CNN) to downscale and convert images from a first color space to a second color space, with pre-training on triplets of original, intermediate, and restored images, allowing efficient upscaling to maintain high resolution without complex computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If complex computational methods like Trainable Guided Filters and Deep Bilateral Learning are used for CNN-based image processing, then image quality and detail preservation are improved, but computational complexity and processing time increase significantly making real-time processing of high-resolution images above 1080p infeasible
Solution Approach 1:
The patent segments the image processing task into distinct stages: downscaling the high-resolution input image to a lower resolution, processing the downscaled image through the CNN with color space conversion, and then upsampling the result back to the original resolution. This segmentation allows the computationally intensive CNN operations to be performed on smaller images while still producing high-resolution output, resolving the contradiction between image quality and computational complexity.
Solution Approach 2:
The patent applies preliminary downscaling action before the main CNN processing step. By reducing the image resolution before feeding it into the CNN, the computational burden is significantly reduced while preserving the essential visual information needed for quality processing. The final upsampling step restores the high resolution, thus achieving both quality preservation and reduced computational complexity.
2Productivity
If downscaling is applied to reduce computational costs, then processing speed and efficiency are improved, but detail loss and quality degradation occur in the processed image
Solution Approach 1:
The patent operates in multiple resolution dimensions by processing the image at a lower resolution dimension during the CNN computation phase, then transforming back to the original high resolution dimension through upsampling. This dimensional transformation allows efficient processing while preserving detail information in the final high-resolution output, resolving the contradiction between processing speed and image detail preservation.
Solution Approach 2:
The downscaling operation acts as an intermediary that enables efficient CNN processing of high-resolution images by creating a computationally manageable intermediate representation. The CNN processes this intermediate low-resolution image, and the results are then transformed back to high resolution, with the intermediary step being key to achieving both speed and quality.
3Manufacturing precision
If the original high-resolution image is processed directly without downscaling, then image detail and quality are maintained, but computational costs and processing time increase making real-time processing infeasible
Solution Approach 1:
The patent segments the processing workflow into downscaling, CNN processing, and upsampling stages, allowing the time-consuming CNN operations to be performed on smaller images while maintaining the ability to produce high-resolution output. This segmentation resolves the contradiction between maintaining image resolution and reducing processing time.
Solution Approach 2:
The patent changes the resolution parameter during processing by downscaling the input image before CNN computation and then upsampling the output. This parameter transformation enables the system to achieve both fast processing (by operating at lower resolution) and high-quality output (by restoring to original resolution), resolving the time-resolution trade-off.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method, apparatus, and computer program product for processing images by using a convolutional neural network (CNN) are proposed. An original image is received from an image source. The original image has a predefined size and high resolution, and is represented in a first color space supported by the image source. Then, an intermediate image is obtained by downscaling the original image in the first color space, and converted from the first color space to a second color space. Next, a restored image is obtained by upscaling the converted intermediate image to the predefined size of the original image. Said upscaling is performed by using the CNN on the original image and the converted intermediate image as inputs and return the restored image. The CNN is pre-trained on a set of triplets, comprising a past original image, a converted past intermediate image and a past restored image.