Adaptive Noise-Guided Deep Learning for Image Super-Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image super-resolution networks struggle with real-world low-resolution images containing unknown degradation such as sensor noise and JPEG compression, lacking scalability and performing poorly due to separate training of denoising and super-resolution tasks.
Innovation Solution
An end-to-end trainable degradation restoration network (DRN) that integrates a noise estimator CNN subnetwork to generate a noise map guiding an SR CNN subnetwork, enhancing its ability to handle noisy images by adjusting intermediate features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If conventional SR networks are trained with downsampled LR images, then training data preparation is simple, but performance on real-world noisy images deteriorates
Solution Approach 1:
The patent changes the training parameters by introducing a noise level parameter and degradation type parameters into the training process. The network is trained with synthetic noisy images where noise level and degradation type are controllable parameters, enabling the model to learn robust features under various noise conditions rather than only simple downsampling artifacts.
Solution Approach 2:
The patent creates synthetic training data by copying and modifying clean HR images through controlled degradation processes (adding noise, compression, downsampling). This synthetic data copying approach allows the network to learn from diverse real-world degradation patterns without requiring actual captured noisy images for training.
2Adaptability or versatility
If denoising and SR tasks are learned separately, then each task can be optimized independently, but overall system complexity increases and performance deteriorates
Solution Approach 1:
The patent merges the denoising and super-resolution tasks into a single unified network architecture. The network simultaneously performs both denoising and SR operations through shared convolutional layers and feature extraction mechanisms, eliminating the need for separate independent networks and reducing overall system complexity while improving performance.
Solution Approach 2:
The patent designs a universal network architecture that can perform multiple functions (denoising and super-resolution) within a single model. The network uses universal feature extraction layers and degradation restoration modules that work for both tasks, making the system more efficient and less complex compared to separate specialized networks.
3Ease of manufacture
If existing SR networks use fixed network parameters, then training is simpler, but adaptability to different noise levels and compression qualities deteriorates
Solution Approach 1:
The patent introduces dynamic adaptability by allowing the network to adjust its behavior based on input characteristics. The network uses learnable parameters that can adapt to different noise levels and degradation types during inference, rather than being fixed for a single condition. This enables the network to scale and adapt to various real-world scenarios without requiring retraining.
Solution Approach 2:
The patent employs parameter change mechanisms where the network can modify its internal parameters or activation states based on the detected noise level and degradation type. This allows the same network architecture to adapt its processing characteristics to match different input conditions, improving scalability without complicating the training process significantly.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A method includes estimating, by a noise estimator (NE) subnetwork of the DRN, an estimated noise map for a noisy input image; and predicting, by the SR subnetwork of the DRN, a clean upscaled image based on the input image and the noise map by, for each of multiple conditional residual dense blocks (CRDBs) stacked within one or more cascade blocks representing the SR subnetwork, adjusting, by a noise control layer of the CRDB that follows a stacked set of a multiple residual dense blocks of the CRDB, feature values of an intermediate feature map associated with the input image by applying (i) a scaling factor and (ii) an offset factor derived from the noise map.