Raw Camera Image Denoising with Multi-Scale Residual Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image denoising techniques, particularly for images captured by digital cameras, struggle to effectively remove noise while preserving texture details, especially under low-light conditions where sensor noise is prominent.
Innovation Solution
The use of Multi-scale Residual Dense Networks (MRDNs) and Multi-scale Residual Dense Block Cascaded U-Nets with Block Connections (MCU-Nets) in convolutional neural networks, combined with atrous spatial pyramid pooling and residual dense blocks, to process raw camera data such as Bayer raw data, enhancing noise reduction and detail preservation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If traditional image denoising techniques are used, then noise removal is achieved, but texture details are lost
Solution Approach 1:
The network divides the denoising task into multiple stages using sequential blocks (RDB1, RDB2, RDB3, RDB4) that progressively process different aspects of noise removal while preserving details at each stage. The multi-scale approach segments the feature space into different resolution levels.
Solution Approach 2:
The patent introduces multi-scale processing by operating at multiple resolution levels simultaneously. The ASPP module adds the dimension of spatial pooling with different rates, allowing the network to capture noise patterns and texture details at various scales without losing information from any single scale.
2Object-affected harmful factors
If strong denoising filters are applied, then noise reduction is improved, but image sharpness deteriorates
Solution Approach 1:
The network applies different processing strengths to different regions and scales of the image. The ASPP module with varying dilation rates (6, 12, 18, 24) applies local quality adjustments at different spatial scales, allowing aggressive denoising in homogeneous regions while preserving sharpness in edge regions.
Solution Approach 2:
The residual dense blocks dynamically adjust the denoising strength based on local image characteristics. The residual connections allow the network to learn adaptive filtering behaviors, applying stronger denoising where needed and weaker filtering where details must be preserved.
3Productivity
If simple convolutional networks are used, then computational efficiency is maintained, but denoising performance is insufficient
Solution Approach 1:
The network performs preliminary feature extraction and noise characterization in the early RDB blocks, preparing processed features that make subsequent denoising more effective. This staged approach allows complex processing to be broken into manageable steps that maintain efficiency.
Solution Approach 2:
The patent combines multiple architectural components (RDB blocks, ASPP module, skip connections) into a composite network structure. Each component contributes specific functionality, and their combination achieves high denoising quality that would be impossible with simple convolutions alone, while the modular structure maintains computational efficiency.
4Loss of information
If multi-scale processing is applied, then detail preservation is improved, but device complexity increases
Solution Approach 1:
The residual dense blocks serve multiple functions simultaneously: they perform denoising, feature extraction, and multi-scale processing through the integrated ASPP module. This multi-functionality reduces the need for separate dedicated components for each task, managing complexity while achieving detailed preservation.
Solution Approach 2:
The ASPP module is nested within the RDB structure, with spatial pooling operations at different rates nested within the same block. This nested organization allows multi-scale processing to be integrated efficiently without requiring separate network branches, managing architectural complexity.
Data Source
AI summary
A method includes: computing noise data by subtracting, by a processing circuit, a noisy image from a corresponding ground truth image; clustering, by the processing circuit, a plurality of noise values of the noise data based on intensity values of the corresponding ground truth image; permuting, by the processing circuit, a plurality of locations of the noise values of the noise data within each cluster; generating, by the processing circuit, a synthetic noise image based on the permuted locations of the noise values; adding, by the processing circuit, the synthetic noise image to the corresponding ground truth image to generate a synthetic noisy image; and augmenting an image dataset for training a neural network to perform image denoising with the synthetic noisy image.