U-Net Image Denoising With Global Context for Mobile Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image denoising and enhancement methods for low-light images fail to effectively integrate global context/color information, leading to poor performance on real images and are computationally expensive, especially on resource-limited devices like mobile phones.
Innovation Solution
A U-net architecture with a global pooling layer and bilinear upsampling is used to integrate global context, reducing computational cost through bilinear downsampling and incorporating a 1×1 convolutional layer, enabling efficient denoising and enhancement on mobile devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If existing denoising and image enhancement approaches are used on low-light images, then image processing is performed, but the processing effects are poor and a large amount of processing resources are required
Solution Approach 1:
The network is segmented into three distinct modules: encoding network for feature extraction, bottleneck network for global context integration, and decoding network for noise map generation. This modular segmentation allows each component to specialize in specific tasks, improving overall processing quality while optimizing resource utilization through targeted computational operations in each segment.
Solution Approach 2:
The bottleneck network transforms local features into global context by projecting them to a lower dimensional space through global pooling, then restoring them to original dimensions. This dimensional transformation enables the integration of global context information across the entire image, significantly improving denoising quality by considering holistic image patterns rather than localized features alone.
2Loss of information
If traditional denoising methods are applied to low-light images, then noise reduction is attempted, but global context/color information is not effectively integrated
Solution Approach 1:
The bottleneck network uses global pooling to project local features to a lower dimensional space, enabling comprehensive integration of global context and color information across the entire image. This dimensional transformation allows the network to capture holistic patterns and color relationships that traditional local-based methods miss, thereby preventing information loss while improving denoising performance.
Solution Approach 2:
The bottleneck network acts as an intermediary between the encoding and decoding networks, receiving local features from the encoder and transforming them into global context representations. This intermediary module bridges the gap between local feature extraction and global context integration, ensuring that color and contextual information are effectively combined before being passed to the decoder for noise map generation.
3Manufacturing precision
If complex network architectures are used for image denoising, then processing accuracy may improve, but computational cost increases significantly on mobile devices
Solution Approach 1:
The network is divided into three functional modules (encoding, bottleneck, decoding), allowing each to perform specialized operations with optimized computational complexity. This segmentation enables mobile devices to process images efficiently by distributing computational tasks across modular components rather than using a monolithic complex architecture.
Solution Approach 2:
The bottleneck network's dimensional transformation approach (projection to lower dimensions via global pooling, then restoration) enables global context integration with reduced computational overhead. By working in a compressed dimensional space, the network achieves accurate denoising results while consuming less energy compared to processing full-resolution features throughout the entire network.
Data Source
AI summary
An image processing method, an electronic device, and a computer-readable storage medium are provided. In the method, a first image is processed through a U-net to obtain a second image, and the second image is a noise map of the first image. The U-net includes an encoding network, a decoding network, and a bottleneck network between the encoding network and the decoding network. The bottleneck network includes a global pooling layer, a bilinear upscaling layer, and a 1×1 convolutional layer. Moreover, a third image is generated according to the first image and the second image, and the third image is a denoised map of the first image.