GAN Image Denoising for the Synthetic-Natural Noise Gap
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image editing systems inaccurately denoise digital images due to the domain gap between synthetic and natural noise, and are inefficient in training processes requiring controlled environments and resource-intensive post-processing.
Innovation Solution
A domain gap generative adversarial network is trained using paired datasets with synthetic noise and unpaired datasets with natural noise, employing one or two discriminators to improve accuracy and efficiency by bridging the domain gap.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If neural networks are trained using synthetic noise, then training efficiency is improved, but denoising accuracy deteriorates due to domain gap between synthetic and natural noise
Solution Approach 1:
The patent introduces a domain gap bridging mechanism that acts as an intermediary between synthetic noise data and natural noise data. This involves using a style transfer network and a domain gap closure network to progressively bridge the domain gap, allowing the model to learn from synthetic data while adapting to natural noise characteristics through the intermediary transformation layers.
Solution Approach 2:
The patent employs parameter change strategies by dynamically adjusting the degree of domain gap closure during training. The model transitions from using purely synthetic noise parameters to progressively incorporating natural noise parameters through the domain gap bridging mechanism, allowing smooth adaptation between different noise domains without abrupt parameter shifts.
2Measurement precision
If individually-generated digital image pairs with natural noise are used, then denoising accuracy may improve, but computational cost and time consumption increase significantly
Solution Approach 1:
The patent uses synthetic noise data as a copy or surrogate for natural noise data. Instead of requiring actual captured image pairs with natural noise for training, the system generates synthetic noisy images that replicate the essential characteristics of natural noise through the domain gap bridging mechanism, significantly reducing the need for expensive controlled photography and post-processing.
Solution Approach 2:
The patent replaces expensive and time-consuming controlled photography sessions with computationally cheaper synthetic noise generation. The synthetic data serves as a disposable training resource that can be generated on-demand without requiring physical camera equipment, controlled environments, or manual post-processing operations.
3Reliability
If strictly controlled photography and post-processing are used to generate training data, then data quality improves, but training time and computing resources increase
Solution Approach 1:
The patent performs preliminary domain gap bridging during the training process itself rather than requiring pre-generated high-quality paired data. The style transfer network and domain gap closure network are trained simultaneously with the main denoising model, preliminarily adapting the synthetic data to match natural noise characteristics before the actual denoising task begins.
Solution Approach 2:
The patent replaces the mechanical photography and post-processing system with a computational domain gap bridging system. Instead of using physical cameras and manual image processing to create training data, the system uses neural networks to computationally transform synthetic data into domains that resemble natural noise, substituting mechanical data collection with computational data generation.
Data Source
AI summary
The present disclosure relates to systems, non-transitory computer-readable media, and methods for training and utilizing a domain gap generative adversarial network. More specifically, in one or more embodiments, the disclosed systems train a domain gap generative adversarial network by generating predicted denoised images from the digital images with synthetic noise and predicted denoised images from the digital images with natural noise. The disclosed systems also utilize a discriminator to generate a first discrimination between the ground-truth digital images for the digital images with synthetic noise and the predicted denoised images for the digital images with synthetic noise, and a second discrimination between the predicted denoised images for the digital images with natural noise and the predicted denoised images for the digital images with synthetic noise. The disclosed system further modify parameters of the domain gap generative adversarial network based on the first discrimination and the second discrimination.