Image Enhancement Models for Non-Normal Sensor Noise
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning-based neural networks struggle with accurately enhancing images with real noise that does not follow a normal distribution, due to the difficulty in obtaining sufficient training pairs of noisy and clean images and pixel position misalignments.
Innovation Solution
A method and apparatus that generate a non-normal noise map using a noise model, based on pixel position information and image capturing parameters, to enhance images with noise following a non-normal distribution, utilizing a machine learning model trained on non-normal noise distributions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If traditional deep learning training is used with presumption of normal distribution noise, then training process is simplified, but image enhancement accuracy for real noise deteriorates
Solution Approach 1:
The patent changes the noise distribution parameter from normal distribution to non-normal distribution (e.g., Poisson distribution) to match real image noise characteristics. This involves modifying the noise generation process in the training pipeline to use appropriate statistical distributions that reflect actual camera sensor noise behavior, thereby improving enhancement accuracy while maintaining training feasibility through automated noise map generation.
2Measurement precision
If training pairs of noisy and clean images are obtained from real images, then training accuracy should improve, but obtaining sufficient training pairs becomes difficult and time-consuming
Solution Approach 1:
The patent performs preliminary action by pre-generating noise maps using noise distribution models before the actual training process. These noise maps are created in advance based on camera parameters and image statistics, allowing the training pipeline to directly use them without requiring time-consuming acquisition and alignment of real noisy-c clean image pairs. This preliminary noise characterization enables faster training while maintaining accuracy.
Solution Approach 2:
The patent creates synthetic copies of noise characteristics by generating noise maps that replicate real noise patterns through statistical modeling. Instead of using actual noisy images captured from real scenes (which require time-consuming acquisition), the system copies the essential noise properties through modeled noise distributions, achieving similar training effectiveness with much faster processing.
3Measurement precision
If training pairs from real images are used, then training data quality improves, but pixel position misalignments occur within training pairs
Solution Approach 1:
The patent introduces noise maps as an intermediary element that decouples the relationship between noisy and clean images. Instead of directly pairing noisy and clean images (which suffer from misalignment), the system uses noise maps generated from clean images as the training target. This intermediary approach eliminates the need for precise spatial alignment between noisy and clean images, as the noise map is derived directly from the clean image's pixel positions and camera parameters.
Data Source
AI summary
A method and apparatus with noise consideration are provided. The method includes generating, using a noise model, a non-normal noise map corresponding to a noise of an input image, and generating an enhanced image of the input image by implementing an image enhancement model based on the input image and the non-normal noise map, where the noise of the input image follows a non-normal distribution.