Image De-noising Model Training with Optical Flow Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image de-noising technologies suffer from poor performance in removing noise from digital images, often resulting in ghosting and poor texture detail resolution.
Innovation Solution
A method and apparatus for acquiring a sample image set by aligning noise-added non-reference images with noise-added reference images using optical flow alignment, and training an image de-noising model with a sample image set that includes aligned non-reference images, noise-added reference images, and non-noise-added reference images, utilizing a novel image de-noising model structure with residual and dense connections.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing image de-noising technology is used, then the processing speed is fast, but the de-noising effect is poor resulting in ghosting and poor texture detail resolution
Solution Approach 1:
The patent segments the image processing task into multiple components: acquiring multiple frame images, performing optical flow alignment to estimate motion, separating noise removal from detail preservation, and using different processing strategies for different image regions. This segmentation allows each component to be optimized independently, resolving the contradiction between de-noising effectiveness and texture detail preservation
Solution Approach 2:
The patent performs preliminary actions by acquiring multiple frame images before de-noising and using optical flow to pre-estimate motion fields. This preliminary motion compensation allows the subsequent de-noising process to work with aligned images, preventing ghosting artifacts while preserving texture details that would otherwise be lost
2Manufacturing precision
If multiple frame images are processed with optical flow alignment, then texture detail resolution is improved, but the processing complexity increases
Solution Approach 1:
The patent introduces optical flow as an intermediary mechanism that bridges multiple frame images and the final de-noised output. The optical flow field acts as a mediator that captures motion information, allowing the system to align images and preserve texture details without requiring complex direct processing of all frame combinations, thus managing complexity while improving resolution
3Reliability
If noise addition processing is applied to training images, then the de-noising model performance is improved, but the training data preparation time increases
Solution Approach 1:
The patent performs preliminary action by pre-processing training images with noise addition and optical flow alignment before model training begins. This advance preparation creates a ready-to-use training dataset that incorporates realistic noise patterns and motion compensation, allowing the model to learn effective de-noising strategies without requiring time-consuming processing during the training phase itself
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The approach significantly improves image de-noising performance by enhancing texture detail resolution and preventing ghosting, as demonstrated by higher Peak Signal to Noise Ratio (PSNR) values and better de-noising effects compared to traditional methods.
Implementation Method 1
aligning each noise-added non-reference image with a noise-added reference image to obtain an aligned non-reference image
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
The present disclosure provides a method and device for acquiring a sample image set. The method includes: acquiring a plurality of frame images continuously shot for a shooting scene; determining, as a reference image, one frame of a plurality of frame images, and determining, as non-reference images, the remaining ones of the plurality of frame images other than the one determined as references images; performing noise addition processing on the format-converted reference image and non-reference image respectively; aligning the each resulting noise-added non-reference image with the noise-added reference image to obtain the aligned non-noise-added non-reference image; and annotating the format-converted reference image with pixel information to obtain non-noise-added reference image and a new type of sample image set, where the sample image set includes: the aligned non-reference image, the noise-added reference image, and the non-noise-added reference image.