Flash-No-Flash Image Fusion for Denoising Without Ghosting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for removing noise from no-flash images using deep neural networks fail to effectively address inconsistent specular highlights and hard shadows in flash images, leading to artifacts such as residual noise and ghosting.
Innovation Solution
An apparatus and method using an artificial neural network module to align and fuse flash and no-flash images, generating a consistent flash image patch through convolution and combining it with the no-flash image using a composite weight to produce a denoised image.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a deep neural network is used to fuse flash and no-flash images for noise removal, then noise removal capability is improved, but artifacts such as residual noise and ghosting are generated due to inconsistent specular highlights and hard shadows in the flash image
Solution Approach 1:
The patent segments the flash image into different regions based on the presence of specular highlights and hard shadows. By identifying and separating these inconsistent regions from the rest of the image, the method applies different processing strategies to different segments, preventing artifacts while maintaining noise removal effectiveness in consistent regions.
Solution Approach 2:
The patent applies local quality adjustment by detecting regions with specular highlights and hard shadows, then applying localized processing to these specific areas. Instead of uniformly processing the entire image, the method adapts the fusion strategy locally to preserve image structure in problematic regions while maintaining noise removal in other areas.
2Measurement precision
If the flash image is directly used for noise removal, then high-frequency details with reduced noise are obtained, but the inconsistent image structure (specular highlights and hard shadows) causes misalignment with the ground truth
Solution Approach 1:
The patent performs preliminary action by detecting and marking regions with specular highlights and hard shadows before the fusion process. This pre-processing step identifies problematic areas that would otherwise cause structure misalignment, allowing the subsequent fusion algorithm to avoid using these regions for noise removal and thus preserve image structure consistency.
3Device complexity
If conventional noise removal methods are used on no-flash images, then processing simplicity is maintained, but the noise removal effectiveness is insufficient compared to methods using flash images
Solution Approach 1:
The patent applies partial action by selectively using only the reliable portions of the flash image (regions without specular highlights or hard shadows) for noise removal. Instead of fully utilizing the flash image which would introduce artifacts, the method partially uses it in a controlled manner, achieving improved noise removal effectiveness while maintaining processing feasibility through targeted application.
Data Source
AI summary
The present disclosure relates to an apparatus and method for improving no-flash image quality by removing noise included in a no-flash image using a flash image. An apparatus for improving no-flash image quality using a flash image according to an example embodiment of the present disclosure includes an artificial neural network module which is trained by receiving an image pair of a flash image and a no-flash image and configured to output a convolutional kernel kc and a composite weight wci from a local flash image patch and a no-flash image patch, a convolution unit configured to convolve the flash image patch and the convolutional kernel to generate a consistent flash image patch, and a combining module configured to combine the no-flash image patch and the consistent flash image patch using the composite weight wci to produce a denoised image.