Demosaicing Raw Images Using Local Convolution Kernels
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current demosaicing methods for digital raw images from optical sensor systems with monochrome photosites filtered by colored filters face challenges such as attenuation of details, 'zipping' effects, and grid or false color issues due to unresolved luminance/chrominance spectral overlap, especially in devices intended for the general public.
Innovation Solution
A demosaicing method using local convolution kernels to transform raw images into low-frequency and high-frequency coefficients, allowing for the reconstruction of luminance and chrominance images, which adapts to local frequency specificities and improves upon traditional global filtering approaches.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If bilinear interpolation is used to reconstruct color planes, then the demosaicing process is simple to implement, but image details are strongly attenuated and zipping effects occur on sharp transitions
Solution Approach 1:
The image processing is segmented into frequency domains using wavelet transforms. The demosaicing process operates separately on low-frequency and high-frequency components, allowing different processing strategies for each band. This segmentation enables preservation of sharp transitions in high-frequency components while maintaining simplicity in low-frequency processing.
Solution Approach 2:
The patent applies local adaptive processing where the demosaicing method varies according to local image characteristics. In regions with sharp transitions, the algorithm preserves high-frequency details by applying appropriate processing only to relevant frequency bands. This local adaptation prevents zipping effects while maintaining overall implementation simplicity.
2Manufacturing precision
If frequency or wavelet transforms with iterative processes are used, then image quality is improved, but the processing becomes too cumbersome for consumer devices
Solution Approach 1:
The complex iterative frequency-domain processing is segmented into separate low-frequency and high-frequency processing paths. Low-frequency components are processed with simple non-iterative methods, while high-frequency components receive targeted processing only where needed. This segmentation dramatically reduces overall processing complexity while maintaining image quality.
Solution Approach 2:
Instead of applying full iterative wavelet processing to the entire image, the patent applies processing only partially - specifically targeting high-frequency components that contain sharp transitions and edge information. This partial action approach achieves quality improvement where most needed while avoiding the computational burden of processing all frequency components iteratively.
3Ease of manufacture
If global filtering is used to separate luminance and chrominance, then the process is linear and easy to implement, but luminance/chrominance spectral overlap causes grid effects and false colors
Solution Approach 1:
The frequency spectrum is segmented into low-frequency and high-frequency bands. Luminance information is primarily extracted from low-frequency components while chrominance information is extracted from high-frequency components. This segmentation reduces spectral overlap between luminance and chrominance, eliminating grid effects and false colors while maintaining implementation simplicity.
Solution Approach 2:
The patent transforms the problem from spatial domain filtering to frequency domain processing using wavelet transforms. By operating in the frequency dimension rather than spatial dimension, the method achieves better separation of luminance and chrominance information without the spectral overlap problems that plague spatial domain global filtering methods.
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
This process of demosaicing a raw digital image (10) in the form of a matrix of pixels each dedicated to a single color among several predetermined colors of colored filters includes a step (118) of obtaining a digital color image of the same resolution as the raw image (10) and in which each pixel results from corresponding pixels of a luminance image (10L) and chrominance images (10C1, 10C2) reconstructed from the raw image (10).It further comprises the following steps: transformation (102) of the raw image (10) by application to its pixels of local convolution kernels (S, V, H, D) taking into account neighboring pixels of different colors, to obtain a low frequency coefficient image (10S) using a low frequency local kernel (S) and high frequency coefficient images (10V, 10H, 10D) using high frequency local kernels (V, H, D); reconstruction (112) of the luminance image (10L) using at least the low frequency coefficient image (10S); reconstruction (104, 114) of the chrominance images (10C1, 10C2) using the high frequency coefficient images (10V, 10H, 10D).