Bayer image noise reduction method based on edge information and brightness information
Through the generalized Anscombe transformation and frequency division idea combining edge information and structural similar information, the problem of Poisson Gaussian noise removal in Bayer images is solved, the signal-to-noise ratio and robustness of Bayer images are improved, and the noise reduction effect is achieved.
Patent Information
- Application Number
- CN202311063214.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-22
- Publication Date
- 2025-02-28
AI Technical Summary
In the prior art, when processing Bayer images, it is difficult to effectively remove Poisson Gaussian noise, and ignores the spectrum correlation between different color channels, resulting in poor noise reduction effect, especially poor robustness under low light or no light conditions.
The generalized Anscombe transform is used to convert the Poisson Gaussian noise in Bayer images into Gaussian noise, and the image is divided into brightness components and color components through frequency division idea, using edge information and structural similar information for noise reduction, and combining guide filtering technology to remove color noise.
It effectively reduces the time cost and calculation complexity of Bayer image noise reduction, improves signal-to-noise ratio and robustness, and improves the visual effect of the image.
Smart Images
Figure CN119540080A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of image processing, and in particular relates to a Bayer image denoising method based on edge information and brightness information. Background Art
[0002] In the prior art, the temporal random noise in the CMOS imaging system is mainly introduced by the CMOS sensor, and its noise model can be summarized as a Poisson-Gaussian noise model, as shown below:
[0003] y=αz+n
[0004] Where y is the observed image of the CMOS sensor, which has a Bayer structure and is also called a Bayer image; α is the gain of the CMOS sensor; n is the mean μ and the variance σ 2 is Gaussian noise; z is a Bayer image containing Poisson noise and obeys the Poisson distribution z~P(x) with x as parameter, that is, its discrete Poisson distribution is
[0005]
[0006] Among them, x i is the expected pixel value at the ith position of the noise-free Bayer image x; M is the number of pixels in the Bayer image. The Poisson noise is caused by the error between the number of photons reaching the CMOS sensor and the theoretical value. Its noise variance depends on the pixel intensity of the noise-free image x, so the noise in the Bayer image is non-uniform. Gaussian noise is mainly caused by the internal circuit of the CMOS sensor, such as thermal noise, dark current noise, etc., and its noise variance does not change with the pixel intensity of the noise-free image x. Poisson-Gaussian noise will affect the subsequent image signal processing (ISP) and the visual effect of the final RGB color image, and will also affect the application value of RGB images in face detection and target recognition. At the same time, the later ISP will change the noise morphology in the Bayer image, making the noise in the image more difficult to remove. To this end, developing an efficient Bayer image denoising algorithm is an essential step.
[0007] Since the non-uniformity of noise and the unique mosaic structure in the Bayer image increase the difficulty of denoising the Bayer image, the traditional Gaussian denoising algorithm is difficult to be directly applied to the denoising of the Bayer image. The prior art proposes an effective algorithm to denoise the Bayer image: first, the green channel image with a high signal-to-noise ratio in the Bayer image is interpolated; then the interpolated green channel image is used as the guide image for other channel images, and the red channel image, green channel image and blue channel image in the Bayer image are filtered using the guide filter; finally, the red channel image, green channel image and blue channel image are reconstructed, thereby completing the noise suppression in the Bayer image.
[0008] In addition, a generalized Anscombe transform method is proposed in the prior art to transform the Poisson Gaussian noise in the image into Gaussian noise with a mean of 0 and a variance of approximately 1, thereby eliminating the dependency between the Poisson noise variance and the noise-free image. In addition, the prior art transforms the image into the frequency domain and finds that its low-frequency component contains the brightness information and most of the energy of the image, while the noise and color difference information in the image are mainly concentrated in the high-frequency area of the image.
[0009] Definition of guided filter: the guided image is g, the input image is y, the output image is f, and it is assumed that the guided image g and the output image f satisfy a linear relationship in the local window, that is, in the local window w centered on pixel k k In, meet:
[0010] f i =a k g i +b k ,i∈w k
[0011] Among them, (a k ,b k ) is in the local window w k In the local window w k In the example, the edge information difference between the output image f and the guidance image g is a k times, that is Thus, the local window w k The linear coefficient (a k ,b k )The problem of solving is transformed into a minimization problem, which can be expressed as
[0012]
[0013] Where ε is the regularization factor to prevent a k The local window w can be solved by linear programming theory.k The optimal value in
[0014]
[0015]
[0016] Among them, μ k and It is the guide image g in the local window w k The mean and variance in |ω| is the local window w k The number of pixels in ; is the input image y in the local window w k cov(g,y) is the mean of the guide image g and the input image y in the local window w k According to the optimal linear coefficient (a k ,b k ) and the linear relationship between the guide image g and the output image f in the local window, so that the linear relationship between the local window w can be obtained. k The output image g is expressed as
[0017]
[0018] Among them, ε controls the smoothness of the image by the guided filter.
[0019] However, the defects of the prior art are:
[0020] 1. For Poisson noise, the existing denoising algorithms can be divided into direct denoising algorithms and variance-stabilized transform denoising algorithms. The direct denoising method is to denoise the Bayer image based on the Poisson distribution of the noise using the maximum a posteriori probability estimation theory. Although the image denoising effect of the direct denoising algorithm is relatively good, the complexity of the algorithm is relatively high and the time cost is relatively large, which is not convenient for real-time image processing. The variance-stabilized transform denoising method is to use the variance-stabilized transform to convert the Poisson noise in the image into Gaussian noise with a mean of 0 and a variance of approximately 1, so that the noise in the image is evenly distributed, which is convenient for using the Gaussian denoising algorithm to denoise the image. In order to reduce the denoising complexity of the Bayer image, the present invention adopts the generalized Anscombe transform to convert the Poisson Gaussian noise in the Bayer image into Gaussian noise with a mean of 0 and a variance of approximately 1.
[0021] 2. Due to the unique mosaic structure of Bayer images, a large number of existing Bayer image denoising algorithms only directly downsample the Bayer image according to the arrangement format of the four color components R, Gr, Gb, and B in the Bayer image; then perform channel-by-channel denoising on the sampled red channel image, green channel image, and blue channel image. This Bayer image denoising method seriously ignores the spectral correlation between different color channels in the Bayer image.
[0022] 3. The relationship between the high-frequency and low-frequency components in the Bayer image and the brightness image and color difference image in the RGB color image is not generally used in Bayer image denoising.
[0023] 4. Existing Bayer image denoising algorithms rarely use noise calibration results.
[0024] 5. The existing Bayer image denoising algorithm has poor robustness in low light or no light conditions, and a large amount of brightness noise and color noise still remain in the denoised Bayer image. In order to better suppress the residual noise on the Bayer image, it is often necessary to increase the denoising intensity of the denoising algorithm, but this usually leads to the loss of texture details in the Bayer.
[0025] In addition, common terminology explanations in the prior art include:
[0026] 1. A simple flowchart of image processing in a CMOS imaging system is shown below: Figure 1 shown.
[0027] 2. Bayer image: A color filter array (CFA) is covered on the surface of the CMOS sensor. The most common CFA format is the Bayer format. The raw data collected by the Bayer format CMOS sensor is called a Bayer image. Figure 1 shown.
[0028] 3. Photon: Due to the particle nature of light, light is composed of discrete particles, so discrete particles are called photons. Summary of the invention
[0029] In order to solve the above problems, the purpose of this application is to:
[0030] 1. The present invention utilizes the results of generalized Anscombe transform and Bayer image noise calibration to convert Poisson-Gaussian noise in Bayer image into Gaussian noise, which is convenient for removing noise in Bayer image and reduces the time cost and computational complexity of Bayer image denoising.
[0031] 2. Since the intensity of noise is often greater than the pixel intensity of the image under dim or no light conditions, a large amount of content in the Bayer image is annihilated by noise. In order to improve the signal-to-noise ratio of the Bayer image, the present invention divides the Bayer image into a brightness component image and a color component image through the idea of frequency division. With respect to the brightness noise of the brightness component image, the present invention uses the edge prior information and non-local structural similarity information of the brightness component image to protect the texture detail features of the image while also greatly attenuating the brightness noise in the brightness component image, thereby improving the robustness of the Bayer image denoising method. In addition, with respect to the color noise in the color component image, the present invention uses the denoised brightness component image as the guide image of the color component image, and uses the idea of channel division to guide filtering of the color component sub-image, thereby improving the subjective quality of the final Bayer image.
[0032] Specifically, the present invention provides a Bayer image denoising method based on edge information and brightness information, the method comprising:
[0033] Firstly, the Bayer image is processed by generalized Anscombe transform, so that the variance of Poisson noise in the Bayer image is transformed into a constant, so that the noise of the transformed Bayer image is approximately considered as Gaussian noise, which is convenient for processing it using Gaussian noise reduction method;
[0034] Secondly, according to the distribution of brightness component and color component in the Bayer image in the frequency domain, the brightness component image and color component image are extracted from the transformed Bayer image respectively, and the denoising model of the brightness component image is constructed by using the structural similarity information and edge information in the brightness component image and the Bayesian estimation theory, and the noise in the brightness image is iteratively suppressed by using the alternating direction multiplier method, so as to remove the brightness noise in the Bayer image;
[0035] Thirdly, according to the arrangement format of the four color components R, Gr, Gb, and B in the Bayer image, the color component image and the denoised brightness component image are decomposed into sub-images of the four channels of R, Gr, Gb, and B respectively, and the color component sub-image is guided filtered by the denoised brightness component sub-image to remove the color noise in the color component and improve the signal-to-noise ratio of the color component image; then, according to the arrangement format of the four color components R, Gr, Gb, and B in the Bayer image, the color sub-images of the four channels are reorganized to obtain the denoised color component image, and then according to the extraction method of the brightness component and the color component, the denoised Bayer image is reconstructed by the denoised brightness component and the color component;
[0036] Finally, the luminance noise and color noise in the Bayer image are effectively suppressed by performing inverse generalized Anscombe transform on the denoised Bayer image.
[0037] The method further comprises:
[0038] S1: Mean value calibrated according to Gaussian noise in Bayer image y and standard deviation Normalize the Gaussian noise:
[0039]
[0040] Where α is the gain value of the CMOS sensor; then the generalized Anscombe transform is used to transform the Bayer image Processing is performed to transform the Poisson Gaussian noise in the Bayer image into Gaussian noise with a mean of 0 and a variance approximately constant. The result is obtained by the following formula:
[0041]
[0042] S2: From the perspective of frequency domain, the Bayer image y is composed of high-frequency components and low-frequency components. The low-frequency components mainly include the brightness component of the Bayer image, and the color component of the Bayer image is mainly concentrated in the high-frequency area of the image. Therefore, the present invention extracts the corresponding brightness component image L and color component image C from the Bayer image y according to the distribution characteristics of the brightness component and color component of the image in the frequency domain and the spectral correlation between the color channels in the Bayer image, where the brightness component image L is expressed as
[0043] L = filter(y,F)
[0044] Among them, filter(*) represents the convolution filtering operation, and F is the filter kernel coefficient matrix, which is expressed as
[0045]
[0046] Since the pixels in the Bayer image carry not only brightness information but also color information of the image, and the size of the brightness component image is exactly the same as that of the Bayer image, the color component image C is obtained by subtracting the brightness component image L from the Bayer image, which can be expressed as
[0047] C=yL
[0048] S3: Since the brightness component extracted by step S2 is disturbed by a large amount of brightness noise, the present invention uses the Bayesian estimation theory to transform the brightness component image denoising problem into a maximum a posteriori estimation problem based on the additive noise model in the brightness component image and the irrelevance between the additive noise and the noise-free brightness component image s. That is, the brightness component image denoising model is composed of the data guarantee term ||Ls||2, the edge feature prior term D G (s) and the non-local structural similarity prior D S (s), which is expressed as
[0049]
[0050] Where ||Q||2 represents the 2-norm operation of the matrix Q, ξ and ζ are the marginal prior terms D G (s) and the non-local structural similarity prior D S Regularization coefficient of (s);
[0051] S4: A large amount of high-frequency noise is concentrated in the color component image C obtained in step S2, so the brightness component image obtained in step S3 is used. Perform guided filtering on the color component image C to improve the visual effect of the Bayer image and suppress the color noise in the image;
[0052] S5: According to the arrangement format of the four color components R, Gr, Gb, and B on the Bayer image, the denoised color sub-images C ch Reconstruct the color component image C of the Bayer image; in addition, according to the relationship between the color component image, the brightness component image and the Bayer image, the color component image C and the brightness component image The two are superimposed on each other to obtain the noise-reduced Bayer image y * ;
[0053] S6: Use the inverse generalized Anscombe transform to denoise the Bayer image y * After processing, the processed Bayer image y is returned to the original pixel space, which is expressed as
[0054]
[0055] At the same time, the Bayer image y is normalized in step S1, so it is necessary to perform a denormalization transformation on the Bayer image y to obtain the final denoised Bayer image That is,
[0056]
[0057] The step S3 further comprises:
[0058] S3.1: The present invention uses a fractional-order total variation model to obtain the edge prior term D of the brightness component image G (s), that is, the edge feature prior information D of the brightness component image s at (i, j) G (s i,j ) is expressed as
[0059]
[0060] in, and It represents the β-fold fractional partial differentials of the pixel point (i, j) in the brightness component image in the horizontal direction h and the vertical direction v, which are respectively expressed as
[0061]
[0062]
[0063] Among them, K is the number of adjacent pixels involved in the calculation of the fractional derivative of the current pixel, is a coefficient, which is expressed as
[0064]
[0065] where Γ(·) is the gamma function;
[0066] S3.2: Since there are many similar structural information in the non-local positions of the image, the strong correlation between the non-local similar structural information in the image and the irrelevance between the additive noise and the noise-free image s are used to suppress the noise in the brightness component image, protect the texture detail information in the image, and improve the visual effect of the image; In order to obtain the non-local structural similarity prior information in the brightness component image, first, the brightness component image L is divided into P reference image blocks of size d×d Secondly, for the reference image block For example, we use the Euclidean distance to find similar blocks of the same size S in the search window centered on it. wl , and the reference image block The q most similar image patches are transformed into column vectors D i and D wl , l = 1, 2, ..., q, and stack them into a matrix D i ={D i ,D w1 ,D w2 ,…,D wq}; According to the matrix D i There is a strong correlation between the column vectors ofi It has low rank characteristics; in the present invention, weighted nuclear norm operation is used to replace the matrix D i Low-rank operation, because the matrix D i The larger singular value represents the edge and texture information of the brightness component image, and the smaller singular value represents the additive noise in the brightness component image. Therefore, the present invention uses the weighted nuclear norm operation to replace the low-rank operation of the matrix, so that the large singular value is attenuated as little as possible, and the small singular value is attenuated as much as possible, so as to protect the texture detail information of the image and suppress the noise in the image; therefore, the non-local structural similarity prior information D of the brightness component image S (s) is expressed as
[0067]
[0068] Among them, ||D i || ω,* Denotes the matrix D i The weighted nuclear norm operation, ω j is the matrix D i The weight corresponding to the j-th singular value of ;
[0069] S3.3 The edge feature prior information D obtained in step S3.2 and step S3.3 G (s) and non-local structural similarity prior information D S (s), the denoising model derived in step S3 is re-expressed as
[0070]
[0071] Among them, M and N are the number of rows and columns of the brightness component image respectively; in order to obtain the optimal solution of the brightness component image The present invention uses variable separation technology to construct a decomposable framework of the denoising model; then uses the alternating direction multiplier method to iteratively converge the decomposable denoising model to find the brightness component image with the best image quality.
[0072] The step S4 further comprises:
[0073] S4.1: According to the arrangement format of the four color components R, Gr, Gb, and B on the Bayer image, the brightness component image And the color component image C is downsampled separately to obtain the brightness sub-images on different channels and color sub-image C ch , ch∈{R,Gr,Gb,B};
[0074] S4.2: The brightness sub-image obtained in step S4.1 As the color sub-image C chThe guidance image is used, and the guidance filter is used to filter the color sub-image C ch Filtering is performed to obtain the denoised color sub-image C ch , ch∈{R,Gr,Gb,B}.
[0075] Therefore, the advantages of the present application are: the present method effectively suppresses the brightness noise and color noise in the Bayer image according to the unique structure of the Bayer image, thereby improving the signal-to-noise ratio in the CMOS imaging system and improving the robustness of the Bayer image denoising method; reducing the time cost and computational complexity of the Bayer image denoising; and improving the visual effect of the Bayer image. BRIEF DESCRIPTION OF THE DRAWINGS
[0076] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.
[0077] Figure 1 It is a schematic diagram of the image processing flow of the CMOS imaging system in the prior art.
[0078] Figure 2 This is a flowchart of the Bayer image denoising method based on edge information and brightness information of the present application. DETAILED DESCRIPTION
[0079] In order to more clearly understand the technical content and advantages of the present invention, the present invention is now further described in detail in conjunction with the accompanying drawings.
[0080] In view of the noise non-uniformity in the Bayer image, the present invention uses the result of noise calibration and generalized Anscombe transform to process the Bayer image, so that the noise variance of the image approaches a constant; at the same time, the present invention protects the texture detail features in the brightness component image based on the edge prior information and non-local structural similarity prior information in the brightness component image, suppresses the brightness noise in the brightness component image, and improves the robustness of the Bayer image denoising algorithm; in addition, the color component image is guided and filtered using the denoised brightness component image to remove the color noise in the color component image. In particular, it includes:
[0081] 1. Application of noise calibration results in Bayer image denoising;
[0082] 2. Establish a relationship between the low-frequency component and high-frequency component in the Bayer image and the brightness component image and the color component image in the image;
[0083] 3. Use the edge prior information and non-local structure similarity prior information in the brightness component image to suppress its brightness noise, effectively improving the signal-to-noise ratio and robustness of the image;
[0084] 4. Use the idea of channel separation to remove color noise in the color component image, that is, use the denoised brightness component sub-image to guide filtering on the color component sub-image.
[0085] The detailed steps of the Bayer image denoising based on edge information and brightness information of the present invention are as follows: Figure 2 As shown:
[0086] Firstly, for the noisy Bayer image, the noise calibration result is used to normalize the Bayer image, and the generalized Anscombe transform is used to process the normalized Bayer image, so that the Poisson Gaussian noise in the Bayer image is approximately transformed into Gaussian noise with a mean of 0 and a constant variance.
[0087] Secondly, according to the distribution of the color component and brightness component in the image in the frequency domain, that is, the brightness component of the image mainly concentrates on the low-frequency component of the image, while the high-frequency component is mainly the color component of the image. Accordingly, a suitable filter is used to extract the brightness component image and the color component image from the Bayer image respectively. Since the brightness component image is interfered by brightness noise, the color component image contains a large amount of color noise, so that the image quality is difficult to meet the requirements; for this reason, the present invention utilizes the Bayesian estimation theory and the edge information and structural similarity information in the brightness component image to effectively suppress the brightness noise in the brightness component image and protect the texture details in the brightness component image;
[0088] Thirdly, the denoised brightness component image is used as the guide image of the color component image, and the denoised brightness component image and the color component image are downsampled according to the arrangement structure of the four colors R, Gr, Gb, and B on the Bayer image to obtain the brightness component sub-image and the color component sub-image of the corresponding channel. In addition, the downsampled brightness component sub-image is used to guide the color component sub-image to suppress the color noise in the color component image.
[0089] Next, the color sub-images of the four channels are reorganized according to the arrangement format of the four color components R, Gr, Gb and B in the Bayer image to obtain the denoised color component image, and then the denoised Bayer image is reconstructed through the denoised brightness component and color component according to the extraction method of the brightness component and the color component;
[0090] Finally, the denoised Bayer image is processed using the inverse form of the generalized Anscombe transform. At the same time, the processed Bayer image needs to be denormalized so that the Bayer image returns to the original pixel space range.
[0091] Specifically, the method further comprises:
[0092] Step S1: According to the mean value of Gaussian noise calibration in Bayer image y and standard deviation Normalize the Gaussian noise:
[0093]
[0094] Where α is the gain value of the CMOS sensor; then the generalized Anscombe transform is used to transform the Bayer image Processing is performed to transform the Poisson Gaussian noise in the Bayer image into Gaussian noise with a mean of 0 and a variance approximately constant. The result is obtained by the following formula:
[0095]
[0096] Step S2: From the frequency domain point of view, the Bayer image y is composed of high-frequency components and low-frequency components. The low-frequency components mainly include the brightness components of the Bayer image, and the color components of the Bayer image are mainly concentrated in the high-frequency area of the image. Therefore, the present invention extracts the corresponding brightness component image L and color component image C from the Bayer image y according to the distribution characteristics of the brightness component and color component of the image in the frequency domain and the spectral correlation between the color channels in the Bayer image, where the brightness component image L is expressed as
[0097] L = filter(y,F)
[0098] Among them, filter(*) represents the convolution filtering operation, and F is the filter kernel coefficient matrix, which is expressed as
[0099]
[0100] Since the pixels in the Bayer image carry not only brightness information but also color information of the image, and the size of the brightness component image is exactly the same as that of the Bayer image, the color component image C is obtained by subtracting the brightness component image L from the Bayer image, which can be expressed as
[0101] C = yL;
[0102] Step S3: Since the brightness component extracted by step S2 is disturbed by a large amount of brightness noise, the present invention uses the Bayesian estimation theory to transform the brightness component image denoising problem into a maximum a posteriori estimation problem based on the additive noise model in the brightness component image and the irrelevance between the additive noise and the noise-free brightness component image s. That is, the brightness component image denoising model is composed of the data guarantee term ||Ls||2, the edge feature prior term DG (s) and the non-local structural similarity prior D S (s), which is expressed as
[0103]
[0104] Where ||Q||2 represents the 2-norm operation of the matrix Q, ξ and ζ are the marginal prior terms D G (s) and the non-local structural similarity prior D S Regularization coefficient of (s);
[0105] Step S3.1: The present invention uses a fractional-order total variation model to obtain the edge prior term D of the brightness component image G (s), that is, the edge feature prior information D of the brightness component image s at (i, j) G (s i,j ) is expressed as
[0106]
[0107] in, and It represents the β-fold fractional partial differentials of the pixel point (i, j) in the brightness component image s in the horizontal direction h and the vertical direction v, which are respectively expressed as
[0108]
[0109]
[0110] Among them, K is the number of adjacent pixels involved in the calculation of the fractional derivative of the current pixel, is a coefficient, which is expressed as
[0111]
[0112] where Γ(·) is the gamma function;
[0113] Step S3.2: Since there are many similar structural information in the non-local position of the image, the strong correlation between the non-local similar structural information in the image and the irrelevance between the additive noise and the noise-free image x are used to suppress the noise in the luminance component image, protect the texture detail information in the image, and improve the visual effect of the image. In order to obtain the non-local structural similarity prior information in the luminance component image, first, the luminance component image L is divided into P reference image blocks of size d×d Secondly, for the reference image block For example, we use the Euclidean distance to find similar blocks of the same size S in the search window centered on it. wl , and the reference image block The q most similar image patches are transformed into column vectors D i and D wl , l = 1, 2, ..., q, and stack them into a matrix D i ={D i ,D w1 ,D w2 ,…,D wq}. According to the matrix D i There is a strong correlation between the column vectors of i The present invention uses weighted nuclear norm operation to replace the matrix D i Low-rank operation, because the matrix D i The larger singular values represent the edge and texture information of the brightness component image, and the smaller singular values represent the additive noise in the brightness component image. Therefore, the present invention uses weighted nuclear norm operation to replace the low-rank operation of the matrix, so that the large singular values are attenuated as little as possible and the small singular values are attenuated as much as possible, thereby protecting the texture detail information of the image and suppressing the noise in the image. Therefore, the non-local structural similarity prior information D of the brightness component image S (s) is expressed as
[0114]
[0115] Among them, ||D i || ω,* Denotes the matrix D i The weighted nuclear norm operation, ω j is the matrix D i The weight corresponding to the j-th singular value of .
[0116] Step S3.3 is based on the edge feature prior information D obtained in step S3.2 and step S3.3. G (s) and non-local structural similarity prior information D S (s), the denoising model derived in step S3 is re-expressed as
[0117]
[0118] Where M and N are the number of rows and columns of the luminance component image, respectively. In order to obtain the optimal solution for the luminance component image The present invention uses variable separation technology to construct a decomposable framework of the denoising model; then uses the alternating direction multiplier method to iteratively converge the decomposable denoising model to find the brightness component image with the best image quality.
[0119] Step S4: A large amount of high-frequency noise is concentrated in the color component image C obtained in step S2, so the brightness component image obtained in step S3 is used Perform guided filtering on the color component image C to improve the visual effect of the Bayer image and suppress the color noise in the image;
[0120] Step S4.1: According to the arrangement format of the four color components R, Gr, Gb, and B on the Bayer image, the brightness component image And the color component image C is downsampled separately to obtain the brightness sub-images on different channels and color sub-image C ch , ch∈{R,Gr,Gb,B};
[0121] Step S4.2: The brightness sub-image obtained in step S4.1 As the color sub-image C ch The guidance image is used, and the guidance filter is used to filter the color sub-image C ch Filtering is performed to obtain the denoised color sub-image C ch , ch∈{R,Gr,Gb,B};
[0122] Step S5: again, according to the arrangement format of the four color components R, Gr, Gb, and B on the Bayer image, the denoised color sub-images C ch Reconstruct the color component image C of the Bayer image; in addition, according to the relationship between the color component image, the brightness component image and the Bayer image, the color component image C and the brightness component image The two are superimposed on each other to obtain the noise-reduced Bayer image y * .
[0123] Step S6: Using the inverse generalized Anscombe transform to transform the denoised Bayer image y * After processing, the processed Bayer image y is returned to the original pixel space, which is expressed as
[0124]
[0125] At the same time, the Bayer image y is normalized in step S1, so it is necessary to perform a denormalization transformation on the Bayer image y to obtain the final denoised Bayer image That is,
[0126]
[0127] In summary, the focus of this method is:
[0128] 1. Apply the noise calibration results to the denoising of Bayer image;
[0129] 2. Using the relationship between the low-frequency component and high-frequency component of the Bayer image and the brightness component and color component of the image, the brightness component image and color component image of the Bayer image are extracted;
[0130] 3. The brightness noise in the brightness component image is suppressed based on the edge prior information and non-local structure similarity prior information in the brightness component image, which effectively improves the signal-to-noise ratio of the brightness component image and the robustness of the Bayer image denoising method;
[0131] 4. Use the denoised brightness component sub-image to perform guided filtering on the color component sub-image to remove the color noise.
[0132] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A Bayer image denoising method based on edge information and brightness information, characterized in that: The method comprises: Firstly, the Bayer image is processed by generalized Anscombe transform, so that the variance of Poisson noise in the Bayer image is transformed into a constant, so that the noise of the transformed Bayer image is considered as Gaussian noise, which is convenient for processing it using Gaussian noise reduction method; Secondly, according to the distribution of brightness component and color component in the Bayer image in the frequency domain, the brightness component image and color component image are extracted from the transformed Bayer image respectively, and the denoising model of the brightness component image is constructed by using the structural similarity information and edge information in the brightness component image and the Bayesian estimation theory, and the noise in the brightness image is iteratively suppressed by using the alternating direction multiplier method, so as to remove the brightness noise in the Bayer image; Thirdly, according to the arrangement format of the four color components R, Gr, Gb, and B in the Bayer image, the color component image and the denoised brightness component image are decomposed into sub-images of four channels R, Gr, Gb, and B respectively, and the color component sub-image is guided filtered by the denoised brightness component sub-image to remove the color noise in the color component and improve the signal-to-noise ratio of the color component image; Next, the color sub-images of the four channels are reorganized according to the arrangement format of the four color components R, Gr, Gb and B in the Bayer image to obtain the denoised color component image, and then the denoised Bayer image is reconstructed through the denoised brightness component and color component according to the extraction method of the brightness component and the color component; Finally, the luminance noise and color noise in the Bayer image are effectively suppressed by performing inverse generalized Anscombe transform on the denoised Bayer image.
2. The Bayer image denoising method based on edge information and brightness information according to claim 1, characterized in that: The method further comprises: S1: Mean value calibrated according to Gaussian noise in Bayer image y and standard deviation Normalize the Gaussian noise: Where α is the gain value of the CMOS sensor; then the generalized Anscombe transform is used to transform the Bayer image Processing is performed to transform the Poisson Gaussian noise in the Bayer image into Gaussian noise with a mean of 0 and a variance approximately constant. The result is obtained by the following formula: S2: From the perspective of frequency domain, the Bayer image y is composed of high-frequency components and low-frequency components. The low-frequency components mainly include the brightness component of the Bayer image, and the color component of the Bayer image is mainly concentrated in the high-frequency area of the image. Therefore, this method extracts the corresponding brightness component image L and color component image C from the Bayer image y according to the distribution characteristics of the brightness component and color component of the image in the frequency domain and the spectral correlation between the color channels in the Bayer image. The brightness component image L is expressed as L = filter(y,F) Among them, filter(*) represents the convolution filtering operation, and F is the filter kernel coefficient matrix, which is expressed as Since the pixels in the Bayer image carry not only brightness information but also color information of the image, and the size of the brightness component image is exactly the same as that of the Bayer image, the color component image C is obtained by subtracting the brightness component image L from the Bayer image, which can be expressed as C = yL; S3: Since the brightness component extracted by step S2 is disturbed by a large amount of brightness noise, this method uses the Bayesian estimation theory to transform the brightness component image denoising problem into a maximum a posteriori estimation problem based on the additive noise model in the brightness component image and the irrelevance between the additive noise and the noise-free brightness component image s. That is, the brightness component image denoising model consists of the data guarantee term ||Ls||2, the edge feature prior term D G (s) and the non-local structural similarity prior D S (s), which is expressed as Where ||Q||2 represents the 2-norm operation of the matrix Q, ξ and ζ are the marginal prior terms D G (s) and the non-local structural similarity prior D S Regularization coefficient of (s); S4: A large amount of high-frequency noise is concentrated in the color component image C obtained in step S2, so the brightness component image obtained in step S3 is used. Perform guided filtering on the color component image C to improve the visual effect of the Bayer image and suppress the color noise in the image; S5: According to the arrangement format of the four color components R, Gr, Gb, and B on the Bayer image, the denoised color sub-images C ch Reconstruct the color component image C of the Bayer image; in addition, according to the relationship between the color component image, the brightness component image and the Bayer image, the color component image C and the brightness component image The two are superimposed on each other to obtain the noise-reduced Bayer image y * ; S6: Use the inverse generalized Anscombe transform to transform the denoised Bayer image y * After processing, the processed Bayer image y is returned to the original pixel space, which is expressed as At the same time, the Bayer image y is normalized in step S1, so it is necessary to perform a denormalization transformation on the Bayer image y to obtain the final denoised Bayer image That is, 3. The Bayer image denoising method based on edge information and brightness information according to claim 2, characterized in that: The step S3 further comprises: S3.1: This method uses a fractional-order total variation model to obtain the edge prior term D of the brightness component image. G (s), that is, the edge feature prior information D of the brightness component image s at (i, j) G (s i,j ) is expressed as in, and It represents the β-fold fractional partial differentials of the pixel point (i, j) in the brightness component image in the horizontal direction h and the vertical direction v, which are respectively expressed as Among them, K is the number of adjacent pixels involved in the calculation of the fractional derivative of the current pixel, is a coefficient, which is expressed as where Γ(·) is the gamma function; S3.2: Since there are many similar structural information in the non-local positions of the image, the strong correlation between the non-local similar structural information in the image and the irrelevance between the additive noise and the noise-free image s are used to suppress the noise in the brightness component image, protect the texture detail information in the image, and improve the visual effect of the image; In order to obtain the non-local structural similarity prior information in the brightness component image, first, the brightness component image L is divided into P reference image blocks of size d×d Secondly, for the reference image block For example, we use the Euclidean distance to find similar blocks of the same size S in the search window centered on it. wl , and the reference image block The q most similar image patches are transformed into column vectors D i and D wl , l = 1, 2, ..., q, and stack them into a matrix D i ={D i ,D w1 ,D w2 ,…,D wq }; According to the matrix D i There is a strong correlation between the column vectors of i It has low rank characteristics; in the present invention, weighted nuclear norm operation is used to replace the matrix D i Low-rank operation, because the matrix D i The larger singular values represent the edge and texture information of the brightness component image, and the smaller singular values represent the additive noise in the brightness component image. Therefore, this method uses weighted nuclear norm operation to replace the low-rank operation of the matrix, so that the large singular values are attenuated as little as possible and the small singular values are attenuated as much as possible, thereby protecting the texture detail information of the image and suppressing the noise in the image; therefore, the non-local structural similarity prior information D of the brightness component image S (s) is expressed as Among them, ||D i || ω,* Denotes the matrix D i The weighted nuclear norm operation, ω j is the matrix D i The weight corresponding to the j-th singular value of ; S3.3 The edge feature prior information D obtained in step S3.2 and step S3.3 G (s) and non-local structural similarity prior information D S (s), the denoising model derived in step S3 is re-expressed as Among them, M and N are the number of rows and columns of the brightness component image respectively; in order to obtain the optimal solution of the brightness component image This method uses variable separation technology to construct a decomposable framework of the denoising model; then the alternating direction multiplier method is used to iteratively converge the decomposable denoising model to find the brightness component image with the best image quality.
4. The Bayer image denoising method based on edge information and brightness information according to claim 2, characterized in that: The step S4 further comprises: S4.1: According to the arrangement format of the four color components R, Gr, Gb, and B on the Bayer image, the brightness component image And the color component image C is downsampled separately to obtain the brightness sub-images on different channels and color sub-image C ch , ch∈{R,Gr,Gb,B}; S4.2: The brightness sub-image obtained in step S4.1 As the color sub-image C ch The guidance image is used, and the guidance filter is used to filter the color sub-image C ch Filtering is performed to obtain the denoised color sub-image C ch , ch∈{R,Gr,Gb,B}.