Methods, apparatus, and devices for processing Bayer images based on G-component guided maps

Abstract

This invention discloses a method, apparatus, and device for processing Bayer images based on G-component guided maps. In the method, for the final denoising component of a G-pixel, the initial denoising component of the center pixel is directly used as the final denoising component of the corresponding G-pixel in the denoised Bayer image. For the final denoising component of non-G pixels, the initial denoising color difference of each pixel within the search window is first calculated. Then, the weighted sum of the initial denoising color difference and the denoising coefficient is used as the final denoising color difference of the center pixel. The final denoising component of the non-G pixel is calculated based on the difference between the initial denoising component and the final denoising color difference. This solves the problem of uneven denoising that occurs when different numbers of G-pixels and non-G pixels are processed using the same denoising method in existing technologies, resulting in better denoising performance and effectively mitigating the grid effect.

Description

Technical Field

[0001] This invention relates to image processing technology, and more specifically to a method, apparatus, and device for processing Bayer images based on G-component guided maps. Background Technology

[0002] With the rapid development of the media age, there are increasingly more ways to capture and record life through photography and videography. However, due to limitations such as the performance of shooting equipment, ambient light, and noise, some captured images contain a lot of noise. The output image format of CMOS image sensors is Bayer image format. Bayer images contain noise, and during subsequent processing in various modules of ISP (Image Signal Processing), the noise in the Bayer image will be affected to varying degrees, making noise reduction in the YUV (luminance and chromatic aberration) domains difficult.

[0003] To obtain a full-color image, the original Bayer image needs to be de-mosaiced and denoised. Some existing solutions directly denoise the original Bayer image. However, due to the characteristics of the original Bayer image—the number of G pixels and R / B pixels is inconsistent—the denoising intensity differs between G pixels and R / B pixels, and a grid effect may occur. Summary of the Invention

[0004] Based on the above situation, the main objective of this invention is to provide a method, apparatus, and device for processing Bayer images based on G-component guided maps, so as to improve the noise reduction effect of Bayer images.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A method for processing Bayer images based on G-component guided maps includes the following steps: Calculating the G-component of each non-G pixel in the original Bayer image through interpolation; the G-components of the non-G pixels and the G-components of the G pixels in the original Bayer image form a G-component guided map; using a first-sized image block centered on each pixel in the G-component guided map as a template window and a second-sized image block as a search window, wherein the second size is larger than the first size; sequentially selecting comparison windows of the same size as the template window centered on each pixel in the search window; calculating the sum of the absolute values ​​of the differences between the G-component of each pixel in each comparison window and the G-component of the corresponding pixel in the template window, using this sum as the noise reduction factor of the central pixel of the comparison window; selecting a noise reduction coefficient from a noise reduction weight table that matches the ratio of the noise reduction factor to the noise reduction intensity of each pixel in the search window, using this coefficient as the noise reduction coefficient of that pixel; wherein, in the noise reduction weight table… The larger the ratio of the pixel's noise reduction factor to its noise reduction intensity, the smaller the corresponding noise reduction coefficient. The weighted sum of the G component of each pixel within a search window centered on the G component guide image, along with the pixel's noise reduction coefficient, is used as the initial noise reduction component of that central pixel. If the central pixel is a G pixel, its initial noise reduction component is used as the final noise reduction component of the corresponding G pixel in the denoised Bayer image. If the central pixel is a non-G pixel, the difference between the G component and the original component of the non-G pixel is calculated within the search window as the initial noise reduction color difference of the non-G pixel. The average of the initial noise reduction color differences of multiple non-G pixels is used to determine the initial noise reduction color difference between a G pixel and another non-G pixel. The weighted sum of the initial noise reduction color difference of each pixel and its noise reduction coefficient is used as the final noise reduction color difference of the central pixel. The difference between the initial noise reduction component of the central pixel and the final noise reduction color difference is used as the final noise reduction component of the corresponding non-G pixel in the denoised Bayer image.

[0007] Preferably, if the center pixel of the search window is an R pixel, determining the initial noise reduction color difference between a G pixel and another non-G pixel based on the average of the initial noise reduction color differences of multiple non-G pixels specifically involves: using the average of the initial noise reduction color differences of a first number of R pixels surrounding each G pixel as the initial noise reduction color difference of the G pixel; and using the average of the initial noise reduction color differences of a second number of R pixels surrounding each B pixel as the initial noise reduction color difference of the B pixel. If the center pixel of the search window is a B pixel, determining the initial noise reduction color difference between a G pixel and another non-G pixel based on the average of the initial noise reduction color differences of multiple non-G pixels specifically involves: using the average of the initial noise reduction color differences of a first number of B pixels surrounding each G pixel as the initial noise reduction color difference of the G pixel; and using the average of the initial noise reduction color differences of a second number of B pixels surrounding each R pixel as the initial noise reduction color difference of the R pixel.

[0008] Preferably, the calculation of the G component of each non-G pixel in the original Bayer image by interpolation specifically includes: calculating the horizontal interpolated G component and the vertical interpolated G component of each non-G pixel in the original Bayer image by interpolation, and calculating the horizontal interpolated non-G component and the vertical interpolated non-G component of each G pixel; calculating the difference between the original component and the horizontal interpolated component of each pixel in the original Bayer image to obtain the horizontal color difference, and the difference between the original component and the vertical interpolated component to obtain the vertical color difference; calculating the absolute value of the difference between the horizontal color differences of two adjacent pixels in the horizontal direction of each pixel in the original Bayer image to obtain the horizontal color difference gradient, and the absolute value of the difference between the vertical color differences of two adjacent pixels in the vertical direction to obtain the vertical color difference gradient; calculating the sum of the horizontal color difference gradients and the sum of the vertical color difference gradients of pixels within a preset window centered on each pixel of the original Bayer image, and using them as the horizontal weight index and the vertical weight index of the center pixel, respectively; searching in the G component weight table for the G component horizontal weight that matches the horizontal weight index of each pixel in the original Bayer image. And the vertical weights of the G components matched with the vertical weight index of each pixel in the original Bayer image; wherein, in the G component weight table, the horizontal weights of the G components are negatively correlated with the sum of the horizontal color difference gradients, and the vertical weights of the G components are negatively correlated with the sum of the vertical color difference gradients; the color difference weighting of each non-G pixel in the original Bayer image is determined, wherein the color difference weighting is the sum of a first product and a second product, the first product being the average horizontal color difference of the even number of pixels centered on the non-G pixel and the central pixel in the horizontal direction, and the average horizontal color difference of the non-G pixel. The product of the horizontal weights of the G component of the G pixel in the total weights, and the second product is the product of the average vertical color difference of the even number of pixels centered on the non-G pixel and the center pixel in the vertical direction, and the percentage of the vertical weight of the G component of the non-G pixel in the total weights, wherein the total weights are the sum of the horizontal weights and vertical weights of the G component of the non-G pixel; the weighted sum of the original component of each non-G pixel in the original Bayer image and the color difference of the non-G pixel is used as the G component guide map of the original Bayer image in the G component of the non-G pixel.

[0009] Preferably, the method further includes the steps of: for each R pixel in the denoised Bayer image, taking the average of the final denoised components of all B pixels within a preset window centered on the R pixel as the B component of the R pixel in the demosaic image; and for each B pixel in the denoised Bayer image, taking the average of the final denoised components of all R pixels within a preset window centered on the B pixel as the R component of the B pixel in the demosaic image.

[0010] Preferably, the method further includes the following steps: For each G pixel in the same row as the R pixel in the denoised Bayer image, the B component of the G pixel in the demosaic image is the sum of a first B sub-component and a second B sub-component. The first B sub-component is the average of the final denoised components of the two B pixels vertically adjacent to the G pixel, and the second B sub-component is the average of the differences between the initial denoised components of the two B pixels vertically adjacent to the G pixel and the initial denoised components of the two B pixels vertically adjacent to the G pixel. For each G pixel in the same row as the B pixel in the denoised Bayer image, the B component of the G pixel in the demosaic image is the sum of a first B sub-component and a second B sub-component. The first B sub-component is the average of the final denoised components of the two B pixels horizontally adjacent to the G pixel, and the second B sub-component is the average of the differences between the initial denoised components of the two B pixels horizontally adjacent to the G pixel and the initial denoised components of the two B pixels horizontally adjacent to the G pixel.

[0011] Preferably, the method further includes the following steps: For each G pixel in the same row as the R pixel in the denoised Bayer image, the R component of the G pixel in the demosaic image is the sum of a first R sub-component and a second R sub-component. The first R sub-component is the average of the final denoised components of the two R pixels that are horizontally adjacent to the G pixel, and the second R sub-component is the average of the differences between the initial denoised components of the two R pixels that are horizontally adjacent to the G pixel and the initial denoised components of the two R pixels that are horizontally adjacent to the G pixel. For each G pixel in the same row as the B pixel in the denoised Bayer image, the R component of the G pixel in the demosaic image is the sum of a first R sub-component and a second R sub-component. The first R sub-component is the average of the final denoised components of the two R pixels that are vertically adjacent to the G pixel, and the second R sub-component is the average of the differences between the initial denoised components of the two R pixels that are vertically adjacent to the G pixel and the initial denoised components of the two R pixels that are vertically adjacent to the G pixel.

[0012] This invention also discloses a Bayer image processing apparatus based on a G-component guided map, comprising: a G-component guided map generation module, used to calculate the G component of each non-G pixel in the original Bayer image by interpolation, wherein the G components of the non-G pixels and the G components of the G pixels in the original Bayer image form a G-component guided map; and a noise reduction coefficient calculation module, used to take a first-sized image block centered on each pixel in the G-component guided map as a template window and a second-sized image block as a search window, wherein the second size is larger than the first size, and sequentially select comparison windows of the same size as the template window centered on each pixel in the search window, calculate the sum of the absolute values ​​of the differences between the G component of each pixel in each comparison window and the G component of the corresponding pixel in the template window, and use this sum as the noise reduction factor of the center pixel of the comparison window, and select a noise reduction coefficient from a noise reduction weight table that matches the ratio of the noise reduction factor to the noise reduction intensity of each pixel in the search window, and use this coefficient as the noise reduction coefficient of that pixel; wherein... In the noise reduction weight table, the larger the ratio of the noise reduction factor to the noise reduction intensity of a pixel, the smaller the corresponding noise reduction coefficient. The final noise reduction component calculation module is used to take the weighted sum of the G component of each pixel in the search window centered on the G component guide map and the noise reduction coefficient of that pixel as the initial noise reduction component of the center pixel. If the center pixel is a G pixel, then the initial noise reduction component of the center pixel is taken as the final noise reduction component of the corresponding G pixel in the noise-reduced Bayer image. If the center pixel is a non-G pixel, then in the search window, the difference between the G component of the non-G pixel and the original component is calculated as the initial noise reduction color difference of the non-G pixel. The initial noise reduction color difference between the G pixel and another non-G pixel is determined according to the average of the initial noise reduction color differences of multiple non-G pixels. The weighted sum of the initial noise reduction color difference of each pixel and the noise reduction coefficient of that pixel is taken as the final noise reduction color difference of the center pixel. The difference between the initial noise reduction component of the center pixel and the final noise reduction color difference is taken as the final noise reduction component of the corresponding non-G pixel in the noise-reduced Bayer image.

[0013] Preferably, if the center pixel of the search window is an R pixel, determining the initial noise reduction color difference between a G pixel and another non-G pixel based on the average of the initial noise reduction color differences of multiple non-G pixels specifically involves: using the average of the initial noise reduction color differences of a first number of R pixels surrounding each G pixel as the initial noise reduction color difference of the G pixel; and using the average of the initial noise reduction color differences of a second number of R pixels surrounding each B pixel as the initial noise reduction color difference of the B pixel. If the center pixel of the search window is a B pixel, determining the initial noise reduction color difference between a G pixel and another non-G pixel based on the average of the initial noise reduction color differences of multiple non-G pixels specifically involves: using the average of the initial noise reduction color differences of a first number of B pixels surrounding each G pixel as the initial noise reduction color difference of the G pixel; and using the average of the initial noise reduction color differences of a second number of B pixels surrounding each R pixel as the initial noise reduction color difference of the R pixel.

[0014] Preferably, the calculation of the G component of each non-G pixel in the original Bayer image by interpolation specifically includes: calculating the horizontal interpolated G component and the vertical interpolated G component of each non-G pixel in the original Bayer image by interpolation, and calculating the horizontal interpolated non-G component and the vertical interpolated non-G component of each G pixel; calculating the difference between the original component and the horizontal interpolated component of each pixel in the original Bayer image to obtain the horizontal color difference, and the difference between the original component and the vertical interpolated component to obtain the vertical color difference; calculating the absolute value of the difference between the horizontal color differences of two adjacent pixels in the horizontal direction of each pixel in the original Bayer image to obtain the horizontal color difference gradient, and the absolute value of the difference between the vertical color differences of two adjacent pixels in the vertical direction to obtain the vertical color difference gradient; calculating the sum of the horizontal color difference gradients and the sum of the vertical color difference gradients of pixels within a preset window centered on each pixel of the original Bayer image, and using them as the horizontal weight index and the vertical weight index of the center pixel, respectively; searching in the G component weight table for the G component horizontal weight that matches the horizontal weight index of each pixel in the original Bayer image. And the vertical weights of the G components matched with the vertical weight index of each pixel in the original Bayer image; wherein, in the G component weight table, the horizontal weights of the G components are negatively correlated with the sum of the horizontal color difference gradients, and the vertical weights of the G components are negatively correlated with the sum of the vertical color difference gradients; the color difference weighting of each non-G pixel in the original Bayer image is determined, wherein the color difference weighting is the sum of a first product and a second product, the first product being the average horizontal color difference of the even number of pixels centered on the non-G pixel and the central pixel in the horizontal direction, and the average horizontal color difference of the non-G pixel. The product of the horizontal weights of the G component of the G pixel in the total weights, and the second product is the product of the average vertical color difference of the even number of pixels centered on the non-G pixel and the center pixel in the vertical direction, and the percentage of the vertical weight of the G component of the non-G pixel in the total weights, wherein the total weights are the sum of the horizontal weights and vertical weights of the G component of the non-G pixel; the weighted sum of the original component of each non-G pixel in the original Bayer image and the color difference of the non-G pixel is used as the G component guide map of the original Bayer image in the G component of the non-G pixel.

[0015] Preferably, it further includes a first demosaic component calculation module: for each R pixel of the denoised Bayer image, the average of the final denoised components of all B pixels within a preset window centered on the R pixel is used as the B component of the R pixel in the demosaic image; for each B pixel of the denoised Bayer image, the average of the final denoised components of all R pixels within a preset window centered on the B pixel is used as the R component of the B pixel in the demosaic image.

[0016] Preferably, the image further includes a second demosaic component calculation module: For each G pixel in the same row as the R pixel in the denoised Bayer image, the B component of the G pixel in the demosaic image is the sum of a first B sub-component and a second B sub-component. The first B sub-component is the average of the final denoised components of the two B pixels vertically adjacent to the G pixel, and the second B sub-component is the average of the differences between the initial denoised components of the two B pixels vertically adjacent to the G pixel and the initial denoised components of the two B pixels vertically adjacent to the G pixel. For each G pixel in the same row as the B pixel in the denoised Bayer image, the B component of the G pixel in the demosaic image is the sum of a first B sub-component and a second B sub-component. The first B sub-component is the average of the final denoised components of the two B pixels horizontally adjacent to the G pixel, and the second B sub-component is the average of the differences between the initial denoised components of the two B pixels horizontally adjacent to the G pixel and the initial denoised components of the two B pixels horizontally adjacent to the G pixel.

[0017] Preferably, the image further includes a third demosaic component calculation module: For each G pixel in the same row as the R pixel in the denoised Bayer image, the R component of the G pixel in the demosaic image is the sum of a first R sub-component and a second R sub-component. The first R sub-component is the average of the final denoised components of the two R pixels that are horizontally adjacent to the G pixel, and the second R sub-component is the average of the differences between the initial denoised components of the two R pixels that are horizontally adjacent to the G pixel and the initial denoised components of the two R pixels that are horizontally adjacent to the G pixel. For each G pixel in the same row as the B pixel in the denoised Bayer image, the R component of the G pixel in the demosaic image is the sum of a first R sub-component and a second R sub-component. The first R sub-component is the average of the final denoised components of the two R pixels that are vertically adjacent to the G pixel, and the second R sub-component is the average of the differences between the initial denoised components of the two R pixels that are vertically adjacent to the G pixel and the initial denoised components of the two R pixels that are vertically adjacent to the G pixel.

[0018] The present invention also discloses a processing device for Bayer images based on G-component guided maps, which processes Bayer images using the aforementioned processing method, or includes the aforementioned processing device.

[0019] The present invention also discloses a computer-readable storage medium having a computer program stored thereon, which, when executed, can implement the processing method as described above.

[0020] The present invention also discloses a processing chip for Bayer images based on G-component guided maps, comprising a processor and a memory, wherein the memory stores a computer program, and the processor is capable of executing the computer program to implement the processing method described above.

[0021] In the Bayer image processing method based on the G-component guided map of this invention, for the final denoising component of a G pixel, the initial denoising component of the center pixel is directly used as the final denoising component of the corresponding G pixel in the denoised Bayer image. For the final denoising component of non-G pixels, the initial denoising color difference of each pixel within the search window needs to be calculated first. Then, the weighted sum of the initial denoising color difference and the denoising coefficient is used as the final denoising color difference of the center pixel. The final denoising component of the non-G pixel is calculated based on the difference between the initial denoising component and the final denoising color difference. This solves the problem of uneven denoising that occurs when different numbers of G pixels and non-G pixels are processed using the same denoising method in the prior art, resulting in better denoising effect and effectively mitigating the grid effect.

[0022] In addition, the G-component guided map obtained by this scheme is more accurate and more conducive to the subsequent processing of Bayer images. For example, the denoised Bayer image obtained by denoising the G-component guided map is better. Furthermore, the color difference weighting obtained based on the G-component guided map is more accurate, and the calculated de-mosaic image is better.

[0023] In addition, using the initial denoising component calculated during the Bayer image processing as the G component in the R / B pixels of the de-mosaic image helps to reduce the amount of computation.

[0024] Other beneficial effects of the present invention will be explained in detail through the introduction of specific technical features and technical solutions in specific embodiments. Those skilled in the art should be able to understand the beneficial technical effects brought about by these technical features and technical solutions through the introduction of these technical features and technical solutions. Attached Figure Description

[0025] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. In the drawings:

[0026] Figure 1 This is a schematic diagram of a Bayer image;

[0027] Figure 2 This is a flowchart of a Bayer image processing method based on a G-component guided map according to an embodiment of the present invention;

[0028] Figure 3 This is a schematic diagram of the template window, search window, and comparison window centered on pixel G43 in an embodiment of the present invention.

[0029] Figure 4 This is a schematic diagram of a template window, a search window, and another comparison window centered on pixel G43 in an embodiment of the present invention.

[0030] Figure 5This is a schematic diagram of a search window centered on pixel R44 in an embodiment of the present invention;

[0031] Figure 6 This is a schematic diagram of a search window centered on pixel B33 in an embodiment of the present invention. Detailed Implementation

[0032] The present invention is described below based on embodiments, but the present invention is not limited to these embodiments. In the following detailed description of the present invention, some specific details are described in detail, but well-known methods, processes, procedures, and elements are not described in detail in order to avoid obscuring the essence of the present invention.

[0033] Furthermore, those skilled in the art should understand that the accompanying drawings provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0034] In the description of this invention, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0035] like Figure 1 As shown, each pixel in the original Bayer image contains only one of the three colors: red, green, and blue (RGB). The data format of the original Bayer image is that the output pixel colors for odd-numbered scan rows alternate between red and green, while the output pixel colors for even-numbered scan rows alternate between green and blue. For ease of description, in this embodiment, pixels in the original Bayer image are divided into R pixels, G pixels, and B pixels according to their colors. R pixels contain an R component (original component), G pixels contain a G component (original component), and B pixels contain a B component (original component). Figure 2 As shown, this invention proposes a method for processing Bayer images based on G component guided maps, comprising the following steps:

[0036] S110, the G component of each non-G pixel (including R and B pixels) in the original Bayer image is calculated through interpolation. The G components of the non-G pixels and the G components of the G pixels in the original Bayer image form a G component guide map. Each pixel in the G component guide map contains a G component, such as... Figure 3 As shown, Figure 3 This is a schematic diagram of a G component guide map. R00 is an R pixel coordinate in the G component guide map, G01 is a G pixel coordinate in the G component guide map, B11 is a B pixel coordinate in the G component guide map, and so on. It will not be elaborated here.

[0037] S120: Using a first-size N image block centered on each pixel in the G-component guiding image as a template window and a second-size K image block as a search window, where the second size K is larger than the first size N, comparison windows of the same size as the template window are selected sequentially, centered on each pixel in the search window. The sum of the absolute values ​​of the differences between the G-component of each pixel in each comparison window and the G-component of the corresponding pixel in the template window is calculated and used as the noise reduction factor SAD of the center pixel of the comparison window. In the noise reduction weight table w_table1, a noise reduction coefficient matching the ratio of the noise reduction factor SAD to the noise reduction intensity sigma of each pixel in the search window is selected and used as the noise reduction coefficient W of that pixel. j，i In the noise reduction weight table w_table1, the larger the ratio of the noise reduction factor SAD to the noise reduction intensity sigma, the smaller the corresponding noise reduction coefficient. The specific calculation formula for the noise reduction factor SAD is shown in (1).

[0038]

[0039] Where (j1, i1) is the center pixel within the template window, and (j2, i2) is the center pixel within the comparison window. The range of values ​​for (j2, i2) is determined by the search window. The above formula takes the center pixel as the origin, with leftward and downward offsets being negative, and rightward and upward offsets being positive. abs(G (j1+j)，(i1+i) -G (j2+j)，(i2+i) ) represents the absolute value of the difference between the G component of any pixel in the template window and the G component of the corresponding pixel in the comparison window (equal to the absolute value of the difference between the G component of each pixel in the comparison window and the G component of the corresponding pixel in the template window). N is the size of the template window. If the template window is 3*3, then the value of N is 3. Substituting the value of N into equation (1), we get equation (2).

[0040]

[0041] Please continue to refer to Figure 3Taking the calculation of the noise reduction coefficient of pixel G43 as an example, the calculation process of the noise reduction weight is explained in detail. A first-size (3*3) image block centered on pixel G43 is used as the template window 100, and a second-size (5*5) image block is used as the search window 200. Taking pixel G21 in the search window 200 as an example, i.e., (j1,i1)=(4,3), (j2,i2)=(2,1), substituting into equation (2), a comparison window 310 of the same size (3*3) as the template window 100 is selected, centered on pixel G21. The sum of the absolute values ​​of the differences between the G component of each pixel in the comparison window 310 and the G component of the corresponding pixel in the template window 100 (G10 and G32 are corresponding positions, B11 and B33 are corresponding positions, and so on) is used as the noise reduction factor SAD of the central pixel G21 of the comparison window 310.

[0042] like Figure 4 As shown, taking the second pixel R22 in the search window 200 as the center, i.e. (j1,i1)=(4,3),(j2,i2)=(2,2) and substituting them into formula (2), a comparison window 320 with the same size as the template window 100 is selected. The absolute value of the difference between the pixel G component at the corresponding position in the comparison window 320 and the template window 200 is calculated to obtain the noise reduction factor SAD corresponding to pixel R22. This process is repeated until every comparison window within the range of the search window 200 is traversed.

[0043] The noise reduction intensity sigma can be selected according to the application scenario. For example, a larger noise reduction intensity can be selected for photos taken in low light conditions, and a smaller noise reduction intensity can be selected for photos taken in bright light conditions, etc. There are no restrictions here. The ratio matching can be achieved by converting the noise reduction factor SAD into a binary number and then shifting the noise reduction intensity sigma bits to the right as the index of the noise reduction weight table w_table1, as shown in equation (3), where >> is the right shift operator. Substituting the noise reduction factor SAD and noise reduction intensity sigma of each pixel in the search window into equation (3) yields the noise reduction coefficient W. j，i ,

[0044] W j，i =w_table1[SAD>>sigma] (3)

[0045] For example, the noise reduction coefficient W of the G43 4，3 To calculate the Sigma at the preset noise reduction intensity, the noise reduction factor SAD corresponding to G21, R22, ..., G65 (the noise reduction factor SAD corresponding to each of the 25 pixels within the search window 100) are sequentially substituted into Formula 3 to finally calculate the noise reduction coefficient W of G43. 4，3 .

[0046] In other embodiments, the noise reduction factor SAD can be converted into a binary number, shifted right by the noise reduction intensity sigma bits, and then rounded down to serve as the index of the noise reduction weight table w_table1.

[0047] S130: The weighted sum of the G component of each pixel in the search window centered on each pixel in the G component guide image and the denoising coefficient of that pixel is used as the initial denoising component of the center pixel. If the center pixel is a G pixel, the initial denoising component of the center pixel is used as the final denoising component of the corresponding G pixel in the denoised Bayer image. If the center pixel is a non-G pixel, the difference between the G component of the non-G pixel and the original component is calculated in the search window as the initial denoising color difference of the non-G pixel. The initial denoising color difference between the G pixel and another non-G pixel is determined based on the average of the initial denoising color differences of multiple non-G pixels. The weighted sum of the initial denoising color difference of each pixel and the denoising coefficient of that pixel is used as the final denoising color difference of the center pixel. The difference between the initial denoising component and the final denoising color difference of the center pixel is used as the final denoising component of the corresponding non-G pixel in the denoised Bayer image.

[0048] Specifically, the G component of each pixel within a search window centered on each pixel in the G component guide map will be... (j1,i1) With the noise reduction coefficient W of this pixel j1,i1 The weighted sum is used as the initial noise reduction component G for the center pixel (j, i). (j,i) As shown in formula (4), where G (j1,i1) For the G component of any pixel within the search window, W (j1，i1) The noise reduction coefficient is obtained in step S120.

[0049] G j，i =∑W ji，i1 *G ji，i1 (4)

[0050] It should be noted that formula (4) is applicable to calculating the initial noise reduction components of all R pixels, B pixels, and G pixels. After obtaining the initial noise reduction components of all pixels, the final noise reduction components of all pixels are then calculated. The calculation methods for the final noise reduction components of G pixels and non-G pixels are different.

[0051] For the final denoising component of pixel G (i.e., the center pixel of the search window is pixel G), as shown in formula (5), the initial denoising component of the center pixel is directly used as the final denoising component of the corresponding pixel G in the denoised Bayer image.

[0052] G′ j，i =G j，i (5)

[0053] Continue with Figure 3Taking G43 as an example, the initial noise reduction component G(4,3) of G43 is used as the final noise reduction component of G43. The calculation method for the final noise reduction components of other G pixels (such as G01, G03, G05, G07, etc.) is similar, and will not be repeated here.

[0054] For the final denoising component of non-G pixels (i.e., the center pixel of the search window is a non-G pixel, taking R pixels as an example), it is necessary to first calculate the initial denoising color difference of each pixel (j1, i1) (including each R pixel, B pixel, and G pixel) within the search window. Then based on the initial noise reduction color difference With noise reduction coefficient W (j1，i1) weighted sum The final noise reduction chromatic aberration of the central pixel Based on the initial noise reduction component G(j,i) and the final noise reduction color difference The final noise reduction components for non-G pixels are calculated. In other words, G pixels do not require calculation of the initial and final noise reduction color differences.

[0055] Initial noise reduction chromatic aberration within the search window centered on non-G pixels The calculation is performed according to the following: pixels that are the same as the center pixel of the search window (R pixel), G pixels within the search window, and another non-G pixel within the search window (B pixel) are processed separately:

[0056] (1) Calculation of the initial noise reduction color difference of pixels (R pixels) with the same center pixel in the search window.

[0057] The initial noise reduction color difference is found in pixels (j1, i1) within the search window that are identical to the center pixel (i.e., both are R pixels). The difference between the G component (the G component in the G component guide diagram) and the original component is shown in equation (6). (j1，1i) For any R pixel (j1, 1i) within the search window, R (j1，i1) This is the original component (R component) of the R pixel (j1, i1).

[0058]

[0059] (2) Calculation of the initial noise-reduced color difference of G pixels within the search window

[0060] The initial denoised color difference of the G pixel within the search window is calculated based on the average of the initial denoised color differences of multiple non-G pixels (R pixels). Specifically, the average of the initial denoised color differences of the R pixels surrounding each G pixel (in this embodiment, the first number is 2) is used as the initial denoised color difference of the G pixel. The initial denoised color difference of the G pixel (j1, i1) located in the odd-numbered scan rows (where the R pixel is located at the left and right adjacent coordinates of the G pixel) within the search window is as shown in formula (7), where (G j1，i1-1 -R j1，i1-1 ), (G j1，i1+1 -R j1，i1+1 ) is G (j1,i1) The initial noise reduction color difference of the left and right adjacent R pixels.

[0061]

[0062] The initial noise reduction color difference for pixel G (j1, i1) located in an even-numbered scan row (where R pixel is located at the upper and lower adjacent coordinates of G pixel) within the search window is shown in formula (8), where (G j1-1，i1 -R j1-1，i1 ), (G j1+1，i1 -R j1+，i1 ) is G j1，1i The initial noise reduction color difference of the adjacent R pixels.

[0063]

[0064] (3) Calculation of another non-G pixel (B pixel) within the search window

[0065] The initial denoised color difference of another non-G pixel (B pixel) within the search window is also calculated based on the average of the initial denoised color differences of multiple non-G pixels (the center pixel of the search window, which is R pixel in this example). Specifically, the initial denoised color difference of the B pixel is calculated based on the average of the initial denoised color differences of a second number (4 in this embodiment) of R pixels surrounding each B pixel. The initial denoised color difference of B pixel B(j1,i1) is shown in formula (9). The coordinates of adjacent R pixels include: (j1-1,i1-1), (j1-1,i1+1), (j1+1,i1-1), (j1+1,i1+1), and the initial denoised color differences of adjacent R pixels are respectively:

[0066] (G j1-1,i1-1 -R j1-1,i1-1 ), (G j1-1,i1+1 -R j1-1,i1+1 ), (G j1+1,i1-1 -R j1+1,i1-1 ), (G j1+1,i1+1 -R j1+1,i1+1 ).

[0067]

[0068] I. Example of a search window with a center pixel of R pixels

[0069] Taking the center pixel of the search window as R pixels (R44) as an example, this explains how to calculate the initial noise reduction color difference of each pixel within the search window. Figure 5 As shown, the initial denoised color difference of any R pixel within the search window 230 is calculated according to formula (6), which is the difference between the G component (G component in the G component guide map) of that pixel and the original component, with the initial denoised color difference of R22. For example, The initial noise reduction color difference calculation method for other R pixels (R24, R42, R26, R42, R44, R46, R62, R64, R66) is the same, and will not be repeated here.

[0070] The initial denoised color difference of the G pixel within the search window 230 is calculated based on the average of the initial denoised color differences of multiple non-G pixels (R pixels). The initial denoised color difference of the G pixel (whose left and right pixels are R pixels) in any odd-numbered scan line within the search window 230 is calculated according to formula (7). Taking G43 as an example, the initial denoised color difference calculation method is as follows: That is, it is calculated based on the average of the initial noise-reduced color differences (G(4,2)-R(4,2)) and (G(4,4)-R(4,4)) of the adjacent R pixels (R42, R44) of pixel G43. The calculation method for the initial noise-reduced color differences of G pixels (G23, G25, G45, G63, G65) in the other odd-numbered scan rows is the same, and will not be repeated here.

[0071] The initial noise reduction of G pixels (where the pixels above and below G pixels are R pixels) in any even-numbered scan line within the search window 230 is calculated according to formula (8). Taking the initial noise reduction color difference calculation method of G32 as an example, That is, it is calculated based on the average of the initial noise-reduced color differences (G(2,2)-R(2,2)) and (G(4,2)-R(4,2)) of the adjacent R pixels (R22, R42) of pixel G32. The calculation method for the initial noise-reduced color differences of G pixels (G34, G36, G52, G54, G56) in the other even-numbered scan rows is the same, and will not be repeated here.

[0072] Within the search window 230, the initial noise reduction of the initial noise reduction color difference of another non-G pixel (any B pixel) is calculated according to formula (9). Taking the color difference of B33 as an example,

[0073] This is calculated based on the average of the initial noise-reduced color differences (G(2,2)-R(2,2)), (G(2,4)-R(2,4)), (G(4,2)-R(4,2)), and (G(4,4)-R(4,4)) of the adjacent R pixels (R22, R24, R42, R44) of pixel B33. The calculation method for the initial noise-reduced color difference of other B pixels (B35, B53, B55) within the search window is similar and will not be repeated here.

[0074] II. Example of a search window with a center pixel of 'B' pixels

[0075] If the center pixel of the search window is pixel B(j1,i1), the initial noise-reduced color difference of pixel B within the search window. The difference between the G component (the G component in the G component guide diagram) and the original component is shown in equation (10). (j1,i1) For any B pixel (j1, i1) within the search window, the G component is... (j1,i1) This is the original component (B component) of pixel (j1, i1) of pixel B.

[0076]

[0077] The initial denoised color difference of the G pixel within the search window is calculated based on the average of the initial denoised color differences of multiple B pixels. Specifically, the initial denoised color difference of the G pixel is calculated based on the average of the initial denoised color differences of a first number (2 in this embodiment) of B pixels surrounding each G pixel. The initial denoised color difference of the G pixel (j1, i1) located in the search window at an even-numbered scan row (where the B pixel is located at the left and right adjacent coordinates of the G pixel) is as shown in formula (7), where (G j1,i1-1 -B j1,i1-1 ), (G j1，i1+1 -B j1，i1+1 ) is G j1,i1 The initial noise reduction color difference of the left and right adjacent B pixels.

[0078]

[0079] The search window is located at pixel (j1, i1) of pixel G in the odd-numbered scan row (where pixel B is located at the upper and lower adjacent coordinates of pixel G). The initial noise reduction color difference is shown in formula (8), where ((G j-1,i1 -B j1-1,i1 ), (G j1+1，i1 -B j1+1，i1 ) is G j1，i1 The initial noise reduction color difference of the adjacent B pixels.

[0080]

[0081] The initial denoising color difference of another non-G pixel (R pixel) within the search window is also calculated based on the average of the initial denoising color differences of multiple non-G pixels (B pixels). Specifically, the initial denoising color difference of the R pixel is calculated based on the average of the initial denoising color differences of a second number (4 in this embodiment) of B pixels surrounding each R pixel. The initial denoising color difference of the R pixel (j1,i1) is shown in formula (9). The coordinates of adjacent B pixels include: (j1-1,i1-1), (j1-1,i1+1), (j1+1,i1-1), (j1+1,i1+1), and the initial denoising color differences of adjacent B pixels are respectively:

[0082] (G j1-1,i1-1 -B j1-1,i1-1 ), (G j1-1,i1+1 -B j1-1,i1+1 ), (G j1+1,i1-1 -B j1+1,i1-1 ), (G j1+1,i1+1 -B j1+1,i1+1 ).

[0083]

[0084] like Figure 6 As shown, taking pixel B33 with center pixel B as an example, the calculation method of the initial noise reduction color difference of each pixel within the search window 240 is explained. The initial noise reduction color difference of any B pixel within the search window 240 is calculated according to formula (10). Taking the initial noise reduction color difference of B11 as an example, The initial denoised color difference of other B pixels (B13, B15, B31, B33, B35, B51, B53, B55) is calculated similarly and will not be repeated here. The initial denoised color difference of any R pixel within the search window 240 is calculated according to formula (13). Taking the initial denoised color difference of R22 as an example,

[0085] The initial noise reduction color difference calculation method for R pixels (R22, R24, R42, R44) within other search windows 240 is similar and will not be repeated here. The initial noise reduction color difference of G pixels (the pixels above and below G pixels are R pixels, and the pixels to the left and right are B pixels) in any even-numbered scan line within search window 240 is calculated according to formula (11). Taking the initial noise reduction color difference of G12 as an example,

[0086] The initial noise reduction color difference calculation method for the G pixels (G14, G32, G34, G52, G54) in the other even-numbered scan rows is similar and will not be repeated here. The initial noise reduction color difference of the G pixels (the pixels above and below the G pixel are B pixels) in any odd-numbered scan row within the search window 240 is calculated according to formula (12). Taking the initial noise reduction color difference of G21 as an example, The initial noise reduction color difference calculation method for the remaining odd-numbered scan rows of G pixels (G23, G25, G41, G43, G45) is the same, and will not be repeated here.

[0087] The final denoised color difference of non-G pixels within the search window is the weighted sum of the initial denoised color difference of each pixel (every R pixel, B pixel, or G pixel within the search window range) and the denoising coefficient of that pixel, which is used as the final denoised color difference of the center pixel.

[0088] If the center pixel of the search window is R pixels, then the final noise reduction color difference of R pixels As shown in formula (14), (j,i) are the coordinates of the center R pixel, and (j1,i1) are the coordinates of any pixel within the search window. W is the initial noise reduction color difference calculated according to formulas (6) to (9). (j1，i1) The noise reduction coefficient is calculated in step S120.

[0089]

[0090] If the center pixel of the search window is pixel B, then pixel B will ultimately have its color difference denoised. As shown in formula (15), (j,i) are the coordinates of the center B pixel, and (j1,i1) are the coordinates of any pixel within the search window. W is the initial noise reduction color difference calculated according to formulas (10) to (13). (j1，i1) The noise reduction coefficient is calculated in step S120.

[0091]

[0092] The difference between the initial denoised component and the final denoised color difference of the center pixel (j, i) (non-G pixel) of the search window is used as the final denoised component corresponding to the non-G pixel in the denoised Bayer image. If the center pixel of the search window is R pixel, the final denoised component R′ is... j，i The calculation is shown in formula (16), where G j,i This is the initial noise reduction component. The final noise reduction color difference is calculated according to formula (14).

[0093]

[0094] If the center pixel of the search window is pixel B, the final noise reduction component B′(j,i) is calculated as shown in formula (16), where G j,i This is the initial noise reduction component. The final noise reduction color difference is calculated according to formula (17).

[0095]

[0096] The final denoising component of each G pixel in the original Bayer image was calculated according to formula (5), the final denoising component of each R pixel in the original Bayer image was calculated according to formula (16), and the final denoising component of each B pixel in the original Bayer image was calculated according to formula (17), thus obtaining the denoised Bayer image.

[0097] As can be seen from the above calculation process, taking a search window size of 5*5 as an example, if the center of the search window is a G pixel, the final noise reduction component of the G pixel is the weighted sum of the initial noise reduction components of all 25 pixels in the search window and the corresponding noise reduction coefficients. If the center of the search window is a non-G pixel, the final noise reduction component of the non-G pixel is the difference between the initial noise reduction component and the final noise reduction color difference. The final noise reduction color difference is the weighted sum of the initial noise reduction color difference of all 25 pixels in the search window and the corresponding noise reduction coefficients. This ensures that the final noise reduction components of the G pixel and the non-G pixel undergo the same number of noise reduction coefficients, making the noise reduction intensity of the G pixel and the non-G pixel consistent, resulting in better noise reduction and effectively mitigating the grid effect.

[0098] In step S110, calculating the G component of each non-G pixel in the original Bayer image through interpolation specifically includes:

[0099] S210, calculate the horizontal interpolation G component and vertical interpolation G component of each non-G pixel (R pixel, B pixel) in the original Bayer image by interpolation, and calculate the horizontal interpolation non-G component and vertical interpolation non-G component of each G pixel. Then calculate the sum of the color difference gradients. The difference between the original component and the horizontal interpolation component of each pixel in the original Bayer image is used to obtain the horizontal color difference, and the difference between the original component and the vertical interpolation component is used to obtain the vertical color difference.

[0100] It should be noted that there are many interpolation methods in the existing technology. This step does not impose specific restrictions on the interpolation calculation method. This embodiment uses gradient interpolation as an example for explanation. Horizontal interpolation is calculated using the horizontal adjacent pixels of the pixel being interpolated, and vertical interpolation is calculated using the vertical adjacent pixels of the pixel being interpolated. The horizontal interpolation G component of pixel R is calculated using the gradient interpolation method as shown in equation (18), and the vertical interpolation G component of pixel R is shown in equation (19), where the coordinates of pixel R are (j, i), and the horizontal interpolation G component is G. j，i H The vertical interpolation G component is G j，i V .

[0101] G j,i H =(G j，i-1 +G j，i+1 ) / 2+(2*Rj，i -R j，i-2 -R j，i+2 ) / 4 (18)

[0102] G j，i v =(G j-1,i +G j+1，i ) / 2+(2*R j，i -R j-2，i -R j+2,i ) / 4 (19)

[0103] Similarly, the horizontal interpolation G component of pixel B is shown in equation (20), and the vertical interpolation G component is shown in equation (21), where the coordinates of pixel B are (j, i), and the horizontal interpolation G component is G. j,i H The vertical interpolation G component is G j,i V .

[0104] G j，i H =(G j，i-1 +G j，i+1 ) / 2+(2*B j，i -B j，i-2 -B j,i+2 ) / 4 (20)

[0105] G j，i v =(G j-1,i +G j,i，i ) / 2+(2*B j，i -B j-2，i -B j+2,,i ) / 4 (21)

[0106] For pixel G, if it is located in an odd-numbered scan line, the R component needs to be interpolated. The coordinates of pixel G are (j, i), and the horizontal interpolation R component is R. j，i H The calculation formula for (horizontal interpolation non-G component) is shown in equation (22), and the vertical interpolation R component R j，i V The calculation formula for (vertical interpolation non-G component) is shown in equation (23).

[0107] R j，i H =(R j，i-1 +R j，i+1 ) / 2+(2*G j,i -G j，i-2 -G j，i+2 ) / 4 (22)

[0108] R j，iV =(R j-1，i +R j+1,i ) / 2+(2*G j，i -G j-2，i -G j+2,i ) / 4 (23)

[0109] For pixel G, if it is located in an even-numbered scan line, the B component needs to be interpolated. The coordinates of pixel G are (j, i), and the horizontal interpolation B component is B. j，i H The calculation formula for (horizontal interpolation non-G component) is shown in equation (24), and the vertical interpolation B component B j，i V The calculation formula for (vertical interpolation non-G component) is shown in equation (25).

[0110] B j，i H = (B j，i-1 +B j，i+1 ) / 2+(2*G j,i -G j，i-2 -G j，i+2 ) / 4 (24)

[0111] B j，i V = (B j-1，i +B j+1,i ) / 2+(2*G j，i -G j-2,i -G j+2,i ) / 4 (25)

[0112] The horizontal interpolation G component and vertical interpolation G component of the R pixel, the horizontal interpolation G component and vertical interpolation G component of the B pixel, and the horizontal interpolation non-G component and vertical interpolation non-G component of the G pixel (i.e., the horizontal interpolation R component and vertical interpolation R component of the G pixel for odd-numbered scan lines, and the horizontal interpolation B component and vertical interpolation B component of the G pixel for even-numbered scan lines) of the original Bayer image were calculated using the above formulas (18) to (25).

[0113] The horizontal color difference is obtained by calculating the difference between the original component and the horizontal interpolation component of each pixel in the original Bayer image, and the vertical color difference is obtained by calculating the difference between the original component and the vertical interpolation component. It should be noted that the horizontal / vertical color difference is the horizontal / vertical interpolation component minus the original component, or it can be the original component minus the horizontal / vertical interpolation component. Formulas (26) to (33) provide a specific explanation for the former case, as follows:

[0114] Horizontal color difference of pixel B (coordinates (j,i)) For horizontal interpolation of G components G j，i HThe difference between the B component and the B component is shown in Equation (26), which represents the vertical color difference of the B pixel. For vertical interpolation G component G j,i V The difference between the component B and component B is shown in equation (27).

[0115]

[0116] Horizontal color difference of pixel R (coordinates (j,i)) For horizontal interpolation of G components G j,i H The difference between the R component and the R component is shown in Equation (28), which represents the vertical color difference of the R pixel. For vertical interpolation G component G j,i V The difference between the component and the R component is shown in equation (29).

[0117]

[0118]

[0119] For pixel G (coordinates (j,i)), if it is located in an odd-numbered scan line, its horizontal color difference... For the G component and the horizontal interpolation R component R j,i H The difference, as shown in equation (30), is its vertical color difference. For the G component and the vertical interpolation R component R j,i V The difference is shown in equation (31).

[0120]

[0121] For pixel G (coordinates (j,i)), if it is located in an even-numbered scan line, its horizontal color difference... For the G component and the horizontal interpolation B component B j，i H The difference, as shown in equation (32), is its vertical color difference. For the G component and the vertical interpolation B component B j，i V The difference is shown in equation (33).

[0122]

[0123] The horizontal and vertical color differences of each pixel are calculated according to equations (26) to (33).

[0124] S220, calculate the absolute value of the difference in horizontal color difference between two adjacent pixels in the horizontal direction of each pixel in the original Bayer image to obtain the horizontal color difference gradient. (As shown in equation (34), equation (34) is applicable to calculating B pixels or G pixels in even-numbered scan rows. The same applies to R pixels and G pixels in odd-numbered scan rows. The specific formula will not be given here.) The vertical color difference gradient is obtained by taking the absolute value of the difference between the vertical color differences of two adjacent pixels in the vertical direction. (As shown in Equation (35), Equation (35) is applicable to calculating B pixels or G pixels of even-numbered scan lines. The same applies to R pixels and G pixels of odd-numbered scan lines. The specific formula will not be given here.)

[0125]

[0126] S230, calculate the sum of the horizontal and vertical color difference gradients of pixels within a preset window centered on each pixel of the original Bayer image, and use these as the horizontal weight index diff for that central pixel. H and vertical weight index diff V Find the horizontal weight index (diff) of each pixel in the original Bayer image in the G component weight table w_table. H (Horizontal index) matching G component horizontal weight W H And the vertical weight index diff of each pixel in the original Bayer image. V (Vertical Index) Matching G Component Vertical Weight W V ; where, in the G component weight table w_table, the G component level weight W H diff with horizontal weight index H They are negatively correlated, with the G component having a vertical weight W. V diff with vertical weight index V They are negatively correlated, with the horizontal weight index diff. H diff with vertical weight index V The calculation formulas are as shown in equations (36) and (37).

[0127]

[0128] Where (j,i) represents any pixel within the preset window, and the range of values ​​for (j,i) is the range of the preset window. This represents the horizontal color difference gradient of any pixel within the preset window. This represents the vertical color gradient of any pixel within the preset window.

[0129] Find the horizontal weight index diff of each pixel in the original Bayer image. H Matched G component level weights W H And the vertical weight index diff of each pixel in the original Bayer image. VMatching G component vertical weight W V The horizontal weight index diff is stored in the G component weight table w_table. H and G component level weight W H The matching relationship between them, and the vertical weight index diff V Perpendicular weights W of the G component V The matching relationships between them are shown in equations (38) and (39).

[0130] W H =w_table[diff H (38)

[0131] W V =w_table[diff V (39)

[0132] S240, determine the color difference weighting for each non-G pixel in the original Bayer image, wherein the color difference weighting is the sum of a first product and a second product, and the first product is the average horizontal color difference of the even number M (M greater than 1) pixels centered on each non-G pixel (taking B pixel as an example, B(j,i)) and the central pixel. The proportion of the horizontal weight of the G component of the non-G pixel in the total weight (W) H / W T The product of the two pixels. The second product is the average of the vertical color differences of the even number of pixels centered on the non-G pixel and the central pixel. The percentage (W) of the vertical weight of the G component relative to the non-G pixel in the total weight. V / W T The product; where the total weight W T The horizontal weight W of the G component of the non-G pixel. H Perpendicular weights W of the G component V sum.

[0133] Taking M equals 2 as an example, the weighted color difference of B pixels is shown in equation (40), where W T =W H +W V The position of pixel B is B(j, i).

[0134]

[0135] Taking M equals 2 as an example, the weighted color difference of R pixels is shown in equation (41).

[0136]

[0137] Among them, W T =W H +WV The position of pixel R is R(j, i).

[0138] S250, the weighted sum of the original component of each non-G pixel in the original Bayer image and the color difference of that non-G pixel is used as the G component guide map of the original Bayer image for that non-G pixel.

[0139] Specifically, the G component of pixel R is shown in formula (42), where R j，i For the original components, Weighted chromatic difference for R pixels.

[0140] Specifically, the G component of pixel B is shown in formula (43), where B j,i For the original components, Weighted chromatic difference for B pixels.

[0141]

[0142] In this embodiment, interpolation methods are used to calculate the horizontal and vertical interpolated G components of each non-G pixel, as well as the horizontal and vertical interpolated non-G components of each G pixel. Then, the horizontal and vertical color difference gradients of each pixel are calculated. Using the sum of the horizontal and vertical color difference gradients within a preset window as indices, the horizontal and vertical weights of the G components are looked up in the G component weight table. The color difference weighting of the non-G pixel is then calculated based on these weights. The original component of each non-G pixel and the weighted color difference of that non-G pixel are summed to obtain the G component of the non-G pixel, thus obtaining the G component guide map. The G component guide map obtained by this scheme is more accurate and more conducive to subsequent Bayer image processing. For example, the denoised Bayer image obtained by denoising the G component guide map is more effective. Furthermore, the color difference weighting obtained based on the G component guide map is more accurate, resulting in a better de-mosaic image.

[0143] In this embodiment of the invention, the Bayer image processing method further includes a demosaicing step of performing a demosaicing process on the denoised Bayer image to obtain a demosaic image. The demosaicing step involves calculating the values ​​of the two missing color components of the R pixel, G pixel, and B pixel, respectively.

[0144] Specifically, using the initial denoising component (calculated according to formula (4)) obtained during the Bayer image processing as the G component in the R / B pixels of the de-mosaic image is beneficial to reduce the amount of computation.

[0145] For each R pixel in the denoised Bayer image, the average of the final denoised components of all B pixels within a preset window centered on that R pixel is taken as the B component of that R pixel in the de-mosaic image. Taking a preset window size of 3*3 pixels as an example, the B component of the R pixel... j,i The calculation formula is shown in equation (44). The coordinates of the center R pixel are (j,i), and the coordinates of all B pixels in the preset window are (j-1,i-1), (j-1,i+1), (j+1,i+1), (j+1,i-1).

[0146] B j，i = (B j-1，i-1 +B j-1,i+1 +B j+1，i+1 +B j+1，i-1 ) / 4 (44)

[0147] For each B pixel in the denoised Bayer image, the average of the final denoised components of all R pixels within a preset window centered on that B pixel is taken as the R component of that B pixel in the de-mosaic image. Taking a preset window size of 3*3 pixels as an example, the R component R of the B pixel... j,i The calculation formula is shown in equation (45). The coordinates of the center B pixel are (j,i), and the coordinates of all R pixels in the preset window are (j-1,i-1), (j-1,i+1), (j+1,i+1), (j+1,i-1).

[0148] R j，i =(R j-1，i-1 +R j-1,i+1 +R j+1,i+1 +R j+1,i-1 ) / 4 (45)

[0149] For each G pixel in the same row as the R pixel in the denoised Bayer image (i.e., G pixels in odd-numbered scan rows), the B component B of that G pixel in the de-mosaic image is... j，i The sum of the first B sub-component and the second B sub-component, where the first B sub-component is the average of the final noise reduction components of the two adjacent B pixels vertically above and below the G pixel (B0). j-1，i +B j+1，i ) / 2, the second B sub-component is the average of the differences between the initial noise reduction components of the two adjacent B pixels in the vertical direction and the initial noise reduction components of the G pixel, respectively (2*G) j，i -G j-1，i -G j+1，i ) / 2, as shown in formula (46).

[0150] B j，i = (B j-1，i +B j+1,i ) / 2+(2*G j,i-G j-1,i -G j+1,i ) / 2 (46)

[0151] For each G pixel (even-numbered scan row) in the denoised Bayer image that is in the same row as the B pixel, that G pixel removes the B component B from the mosaic image. j,i The sum of the first B sub-component and the second B sub-component, where the first B sub-component is the average of the final noise reduction components of the two B pixels horizontally adjacent to the G pixel (B0). j,i-1 +B j,i+1 The second B sub-component is the average of the differences between the initial noise reduction components of the two B pixels on the left and right sides in the horizontal direction, minus the initial noise reduction components of the G pixel (2*G). j,i -G j,i-1 -G j,i+1 ) / 2.

[0152] B j，i = (B j，i-1 +B j,i+1 ) / 2+(2*G j，i -G j,i-1 -G j,i+1 ) / 2 (47)

[0153] For each G pixel in the same row as the R pixel in the denoised Bayer image, the R component of that G pixel in the desacrifice image is the sum of the first R sub-component and the second R sub-component, where the first R sub-component (R... j，i-1 +R j，i+1 ) / 2 is the average of the final noise reduction components of the two adjacent R pixels in the horizontal direction of the G pixel, and the second R sub-component (2*G) j，i -G j，i-1 -G j，i+1 ) / 2 is the average of the difference between the initial noise reduction components of the two R pixels on the left and right sides in the horizontal direction, respectively, and the initial noise reduction components of the G pixel.

[0154] R j,i =(R j,i-1 +R j,i+1 ) / 2+(2*G j,i -G j,i -1-G j,i+1 ) / 2 (48)

[0155] For each G pixel in the same row as a B pixel in the denoised Bayer image, the R component in the desacrifice image of that G pixel is the sum of the first B sub-component and the second B sub-component. The first B sub-component (R(j-1,i)+R(j+1,i)) / 2 is the average of the final denoised components of the two adjacent R pixels vertically above and below that G pixel. The second B sub-component (2*G) / 2 is the average of the denoised components of the two adjacent R pixels in the vertical direction of that G pixel. j，i-G(j-1,i)-G(j+1,i) is the average of the difference between the initial noise reduction components of the two adjacent R pixels in the vertical direction and the initial noise reduction components of the initial noise reduction components of the G pixel.

[0156] R j，i =(R j-1,i +R j+1,i ) / 2+(2*G j,i -G j-1,i -G j+1,i ) / 2 (49)

[0157] This invention also discloses a Bayer image processing apparatus based on a G-component guided map, comprising: a G-component guided map generation module, used to calculate the G component of each non-G pixel in the original Bayer image by interpolation, wherein the G components of the non-G pixels and the G components of the G pixels in the original Bayer image form a G-component guided map; and a noise reduction coefficient calculation module, used to take a first-sized image block centered on each pixel in the G-component guided map as a template window and a second-sized image block as a search window, wherein the second size is larger than the first size, and sequentially select comparison windows of the same size as the template window centered on each pixel in the search window, calculate the sum of the absolute values ​​of the differences between the G component of each pixel in each comparison window and the G component of the corresponding pixel in the template window, and use this sum as the noise reduction factor of the center pixel of the comparison window, and select a noise reduction coefficient from a noise reduction weight table that matches the ratio of the noise reduction factor to the noise reduction intensity of each pixel in the search window, and use this coefficient as the noise reduction coefficient of that pixel; wherein... In the noise reduction weight table, the larger the ratio of the noise reduction factor to the noise reduction intensity of a pixel, the smaller the corresponding noise reduction coefficient. The final noise reduction component calculation module is used to take the weighted sum of the G component of each pixel in the search window centered on the G component guide map and the noise reduction coefficient of that pixel as the initial noise reduction component of the center pixel. If the center pixel is a G pixel, then the initial noise reduction component of the center pixel is taken as the final noise reduction component of the corresponding G pixel in the noise-reduced Bayer image. If the center pixel is a non-G pixel, then in the search window, the difference between the G component of the non-G pixel and the original component is calculated as the initial noise reduction color difference of the non-G pixel. The initial noise reduction color difference between the G pixel and another non-G pixel is determined according to the average of the initial noise reduction color differences of multiple non-G pixels. The weighted sum of the initial noise reduction color difference of each pixel and the noise reduction coefficient of that pixel is taken as the final noise reduction color difference of the center pixel. The difference between the initial noise reduction component of the center pixel and the final noise reduction color difference is taken as the final noise reduction component of the corresponding non-G pixel in the noise-reduced Bayer image.

[0158] The present invention also discloses a processing device for Bayer images based on G-component guided maps, which processes Bayer images using the aforementioned processing method, or includes the aforementioned processing device.

[0159] The present invention also discloses a computer-readable storage medium having a computer program stored thereon, which, when executed, can implement the processing method as described above.

[0160] The present invention also discloses a processing chip for Bayer images based on G-component guided maps, comprising a processor and a memory, wherein the memory stores a computer program, and the processor is capable of executing the computer program to implement the processing method described above.

[0161] Those skilled in the art will understand that, without conflict, the above-mentioned preferred solutions can be freely combined and superimposed.

[0162] It should be understood that the above embodiments are merely exemplary and not restrictive. Various obvious or equivalent modifications or substitutions that can be made by those skilled in the art regarding the above details without departing from the basic principles of the present invention will be included within the scope of the claims of the present invention.

Claims

1. A method for processing Bayer images based on G-component guided maps, characterized in that, Includes the following steps: The G component of each non-G pixel in the original Bayer image is calculated by interpolation, and the G component of the non-G pixel and the G component of the G pixel in the original Bayer image form a G component guide map. Using a first-sized image block centered on each pixel in the G-component guiding image as a template window and a second-sized image block as a search window (where the second size is larger than the first size), comparison windows of the same size as the template window are sequentially selected, centered on each pixel in the search window. The sum of the absolute values ​​of the differences between the G-component of each pixel in each comparison window and the G-component of the corresponding pixel in the template window is calculated and used as the noise reduction factor of the central pixel of the comparison window. A noise reduction coefficient matching the ratio of the noise reduction factor to the noise reduction intensity of each pixel in the search window is selected from the noise reduction weight table and used as the noise reduction coefficient of that pixel. The larger the ratio of the noise reduction factor to the noise reduction intensity of a pixel in the noise reduction weight table, the smaller the corresponding noise reduction coefficient. The weighted sum of the G component of each pixel in the search window centered on the G component guide image and the denoising coefficient of that pixel is used as the initial denoising component of the center pixel. If the center pixel is a G pixel, the initial denoising component of the center pixel is used as the final denoising component of the corresponding G pixel in the denoised Bayer image. If the center pixel is a non-G pixel, the difference between the G component of the non-G pixel and the original component is calculated in the search window as the initial denoising color difference of the non-G pixel. The initial denoising color difference between the G pixel and another non-G pixel is determined based on the average of the initial denoising color differences of multiple non-G pixels. The weighted sum of the initial denoising color difference of each pixel and the denoising coefficient of that pixel is used as the final denoising color difference of the center pixel. The difference between the initial denoising component of the center pixel and the final denoising color difference is used as the final denoising component of the corresponding non-G pixel in the denoised Bayer image.

2. The processing method according to claim 1, characterized in that, If the center pixel of the search window is an R pixel, the initial noise reduction color difference between a G pixel and another non-G pixel is determined based on the average of the initial noise reduction color differences of multiple non-G pixels as follows: The initial noise reduction color difference of each G pixel is determined by the average of the initial noise reduction color differences of the first number of R pixels surrounding each G pixel; the initial noise reduction color difference of each B pixel is determined by the average of the initial noise reduction color differences of the second number of R pixels surrounding each B pixel. If the center pixel of the search window is pixel B, the initial noise reduction color difference between pixel G and another non-G pixel is determined based on the average of the initial noise reduction color differences of multiple non-G pixels as follows: The initial denoising color difference of each G pixel is determined by the average of the initial denoising color differences of the first number of B pixels surrounding it; and the initial denoising color difference of each R pixel is determined by the average of the initial denoising color differences of the second number of B pixels surrounding it.

3. The processing method according to claim 1, characterized in that, The calculation of the G component of each non-G pixel in the original Bayer image by interpolation specifically includes: The horizontal interpolation G component and the vertical interpolation G component of each non-G pixel in the original Bayer image are calculated by interpolation, and the horizontal interpolation non-G component and the vertical interpolation non-G component of each G pixel are calculated. Then, the difference between the original component and the horizontal interpolation component of each pixel in the original Bayer image is calculated to obtain the horizontal color difference, and the difference between the original component and the vertical interpolation component is calculated to obtain the vertical color difference. The horizontal color difference gradient is obtained by calculating the absolute value of the difference between the horizontal color difference of two adjacent pixels in the horizontal direction of each pixel in the original Bayer image, and the vertical color difference gradient is obtained by calculating the absolute value of the difference between the vertical color difference of two adjacent pixels in the vertical direction. Calculate the sum of the horizontal and vertical color difference gradients of pixels within a preset window centered on each pixel of the original Bayer image. These sums are used as the horizontal and vertical weight indices of the central pixel, respectively. Then, search the G-component weight table for the G-component horizontal weight that matches the horizontal weight index of each pixel in the original Bayer image and the G-component vertical weight that matches the vertical weight index of each pixel in the original Bayer image. In the G-component weight table, the G-component horizontal weight is negatively correlated with the sum of the horizontal color difference gradients, and the G-component vertical weight is negatively correlated with the sum of the vertical color difference gradients. Determine the color difference weighting for each non-G pixel in the original Bayer image, wherein the color difference weighting is the sum of a first product and a second product. The first product is the product of the average horizontal color difference of an even number of pixels centered on the non-G pixel and the center pixel, and the proportion of the horizontal weight of the G component of the non-G pixel in the total weight. The second product is the product of the average vertical color difference of an even number of pixels centered on the non-G pixel and the center pixel, and the proportion of the vertical weight of the G component of the non-G pixel in the total weight. The total weight is the sum of the horizontal weight of the G component and the vertical weight of the G component of the non-G pixel. The weighted sum of the original component of each non-G pixel in the original Bayer image and the color difference of that non-G pixel is used as the G component guide map of the original Bayer image for that non-G pixel.

4. The processing method according to claim 2, characterized in that, It also includes the following steps: For each R pixel in the denoised Bayer image, the average of the final denoised components of all B pixels within a preset window centered on that R pixel is taken as the B component of that R pixel in the de-mosaic image. For each B pixel in the denoised Bayer image, the average of the final denoised components of all R pixels within a preset window centered on that B pixel is taken as the R component of that B pixel in the de-mosaic image.

5. The processing method according to claim 4, characterized in that, It also includes the following steps: For each G pixel in the same row as the R pixel in the denoised Bayer image, the B component of the G pixel in the demosaic image is the sum of the first B sub-component and the second B sub-component. The first B sub-component is the average of the final denoised components of the two B pixels that are vertically adjacent to the G pixel. The second B sub-component is the average of the difference between the initial denoised components of the G pixel and the initial denoised components of the two B pixels that are vertically adjacent to the G pixel. For each G pixel in the same row as the B pixel in the denoised Bayer image, the B component in the demosaic image of the G pixel is the sum of the first B sub-component and the second B sub-component. The first B sub-component is the average of the final denoised components of the two B pixels that are horizontally adjacent to the G pixel. The second B sub-component is the average of the difference between the initial denoised components of the G pixel and the initial denoised components of the two B pixels that are horizontally adjacent to the G pixel.

6. The processing method according to claim 4, characterized in that, It also includes the following steps: For each G pixel in the same row as the R pixel in the denoised Bayer image, the R component of the G pixel in the demosaic image is the sum of the first R sub-component and the second R sub-component. The first R sub-component is the average of the final denoised components of the two R pixels that are horizontally adjacent to the G pixel. The second R sub-component is the average of the difference between the initial denoised components of the G pixel and the initial denoised components of the two R pixels that are horizontally adjacent to the G pixel. For each G pixel in the same row as the B pixel in the denoised Bayer image, the R component in the demosaic image of the G pixel is the sum of a first R sub-component and a second R sub-component. The first R sub-component is the average of the final denoised components of the two R pixels that are vertically adjacent to the G pixel. The second R sub-component is the average of the difference between the initial denoised components of the G pixel and the initial denoised components of the two R pixels that are vertically adjacent to the G pixel.

7. A processing apparatus for Bayer images based on G-component guided maps, characterized in that, include: The G-component guide map generation module is used to calculate the G component of each non-G pixel in the original Bayer image by interpolation. The G component of the non-G pixel and the G component of the G pixel in the original Bayer image form a G-component guide map. The noise reduction coefficient calculation module is used to take a first-sized image block centered on each pixel in the G-component guiding image as a template window and a second-sized image block as a search window, wherein the second size is larger than the first size. It sequentially selects comparison windows of the same size as the template window, centered on each pixel in the search window, and calculates the sum of the absolute values ​​of the differences between the G-component of each pixel in each comparison window and the G-component of the corresponding pixel in the template window. This sum is used as the noise reduction factor of the central pixel of the comparison window. The module then selects a noise reduction coefficient from the noise reduction weight table that matches the ratio of the noise reduction factor to the noise reduction intensity of each pixel in the search window, and uses this coefficient as the noise reduction coefficient for that pixel. The larger the ratio of the noise reduction factor to the noise reduction intensity of a pixel in the noise reduction weight table, the smaller the corresponding noise reduction coefficient. The final noise reduction component calculation module is used to take the weighted sum of the G component of each pixel in the search window centered on each pixel in the G component guide image and the noise reduction coefficient of that pixel as the initial noise reduction component of the center pixel. If the center pixel is a G pixel, the initial noise reduction component of the center pixel is taken as the final noise reduction component of the corresponding G pixel in the noise-reduced Bayer image. If the center pixel is a non-G pixel, the module calculates the difference between the G component and the original component of the non-G pixel in the search window as the initial noise reduction color difference of the non-G pixel. The module also determines the initial noise reduction color difference between the G pixel and another non-G pixel based on the average of the initial noise reduction color differences of multiple non-G pixels. The module takes the weighted sum of the initial noise reduction color difference of each pixel and the noise reduction coefficient of that pixel as the final noise reduction color difference of the center pixel. Finally, the module takes the difference between the initial noise reduction component of the center pixel and the final noise reduction color difference as the final noise reduction component of the corresponding non-G pixel in the noise-reduced Bayer image.

8. The processing apparatus according to claim 7, characterized in that, If the center pixel of the search window is an R pixel, the initial noise reduction color difference between a G pixel and another non-G pixel is determined based on the average of the initial noise reduction color differences of multiple non-G pixels as follows: The initial noise reduction color difference of each G pixel is determined by the average of the initial noise reduction color differences of the first number of R pixels surrounding each G pixel; the initial noise reduction color difference of each B pixel is determined by the average of the initial noise reduction color differences of the second number of R pixels surrounding each B pixel. If the center pixel of the search window is pixel B, the initial noise reduction color difference between pixel G and another non-G pixel is determined based on the average of the initial noise reduction color differences of multiple non-G pixels as follows: The initial denoising color difference of each G pixel is determined by the average of the initial denoising color differences of the first number of B pixels surrounding it; and the initial denoising color difference of each R pixel is determined by the average of the initial denoising color differences of the second number of B pixels surrounding it.

9. The processing apparatus according to claim 7, characterized in that, The calculation of the G component of each non-G pixel in the original Bayer image by interpolation specifically includes: The horizontal interpolation G component and the vertical interpolation G component of each non-G pixel in the original Bayer image are calculated by interpolation, and the horizontal interpolation non-G component and the vertical interpolation non-G component of each G pixel are calculated. Then, the difference between the original component and the horizontal interpolation component of each pixel in the original Bayer image is calculated to obtain the horizontal color difference, and the difference between the original component and the vertical interpolation component is calculated to obtain the vertical color difference. The horizontal color difference gradient is obtained by calculating the absolute value of the difference between the horizontal color difference of two adjacent pixels in the horizontal direction of each pixel in the original Bayer image, and the vertical color difference gradient is obtained by calculating the absolute value of the difference between the vertical color difference of two adjacent pixels in the vertical direction. Calculate the sum of the horizontal and vertical color difference gradients of pixels within a preset window centered on each pixel of the original Bayer image. These sums are used as the horizontal and vertical weight indices of the central pixel, respectively. Then, search the G-component weight table for the G-component horizontal weight that matches the horizontal weight index of each pixel in the original Bayer image and the G-component vertical weight that matches the vertical weight index of each pixel in the original Bayer image. In the G-component weight table, the G-component horizontal weight is negatively correlated with the sum of the horizontal color difference gradients, and the G-component vertical weight is negatively correlated with the sum of the vertical color difference gradients. Determine the color difference weighting for each non-G pixel in the original Bayer image, wherein the color difference weighting is the sum of a first product and a second product. The first product is the product of the average horizontal color difference of an even number of pixels centered on the non-G pixel and the center pixel, and the proportion of the horizontal weight of the G component of the non-G pixel in the total weight. The second product is the product of the average vertical color difference of an even number of pixels centered on the non-G pixel and the center pixel, and the proportion of the vertical weight of the G component of the non-G pixel in the total weight. The total weight is the sum of the horizontal weight of the G component and the vertical weight of the G component of the non-G pixel. The weighted sum of the original component of each non-G pixel in the original Bayer image and the color difference of that non-G pixel is used as the G component guide map of the original Bayer image for that non-G pixel.

10. The processing apparatus according to claim 8, characterized in that, It also includes the first demosaic component calculation module: For each R pixel in the denoised Bayer image, the average of the final denoised components of all B pixels within a preset window centered on that R pixel is taken as the B component of that R pixel in the de-mosaic image. For each B pixel in the denoised Bayer image, the average of the final denoised components of all R pixels within a preset window centered on that B pixel is taken as the R component of that B pixel in the de-mosaic image.

11. The processing apparatus according to claim 10, characterized in that, It also includes a second demosaic component calculation module: For each G pixel in the same row as the R pixel in the denoised Bayer image, the B component of the G pixel in the demosaic image is the sum of the first B sub-component and the second B sub-component. The first B sub-component is the average of the final denoised components of the two B pixels that are vertically adjacent to the G pixel. The second B sub-component is the average of the difference between the initial denoised components of the G pixel and the initial denoised components of the two B pixels that are vertically adjacent to the G pixel. For each G pixel in the same row as the B pixel in the denoised Bayer image, the B component in the demosaic image of the G pixel is the sum of the first B sub-component and the second B sub-component. The first B sub-component is the average of the final denoised components of the two B pixels that are horizontally adjacent to the G pixel. The second B sub-component is the average of the difference between the initial denoised components of the G pixel and the initial denoised components of the two B pixels that are horizontally adjacent to the G pixel.

12. The processing apparatus according to claim 10, characterized in that, It also includes a third demosaic component calculation module: For each G pixel in the same row as the R pixel in the denoised Bayer image, the R component of the G pixel in the demosaic image is the sum of the first R sub-component and the second R sub-component. The first R sub-component is the average of the final denoised components of the two R pixels that are horizontally adjacent to the G pixel. The second R sub-component is the average of the difference between the initial denoised components of the G pixel and the initial denoised components of the two R pixels that are horizontally adjacent to the G pixel. For each G pixel in the same row as the B pixel in the denoised Bayer image, the R component in the demosaic image of the G pixel is the sum of a first R sub-component and a second R sub-component. The first R sub-component is the average of the final denoised components of the two R pixels that are vertically adjacent to the G pixel. The second R sub-component is the average of the difference between the initial denoised components of the G pixel and the initial denoised components of the two R pixels that are vertically adjacent to the G pixel.

13. A processing device for Bayer images based on G-component guided maps, characterized in that, The Bayer image is processed using the processing method described in any one of claims 1-6, or the processing apparatus described in any one of claims 7-12 is also used.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it can implement the processing method as described in any one of claims 1-6.

15. A Bayer image processing chip based on G-component guided maps, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program, and the processor is capable of executing the computer program to implement the processing method as described in any one of claims 1-6.

Images