Bayer image processing method, device, storage medium, chip and equipment

Abstract

The present invention discloses a Bayer image processing method, apparatus, storage medium, chip and device. The method uses an interpolation method to calculate the horizontal interpolation G component and vertical interpolation G component of each non-G pixel, as well as the horizontal interpolation non-G component and vertical interpolation non-G component of each G pixel. The horizontal color difference gradient and vertical color difference gradient of each pixel are then calculated. The G component horizontal weight and the G component vertical weight are searched in a G component weight table based on the sum of the horizontal color difference gradient and the sum of the vertical color difference gradient in a preset window as an index. The color difference weight of the non-G pixel is then calculated based on the horizontal weight and the vertical weight. The G component of the non-G pixel is obtained by summing the original component of each non-G pixel and the color difference weight of the non-G pixel to obtain a G component guide map. The G component guide map obtained by this solution is more accurate and more conducive to subsequent Bayer image processing.

Description

Technical Field

[0001] The present invention relates to image processing technology, and in particular to a Bayer image processing method, device, storage medium, chip and equipment. Background Art

[0002] With the rapid development of the media age, more and more ways to capture images and record life, such as video and recording, are becoming available. However, due to limitations in camera performance, ambient light, and noise, some captured images contain significant noise. CMOS image sensors output images in the Bayer image format, which is subject to noise. This noise is subsequently affected to varying degrees during processing in various Image Signal Processing (ISP) modules, making noise reduction in the YUV (luminance and color) domain difficult.

[0003] To obtain a full-color image, the original Bayer image must be demosaiced and subjected to noise reduction. Some existing solutions first interpolate the non-G pixels of the original Bayer image to obtain a G component guidance map, and then filter the original Bayer image directly based on this G component guidance map. However, this method results in an inaccurate G component guidance map, which affects subsequent filtering based on this G component guidance map. Summary of the Invention

[0004] Based on the above situation, the main purpose of the present invention is to provide a Bayer image processing method, device, storage medium, chip and equipment to provide a more accurate G component guide map to improve the effect of subsequent Bayer image processing.

[0005] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] A Bayer image processing method includes the following steps:

[0007] Calculating a horizontal interpolation G component and a vertical interpolation G component of each non-G pixel in the original Bayer image by interpolation, and calculating a horizontal interpolation non-G component and a vertical interpolation non-G component of each G pixel, and then calculating a difference between an original component and a horizontal interpolation component of each pixel in the original Bayer image to obtain a horizontal color difference, and a difference between an original component and a vertical interpolation component to obtain a vertical color difference;

[0008] Calculating the absolute value of the horizontal color difference between the left and right adjacent pixels of each pixel in the original Bayer image to obtain the horizontal color difference gradient, and calculating the absolute value of the vertical color difference between the upper and lower adjacent pixels in the vertical direction to obtain the vertical color difference gradient;

[0009] Calculating the sum of the horizontal color difference gradients and the sum of the vertical color difference gradients of pixels within a preset window centered on each pixel of the original Bayer image, respectively serving as the horizontal weight index and the vertical weight index of the central pixel, and searching a G component horizontal weight that matches the horizontal weight index of each pixel of the original Bayer image and a G component vertical weight that matches the vertical weight index of each pixel of the original Bayer image in a G component weight table; wherein, in the G component weight table, the G component horizontal weight is negatively correlated with the sum of the horizontal color difference gradients, and the G component vertical weight is negatively correlated with the sum of the vertical color difference gradients;

[0010] Determining a color difference weight for each non-G pixel in the original Bayer image, wherein the color difference weight is the sum of a first product and a second product, the first product being the product of an average of horizontal color differences of an even number of pixels centered on the non-G pixel and the center pixel in a horizontal direction and a proportion of the horizontal weight of the G component of the non-G pixel in a total weight, and the second product being the product of an average of vertical color differences of an even number of pixels centered on the non-G pixel and the center pixel in a vertical direction and a proportion of the vertical weight of the G component of the non-G pixel in a total weight, wherein the total weight is the sum of the horizontal weight and the vertical weight of the G component of the non-G pixel;

[0011] The weighted sum of the original component of each non-G pixel in the original Bayer image and the color difference of the non-G pixel is used as the G component of the G component guide map of the original Bayer image at the non-G pixel to obtain the G component guide map.

[0012] Preferably, the method further comprises the following steps:

[0013] Using an image block of a first size centered on each pixel in the G component guide image as a template window and an image block of a second size as a search window, wherein the second size is larger than the first size;

[0014] Selecting comparison windows of the same size as the template window in sequence with a set pixel in the search window as the center, and calculating the sum of the absolute values ​​of the differences between the G component of each pixel in each comparison window and the G component of the pixel at the corresponding position in the template window as the search absolute difference sum of the center pixel of the comparison window;

[0015] performing edge detection on the G component guide image to obtain an edge value of each pixel, comparing the edge value of the pixel with a first edge threshold and a second edge threshold, respectively, and determining the pixel as a weak edge, a flat area, and a strong edge when the edge value is less than the second edge threshold, between the second edge threshold and the first edge threshold, and greater than the first edge threshold, respectively, and setting a noise reduction intensity for the pixel to a third intensity, the first intensity, and the second intensity, respectively; wherein the first intensity, the third intensity, and the second intensity decrease in magnitude in sequence;

[0016] Selecting a noise reduction coefficient that matches the ratio of the search absolute difference and the noise reduction intensity of a set pixel in the search window from the noise reduction weight table as the noise reduction coefficient of the pixel; wherein a larger ratio of the search absolute difference and the noise reduction intensity of the pixel in the noise reduction weight table corresponds to a smaller noise reduction coefficient;

[0017] The weighted sum of the original component of the pixel in the search window centered at each set pixel in the original Bayer image and the noise reduction coefficient of the pixel is used as the noise reduction component of the pixel, thereby obtaining a noise-reduced Bayer image.

[0018] Preferably, performing edge detection on the G component guide map to obtain the edge value of each pixel specifically includes the following steps:

[0019] Select several groups of symmetrical pixel pairs in each direction centered on each pixel of the G component guide map, calculate the absolute value of the difference between the two pixels in the symmetrical pixel pair, and calculate the sum of the absolute values ​​of several groups of symmetrical pixel pairs in each direction as the sub-edge value in that direction, and then calculate the sum of the sub-edge values ​​in several directions to obtain the edge value of the pixel.

[0020] Preferably, the third strength is equal to the product of the second strength coefficient and the second strength plus the product of the first strength coefficient and the first strength;

[0021] The first intensity coefficient is equal to 1 minus the second intensity coefficient, and the second intensity coefficient is equal to the ratio of the difference between the edge value of the pixel and the first edge threshold and the difference between the second edge threshold and the first edge threshold.

[0022] Preferably, if the difference between the horizontal weight index and the vertical weight index of a non-G pixel in the original Bayer image is greater than a preset value, the color difference weighting of the non-G pixel is calculated using only the smaller of the horizontal weight index and the vertical weight index.

[0023] Preferably, the method further comprises the following steps:

[0024] Searching, in a mosaic G component weight table, a mosaic G component horizontal weight that matches the horizontal weight index of each pixel of the original Bayer image, and a mosaic G component vertical weight that matches the vertical weight index of each pixel of the original Bayer image; wherein, in the mosaic G component weight table, the mosaic G component horizontal weight is negatively correlated with the horizontal weight index, and the mosaic G component vertical weight is negatively correlated with the vertical weight index;

[0025] The product of the average of the horizontal color differences of the pixels in an even number of horizontal directions centered on each non-G pixel in the denoised Bayer image and the central pixel and the proportion of the horizontal weight of the mosaic G component of the non-G pixel in the total mosaic weight is summed, and the product of the average of the vertical color differences of the pixels in an even number of vertical directions centered on the non-G pixel and the central pixel and the proportion of the vertical weight of the mosaic G component of the non-G pixel in the total mosaic weight is summed to obtain a mosaic color difference weight of the non-G pixel; wherein the total mosaic weight is the sum of the horizontal weight and the vertical weight of the mosaic G component of the non-G pixel;

[0026] The weighted sum of the noise reduction component of each non-G pixel in the noise reduction Bayer image and the mosaic color difference of the non-G pixel is used as the G component of the demosaiced image at the non-G pixel.

[0027] Preferably, the weighted sum of the noise reduction component of each non-G pixel in the noise reduction Bayer image and the color difference of the non-G pixel is used as the G component of the demosaiced image at the non-G pixel.

[0028] Preferably, the average value of the G component of all pixels in the comparison window is used as the brightness value of the central pixel; the ratio of the brightness value of the central pixel in the comparison window to the preset brightness value is rounded to obtain a brightness index; a brightness compensation / suppression coefficient matching the brightness index of the central pixel in the comparison window is searched in a brightness compensation array, and the product of the search absolute difference sum of the central pixel in the comparison window and the brightness compensation / suppression coefficient is used as a new search absolute difference sum; wherein the brightness index and the brightness compensation coefficient in the brightness compensation array are negatively correlated;

[0029] A noise reduction coefficient that matches the ratio of the new search absolute difference and the noise reduction intensity of the set pixel in the search window is selected from the noise reduction weight table as the noise reduction coefficient of the pixel.

[0030] The present invention also discloses a Bayer image processing device, comprising:

[0031] a color difference calculation module, configured to calculate, by interpolation, a horizontal interpolation G component and a vertical interpolation G component of each non-G pixel in the original Bayer image, and calculate a horizontal interpolation non-G component and a vertical interpolation non-G component of each G pixel, and then calculate the difference between the original component and the horizontal interpolation component of each pixel in the original Bayer image to obtain a horizontal color difference, and the difference between the original component and the vertical interpolation component to obtain a vertical color difference;

[0032] A color difference gradient calculation module is used to calculate the absolute value of the horizontal color difference difference between the two adjacent pixels on the left and right in the horizontal direction of each pixel of the original Bayer image to obtain the horizontal color difference gradient, and the absolute value of the vertical color difference difference between the two adjacent pixels on the top and bottom in the vertical direction to obtain the vertical color difference gradient;

[0033] A weight calculation module is configured to calculate the sum of the horizontal color difference gradients and the sum of the vertical color difference gradients of pixels within a preset window centered on each pixel of the original Bayer image, respectively serving as the horizontal weight index and the vertical weight index of the central pixel, and search a G component horizontal weight that matches the horizontal weight index of each pixel of the original Bayer image, and a G component vertical weight that matches the vertical weight index of each pixel of the original Bayer image in a G component weight table; wherein, in the G component weight table, the G component horizontal weight is negatively correlated with the sum of the horizontal color difference gradients, and the G component vertical weight is negatively correlated with the sum of the vertical color difference gradients;

[0034] a color difference weighted calculation module, configured to determine a color difference weight for each non-G pixel in the original Bayer image, wherein the color difference weight is the sum of a first product and a second product, the first product being the product of an average of horizontal color differences of an even number of pixels centered on the non-G pixel and the center pixel in the horizontal direction and a proportion of the horizontal weight of the G component of the non-G pixel in the total weight; and the second product being the product of an average of vertical color differences of an even number of pixels centered on the non-G pixel and the center pixel in the vertical direction and a proportion of the vertical weight of the G component of the non-G pixel in the total weight, wherein the total weight is the sum of the horizontal weight and the vertical weight of the G component of the non-G pixel;

[0035] The G component guide map generation module is used to take the weighted sum of the original component of each non-G pixel in the original Bayer image and the color difference of the non-G pixel as the G component of the G component guide map of the original Bayer image at the non-G pixel to obtain the G component guide map.

[0036] Preferably, the generating device further comprises the following:

[0037] a search window generating module, configured to use an image block of a first size centered on each pixel in the G component guide image as a template window and an image block of a second size as a search window, wherein the second size is larger than the first size;

[0038] a search absolute difference sum calculation module, configured to sequentially select comparison windows of the same size as the template window with a set pixel in the search window as the center, and calculate the sum of the absolute values ​​of the differences between the G component of each pixel in each comparison window and the G component of the pixel at the corresponding position in the template window as the search absolute difference sum of the center pixel of the comparison window;

[0039] a noise reduction intensity generation module, configured to perform edge detection on the G component guide map to obtain an edge value for each pixel, compare the edge value of the pixel with a first edge threshold and a second edge threshold, respectively, and determine the pixel as a weak edge, a flat area, and a strong edge if the edge value is less than the second edge threshold, between the second edge threshold and the first edge threshold, and greater than the first edge threshold, respectively, and set the noise reduction intensity of the pixel to a third intensity, the first intensity, and the second intensity, respectively; wherein the first intensity, the third intensity, and the second intensity decrease in magnitude in sequence;

[0040] a noise reduction coefficient generation module, configured to select, from a noise reduction weight table, a noise reduction coefficient that matches the ratio of the search absolute difference and the noise reduction intensity of a set pixel in the search window as the noise reduction coefficient of the pixel; wherein, a larger ratio of the search absolute difference and the noise reduction intensity of the pixel in the noise reduction weight table corresponds to a smaller noise reduction coefficient;

[0041] The denoised Bayer image generation module is used to take the weighted sum of the original component of the pixel in the search window centered on each set pixel in the original Bayer image and the noise reduction coefficient of the pixel as the noise reduction component of the pixel, thereby obtaining the denoised Bayer image.

[0042] Preferably, performing edge detection on the G component guide map to obtain an edge value of each pixel specifically includes the following steps:

[0043] Select several groups of symmetrical pixel pairs in each direction centered on each pixel of the G component guide map, calculate the absolute value of the difference between the two pixels in the symmetrical pixel pair, and calculate the sum of the absolute values ​​of several groups of symmetrical pixel pairs in each direction as the sub-edge value in that direction, and then calculate the sum of the sub-edge values ​​in several directions to obtain the edge value of the pixel.

[0044] Preferably, the third strength is equal to the product of the second strength coefficient and the second strength plus the product of the first strength coefficient and the first strength;

[0045] The first intensity coefficient is equal to 1 minus the second intensity coefficient, and the second intensity coefficient is equal to the ratio of the difference between the edge value of the pixel and the first edge threshold and the difference between the second edge threshold and the first edge threshold.

[0046] Preferably, if the difference between the horizontal weight index and the vertical weight index of a non-G pixel in the original Bayer image is greater than a preset value, the color difference weighting of the non-G pixel is calculated using only the smaller of the horizontal weight index and the vertical weight index.

[0047] Preferably, the generating device further comprises:

[0048] a mosaic G component weight calculation module, configured to search a mosaic G component horizontal weight that matches the horizontal weight index of each pixel of the original Bayer image, and a mosaic G component vertical weight that matches the vertical weight index of each pixel of the original Bayer image in a mosaic G component weight table; wherein, in the mosaic G component weight table, the mosaic G component horizontal weight is negatively correlated with the horizontal weight index, and the mosaic G component vertical weight is negatively correlated with the vertical weight index;

[0049] A mosaic color difference weighted calculation module is configured to sum the product of the average of the horizontal color differences of the pixels in an even number of horizontal directions centered on each non-G pixel in the denoised Bayer image and the central pixel and the proportion of the horizontal weight of the mosaic G component of the non-G pixel in the total mosaic weight, and the product of the average of the vertical color differences of the pixels in an even number of vertical directions centered on the non-G pixel and the central pixel and the proportion of the vertical weight of the mosaic G component of the non-G pixel in the total mosaic weight, to obtain the mosaic color difference weight of the non-G pixel; wherein the total mosaic weight is the sum of the horizontal weight and the vertical weight of the mosaic G component of the non-G pixel;

[0050] The demosaiced image generating module is used to take the weighted sum of the noise reduction component of each non-G pixel in the noise reduction Bayer image and the mosaic color difference of the non-G pixel as the G component of the demosaiced image at the non-G pixel.

[0051] Preferably, the weighted sum of the noise reduction component of each non-G pixel in the noise reduction Bayer image and the color difference of the non-G pixel is used as the G component of the demosaiced image at the non-G pixel.

[0052] Preferably, the average value of the G component of all pixels in the comparison window is used as the brightness value of the central pixel; the ratio of the brightness value of the central pixel in the comparison window to the preset brightness value is rounded to obtain a brightness index; a brightness compensation / suppression coefficient matching the brightness index of the central pixel in the comparison window is searched in a brightness compensation array, and the product of the search absolute difference sum of the central pixel in the comparison window and the brightness compensation / suppression coefficient is used as a new search absolute difference sum; wherein the brightness index and the brightness compensation coefficient in the brightness compensation array are negatively correlated;

[0053] A noise reduction coefficient that matches the ratio of the new search absolute difference and the noise reduction intensity of the set pixel in the search window is selected from the noise reduction weight table as the noise reduction coefficient of the pixel.

[0054] The present invention also discloses a Bayer image processing device, which processes the Bayer image using the method described above, or the processing apparatus described above.

[0055] The present invention also discloses a computer-readable storage medium on which a computer program is stored. When the computer program is executed, the processing method can be implemented.

[0056] The present invention also discloses a Bayer image processing chip, comprising a processor and a memory, wherein a computer program is stored in the memory, and the processor can execute the computer program to implement the method described.

[0057] In the Bayer image processing method, an interpolation method is used to calculate the horizontal interpolation G component and the vertical interpolation G component of each non-G pixel, as well as the horizontal interpolation non-G component and the vertical interpolation non-G component of each G pixel. Then, the horizontal color difference gradient and the vertical color difference gradient of each pixel are calculated. The G component horizontal weight and the G component vertical weight are searched in the G component weight table based on the sum of the horizontal color difference gradient and the sum of the vertical color difference gradient in a preset window. Then, the color difference weight of the non-G pixel is calculated based on the horizontal weight and the vertical weight of the G component. The G component of the non-G pixel is obtained by summing the original component of each non-G pixel and the color difference weight of the non-G pixel to obtain a G component guidance map. The G component guidance map obtained by this scheme is more accurate and more conducive to subsequent Bayer image processing. For example, the denoised Bayer image obtained by denoising the G component guidance map has a better effect. In addition, the color difference weight obtained based on the G component guidance map is more accurate, and the calculated demosaiced image has a better effect.

[0058] In some schemes, the weight index (horizontal, vertical) and color difference weighting used in the calculation of the G component guide map can be reused in the calculation of the demosaiced image, greatly reducing the amount of calculation while ensuring the processing effect.

[0059] In some solutions, by performing adaptive intensity denoising on different types of edges of the G component guide map, that is, the flatter the area, the greater the denoising intensity, thereby preserving the image edges as much as possible while ensuring the denoising effect.

[0060] In addition, the color difference weighting is used as one of the components, and the edge part of the G component guide map is directional interpolated by setting the edge threshold to obtain the noise reduction component of the noise reduction Bayer image, which can greatly reduce the false color and zipper effect.

[0061] Other beneficial effects of the present invention will be explained through the introduction of specific technical features and technical solutions in the specific implementation methods. Those skilled in the art should be able to understand the beneficial technical effects brought about by the introduction of these technical features and technical solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings.

[0063] Figure 1 is a schematic diagram of a Bayer image;

[0064] Figure 2 Flowchart of a Bayer image processing method according to an embodiment of the present invention;

[0065] Figure 3 Schematic diagram of a Bayer image in an embodiment of the present invention;

[0066] Figure 4 Flowchart of a Bayer image processing method according to an embodiment of the present invention;

[0067] Figure 5 Schematic diagram of a template window, a search window, and a comparison window in a Bayer image processing method according to an embodiment of the present invention;

[0068] Figure 6 Schematic diagram of a template window, a search window, and another comparison window in a Bayer image processing method according to an embodiment of the present invention;

[0069] Figure 7 A schematic diagram of a sub-edge value in a Bayer image processing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0070] The present invention is described below based on the following embodiments, but the present invention is not limited to these embodiments. In the following detailed description of the present invention, some specific details are described in detail. In order to avoid obscuring the essence of the present invention, well-known methods, processes, procedures, and components are not described in detail.

[0071] Furthermore, persons of ordinary skill in the art will appreciate that the figures provided herein are for illustration purposes only and are not necessarily drawn to scale.

[0072] Unless the context clearly requires otherwise, throughout the specification and claims, the words "include," "comprising," and similar words should be construed in an inclusive sense rather than an exclusive or exhaustive sense; that is, in the sense of "including but not limited to."

[0073] In the description of the present invention, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood to indicate or imply relative importance. In addition, in the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0074] like Figure 1 As shown, each pixel of the original Bayer image has only one of the three colors of red, green and blue (RGB). The data format of the original Bayer image is that the output pixel colors of the odd-numbered scanning lines are red and green alternately, and the output pixel colors of the even-numbered scanning lines are green and blue alternately. For the convenience of description, in this embodiment, the pixels are divided into R pixels, G pixels and B pixels according to the colors of the pixels in the original Bayer image. There is an R component (original component) in the R pixel, a G component (original component) in the G pixel, and a B component (original component) in the B pixel. Figure 2 As shown, the present invention proposes a Bayer image processing method for processing the Bayer image, which includes the following steps:

[0075] S110, calculating the horizontal interpolation G component and the vertical interpolation G component of each non-G pixel (R pixel, B pixel) in the original Bayer image by interpolation, and calculating the horizontal interpolation non-G component and the vertical interpolation non-G component of each G pixel, and then calculating the sum of the color difference gradients of each pixel in the original Bayer image to obtain the horizontal color difference by the difference between the original component and the horizontal interpolation component, and to obtain the vertical color difference by the difference between the original component and the vertical interpolation component.

[0076] It should be noted that there are many interpolation methods in the prior art. This step does not impose any specific restrictions on the interpolation calculation method. This embodiment uses gradient interpolation as an example for explanation. Horizontal interpolation is calculated using the adjacent pixels in the horizontal direction of the interpolated pixel, and vertical interpolation is calculated using the adjacent pixels in the vertical direction of the interpolated pixel. The horizontal interpolation G component of the R pixel is calculated using the gradient interpolation method as shown in formula (1), and the vertical interpolation G component of the R pixel is shown in formula (2), where the coordinates of the R pixel are (j, i), and the horizontal interpolation G component is G j，i H , the vertical interpolation G component is Gj，i V .

[0077] G j，i H =(G j，i-1 +G j，i+1 ) / 2+(2*R j，i -R j，i-2 -R j，i+2 ) / 4 (1)

[0078] G j，i V =(G j-1 , i+G j+1 ,i) / 2+(2*R j，i -R j-2，i -R j+2，i ) / 4 (2)

[0079] Similarly, the horizontal interpolation G component of the B pixel is shown in formula (3), and the vertical interpolation G component is shown in formula (4), where the coordinates of the B pixel are (j, i), and the horizontal interpolation G component is G j，i H , the vertical interpolation G component is G j，i V .

[0080] G j，i H =(G j，i-1 +G j，i+1 ) / 2+(2*B j，i -B j，i-2 -B j，i+2 ) / 4 (3)

[0081] G j，i V =(G j-1，i +G j+1，i ) / 2+(2*B j，i -B j-2，i -B j+2，i ) / 4 (4)

[0082] If the G pixel is located in an odd scan line, the R component needs to be interpolated. The coordinates of the G pixel are (j, i), and the horizontal interpolation R component R j，i H The calculation formula of (horizontal interpolation non-G component) is shown in formula (5), and the vertical interpolation R component R j，i V The calculation formula of (vertical interpolation non-G component) is shown in formula (6).

[0083] R j，i H =(R j，i-1 +Rj，i+1 ) / 2+(2*G j，i -G j，i-2 -G j，i+2 ) / 4 (5)

[0084] R j，i V =(R j-1，i +R j+1，i ) / 2+(2*G j，i -G j-2，i -G j+2，i ) / 4 (6)

[0085] If the G pixel is located in an even scan line, the B component needs to be interpolated. The coordinates of the G pixel are (j, i), and the horizontal interpolation B component B j，i H The calculation formula for (horizontally interpolating non-G components) is shown in equation (7), and the vertical interpolation B component B j，i V The calculation formula of (vertical interpolation non-G component) is shown in formula (8),

[0086] B j，i H =(B j，i-1 +B j，i+1 ) / 2+(2*G j，i -G j，i-2 -G j，i+2 ) / 4 (7)

[0087] B j，i V =(B j-1，i +B j+1，i ) / 2+(2*G j，i -G j-2，i -G j+2，i ) / 4 (8)

[0088] The horizontal interpolation G component and vertical interpolation G component of the R pixel, the horizontal interpolation G component and vertical interpolation G component of the B pixel, and the horizontal interpolation non-G component and vertical interpolation non-G component of the G pixel of the original Bayer image are calculated by the above formulas 1 to 8 (that is, the horizontal interpolation R component and vertical interpolation R component of the G pixel for the odd scanning lines, and the horizontal interpolation B component and vertical interpolation B component of the G pixel for the even scanning lines).

[0089] The horizontal color difference is calculated by calculating the difference between the original component and the horizontal interpolated component of each pixel in the original Bayer image, and the vertical color difference is calculated by calculating the difference between the original component and the vertical interpolated component. It should be noted that the horizontal / vertical color difference is the horizontal / vertical interpolated component minus the original component, or it can be the original component minus the horizontal / vertical interpolated component. Formulas (9) to (16) specifically explain the former case, as follows:

[0090] Horizontal color difference of pixel B (coordinates (j, i)) is the horizontal interpolation G component G j，i H The difference between the B component and the vertical color difference of the B pixel is shown in formula (9). is the vertical interpolation G component G j，i V The difference between the A component and the B component is shown in formula (10).

[0091]

[0092]

[0093] Horizontal color difference of R pixel (coordinate (j, i)) is the horizontal interpolation G component G j，i H The difference between the R component and the vertical color difference of the R pixel is shown in formula (11). is the vertical interpolation G component G j，i V The difference between the R component and the y component is shown in formula (12).

[0094]

[0095]

[0096] G pixel (coordinates (j, i)), if it is located in an odd scanning line, its horizontal color difference The G component and the horizontal interpolation R component R j，i H As shown in formula (13), the vertical chromatic aberration is the G component and the vertical interpolation R component R j，i V The difference is shown in formula (14).

[0097]

[0098]

[0099] G pixel (coordinates (j, i)), if it is located in an even scan line, its horizontal color difference G component and horizontal interpolation B component Bj，i H As shown in formula (15), the vertical chromatic aberration G component and vertical interpolation B component B j，i V The difference is shown in formula (16).

[0100]

[0101]

[0102] The horizontal color difference and vertical color difference of each pixel are calculated according to equations (9) to (16).

[0103] S120, calculating the absolute value of the horizontal color difference between the two adjacent pixels on the left and right of each pixel in the original Bayer image in the horizontal direction to obtain the horizontal color difference gradient (As shown in formula (17), formula (17) is applicable to the calculation of B pixels or G pixels of even scanning lines. The same is true for R pixels and G pixels of odd scanning lines. The specific formula is not given here.) and the absolute value of the vertical color difference between the two adjacent pixels above and below in the vertical direction is used to obtain the vertical color difference gradient (As shown in formula (18), formula (18) is applicable to calculating B pixels or G pixels of even-numbered scanning lines. The same is true for R pixels and G pixels of odd-numbered scanning lines. The specific formula is not given here).

[0104]

[0105]

[0106] S130, calculating the sum of the horizontal color difference gradient and the sum of the vertical color difference gradient of the pixels in the preset window centered on each pixel of the original Bayer image, and using them as the horizontal weight index diff of the central pixel respectively H , and vertical weight index diff V . Find the horizontal weight index diff for each pixel in the original Bayer image in the G component weight table w_table H (Horizontal index) The horizontal weight W of the matched G component H , and the vertical weight index diff of each pixel with the original Bayer image V (Vertical index) The vertical weight W of the matched G component V ; Among them, in the G component weight table w_table, the G component horizontal weight WH and the horizontal weight index diff H There is a negative correlation, the vertical weight W of the G component V Diff with vertical weight index VNegatively correlated, horizontal weight index diff H Diff with vertical weight index V The calculation formulas are as described in formulas (19) and (20).

[0107]

[0108]

[0109] Wherein, (j, i) represents any pixel in the preset window, and the value range of (j, i) is the preset window range. Represents the horizontal color gradient of any pixel in the preset window. Represents the vertical color gradient of any pixel within the preset window.

[0110] Find the horizontal weight index diff for each pixel with the original Bayer image H Matched G component horizontal weight W H , and the vertical weight index diff of each pixel with the original Bayer image V Matched G component vertical weight W V By setting the G component weight table w_table to store the horizontal weight index diff H and G component horizontal weight W H The matching relationship between them, and the vertical weight index diff V Weight W perpendicular to the G component V The matching relationship between them is shown in equations (21) and (22).

[0111] W H =w_table[diff H ] (twenty one)

[0112] W V =w_table[diff V ] (twenty two)

[0113] S140, determining a color difference weight for each non-G pixel in the original Bayer image, wherein the color difference weight is the sum of a first product and a second product, and the first product is the average of the horizontal color differences of an even number M (M greater than 1) of pixels in the horizontal direction centered on each non-G pixel (taking a B pixel as an example, B(j, i)) and the center pixel. The proportion of the G component horizontal weight of the non-G pixel in the total weight W H / W T The second product is the average of the vertical color differences of the even number of pixels centered on the non-G pixel and the central pixel in the vertical direction. The vertical weight of the G component of the non-G pixel accounts for W of the total weight V / W T The product of; where the total weight W T is the horizontal weight W of the G component of the non-G pixel H Weight W perpendicular to the G component V sum.

[0114] Taking M equal to 2 as an example, the color difference weighting of B pixel is shown in formula (23), where W T =W H +W V , the position of the B pixel is B(j,i).

[0115]

[0116] Taking M equal to 2 as an example, the color difference weighting of R pixel is shown in formula (24):

[0117]

[0118] Among them, W T =W H +W V , the position of the R pixel is R(j,i).

[0119] In some embodiments, if the difference between the horizontal weight index and the vertical weight index of a non-G pixel in the original Bayer image is greater than a preset value, the color difference weighting of the non-G pixel is calculated using only the smaller of the horizontal weight index and the vertical weight index. Figure 3 As shown, for non-G pixels (j, i), the difference between the horizontal weight index and the vertical weight index is greater than the preset value, and the vertical weight index is larger, then the color difference weighted value calculated based on the horizontal weight index is more accurate, and directly calculating the color difference weighting using the horizontal weight index (the smaller one) helps to alleviate false colors at the edges of the image.

[0120] S150 , taking the weighted sum of the original component of each non-G pixel in the original Bayer image and the color difference of the non-G pixel as the G component of the original Bayer image at the G component of the non-G pixel to obtain a G component guidance map.

[0121] Specifically, the original component R of the R pixel j，i Weighted color difference with R pixel The sum G (j，i) , as shown in formula (25), where the color difference is weighted is the color difference weight calculated according to formula (24).

[0122] The original component B of the B pixel j，i Weighted color difference with R pixel The sum G(j，i) , as shown in formula (26), where the color difference weighted is the color difference weight calculated according to formula (23).

[0123]

[0124]

[0125] The Bayer image processing method in this embodiment uses an interpolation method to calculate the horizontal interpolation G component and the vertical interpolation G component of each non-G pixel, as well as the horizontal interpolation non-G component and the vertical interpolation non-G component of each G pixel, and then calculates the horizontal color difference gradient and the vertical color difference gradient of each pixel. The G component horizontal weight and the G component vertical weight are searched in the G component weight table based on the sum of the horizontal color difference gradient and the sum of the vertical color difference gradient in a preset window as an index. The color difference weight of the non-G pixel is then calculated based on the horizontal weight and the vertical weight of the G component. The G component of the non-G pixel is obtained by summing the original component of each non-G pixel and the color difference weight of the non-G pixel to obtain a G component guide map. The G component guide map obtained by this solution is more conducive to subsequent noise reduction processing, and the denoised Bayer image obtained by denoising the G component guide map has a better effect.

[0126] After obtaining the G component guidance map, the G component guidance map can be used for noise reduction, such as Figure 4 As shown, the specific steps include:

[0127] S210: Using an image block of a first size N centered around each pixel in the G component guide image as a template window and an image block of a second size K as a search window, wherein the second size is larger than the first size. Figure 5 As shown, Figure 5 It is a G component guide map. Each pixel in the G component guide map includes a G component (not shown in the figure). Taking pixel B33 as an example, the first size N is 3*3 pixels, the second size K is 5*5 pixels, and the image block with 3*3 pixels centered on B33 is the template window 100, and the image block with a size of 5*5 pixels is the search window 200.

[0128] S220, selecting comparison windows of the same size as the template window in sequence with the set pixel in the search window as the center, and calculating the sum of the absolute values ​​of the differences between the G component of each pixel in each comparison window and the G component of the pixel at the corresponding position of the template window as the search absolute difference and SAD of the center pixel of the comparison window.

[0129] Set the pixel to the same color as the center pixel in the search window, such as G pixel, B pixel, or R pixel. Figure 5As shown in the figure, when calculating the G component of the G guide map of the B pixel B33, only B11, B13, B15, B31, B33, B35, B51, B53 and B55, which are all B pixels within the search window range, are selected as center pixels to determine the comparison window, so as to reduce the number of comparison windows and achieve the effect of simplifying the calculation.

[0130] The calculation formula for searching absolute difference and SAD is shown in formula (27).

[0131]

[0132] Where (j1, i1) is the center pixel in the template window, (j2, i2) is the center pixel in the comparison window (i.e., centered on this pixel), and the value range of (j2, i2) is determined by the search window. The above formula takes the center pixel as the origin, and pixels offset to the left and downward are negative, while pixels offset to the right and upward are positive. abs(G (j1+j)，(i1+i) -G (j2+j)，(i2+i) ) represents the absolute value of the difference between the G component of any pixel in the template window and the G component of the pixel at the corresponding position in the comparison window (equal to the absolute value of the difference between the G component of each pixel in the comparison window and the G component of the pixel at the corresponding position in the template window), N is determined according to the size of the template window. If the template window is 3*3, the value of N is 3.

[0133] Please continue to refer to Figure 5 Continuing with pixel B33 as an example, with the first pixel B11 in the search window as the center, select a comparison window 310 with the same size as the template window (3*3 pixels), and calculate the absolute value of the difference between the G component of the pixels at the corresponding positions in the comparison window 310 and the template window 200 (R00 and R22 are the corresponding positions, G01 and G23 are the corresponding positions, and so on), and obtain the search absolute difference and SAD corresponding to pixel B11, as shown in formula (28).

[0134]

[0135] like Figure 6 As shown, with pixel B13 in the search window as the center, a comparison window 320 of the same size as the template window is selected, and the absolute value of the difference between the G component of the corresponding pixel in the comparison window 320 and the template window 200 is calculated to obtain the search absolute difference and SAD corresponding to pixel B13. This process is repeated until every comparison window within the search window 200 is traversed.

[0136] S230, perform edge detection on the G component guide map to obtain the edge value edge of each pixel, compare the edge value of the pixel with the first edge threshold T1 and the second edge threshold T2 respectively, and judge the pixel as a weak edge, a flat area, and a strong edge when the edge value edge is less than the second edge threshold T2, between the second edge threshold T2 and the first edge threshold T1, and greater than the first edge threshold T1, and set the noise reduction intensity sigma of the pixel to the third intensity sigma3, the first intensity sigma1, and the second intensity sigma2 respectively; wherein the sizes of the first intensity sigma1 (corresponding to the flat area), the third intensity sigma3 (corresponding to the weak edge), and the second intensity sigma2 (corresponding to the strong edge) decrease in sequence.

[0137] In step S230, edge detection is performed on the G component guide image to obtain the edge value of each pixel, which specifically includes the following steps: selecting a plurality of symmetrical pixel pairs in each direction centered on each pixel of the G component guide image, calculating the absolute value of the difference between the two pixels in the symmetrical pixel pairs, and calculating the sum of the absolute values ​​of the plurality of symmetrical pixel pairs in each direction as the sub-edge value in the direction, and then calculating the sum of the sub-edge values ​​in the plurality of directions to obtain the edge value of the pixel. Figure 7 As shown, taking pixel (j, i) as an example, each direction includes the direction from east to west, from north to south, from northeast to southwest, and from northwest to southeast. Taking the direction from east to west, two groups of symmetrical pixel pairs as an example, the first group of symmetrical pixel pairs are the points (j, i-1) and (j, i+1) with a distance of 1 pixel from (j, i) to the center, and the second group of symmetrical pixel pairs are the points (j, i-2) and (j, i+2) with a distance of 2 pixels from (j, i) to the center. The sub-edge value edge1 from east to west is (j，i) As shown in formula (29), it is the absolute value of the difference between the G components of two pixels in a symmetrical pixel pair, and the absolute values ​​of all symmetrical pixel pairs are summed. The calculation method of other sub-edge values ​​is similar, as shown in formulas (30) to (32), which will not be repeated here.

[0138] edge1 (j，i) =abs(G (j，i-1) -G (j，i+1) )+abs(G (j，i-2) -G (j，i+2) ) (29)

[0139] edge2 (j，i) =abs(G (j-1，i) -G (j+1，i) )+abs(G (j-2，i) -G (j+2，i) ) (30)

[0140] edge3 (j，i) =abs(G (j-1，i+1) -G (j+1，i-1) )+abs(G (j-2，i+2) -G (j+2，i-2) ) (31)

[0141] edge4 (j，i) =abs(G (j-1，i-1) -G (j+1，i+1) )+abs(G (j-2，i-2) -G (j+2，i+2) ) (32)

[0142] In some embodiments, the third intensity sigma3 is equal to the product of the second intensity coefficient ratio and the second intensity sigma2 (sigma2*ratio) plus the product of the first intensity coefficient and the first intensity sigma1, wherein the first intensity coefficient is equal to 1 minus the second intensity coefficient ratio, see formula (33) for details;

[0143] sigma3=sigma1*(1-ratio)+sigma2*ratio (33)

[0144] The second intensity coefficient is equal to the ratio of the difference between the edge value of the pixel and the first edge threshold value to the difference between the second edge threshold value and the first edge threshold value: ratio = (edge (j，i) -T1) / (T2-T1).

[0145] S240, selecting a noise reduction coefficient W that matches the ratio of the search absolute difference of the set pixel in the search window to the noise reduction strength sigma in the noise reduction weight table w_table1 j，i , as the noise reduction coefficient W of the pixel j，i The larger the ratio of the search absolute difference of pixels in the denoising weight table to the denoising intensity, the smaller the corresponding denoising coefficient. By adaptively denoising different types of edges in the G component guide map, i.e., increasing the denoising intensity for flatter areas, this approach preserves image edges as much as possible while ensuring effective denoising.

[0146] In some embodiments, the search absolute difference and SAD may be converted into binary numbers, and then the noise reduction strength sigma bits may be right-shifted as an index of the noise reduction weight table w_table1, as shown in formula (34), where >> is a right-shift operator.

[0147] W j，i =w_table1[SAD>>sigma] (34)

[0148] In other embodiments, the matching method may also convert the search absolute difference and SAD into binary, right-shift by the noise reduction intensity sigma bit, and then round it to the integer value as the index. In other embodiments, the ratio of the search absolute difference and SAD to the noise reduction intensity sigma for each pixel may be used as the index. All of the above methods can directly or indirectly reflect the search absolute difference and SAD to the noise reduction intensity sigma ratio.

[0149] S250, the original component src of the pixels in the search window centered at each set pixel in the original Bayer image (j1，i1) and the noise reduction coefficient W of the pixel (j1，i1) The weighted sum of is taken as the denoising component of the pixel, thereby obtaining the denoised Bayer image, as shown in formula (35).

[0150] dst(j, i) = ∑W (j1，i1) *src (j1，i1) (35)

[0151] Where (j1, i1) is the set pixel in the search window, and the value range of (j1, i1) is determined by the selection method of the search window and the set pixel. (j1，i1) Set the original component of the pixel in the search window, W (j1，i1) Set the noise reduction coefficient W corresponding to the pixel in the search window (j1，i1) .

[0152] If the pixel is set as each pixel of the search window, the search window includes 25 set pixels, that is, 25 comparison windows are determined with the 25 pixels as the center, and the search absolute difference and SAD and noise reduction strength sigma corresponding to each pixel in the search window are searched. The matching noise reduction coefficient W is found according to the ratio of the search absolute difference and SAD and noise reduction strength sigma. (j1，i1) , substitute into formula (35) for weighted summation, solve the noise reduction component of the central pixel, and thus obtain the noise reduction Bayer image. If the pixels are set to the same color pixels, Figure 5 Taking B33 in the example, the search window includes 9 set pixels. The SAD of the 9 comparison windows and the noise reduction strength sigma of the set pixels are determined with the 9 pixels as the center. The matching noise reduction coefficient W is found based on the search absolute difference and the ratio of SAD to noise reduction strength sigma. (j1，i1) , substitute into formula (35) for weighted summation.

[0153] In some embodiments, a brightness compensation / suppression step is further included, specifically comprising:

[0154] S310, taking the mean value of the G components of all pixels in the comparison window as the brightness value L(j, i) of the center pixel; rounding the ratio of the brightness value of the center pixel in the comparison window to the preset brightness value to obtain a brightness index L′(j, i); searching the brightness compensation / suppression coefficient luminance_compensation[L′] that matches the brightness index of the center pixel in the comparison window in the brightness compensation array, and taking the product of the search absolute difference sum of the center pixel in the comparison window and the brightness compensation / suppression coefficient SAD*luminance_compensation[L′] as the new search absolute difference sum SAD'; wherein, the brightness index L′(j, i) in the brightness compensation array and the brightness compensation coefficient luminance_compensation[L′] are negatively correlated.

[0155] Taking the comparison window size of 3*3 pixels as an example, the coordinates of the center pixel of the comparison window are (j, i), and the calculation formula of the brightness value is L (j，i) As shown in formula (36), where G (j+j1，i+i1) Represents the G component value of any pixel in the comparison window in the G component guidance map.

[0156]

[0157] The new calculation formula for search absolute difference and SAD' is shown in Equation (37).

[0158] SAD′=SAD*luminance_compensation[L′] (37)

[0159] If the brightness preset value is 32, the number of brightness compensation / suppression coefficients in the brightness compensation array is 8; if the brightness preset value is 16, the number of brightness compensation / suppression coefficients in the brightness compensation array is 16, and so on.

[0160] S320 , selecting a noise reduction coefficient that matches the new search absolute difference and the ratio of the SAD′ to the noise reduction intensity sgima of the set pixel in the search window from the noise reduction weight table as the noise reduction coefficient of the pixel.

[0161] In an embodiment of the present invention, the Bayer image processing method further includes a demosaicing step for the denoised Bayer image. The demosaicing step is to respectively calculate the values ​​of the two other color components of the missing R pixels, G pixels, and B pixels. The method for calculating the G component of non-G pixels specifically includes the following steps:

[0162] S410, searching the mosaic G component weight table wc_table for the horizontal weight index diff of each pixel in the original Bayer image HThe horizontal weight W of the matched mosaic G component c H =wc_table[diff H ], and the vertical weight index diff of each pixel with the original Bayer image V Matched mosaic G component vertical weight W c V =wc_table[diff V ]; Among them, in the mosaic G component weight table, the horizontal weight of the mosaic G component is negatively correlated with the horizontal weight index, and the vertical weight of the mosaic G component is negatively correlated with the vertical weight index.

[0163] In this embodiment, in order to reduce the amount of calculation, the horizontal weight index diff of the original Bayer image calculated in step 130 is directly used. H Diff with vertical weight index V As the index value. It is understood by those skilled in the art that the horizontal weight index diff can also be recalculated according to the denoised Bayer image. H Diff with vertical weight index V , no restriction is made here.

[0164] S420, calculate the average horizontal color difference between an even number (P) of pixels in the horizontal direction centered on each non-G pixel in the denoised Bayer image (taking B pixel as an example, B(j, i)) and the central pixel The proportion of the horizontal weight of the mosaic G component of the non-G pixel in the total weight of the mosaic W' H / W′ T The product of the vertical color difference of the even number (P) of pixels in the vertical direction centered on the non-G pixel and the center pixel The product of the vertical weight of the mosaic G component of the non-G pixel in the total weight of the mosaic W' V / W′ T Sum up and get the mosaic color difference weight of the non-G pixel Among them, the total weight of mosaic W′ T is the horizontal weight W′ of the mosaic G component of the non-G pixel H Weight W′ perpendicular to the mosaic G component V sum.

[0165] Taking P equal to 3 as an example, the B pixel mosaic color difference weighted As shown in formula (38), where W′ T =W′ H +W′ v , the position of the B pixel is B(j,i).

[0166]

[0167] S430: The weighted sum of the noise reduction component of each non-G pixel in the noise reduction Bayer image and the mosaic color difference of the non-G pixel is used as the G component of the demosaic image at the non-G pixel. Taking the B pixel as an example, the B pixel coordinates are (j, i), and the noise reduction component is B' j，i , weight the mosaic color difference in step S420 Substituting into equation (39) yields the G component of the B pixel. Calculating the G component of the R pixel is similar and will not be repeated here.

[0168]

[0169] In other embodiments, to reduce the amount of calculation, the weighted sum of the noise reduction component of each non-G pixel in the noise reduction Bayer image and the color difference of the non-G pixel can be used as the G component of the demosaiced image at the non-G pixel.

[0170] In addition to calculating the G component of the non-G pixel, another component of the non-G pixel must also be calculated, such as the B component of the R pixel, or the R component of the B pixel.

[0171] The B component in the R pixel is calculated by using the G component in the R pixel calculated in step S430 and the color difference between the G component and the B component adjacent to the R pixel. The specific calculation method is shown in formula (40).

[0172]

[0173] Among them, the current coordinate of R pixel is (ji), G′ j，i is the G component of the demosaiced image interpolated in step S430, is the color difference calculated in step S140.

[0174] Similarly, the R component in the B pixel is calculated by interpolating the G component calculated in step S430 and calculating the color difference between the G component and the R component around the B pixel. The specific calculation method is shown in formula (41).

[0175]

[0176] Among them, the current coordinate of B pixel is (ji), G′ j，i is the G component of the demosaiced image interpolated in step S430, is the color difference calculated in step S140.

[0177] For the G pixels in the denoised Bayer image, its R and B components also need to be calculated. For G pixels in odd scan lines, the B component is calculated using the B and G components of its vertically adjacent B pixels (with coordinates (j-1, i) and (j+1, i)). The specific calculation formula is shown in Equation (42).

[0178] B′ j，i =(B′ j-1，i +B′ j+1，i ) / 2+(2*G′ j，i -G′ j-1，i -G′ j+1，i ) / 2 (42)

[0179] Among them, the coordinates of G pixel are (j, i), B′ j-1，i , B′ j+1，i is the value of the B component of the B pixel of the adjacent B pixel (coordinates are (j-1, i) and (j+1, i)) in the denoised Bayer image, G′ j，i , G′ j-1，i , G′ j+1，i is the G component of the demosaiced image calculated in step S430.

[0180] For the G pixels of odd scanning lines, the R component is calculated using the values ​​of the R and G components of the R pixels adjacent to it in the horizontal direction (with coordinates of (j, i-1) and (j, i+1)). The specific calculation formula is shown in formula (43).

[0181] R′ j，i =(R′ j，i-1 +R′ j，i+1 ) / 2+(2*G′ j，i -G′ j，i-1 -G′ j，i+1 ) / 2 (43)

[0182] Among them, the coordinates of G pixel are (j, i), R′ j，i-1 and R′ j，i+1 is the value of the R component of the R pixel with adjacent R pixel coordinates ((j, i-1) and (j, i+1)) in the denoised Bayer image, G′ j，i , G′ j，i-1 , G′ j，i+1 is the G component of the demosaiced image calculated in step S430.

[0183] For G pixels in even scan lines, the B component is calculated using its adjacent B pixels in the horizontal direction. The specific calculation formula is shown in formula (44).

[0184] B′ j，i =(B′j，i-1 +B′ j，i+1 ) / 2+(2*G′ j，i -G′ j，i-1 -G′ j，i+1 ) / 2 (44)

[0185] Among them, the coordinates of G pixel are (j, i), B′ j，i-1 , B′ j，i+1 is the B component of the adjacent B pixels (coordinates are (j, i-1) and (j, i+1)) in the denoised Bayer image, G′ j，i , G′ j，i-1 , G′ j，i+1 is the G component of the demosaiced image calculated in step S430.

[0186] For the G pixels in the even scanning lines, the R component is calculated using the adjacent R pixels in the vertical direction. The specific calculation formula is shown in formula (45).

[0187] R′ j，i =(R′ j-1，i +R′ j+1，i ) / 2+(2*G′ j，i -G′ j-1，i -G′ j+1，i ) / 2 (45)

[0188] Among them, the coordinates of G pixel are (j, i), R′ j-1，i , R′ j+1，i is the R component of the adjacent R pixels (coordinates are (j-1, i) and (j+1, i)) in the denoised Bayer image, G′ j，i , G′ j-1，i , G′ j+1，i The G component of the demosaiced image calculated in step S430.

[0189] At this point, the demosaiced image calculated using the denoised Bayer image is obtained.

[0190] The present invention also discloses a Bayer image processing device, comprising:

[0191] a color difference calculation module, configured to calculate, by interpolation, a horizontal interpolation G component and a vertical interpolation G component of each non-G pixel in the original Bayer image, and calculate a horizontal interpolation non-G component and a vertical interpolation non-G component of each G pixel, and then calculate the difference between the original component and the horizontal interpolation component of each pixel in the original Bayer image to obtain a horizontal color difference, and the difference between the original component and the vertical interpolation component to obtain a vertical color difference;

[0192] A color difference gradient calculation module is used to calculate the absolute value of the horizontal color difference difference between the two adjacent pixels on the left and right in the horizontal direction of each pixel of the original Bayer image to obtain the horizontal color difference gradient, and the absolute value of the vertical color difference difference between the two adjacent pixels on the top and bottom in the vertical direction to obtain the vertical color difference gradient;

[0193] A weight calculation module is configured to calculate the sum of the horizontal color difference gradients and the sum of the vertical color difference gradients of pixels within a preset window centered on each pixel of the original Bayer image, respectively serving as the horizontal weight index and the vertical weight index of the central pixel, and search a G component horizontal weight that matches the horizontal weight index of each pixel of the original Bayer image, and a G component vertical weight that matches the vertical weight index of each pixel of the original Bayer image in a G component weight table; wherein, in the G component weight table, the G component horizontal weight is negatively correlated with the sum of the horizontal color difference gradients, and the G component vertical weight is negatively correlated with the sum of the vertical color difference gradients;

[0194] a color difference weighted calculation module, configured to determine a color difference weight for each non-G pixel in the original Bayer image, wherein the color difference weight is the sum of a first product and a second product, the first product being the product of an average of horizontal color differences of an even number of pixels centered on the non-G pixel and the center pixel in the horizontal direction and a proportion of the horizontal weight of the G component of the non-G pixel in the total weight; and the second product being the product of an average of vertical color differences of an even number of pixels centered on the non-G pixel and the center pixel in the vertical direction and a proportion of the vertical weight of the G component of the non-G pixel in the total weight, wherein the total weight is the sum of the horizontal weight and the vertical weight of the G component of the non-G pixel;

[0195] The G component guide map generation module is used to take the weighted sum of the original component of each non-G pixel in the original Bayer image and the color difference of the non-G pixel as the G component of the G component guide map of the original Bayer image at the non-G pixel to obtain the G component guide map.

[0196] The present invention also discloses a Bayer image processing device, which processes the Bayer image using the method described above, or the processing apparatus described above.

[0197] The present invention also discloses a computer-readable storage medium on which a computer program is stored. When the computer program is executed, the processing method can be implemented.

[0198] The present invention also discloses a Bayer image processing chip, comprising a processor and a memory, wherein a computer program is stored in the memory, and the processor can execute the computer program to implement the method described.

[0199] It will be understood by those skilled in the art that, under the premise of no conflict, the above-mentioned preferred embodiments can be freely combined and superimposed. Among them, the flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or part of the code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. The numbering of each step in this article is only for the convenience of description and reference, and is not used to limit the order of execution. The specific execution order is determined by the technology itself, and those skilled in the art can determine various allowable and reasonable orders based on the technology itself.

[0200] It should be noted that the use of step numbers (letters or numbers) to refer to certain specific method steps in the present invention is solely for the purpose of descriptive convenience and brevity, and is in no way intended to limit the order of these method steps. Those skilled in the art will appreciate that the order of the relevant method steps is determined by the technology itself and should not be unduly limited by the presence of step numbers. Those skilled in the art can determine various permissible and reasonable step orders based on the technology itself.

[0201] Those skilled in the art will appreciate that, provided there is no conflict, the above preferred solutions can be freely combined and superimposed.

[0202] It should be understood that the above-mentioned embodiments are merely illustrative and non-restrictive. Without departing from the basic principles of the present invention, various obvious or equivalent modifications or substitutions that can be made by those skilled in the art to the above-mentioned details will be included in the scope of the claims of the present invention.

Claims

1. A Bayer image processing method, characterized in that: The steps include: Calculating a horizontal interpolation G component and a vertical interpolation G component of each non-G pixel in the original Bayer image by interpolation, and calculating a horizontal interpolation non-G component and a vertical interpolation non-G component of each G pixel, and then calculating a difference between an original component and a horizontal interpolation component of each pixel in the original Bayer image to obtain a horizontal color difference, and a difference between an original component and a vertical interpolation component to obtain a vertical color difference; Calculating the absolute value of the horizontal color difference between the left and right adjacent pixels of each pixel in the original Bayer image to obtain the horizontal color difference gradient, and calculating the absolute value of the vertical color difference between the upper and lower adjacent pixels in the vertical direction to obtain the vertical color difference gradient; Calculating the sum of the horizontal color difference gradients and the sum of the vertical color difference gradients of pixels within a preset window centered on each pixel of the original Bayer image, respectively serving as the horizontal weight index and the vertical weight index of the central pixel, and searching a G component horizontal weight that matches the horizontal weight index of each pixel of the original Bayer image and a G component vertical weight that matches the vertical weight index of each pixel of the original Bayer image in a G component weight table; wherein, in the G component weight table, the G component horizontal weight is negatively correlated with the sum of the horizontal color difference gradients, and the G component vertical weight is negatively correlated with the sum of the vertical color difference gradients; Determining a color difference weight for each non-G pixel in the original Bayer image, wherein the color difference weight is the sum of a first product and a second product, the first product being the product of an average of horizontal color differences of an even number of pixels centered on the non-G pixel and the center pixel in a horizontal direction and a proportion of the horizontal weight of the G component of the non-G pixel in a total weight, and the second product being the product of an average of vertical color differences of an even number of pixels centered on the non-G pixel and the center pixel in a vertical direction and a proportion of the vertical weight of the G component of the non-G pixel in a total weight, wherein the total weight is the sum of the horizontal weight and the vertical weight of the G component of the non-G pixel; The weighted sum of the original component of each non-G pixel in the original Bayer image and the color difference of the non-G pixel is used as the G component of the G component guide map of the original Bayer image at the non-G pixel to obtain the G component guide map.

2. The processing method according to claim 1, characterized in that The following steps are also included: Using an image block of a first size centered on each pixel in the G component guide image as a template window and an image block of a second size as a search window, wherein the second size is larger than the first size; Selecting comparison windows of the same size as the template window in sequence with a set pixel in the search window as the center, and calculating the sum of the absolute values ​​of the differences between the G component of each pixel in each comparison window and the G component of the pixel at the corresponding position in the template window as the search absolute difference sum of the center pixel of the comparison window; performing edge detection on the G component guide image to obtain an edge value of each pixel, comparing the edge value of the pixel with a first edge threshold and a second edge threshold, respectively, and determining the pixel as a weak edge, a flat area, and a strong edge when the edge value is less than the second edge threshold, between the second edge threshold and the first edge threshold, and greater than the first edge threshold, respectively, and setting a noise reduction intensity for the pixel to a third intensity, the first intensity, and the second intensity, respectively; wherein the first intensity, the third intensity, and the second intensity decrease in magnitude in sequence; Selecting a noise reduction coefficient that matches the ratio of the search absolute difference and the noise reduction intensity of a set pixel in the search window from the noise reduction weight table as the noise reduction coefficient of the pixel; wherein a larger ratio of the search absolute difference and the noise reduction intensity of the pixel in the noise reduction weight table corresponds to a smaller noise reduction coefficient; The weighted sum of the original component of the pixel in the search window centered at each set pixel in the original Bayer image and the noise reduction coefficient of the pixel is used as the noise reduction component of the pixel, thereby obtaining a noise-reduced Bayer image.

3. The processing method according to claim 2, characterized in that Performing edge detection on the G component guide image to obtain the edge value of each pixel specifically includes the following steps: Select several groups of symmetrical pixel pairs in each direction centered on each pixel of the G component guide map, calculate the absolute value of the difference between the two pixels in the symmetrical pixel pair, and calculate the sum of the absolute values ​​of several groups of symmetrical pixel pairs in each direction as the sub-edge value in that direction, and then calculate the sum of the sub-edge values ​​in several directions to obtain the edge value of the pixel.

4. The processing method according to claim 3, characterized in that The third strength is equal to the product of the second strength coefficient and the second strength plus the product of the first strength coefficient and the first strength; The first intensity coefficient is equal to 1 minus the second intensity coefficient, and the second intensity coefficient is equal to the ratio of the difference between the edge value of the pixel and the first edge threshold and the difference between the second edge threshold and the first edge threshold.

5. The processing method according to claim 1, characterized in that If the difference between the horizontal weight index and the vertical weight index of a non-G pixel in the original Bayer image is greater than a preset value, the color difference weighting of the non-G pixel is calculated using only the smaller of the horizontal weight index and the vertical weight index.

6. The processing method according to claim 2, characterized in that The following steps are also included: Searching, in a mosaic G component weight table, a mosaic G component horizontal weight that matches the horizontal weight index of each pixel of the original Bayer image, and a mosaic G component vertical weight that matches the vertical weight index of each pixel of the original Bayer image; wherein, in the mosaic G component weight table, the mosaic G component horizontal weight is negatively correlated with the horizontal weight index, and the mosaic G component vertical weight is negatively correlated with the vertical weight index; The product of the average of the horizontal color differences of the pixels in an even number of horizontal directions centered on each non-G pixel in the denoised Bayer image and the central pixel and the proportion of the horizontal weight of the mosaic G component of the non-G pixel in the total mosaic weight is summed, and the product of the average of the vertical color differences of the pixels in an even number of vertical directions centered on the non-G pixel and the central pixel and the proportion of the vertical weight of the mosaic G component of the non-G pixel in the total mosaic weight is summed to obtain a mosaic color difference weight of the non-G pixel; wherein the total mosaic weight is the sum of the horizontal weight and the vertical weight of the mosaic G component of the non-G pixel; The weighted sum of the noise reduction component of each non-G pixel in the noise reduction Bayer image and the mosaic color difference of the non-G pixel is used as the G component of the demosaiced image at the non-G pixel.

7. The processing method according to claim 2, characterized in that The weighted sum of the noise reduction component of each non-G pixel in the noise reduction Bayer image and the color difference of the non-G pixel is used as the G component of the demosaiced image at the non-G pixel.

8. The processing method according to claim 2, characterized in that: The average value of the G components of all pixels in the comparison window is used as the brightness value of the central pixel; the ratio of the brightness value of the central pixel in the comparison window to the brightness preset value is rounded to obtain a brightness index; searching, in a brightness compensation array, for a brightness compensation / suppression coefficient that matches the brightness index of the center pixel in the comparison window, and using the product of the search absolute difference sum of the center pixel in the comparison window and the brightness compensation / suppression coefficient as a new search absolute difference sum; wherein the brightness index in the brightness compensation array and the brightness compensation coefficient are negatively correlated; A noise reduction coefficient that matches the ratio of the new search absolute difference and the noise reduction intensity of the set pixel in the search window is selected from the noise reduction weight table as the noise reduction coefficient of the pixel.

9. A Bayer image processing device, characterized in that: include: a color difference calculation module, configured to calculate, by interpolation, a horizontal interpolation G component and a vertical interpolation G component of each non-G pixel in the original Bayer image, and calculate a horizontal interpolation non-G component and a vertical interpolation non-G component of each G pixel, and then calculate the difference between the original component and the horizontal interpolation component of each pixel in the original Bayer image to obtain a horizontal color difference, and the difference between the original component and the vertical interpolation component to obtain a vertical color difference; A color difference gradient calculation module is used to calculate the absolute value of the horizontal color difference difference between the two adjacent pixels on the left and right in the horizontal direction of each pixel of the original Bayer image to obtain the horizontal color difference gradient, and the absolute value of the vertical color difference difference between the two adjacent pixels on the top and bottom in the vertical direction to obtain the vertical color difference gradient; A weight calculation module is configured to calculate the sum of the horizontal color difference gradients and the sum of the vertical color difference gradients of pixels within a preset window centered on each pixel of the original Bayer image, respectively serving as the horizontal weight index and the vertical weight index of the central pixel, and search a G component horizontal weight that matches the horizontal weight index of each pixel of the original Bayer image, and a G component vertical weight that matches the vertical weight index of each pixel of the original Bayer image in a G component weight table; wherein, in the G component weight table, the G component horizontal weight is negatively correlated with the sum of the horizontal color difference gradients, and the G component vertical weight is negatively correlated with the sum of the vertical color difference gradients; a color difference weighted calculation module, configured to determine a color difference weight for each non-G pixel in the original Bayer image, wherein the color difference weight is the sum of a first product and a second product, the first product being the product of an average of horizontal color differences of an even number of pixels centered on the non-G pixel and the center pixel in the horizontal direction and a proportion of the horizontal weight of the G component of the non-G pixel in the total weight; and the second product being the product of an average of vertical color differences of an even number of pixels centered on the non-G pixel and the center pixel in the vertical direction and a proportion of the vertical weight of the G component of the non-G pixel in the total weight, wherein the total weight is the sum of the horizontal weight and the vertical weight of the G component of the non-G pixel; The G component guide map generation module is used to take the weighted sum of the original component of each non-G pixel in the original Bayer image and the color difference of the non-G pixel as the G component of the G component guide map of the original Bayer image at the non-G pixel to obtain the G component guide map.

10. The processing device according to claim 9, characterized in that Also includes the following: a search window generating module, configured to use an image block of a first size centered on each pixel in the G component guide image as a template window and an image block of a second size as a search window, wherein the second size is larger than the first size; a search absolute difference sum calculation module, configured to sequentially select comparison windows of the same size as the template window with a set pixel in the search window as the center, and calculate the sum of the absolute values ​​of the differences between the G component of each pixel in each comparison window and the G component of the pixel at the corresponding position in the template window as the search absolute difference sum of the center pixel of the comparison window; a noise reduction intensity generation module, configured to perform edge detection on the G component guide map to obtain an edge value for each pixel, compare the edge value of the pixel with a first edge threshold and a second edge threshold, respectively, and determine the pixel as a weak edge, a flat area, and a strong edge if the edge value is less than the second edge threshold, between the second edge threshold and the first edge threshold, and greater than the first edge threshold, respectively, and set the noise reduction intensity of the pixel to a third intensity, the first intensity, and the second intensity, respectively; wherein the first intensity, the third intensity, and the second intensity decrease in magnitude in sequence; a noise reduction coefficient generation module, configured to select, from a noise reduction weight table, a noise reduction coefficient that matches the ratio of the search absolute difference and the noise reduction intensity of a set pixel in the search window as the noise reduction coefficient of the pixel; wherein, a larger ratio of the search absolute difference and the noise reduction intensity of the pixel in the noise reduction weight table corresponds to a smaller noise reduction coefficient; The denoised Bayer image generation module is used to take the weighted sum of the original component of the pixel in the search window centered on each set pixel in the original Bayer image and the noise reduction coefficient of the pixel as the noise reduction component of the pixel, thereby obtaining the denoised Bayer image.

11. The processing device according to claim 10, characterized in that Performing edge detection on the G component guide image to obtain the edge value of each pixel specifically includes the following steps: Select several groups of symmetrical pixel pairs in each direction centered on each pixel of the G component guide map, calculate the absolute value of the difference between the two pixels in the symmetrical pixel pair, and calculate the sum of the absolute values ​​of several groups of symmetrical pixel pairs in each direction as the sub-edge value in that direction, and then calculate the sum of the sub-edge values ​​in several directions to obtain the edge value of the pixel.

12. The processing device according to claim 11, characterized in that The third strength is equal to the product of the second strength coefficient and the second strength plus the product of the first strength coefficient and the first strength; The first intensity coefficient is equal to 1 minus the second intensity coefficient, and the second intensity coefficient is equal to the ratio of the difference between the edge value of the pixel and the first edge threshold and the difference between the second edge threshold and the first edge threshold.

13. The processing device according to claim 9, characterized in that If the difference between the horizontal weight index and the vertical weight index of a non-G pixel in the original Bayer image is greater than a preset value, the color difference weighting of the non-G pixel is calculated using only the smaller of the horizontal weight index and the vertical weight index.

14. The processing device according to claim 10, characterized in that Also includes: a mosaic G component weight calculation module, configured to search a mosaic G component horizontal weight that matches the horizontal weight index of each pixel of the original Bayer image, and a mosaic G component vertical weight that matches the vertical weight index of each pixel of the original Bayer image in a mosaic G component weight table; wherein, in the mosaic G component weight table, the mosaic G component horizontal weight is negatively correlated with the horizontal weight index, and the mosaic G component vertical weight is negatively correlated with the vertical weight index; A mosaic color difference weighted calculation module is configured to sum the product of the average of the horizontal color differences of the pixels in an even number of horizontal directions centered on each non-G pixel in the denoised Bayer image and the central pixel and the proportion of the horizontal weight of the mosaic G component of the non-G pixel in the total mosaic weight, and the product of the average of the vertical color differences of the pixels in an even number of vertical directions centered on the non-G pixel and the central pixel and the proportion of the vertical weight of the mosaic G component of the non-G pixel in the total mosaic weight, to obtain the mosaic color difference weight of the non-G pixel; wherein the total mosaic weight is the sum of the horizontal weight and the vertical weight of the mosaic G component of the non-G pixel; The demosaiced image generating module is used to take the weighted sum of the noise reduction component of each non-G pixel in the noise reduction Bayer image and the mosaic color difference of the non-G pixel as the G component of the demosaiced image at the non-G pixel.

15. The processing device according to claim 10, characterized in that The weighted sum of the noise reduction component of each non-G pixel in the noise reduction Bayer image and the color difference of the non-G pixel is used as the G component of the demosaiced image at the non-G pixel.

16. The processing device according to claim 10, characterized in that The average value of the G components of all pixels in the comparison window is used as the brightness value of the central pixel; the ratio of the brightness value of the central pixel in the comparison window to the brightness preset value is rounded to obtain a brightness index; searching, in a brightness compensation array, for a brightness compensation / suppression coefficient that matches the brightness index of the center pixel in the comparison window, and using the product of the search absolute difference sum of the center pixel in the comparison window and the brightness compensation / suppression coefficient as a new search absolute difference sum; wherein the brightness index in the brightness compensation array and the brightness compensation coefficient are negatively correlated; A noise reduction coefficient that matches the ratio of the new search absolute difference and the noise reduction intensity of the set pixel in the search window is selected from the noise reduction weight table as the noise reduction coefficient of the pixel.

17. A Bayer image processing device, characterized in that: The Bayer image is processed using the processing method according to any one of claims 1 to 8, or includes the processing device according to any one of claims 9 to 16.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, it can implement the processing method according to any one of claims 1 to 8.

19. A Bayer image processing chip, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and the processor is capable of executing the computer program to implement the method according to any one of claims 1 to 8.

Images