Bayer image processing method, device, electronic device and storage medium

Abstract

The present invention provides a method, device, electronic device and storage medium for Bayer image processing, wherein the Bayer image processing method includes: obtaining an image to be processed; interpolating designated pixels in the image to be processed to obtain green pixel values ​​of the designated pixels; filtering the green pixel values ​​of all pixels; and interpolating the pixel values ​​of the remaining colors of each pixel based on the filtered green pixel values ​​and the pixel values ​​of the designated colors of the designated pixels to obtain the final red pixel value, green pixel value and blue pixel value of each pixel. The present invention can effectively interpolate Bayer images and remove noise in flat areas, retain image edge information, and restore the true color information of the image to the maximum extent. This denoising method is not limited to use after G pixel interpolation is completed, but can also be used for Bayer image denoising.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a Bayer image processing method, device, electronic equipment and storage medium. Background Art

[0002] Currently commonly used image sensors, such as CCDs or CMOS, can only sense light intensity but cannot distinguish color. In practical applications, a color filter array (CFA) is typically placed in front of the photoreceptor. Each pixel records only one color component: red (green or blue). Therefore, the remaining two color components of the pixel must be interpolated to obtain the true color of the image. This type of CFA containing only red (green or blue) pixels is also called a Bayer CFA. Its smallest unit is a 2x2 unit, containing one red pixel, one blue pixel, and two green pixels.

[0003] The CFA interpolation algorithm is generally divided into two steps. First, the green pixel is interpolated using information from the surrounding pixels. Since green pixels account for 1 / 2 of the image, and the other two pixels account for 1 / 4, the green pixel is generally interpolated first, and then the red or blue pixel is interpolated based on the green pixel. Using the green pixel to interpolate the red or blue pixel can quickly obtain the true image color. Therefore, the accuracy of the green pixel interpolation result directly affects whether the Bayer image can be accurately converted to an RGB image. Typically, the Bayer image to be processed has a certain amount of noise, and the first step of green pixel interpolation also introduces noise. The process of using the green pixel interpolation result to calculate the other two pixels will produce error transmission, affecting the final interpolation results of the red and blue pixels. Summary of the Invention

[0004] The present invention provides a Bayer image processing method, apparatus, electronic device, and storage medium that effectively interpolate Bayer images and remove noise from flat areas, preserving image edge information and maximally restoring the image's true color information. This noise reduction method is not limited to use after green pixel interpolation and can also be applied to Bayer image noise reduction.

[0005] According to a first aspect of the present invention, a method for Bayer image processing is provided, the method comprising:

[0006] Get the image to be processed;

[0007] interpolating designated pixels in the image to be processed to obtain green pixel values ​​of the designated pixels, where the pixel values ​​of designated colors of the designated pixels are determined, wherein the designated colors of some designated pixels are red and some designated pixels are blue;

[0008] Filter the green pixel values ​​of all pixels;

[0009] Based on the filtered green pixel value and the pixel value of the designated pixel point, the pixel values ​​of the remaining colors of each pixel point are interpolated to obtain the final red pixel value, green pixel value and blue pixel value of each pixel point.

[0010] According to a second aspect of the present invention, there is provided a device for Bayer image processing, comprising:

[0011] An acquisition module, used for acquiring an image to be processed;

[0012] A green interpolation module is used to interpolate the green pixel values ​​of the designated pixels in the image to be processed to obtain the green pixel values ​​of the designated pixels, wherein the pixel values ​​of the designated pixels are determined, wherein some designated pixels are red and some designated pixels are blue;

[0013] A filtering module is used to filter the green pixel values ​​of all pixels;

[0014] The red and blue interpolation module is used to interpolate the pixel values ​​of the remaining colors of each pixel point based on the filtered green pixel value and the pixel value of the specified pixel point to obtain the final red pixel value, green pixel value and blue pixel value of each pixel point.

[0015] According to a third aspect of the present invention, there is provided an electronic device comprising a memory and a processor.

[0016] The memory is used to store code;

[0017] The processor is used to execute the code in the memory to implement the method involved in the first aspect and its optional solutions.

[0018] According to a fourth aspect of the present invention, there is provided a storage medium having a program stored thereon, wherein the program, when executed by a processor, implements the method involved in the first aspect and its optional solutions.

[0019] The Bayer image processing method, apparatus, electronic device, and storage medium provided by the present invention implement filtering and noise reduction on the green pixel values ​​after interpolating the missing green pixel values ​​of the designated pixel point in the Bayer image, reducing the impact of noise. The blue and red pixel values ​​of the designated pixel point are then calculated based on the filtered green pixel values. This method effectively interpolates the Bayer image and removes noise from flat areas, preserving image edge information, maximizing the restoration of the image's true color information, and improving the accuracy of RGB images. The Bayer image processing method can also be implemented using smaller chips to achieve noise reduction. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 is a schematic diagram of a sliding window of an image to be processed in one embodiment of the present invention;

[0022] Figure 2 1 is a flow chart of a Bayer image processing method according to an embodiment of the present invention;

[0023] Figure 3 1 is a flow chart of S12 in the Bayer image processing method according to one embodiment of the present invention;

[0024] Figure 4 is a schematic diagram of the correlation between the first fusion reference information and the second fusion reference information in one embodiment of the present invention;

[0025] Figure 5 is a schematic flow chart of a green pixel filtering process for a specified pixel point in one embodiment of the present invention;

[0026] Figure 6 is a flow chart of step S131 in one embodiment of the present invention;

[0027] Figure 7 1 is a schematic diagram of program modules of a Bayer image processing apparatus 200 according to an embodiment of the present invention;

[0028] Figure 8 A schematic diagram of submodules of the green pixel interpolation module 202 according to an embodiment of the present invention;

[0029] Figure 9 Schematic diagram of submodules of the green pixel filtering module 203 in one embodiment of the present invention;

[0030] Figure 10 The first embodiment of the present invention is a schematic structural diagram of an electronic device 30. DETAILED DESCRIPTION

[0031] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0032] The terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way are interchangeable where appropriate, so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or apparatus.

[0033] The following specific embodiments are used to describe the technical solution of the present invention in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0034] Please refer to Figure 1 , a schematic diagram of a sliding window of an image to be processed in one embodiment of the present invention;

[0035] The image to be processed uses a sliding window to select a designated pixel point, the center point of the sliding window is the designated pixel point, and the size of the sliding window is 2×r+1, where r is the radius of the sliding window.

[0036] Please refer to Figure 2 , a schematic flow chart of a method for Bayer image processing in one embodiment of the present invention, the method comprising:

[0037] Step S11: obtaining an image to be processed;

[0038] The image to be processed is: the original image obtained by a color filter array (CFA) placed in front of the photoreceptor or an image obtained based on the original image, wherein each pixel records only one color component of the three colors: red, green, or blue, wherein the number of green pixels accounts for 1 / 2 of the total number of pixels, and the number of red pixels and blue pixels each accounts for 1 / 4 of the total number of pixels; the image to be processed can be understood as a Bayer image, wherein the distribution of red pixels, blue pixels, and green pixels can refer to the conventional understanding of Bayer images in this field.

[0039] Step S12: interpolating a designated pixel point in the image to be processed to obtain a green pixel value of the designated pixel point;

[0040] The pixel values ​​of the designated colors of the designated pixels are determined, wherein the designated colors of some designated pixels are red, and the designated colors of some designated pixels are blue;

[0041] Step S13: filtering the green pixel values ​​of all pixels;

[0042] Step S14: Based on the filtered green pixel value and the pixel value of the designated color of the designated pixel point, the pixel values ​​of the remaining colors of each pixel point are interpolated to obtain the red pixel value, green pixel value and blue pixel value of each pixel point.

[0043] In the above scheme, after Bayer image interpolation calculates the missing green pixel value for the specified pixel, filtering and denoising the green pixel value is performed to reduce the impact of noise. The blue and red pixel values ​​for the specified pixel are then calculated based on the filtered green pixel value. This method effectively removes noise from flat areas, preserves image edge information, and maximizes the restoration of true color information, improving the accuracy of RGB images.

[0044] Please refer to Figure 3 , a schematic flow chart of S12 in the Bayer image processing method in one embodiment of the present invention includes:

[0045] S121: Calculating a first color difference in a first direction and a second color difference in a second direction of a pixel value of each pixel point in the image to be processed;

[0046] The pixel values ​​of the designated colors of the designated pixels are determined, wherein the designated colors of some designated pixels are red, and the designated colors of some designated pixels are blue;

[0047] S122: Determine a first directional difference value, a second directional difference value, and a non-directional difference value based on the first color difference and the second color difference;

[0048] Specifically, for example, based on the first color difference and the second color difference, a target difference value of the specified pixel point under the influence of the pixel value change in the first direction is evaluated to obtain a first directional difference value; the target difference value of the specified pixel point under the influence of the pixel value change in the second direction is evaluated to obtain a second directional difference value; and the influence of the surrounding pixel value change on the target difference value of the specified pixel point is evaluated to obtain a non-directional difference value; the target difference value represents the difference between the pixel value of the specified pixel point and the green pixel value;

[0049] S123: Determine difference fusion reference information of the designated pixel based on the first color difference and the second color difference, where the difference fusion reference information represents the influence of the first directional difference, the second directional difference, and the non-directional difference on a fused target difference.

[0050] S124: Based on the fusion reference information of the designated pixel point, fuse the first directional difference, the second directional difference, and the non-directional difference to obtain a fused target difference value of the designated pixel point;

[0051] S125: Determine the green pixel value of the designated pixel point based on the fused target difference.

[0052] S121 includes: calculating a first color difference in the first direction and a second color difference in the second direction of the pixel value at the position of the specified pixel point in the image to be processed based on the following formula:

[0053] DH i,j =C i,j -(G i,j-1 +G i,j+1 ) / 2;

[0054] DV i,j =C i,j -(G i-1,j +G i+1,j ) / 2;

[0055] DH i,j-1 =G i,j-1 -(C i,j-2 +C i,j ) / 2;

[0056] DV i-1,j =G i-1,j -(C i-2,j +C i,j ) / 2;

[0057] in:

[0058] C i，jis the specified pixel value of the specified pixel point at position (i, j) in the image to be processed. If the specified pixel point is a red pixel point, C in the formula i，j The value is the red pixel value. If the specified pixel point is a blue pixel point, C in the formula i，j The value is the blue pixel value;

[0059] DH i,j is the first color difference of the specified pixel at position (i, j) in the image to be processed;

[0060] DV i,j is the second color difference of the specified pixel at position (i, j) in the image to be processed;

[0061] C i,j A pixel value representing a specified color of a specified pixel at position (i, j) in the image to be processed;

[0062] C i,j-2 Represents the pixel value of the specified color at the position (i, j-2) in the image to be processed;

[0063] C i-2,j Represents the pixel value of the specified color at the position (i-2, j) in the image to be processed;

[0064] G i,j+1 is the green pixel value at position (i, j+1) in the image to be processed;

[0065] G i-1,j is the green pixel value at position (i-1, j) in the image to be processed;

[0066] G i+1,j is the green pixel value at position (i+1, j) in the image to be processed;

[0067] In step S122, the eastward color difference, westward color difference, northward color difference, and southward color difference of the designated pixel point may be calculated based on the first color difference and the second color difference. The calculating step includes:

[0068] Calculate the statistical value of the first color difference of the specified pixel and one or more pixel points along the east direction to obtain the east color difference; the east direction refers to the positive direction of the first direction; the corresponding calculation formula can be, for example:

[0069] deltaE i,j =(DH i,j+2 +DH i,j+1 +DH i,j ) / 3;

[0070] Among them, deltaE i,jis the eastward difference at position (i, j) in the image to be processed, DH i,j+2 is the first color difference at position (i, j+2) in the image to be processed, DH i,j+1 is the first color difference at position (i, j+1) in the image to be processed, DH i，j is the first color difference at position (i, j) in the image to be processed;

[0071] Calculate the statistical value of the first color difference between the specified pixel and one or more pixels along the west direction to obtain a westward difference; the westward direction refers to the negative direction of the first direction; the corresponding calculation formula can be, for example:

[0072] deltaW i,j =(DH i,j-2 +DH i,j-1 +DH i,j ) / 3;

[0073] in,

[0074] deltaW i,j is the westward difference at position (i, j) in the image to be processed;

[0075] DH i,j-2 is the first color difference at position (i, j-2) in the image to be processed;

[0076] DH i,j-1 is the first color difference at position (i, j-1) in the image to be processed;

[0077] DH i,j is the first color difference at position (i, j) in the image to be processed;

[0078] Calculating a statistical value of a first color difference between the designated pixel and one or more pixels along the south direction to obtain a south direction difference; wherein the south direction refers to a positive direction of the second direction;

[0079] deltaS i,j =(DV i+2,j +DV i+1,j +DV i,j ) / 3;

[0080] in,

[0081] deltaS i,j is the south difference at position (i, j) in the image to be processed;

[0082] DV i+2,j is the second color difference at position (i+2, j) in the image to be processed;

[0083] DV i+1，j is the second color difference at position (i+1, j) in the image to be processed;

[0084] DV i，j is the second color difference at position (i, j) in the image to be processed;

[0085] Calculating a statistical value of a first color difference between the designated pixel and one or more pixels along the north direction to obtain a north direction difference; wherein the north direction refers to the negative direction of the second direction;

[0086] deltaN i,j =(DV i-2,j +DV i-1,j +DV i,j ) / 3;

[0087] in,

[0088] deltaN i,j is the north difference at position (i, j) in the image to be processed;

[0089] DV i-2,j is the second color difference at position (i-2, j) in the image to be processed;

[0090] DV i-1,j is the second color difference at position (i-1, j) in the image to be processed;

[0091] DV i,j is the second color difference at position (i, j) in the image to be processed;

[0092] In step S122, eastward fusion reference information, westward fusion reference information, southward reference information, and northward reference information of the designated pixel may be calculated based on the first color difference and the second color difference. The calculation process includes:

[0093] Calculate the eastward fusion reference information of the specified pixel point:

[0094] wE i,j =|DH i,j |+|DH i,j+2 -DH i,j+1 |+α1×|DH i,j -DH i,j+1 |+α2

[0095] ×|DH i-1,j -DH i-1,j+1 |+α3×|DH i+1,j -DH i+1,j+1 |

[0096] in,

[0097] wE i,j Representing the eastward fusion reference information of the designated pixel point at position (i, j) in the image to be processed;

[0098] DH i,j Characterizing the first color difference at position (i, j) in the image to be processed;

[0099] DH i,j+2 Characterizes the first color difference at position (i, j+2) in the image to be processed;

[0100] DH i,j+1 Characterizes the first color difference at position (i, j+1) in the image to be processed;

[0101] DH i,j Characterizing the first color difference at position (i, j) in the image to be processed;

[0102] DH i-1,j Characterizes the first color difference at position (i-1, j) in the image to be processed;

[0103] DH i-1,j+1 Characterizes the first color difference at position (i-1, j+1) in the image to be processed;

[0104] DH i+1,j Characterizes the first color difference at position (i+1, j) in the image to be processed;

[0105] DH i+1,j+1 Characterizes the first color difference at the position (i+1, j+1) in the image to be processed;

[0106] a1, a2, a3 are coefficients used to calculate the fusion reference information, 0≤a1, a2, a3≤1;

[0107] Calculate the westward fusion reference information of the specified pixel point:

[0108] wW i,j =|DH i,j |+|DH i,j-2 -DH i,j-1 |+α1×|DH i,j -DH i,j-1 |+α2

[0109] ×|DH i-1,j -DH i-1,j-1 |+α3×|DH i+1,j -DH i+1,j-1 |

[0110] wWi,j Representing the westward fusion reference information of the designated pixel point at position (i, j) in the image to be processed;

[0111] DH i,j Characterizing the first color difference at position (i, j) in the image to be processed;

[0112] DH i,j-2 Characterizes the first color difference at position (i, j-2) in the image to be processed;

[0113] DH i,j-1 Characterizes the first color difference at position (i, j-1) in the image to be processed;

[0114] DH i-1,j Characterizes the first color difference at position (i-1, j) in the image to be processed;

[0115] DH i-1,j-1 Characterizes the first color difference at the position (i-1, j-1) in the image to be processed;

[0116] DH i+1,j Characterizes the first color difference at position (i+1, j) in the image to be processed;

[0117] a1, a2, a3 are coefficients used to calculate the fusion reference information, 0≤a1, a2, a3≤1;

[0118] Calculate the southbound fusion reference information of the specified pixel:

[0119] wxya i,j =|DV i,j |+|DV i+2,j -DV i+1,j |+α1×|DV i,j -DV i+1,j |+α2

[0120] ×|DV i,j-1 -DV i+1,j-1 |+α3×|DV i,j+1 -DV i+1,j+1 |

[0121] wxya i,j Representing the southward fusion reference information of the designated pixel point at position (i, j) in the image to be processed;

[0122] DV i+2,j Characterizes the second color difference at position (i+2, j) in the image to be processed;

[0123] DV i+1,jCharacterizes the second color difference at position (i+1, j) in the image to be processed;

[0124] DV i,j Characterizing the second color difference at position (i, j) in the image to be processed;

[0125] DV i+1,j-1 Characterizes the second color difference at the position (i+1, j-1) in the image to be processed;

[0126] DV i,j-1 Characterizes the second color difference at the position (i, j-1) in the image to be processed;

[0127] DV i+1,j+1 Characterizes the second color difference at the position (i+1, j+1) in the image to be processed;

[0128] DV i,j+1 Characterizes the second color difference at position (i, j+1) in the image to be processed;

[0129] a1, a2, a3 are coefficients used to calculate the fusion reference information, 0≤a1, a2, a3≤1;

[0130] Calculate the north fusion reference information of the specified pixel:

[0131] wN i,j =|DV i,j |+|DV i-2,j -DV i-1,j |+α1×|DV i,j -DV i-1,j |+α2

[0132] ×|DV i,j-1 -DV i-1,j-1 |+α3×|DV i,j+1 -DH i-1,j+1 |

[0133] wN i,j Characterizing the north fusion reference information of the designated pixel point at position (i, j) in the image to be processed;

[0134] DV i,j Characterizing the second color difference at position (i, j) in the image to be processed;

[0135] DV i-1,j Characterizes the second color difference at position (i-1, j) in the image to be processed;

[0136] DV i-2,j Characterizing the second color difference at position (i-2, j) in the image to be processed;

[0137] DV i,j-1 Characterizes the second color difference at the position (i, j-1) in the image to be processed;

[0138] DV i-1,j-1 Characterizes the second color difference at the position (i-1, j-1) in the image to be processed;

[0139] DV i,j+1 Characterizes the second color difference at position (i, j+1) in the image to be processed;

[0140] DV i-1,j+1 Characterizes the second color difference at position (i-1, j+1) in the image to be processed;

[0141] a1, a2, a3 are coefficients used to calculate the fusion reference information, 0≤a1, a2, a3≤1;

[0142] The size of the sliding window used for fusion reference information calculation is not unique.

[0143] In step S122, the eastward chromatic aberration and the westward chromatic aberration may be weightedly summed based on the eastward fusion reference information and the westward fusion reference information to obtain the first directional difference value:

[0144]

[0145] in:

[0146] DeltaH i,j A first directional difference value representing the specified pixel point at position (i, j) in the image to be processed;

[0147] deltaX i,j Represents the X-direction difference of the position (i, j) in the image to be processed, where X is the east and west directions;

[0148] wxya i,j Represents the X-direction fusion reference information of the position (i, j) in the image to be processed, where the X direction is east and west;

[0149] Based on the southbound fusion reference information and the northbound fusion reference information, a weighted sum is performed on the southbound color difference and the northbound color difference to obtain the second directional difference value:

[0150]

[0151] in:

[0152] DeltaV i,jA second directional difference value representing the specified pixel point at position (i, j) in the image to be processed;

[0153] deltaX i,j Represents the X-direction difference of the position (i, j) in the image to be processed, where X is the south and north directions;

[0154] wxya i,j Represents the X-direction fusion reference information of the (i, j) position in the image to be processed, where X represents the south and north directions;

[0155] The non-directional difference is calculated based on the following formula:

[0156]

[0157] in:

[0158] DeltaHV i,j Characterizes the directionless difference of the specified pixel point at position (i, j) in the image to be processed;

[0159] deltaX i,j Represents the X-direction difference of the position (i, j) in the image to be processed, where X is east, west, south and north;

[0160] wxya i,j The X-direction fusion reference information of the position (i, j) in the image to be processed is represented, where the directions of X are east, west, south and north.

[0161] Step S123: Calculating the difference fusion reference information of the designated pixel based on the first color difference and the second color difference.

[0162] The calculation process includes:

[0163] The first and second gradients are calculated based on the following formulas:

[0164] diffH i,j =|DH i,j-1 -DH i,j+1 |

[0165] diffV i,j =|DV i-1,j -DV i+1,j |

[0166] in,

[0167] diffH i,j Characterizing the first gradient of the designated pixel at position (i, j) in the image to be processed;

[0168] diffVi,j Characterizing the second gradient of the designated pixel at position (i, j) in the image to be processed;

[0169] DH i,j-1 Characterizes the first color difference at position (i, j-1) in the image to be processed;

[0170] DH i,j+1 Characterizes the first color difference at position (i, j+1) in the image to be processed;

[0171] DV i-1,j Characterizes the second color difference at position (i-1, j) in the image to be processed;

[0172] DV i+1,j Characterizes the second color difference at position (i+1, j) in the image to be processed;

[0173] The first gradient threshold upper limit, the first gradient threshold lower limit, the second gradient threshold upper limit, and the second gradient threshold lower limit are calculated based on the following formula:

[0174] thL1=diffV i,j ×coef1;thH1=diffV i,j ×coef2

[0175] thL2=diffV i,j ×coef3;thH2=diffV i,j ×coef4

[0176] Among them, cofe1 <cofe2,cofe3<cofe4,

[0177] cofe1≥1,cofe2≥1,0≤cofe3≤1,0≤cofe4≤1

[0178] in,

[0179] thL1 represents the lower limit of the first gradient threshold of the designated pixel point in the image to be processed;

[0180] thH1 represents the upper limit of the first gradient threshold of the designated pixel point in the image to be processed;

[0181] thL2 represents the lower limit of the second gradient threshold of the designated pixel in the image to be processed;

[0182] thH2 represents the upper limit of the second gradient threshold of the designated pixel in the image to be processed;

[0183] diffV i,jCharacterizing the second gradient of the designated pixel at position (i, j) in the image to be processed;

[0184] coef1 and coef2 represent the evaluation coefficients of the first gradient threshold in the image to be processed;

[0185] coef3 and coef4 represent the evaluation coefficients of the second gradient threshold in the image to be processed;

[0186] The first fusion reference information and the second fusion reference information are calculated based on the following formula:

[0187]

[0188] in:

[0189] alphaH i,j The first fusion reference information representing the designated pixel point in the image to be processed;

[0190] alphaV i,j The second fusion reference information representing the designated pixel point in the image to be processed;

[0191] diffH i,j Characterizing the first gradient of the designated pixel at position (i, j) in the image to be processed;

[0192] diffV i,j Characterizing the second gradient of the designated pixel at position (i, j) in the image to be processed;

[0193] thL1 represents the lower limit of the first gradient threshold of the designated pixel point in the image to be processed;

[0194] thH1 represents the upper limit of the first gradient threshold of the designated pixel point in the image to be processed;

[0195] thL2 represents the lower limit of the second gradient threshold of the designated pixel in the image to be processed;

[0196] thH2 represents the upper limit of the second gradient threshold of the designated pixel in the image to be processed.

[0197] Step S124: Based on the fusion reference information of the designated pixel point, the first directional difference, the second directional difference, and the non-directional difference are fused to obtain a fused target difference value of the designated pixel point. The calculation process includes:

[0198]

[0199] in:

[0200] Delta i,j Represents the fused target difference value of the specified pixel point at position (i, j) in the image to be processed;

[0201] DeltaH i,j The first directional difference value representing the specified pixel point at position (i, j) in the image to be processed;

[0202] DeltaV i,j The second directional difference value representing the specified pixel point at position (i, j) in the image to be processed;

[0203] DeltaHV i,j The directionless difference value representing the specified pixel point at position (i, j) in the image to be processed;

[0204] alphaH i,j The first fusion reference information representing the specified pixel point at position (i, j) in the image to be processed;

[0205] alphaV i,j The second fusion reference information representing the specified pixel point at position (i, j) in the image to be processed;

[0206] diffH i,j Characterizing the first gradient of the designated pixel at position (i, j) in the image to be processed;

[0207] diffV i,j Characterizing the second gradient of the designated pixel at position (i, j) in the image to be processed;

[0208] thL1 represents the lower limit of the first gradient threshold of the designated pixel point in the image to be processed;

[0209] thH1 represents the upper limit of the first gradient threshold of the designated pixel point in the image to be processed;

[0210] thL2 represents the lower limit of the second gradient threshold of the designated pixel in the image to be processed;

[0211] thH2 represents the upper limit of the second gradient threshold of the designated pixel in the image to be processed.

[0212] Please refer to Figure 4 , a schematic diagram of the correlation between the first fusion reference information and the second fusion reference information in one embodiment of the present invention.

[0213] The target difference after fusion Delta i,j The threshold used in the calculation process includes the first fusion reference information alphaH i,j and the second fusion reference information alphaV i,j ,When the threshold is multiple, the curve segments also increase accordingly.

[0214] In another embodiment, the calculation results of several sliding windows may be combined to obtain the first gradient difference diffH i,j and the second gradient difference diffV i,j .

[0215] Optionally, when the gradient threshold is 1, the target difference is calculated as follows:

[0216]

[0217] The alpha is a configuration parameter. Generally, the value of alpha is 0 to 1.

[0218] DeltaH i,j The first directional difference value representing the specified pixel point at position (i, j) in the image to be processed;

[0219] DeltaV i,j The second directional difference value representing the specified pixel point at position (i, j) in the image to be processed;

[0220] DeltaHV i,j The directionless difference value representing the specified pixel point at position (i, j) in the image to be processed;

[0221] Step S125: Determine the green pixel value of the designated pixel based on the fused target difference. The calculation process includes:

[0222] Calculate the sum of the fused target difference value of the specified pixel point and the original pixel value of the specified pixel point to obtain the green pixel value of the specified pixel point:

[0223] G i,j =C i,j +Delta i,j ;

[0224] in:

[0225] G i,j The green pixel value of the designated pixel at position (i, j) in the image to be processed, i.e., the interpolation result of the green pixel;

[0226] Ci,j Represents the pixel value of the specified pixel at position (i, j) in the image to be processed;

[0227] Delta i,j The fused target difference value of the designated pixel point at the (i, j) position in the image to be processed is represented.

[0228] In an embodiment of the present invention, the interpolation direction of the green pixel is selected based on the fusion reference information of the designated pixel point. The selection of the threshold and the relationship between the thresholds are key aspects of the present invention. The threshold can be single or multiple. The present invention calculates the target difference using the fusion reference information for two thresholds and a single threshold, respectively. Regardless of the threshold selection method, it complies with the principles of the present invention and achieves the purpose of selecting the interpolation direction and fusing the interpolation results.

[0229] Please refer to Figure 5 , a schematic flow chart of a green pixel filtering method for a specified pixel point according to one embodiment of the present invention. Since the interpolation of red and blue pixels depends on the green pixels, the accuracy of the green pixel interpolation result directly affects the image quality. Therefore, after the green pixel interpolation is completed, noise processing is performed on all green pixels to reduce the impact of noise on the conversion of the Bayer image into an RGB image.

[0230] Step S13 may include:

[0231] Step S131: determining a green pixel correction value of the designated pixel point based on the green pixel value and the original pixel value of each pixel point in the image to be processed;

[0232] Step S132: determining filtering fusion reference information for each pixel point based on the original pixel value at the location of each pixel point in the image to be processed; the filtering fusion reference information represents the influence of the green pixel value and the green pixel correction value of the corresponding pixel point on the fused green pixel value;

[0233] Step S133: Based on the filtering fusion reference information, fuse the green pixel value and the green pixel correction value to obtain a fused green pixel value of the corresponding pixel point, and use the fused green pixel value as the filtered green pixel value.

[0234] Please refer to Figure 6 , a flow chart of step S131 in one embodiment of the present invention;

[0235] The step S131 includes:

[0236] S1311: Based on the green pixel value at the position of each pixel point in the image to be processed, evaluating the mean of the corresponding pixel points under the influence of the change of the green pixel value in the first direction to obtain a first directional mean value; evaluating the mean of the specified pixel points under the influence of the change of the green pixel value in the second direction to obtain a second directional mean value;

[0237] The first directional mean is calculated based on the following formula:

[0238]

[0239] in:

[0240] GH i,j represents the mean of the corresponding pixels under the influence of the change of the green pixel value in the first direction, that is, the first direction mean;

[0241] G i,j-1 represents the green pixel value at position (i, j-1) in the image to be processed;

[0242] G i,j represents the green pixel value at position (i, j) in the image to be processed;

[0243] G i,j+1 represents the green pixel value at position (i, j+1) in the image to be processed;

[0244] The second directivity mean is calculated based on the following formula:

[0245]

[0246] GV i,j represents the mean of the corresponding pixels under the influence of the change of the green pixel value in the second direction, that is, the second direction mean;

[0247] G i,j represents the green pixel value at position (i, j) in the image to be processed;

[0248] G i-1,j represents the green pixel value at position (i-1, j) in the image to be processed;

[0249] G i+1,j represents the green pixel value at position (i+1, j) in the image to be processed;

[0250] S1312: Determine first directional filtering fusion reference information of the first direction and second directional filtering fusion reference information of the second direction based on the original pixel value of each pixel point in the image to be processed;

[0251] The first directional filtering fusion reference information and the second directional filtering fusion reference information are calculated based on the following formula:

[0252]

[0253] WH i,j The first directional filtering fusion reference information representing the position (i, j) in the image to be processed;

[0254] WV i,j The second directional filtering fusion reference information representing the position (i, j) in the image to be processed;

[0255] I i,j represents the original pixel value at position (i, j) in the image to be processed;

[0256] I i-1,j-1 represents the original pixel value at position (i-1, j-1) in the image to be processed;

[0257] I i-1,j Represents the original pixel value at position (i-1, j) in the image to be processed;

[0258] I i-1,j+1 Represents the original pixel value at position (i-1, j+1) in the image to be processed;

[0259] I i,j-1 Represents the original pixel value at position (i, j-1) in the image to be processed;

[0260] I i,j+1 Represents the original pixel value at position (i, j+1) in the image to be processed;

[0261] I i+1,j-1 represents the original pixel value at position (i+1, j-1) in the image to be processed;

[0262] I i+1,j represents the original pixel value at position (i+1, j) in the image to be processed;

[0263] I i+1,j+1 represents the original pixel value at position (i+1, j+1) in the image to be processed;

[0264] S1313: Determine first directional filtering reference information based on first filtering fusion reference information of one or more pixels, and determine second directional filtering reference information based on second filtering fusion reference information of one or more pixels;

[0265] The first directional filtering reference information and the second directional filtering reference information may be calculated based on the following formula:

[0266]

[0267] in:

[0268] The first directional filtering reference information representing the specified pixel point at the position (i, j) in the image to be processed;

[0269] Second directional filtering reference information representing the specified pixel point at position (i, j) in the image to be processed;

[0270] r1 is the radius, which represents the calculation the number of pixels in the positive direction of the first direction, the negative direction of the first direction, the positive direction of the second direction, and the negative direction of the second direction selected when ;

[0271] N represents the total number of pixels within the radius r1;

[0272] WH m,n The first direction filtering reference information representing the pixel points within the radius r1 in the image to be processed, wherein: -r1≤m≤r1, -r1≤n≤r1;

[0273] WV m,n The second direction filtering reference information representing the pixel points within the radius r in the image to be processed, wherein: -r1≤m≤r1, -r1≤n≤r1;

[0274] S1314: Calculate a green pixel correction value of the designated pixel based on the first directional mean value, the second directional mean value, the first directional filter reference information, and the second directional filter reference information;

[0275] The specific calculation can be based on the following formula:

[0276]

[0277] in:

[0278] Gest i,j represents the green pixel correction value at position (i, j) in the image to be processed;

[0279] The first directional filtering reference information representing the position (i, j) in the image to be processed;

[0280] The second directional filtering reference information representing the position (i, j) in the image to be processed;

[0281] GH i,j Characterizing the first directional mean value of the (i, j) position in the image to be processed;

[0282] GV i,j The second directional mean value of the position (i, j) in the image to be processed is characterized.

[0283] Step S132 may further include:

[0284] Calculating the mean and variance of the green pixel correction value of the designated pixel point based on the green pixel correction value of the designated pixel point, based on the following formula;

[0285]

[0286] in:

[0287] represents the mean value of the green pixel correction value of the designated pixel at position (i, j) in the image to be processed;

[0288] r1 is the radius, which represents the calculation the number of pixels in the positive direction of the first direction, the negative direction of the first direction, the positive direction of the second direction, and the negative direction of the second direction selected when ;

[0289] N represents the total number of pixels within the radius r1;

[0290] Gest m,n It represents the green pixel correction value of the pixel points within the radius r1 in the image to be processed, wherein: -r1≤m≤r, -r1≤n≤r1;

[0291] var i,j Characterizes the variance of the green pixel correction value of the specified pixel point at position (i, j) in the image to be processed;

[0292] The present invention uses the mean value to participate in the calculation of the variance var as an example, which can simplify the calculation. The calculation method can also use the expected value of the green pixel correction value instead of the mean value to calculate the variance var.

[0293] Based on the mean and variance of the green pixel correction values, the filtering fusion reference information of the specified pixel point is determined, and the filtering fusion reference information is calculated based on the following formula:

[0294]

[0295] in:

[0296] var i,j Characterizes the variance of the green pixel correction value of the specified pixel point at position (i, j) in the image to be processed;

[0297] var n Characterizes the noise variance of the specified pixel at position (i, j) in the image to be processed, where the noise variance can be calculated based on the brightness or gradient of the image to be processed;

[0298] alpha represents the filtering fusion reference information of the specified pixel point at position (i, j) in the image to be processed;

[0299] In another embodiment, the noise is Gaussian noise, and the calculation formula of the filtering fusion reference information is as follows:

[0300]

[0301] alpha represents the filtering fusion reference information of the (i, j) position in the image to be processed;

[0302] Step S133: Based on the filtered fusion reference information, the green pixel value and the green pixel correction value are fused to obtain a fused green pixel value of the corresponding pixel point, and the fused green pixel value is used as the filtered green pixel value. The filtered green pixel value G' is calculated based on the following formula: i,j :

[0303]

[0304] in:

[0305] G' i,j represents the filtered green pixel value at position (i, j) in the image to be processed;

[0306] G i,j represents the green pixel value at position (i, j) in the image to be processed;

[0307] alpha represents the filtering fusion reference information of the (i, j) position in the image to be processed;

[0308] It represents the mean of the green pixel correction values ​​at the (i, j) position in the image to be processed.

[0309] The filtering method mentioned in the present invention can also be used in Bayer domain image filtering. The filtering fusion reference information fuses the green pixel points around the specified pixel point or is calculated in other ways. It is not limited to using the green pixel value obtained by the method of the present invention and then using this method for filtering.

[0310] Step S14: Based on the filtered green pixel value and the pixel value of the designated pixel point, interpolate the pixel values ​​of the remaining colors of each pixel point to obtain the red pixel value, green pixel value and blue pixel value of each pixel point.

[0311] For example, if the center point of the sliding window is a blue pixel, the calculation formula for the red pixel value is as follows:

[0312]

[0313] in:

[0314] R i,j Represents the red pixel value at position (i, j) in the image to be processed;

[0315] G' i,j represents the filtered green pixel value at position (i, j) in the image to be processed;

[0316] G' i-1,j-1 represents the filtered green pixel value at position (i-1, j-1) in the image to be processed;

[0317] G' i-1,j+1 represents the filtered green pixel value at position (i-1, j+1) in the image to be processed;

[0318] G' i+1,j-1 represents the filtered green pixel value at position (i+1, j-1) in the image to be processed;

[0319] G' i+1,j+1 represents the filtered green pixel value at position (i+1, j+1) in the image to be processed;

[0320] R i-1,j-1 Represents the red pixel value at position (i-1, j-1) in the image to be processed;

[0321] R i-1,j+1 represents the red pixel value at position (i-1, j+1) in the image to be processed;

[0322] R i+1,j-1 represents the red pixel value at position (i+1, j-1) in the image to be processed;

[0323] Ri+1,j+1 It represents the red pixel value at the (i+1, j+1) position in the image to be processed.

[0324] If the center point of the sliding window is a red pixel point, the calculation formula for the blue pixel value can be obtained by replacing the red pixel value in the above calculation formula with the blue pixel value, thereby calculating the corresponding blue pixel value.

[0325] In one pixel interpolation method, if the center point of the sliding window is a green pixel point Gb, Gb can be understood as the green component adjacent to the blue component in the horizontal direction. The calculation formula for the red pixel value and the blue pixel value of the center point of the sliding window is as follows:

[0326]

[0327] R i,j Represents the red pixel value at position (i, j) in the image to be processed;

[0328] G' i,j Characterizes the post-wave green pixel value of the (i, j) position in the image to be processed;

[0329] R i-1,j Represents the red pixel value at position (i-1, j) in the image to be processed;

[0330] R i+1,j represents the red pixel value at position (i+1, j) in the image to be processed;

[0331] G' i-1,j represents the filtered green pixel value at position (i-1, j) in the image to be processed;

[0332] G' i+1,j represents the filtered green pixel value at position (i+1, j) in the image to be processed;

[0333] B i,j represents the blue pixel value at position (i, j) in the image to be processed;

[0334] B i,j-1 represents the blue pixel value at position (i, j-1) in the image to be processed;

[0335] B i,j+1 represents the blue pixel value at position (i, j+1) in the image to be processed;

[0336] G' i,j-1 represents the filtered green pixel value at position (i, j-1) in the image to be processed;

[0337] G'i,j+1 represents the filtered green pixel value at position (i, j+1) in the image to be processed;

[0338] If the center point of the sliding window is a green pixel Gr, Gr can be understood as the green component adjacent to the red component in the horizontal direction. The calculation formula for the red pixel value and the blue pixel value of the center point of the sliding window is as follows:

[0339]

[0340] R i,j Represents the red pixel value at position (i, j) in the image to be processed;

[0341] G' i,j represents the filtered green pixel value at position (i, j) in the image to be processed;

[0342] R i-1,j Represents the red pixel value at position (i-1, j) in the image to be processed;

[0343] R i+1,j represents the red pixel value at position (i+1, j) in the image to be processed;

[0344] G' i-1,j represents the filtered green pixel value at position (i-1, j) in the image to be processed;

[0345] G' i+1,j represents the filtered green pixel value at position (i+1, j) in the image to be processed;

[0346] B i,j represents the blue pixel value at position (i, j) in the image to be processed;

[0347] B i,j-1 represents the blue pixel value at position (i, j-1) in the image to be processed;

[0348] B i,j+1 represents the blue pixel value at position (i, j+1) in the image to be processed;

[0349] G' i,j-1 represents the filtered green pixel value at position (i, j-1) in the image to be processed;

[0350] G' i,j+1 It represents the filtered green pixel value at position (i, j+1) in the image to be processed.

[0351] Please refer to Figure 7, a Bayer image processing apparatus 200 is provided, comprising:

[0352] An acquisition module 201 is used to acquire an image to be processed;

[0353] A green interpolation module 202 is configured to interpolate designated pixels in the image to be processed to obtain green pixel values ​​of the designated pixels, where the pixel values ​​of designated colors of the designated pixels are already determined, wherein the designated colors of some designated pixels are red and some are blue;

[0354] Filtering module 203, used to filter the green pixel values ​​of all pixels;

[0355] A red and blue interpolation module 204 is configured to interpolate the pixel values ​​of the remaining colors of each pixel based on the filtered green pixel value and the original pixel value of the designated pixel to obtain a red pixel value, a green pixel value, and a blue pixel value for each pixel;

[0356] Please refer to Figure 8 , a schematic diagram of submodules of the green interpolation module 202;

[0357] The color difference calculation submodule 2021 is configured to calculate a first color difference in a first direction and a second color difference in a second direction for the pixel value of each pixel point in the image to be processed; the pixel value of the designated color of the designated pixel point is determined, wherein some designated pixels are red and some designated pixels are blue;

[0358] a directional difference calculation submodule 2022 for evaluating, based on the first color difference and the second color difference, a target difference for a specified pixel point under the influence of pixel value changes in the first direction to obtain a first directional difference; evaluating the target difference for the specified pixel point under the influence of pixel value changes in the second direction to obtain a second directional difference; and evaluating the target difference for the specified pixel point under the influence of surrounding pixel value changes to obtain a non-directional difference; the target difference representing the difference between the original pixel value and the green pixel value of the specified pixel point;

[0359] A difference fusion reference information calculation submodule 2023 is configured to determine difference fusion reference information of the specified pixel point based on the first color difference and the second color difference, wherein the difference fusion reference information represents the influence of the first directional difference, the second directional difference, and the non-directional difference on the fused target difference;

[0360] The target difference calculation submodule 2024 fuses the first directional difference, the second directional difference, and the non-directional difference based on the fusion reference information of the designated pixel to obtain a fused target difference of the designated pixel;

[0361] The green pixel fusion submodule 2025 determines the green pixel value of the designated pixel point based on the fused target difference.

[0362] Please refer to Figure 9 , a schematic diagram of submodules of the filtering module 203;

[0363] A green pixel correction value calculation submodule 2031 is configured to determine a green pixel correction value of a designated pixel point based on the green pixel value and the original pixel value of each pixel point;

[0364] The filter fusion reference information calculation submodule 2032 is configured to determine filter fusion reference information for each pixel based on the green pixel value at the location of each pixel; the filter fusion reference information represents the impact of the green pixel value and the green pixel correction value of the corresponding pixel on the fused green pixel value;

[0365] The green pixel filtering and fusion submodule 2033 is used to fuse the green pixel value and the green pixel correction value based on the filtering and fusion reference information to obtain the fused green pixel value of the corresponding pixel point, and use the fused green pixel value as the filtered green pixel value.

[0366] The Bayer image processing method, apparatus, electronic device, and storage medium provided by the present invention interpolate the missing green pixels of a specified pixel in a Bayer image to obtain a grayscale image. Noise reduction is performed on the grayscale image to reduce the impact of noise. The blue and red pixel values ​​of the specified pixel are then calculated based on the interpolated green pixel values ​​of the specified pixel. This method improves the accuracy of RGB images and achieves noise reduction at the expense of smaller chip implementations.

[0367] Please refer to Figure 10 , provides an electronic device 30, including:

[0368] processor 31; and

[0369] a memory 32 for storing executable instructions of the processor;

[0370] The processor 31 is configured to execute the above-mentioned method by executing the executable instructions.

[0371] The processor 31 can communicate with the memory 32 via a bus 33 .

[0372] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned method when executed by a processor.

[0373] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0374] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A Bayer image processing method, characterized in that: include: Get the image to be processed; interpolating designated pixels in the image to be processed to obtain green pixel values ​​of the designated pixels, where the pixel values ​​of designated colors of the designated pixels are determined, wherein the designated colors of some designated pixels are red and some designated pixels are blue; Filter the green pixel values ​​of all pixels; Based on the filtered green pixel value and the pixel value of the designated color of the designated pixel point, interpolating the pixel values ​​of the remaining colors of each pixel point to obtain the final red pixel value, green pixel value and blue pixel value of each pixel point; Interpolating a specified pixel point in the image to be processed to obtain a green pixel value of the specified pixel point includes: Calculating a first color difference in a first direction and a second color difference in a second direction of a pixel value of each pixel point in the image to be processed; Based on the first color difference and the second color difference, evaluating a target difference value of the designated pixel point under the influence of pixel value changes in the first direction to obtain a first directional difference value; evaluating the target difference value of the designated pixel point under the influence of pixel value changes in the second direction to obtain a second directional difference value; and evaluating the target difference value of the designated pixel point under the influence of surrounding pixel value changes to obtain a non-directional difference value; the target difference value represents the difference between the pixel value of the designated color and the green pixel value of the designated pixel point; Determining difference fusion reference information of the designated pixel based on the first color difference and the second color difference, where the difference fusion reference information represents the influence of the first directional difference, the second directional difference, and the non-directional difference on a fused target difference; Based on the fusion reference information of the designated pixel point, the first directional difference, the second directional difference and the non-directional difference are fused to obtain a fused target difference value of the designated pixel point; Based on the fused target difference, a green pixel value of the designated pixel is determined.

2. The Bayer image processing method according to claim 1, wherein: Calculating a first color difference in a first direction and a second color difference in a second direction of a pixel value at a position of a specified pixel point in the image to be processed, comprising: The first color difference and the second color difference are calculated based on the following formula: DH i,j =C i,j -(G i,j-1 +G i,j+1 ) / 2; DV i,j =C i,j -(G i-1,j +G i+1,j ) / 2; DH i,j-1 =G i,j-1 -(C i,j-2 +C i,j ) / 2; DV i-1,j =G i-1,j -(C i-2,j +C i,j ) / 2; in: DH i,j Characterizing the first color difference at position (i, j) in the image to be processed; DV i,j Characterizing the second color difference at position (i, j) in the image to be processed; C i,j A pixel value representing a specified color of a specified pixel at position (i, j) in the image to be processed; C i,j-2 Represents the pixel value of the specified color at the position (i, j-2) in the image to be processed; C i-2,j Represents the pixel value of the specified color at the position (i-2, j) in the image to be processed; G i,j-1 represents the green pixel value at position (i, j-1) in the image to be processed; G i,j+1 represents the green pixel value at position (i, j+1) in the image to be processed; G i-1,j represents the green pixel value at position (i-1, j) in the image to be processed; G i+1,j It represents the green pixel value at position (i+1, j) in the image to be processed.

3. The Bayer image processing method according to claim 1, wherein: Calculating the first directional difference value of the designated pixel point, the second directional difference value of the designated pixel point, and the non-directional difference value of the designated pixel point, including: Based on the first color difference and the second color difference, calculating a statistical value of the first color difference of the designated pixel and one or more pixel points along the east direction to obtain an east direction difference; wherein the east direction refers to the positive direction of the first direction; Based on the first color difference and the second color difference, calculating a statistical value of the first color difference of the designated pixel and one or more pixel points along the west direction to obtain a westward difference value; wherein the westward direction refers to the negative direction of the first direction; Based on the first color difference and the second color difference, calculating a statistical value of the first color difference of the designated pixel and one or more pixel points along the north direction to obtain a north direction difference; wherein the north direction refers to the positive direction of the second direction; Based on the first color difference and the second color difference, calculating a statistical value of the first color difference of the designated pixel and one or more pixel points along the south direction to obtain a south direction difference; wherein the south direction refers to the negative direction of the second direction; Determining eastward fusion reference information, westward fusion reference information, southward fusion reference information, and northward fusion reference information based on the first color difference and the second color difference; Based on the eastward fusion reference information and the westward fusion reference information, performing a weighted summation on the eastward difference and the westward difference to obtain the first directional difference; Based on the southbound fusion reference information and the northbound fusion reference information, performing a weighted summation on the southbound difference and the northbound difference to obtain the second directional difference; Based on the eastward fusion reference information, the westward fusion reference information, the southward fusion reference information, and the northward fusion reference information, a weighted sum is performed on the eastward difference, the westward difference, the southward difference, and the northward difference to obtain the directionless difference.

4. The Bayer image processing method according to claim 3, wherein: The easting difference, the westing difference, the southing difference, and the northing difference are calculated based on the following formula: deltaE i,j =(DH i,j+2 +DH i,j+1 +DH i,j ) / 3; deltaW i,j =(DH i,j-2 +DH i,j-1 +DH i,j ) / 3; deltaS i,j =(DV i+2,j +DV i+1,j +DV i,j ) / 3; deltaN i,j =(DV i-2,j +DV i-1,j +DV i,j ) / 3; in: deltaE i,j Characterizes the easting difference of the position (i, j) in the image to be processed; deltaW i,j Characterizes the westward difference of the position (i, j) in the image to be processed; deltaS i,j Characterizes the south difference at position (i, j) in the image to be processed; deltaN i,j Characterizes the north difference of the position (i, j) in the image to be processed; DH i,j+2 Characterizes the first color difference at position (i, j+2) in the image to be processed; DH i,j+1 Characterizes the first color difference at position (i, j+1) in the image to be processed; DH i,j Characterizing the first color difference at position (i, j) in the image to be processed; DH i,j-2 Characterizes the first color difference at position (i, j-2) in the image to be processed; DH i,j-1 Characterizes the first color difference at position (i, j-1) in the image to be processed; DV i+2,j Characterizes the second color difference at position (i+2, j) in the image to be processed; DV i+1,j Characterizes the second color difference at position (i+1, j) in the image to be processed; DV i,j Characterizing the second color difference at position (i, j) in the image to be processed; DV i-2,j Characterizing the second color difference at position (i-2, j) in the image to be processed; DV i-1,j Characterizes the second color difference at position (i-1, j) in the image to be processed; The eastbound fusion reference information, the westbound fusion reference information, the southbound fusion reference information, and the northbound fusion reference information are calculated based on the following formula: wE i,j =|DH i,j |+|DH i,j+2 -DH i,j+1 |+α1×|DH i,j -DH i,j+1 |+α2×|DH i-1,j -DH i-1,j+1 |+α3×|DH i+1,j -DH i+1,j+1 | wW i,j =|DH i,j |+|DH i,j-2 -DH i,j-1 |+α1×|DH i,j -DH i,j-1 |+α2×|DH i-1,j -DH i-1,j-1 |+α3×|DH i+1,j -DH i+1,j-1 | wS i,j =|DV i,j |+|DV i+2,j -DV i+1,j |+α1×|DV i,j -DV i+1,j |+α2×|DV i,j-1 -DV i+1,j-1 |+α3×|DV i,j+1 -DV i+1,j+1 | wN i,j =|DV i,j |+|DV i-2,j -DV i-1,j |+α1×|DV i,j -DV i-1,j |+α2×|DV i,j-1 -DV i-1,j-1 |+α3×|DV i,j+1 -DH i-1,j+1 | in: wE i,j Representing the eastward fusion reference information of the position (i, j) in the image to be processed; wW i,j Characterizing the westward fusion reference information of the position (i, j) in the image to be processed; wxya i,j Representing the southbound fusion reference information of the position (i, j) in the image to be processed; wN i,j Characterizing the north fusion reference information of the position (i, j) in the image to be processed; DH i,j Characterizing the first color difference at position (i, j) in the image to be processed; DV i,j Characterizing the second color difference at position (i, j) in the image to be processed; DH i,j+2 Characterizes the first color difference at position (i, j+2) in the image to be processed; DH i,j+1 Characterizes the first color difference at position (i, j+1) in the image to be processed; DH i,j Characterizing the first color difference at position (i, j) in the image to be processed; DH i,j-1 Characterizes the first color difference at position (i, j-1) in the image to be processed; DH i,j-2 Characterizes the first color difference at position (i, j-2) in the image to be processed; DH i+1,j Characterizes the first color difference at position (i+1, j) in the image to be processed; DH i+1,j+1 Characterizes the first color difference at the position (i+1, j+1) in the image to be processed; DH i+1,j-1 Characterizes the first color difference at position (i+1, j-1) in the image to be processed; DH i-1,j Characterizes the first color difference at position (i-1, j) in the image to be processed; DH i-1,j+1 Characterizes the first color difference at position (i-1, j+1) in the image to be processed; DH i-1,j-1 Characterizes the first color difference at the position (i-1, j-1) in the image to be processed; DV i+2,j Characterizes the second color difference at position (i+2, j) in the image to be processed; DV i+1,j Characterizes the second color difference at position (i+1, j) in the image to be processed; DV i,j Characterizing the second color difference at position (i, j) in the image to be processed; DV i-1,j Characterizes the second color difference at position (i-1, j) in the image to be processed; DV i-2,j Characterizing the second color difference at position (i-2, j) in the image to be processed; DV i+1,j-1 Characterizes the second color difference at the position (i+1, j-1) in the image to be processed; DV i,j-1 Characterizes the second color difference at the position (i, j-1) in the image to be processed; DV i-1,j-1 Characterizes the second color difference at the position (i-1, j-1) in the image to be processed; DV i+1,j+1 Characterizes the second color difference at the position (i+1, j+1) in the image to be processed; DV i,j+1 Characterizes the second color difference at position (i, j+1) in the image to be processed; DV i-1,j+1 Characterizes the second color difference at position (i-1, j+1) in the image to be processed; a1, a2, and a3 are preset coefficients used to calculate the fusion reference information. And 0≤a1, a2, a3≤1; The first directivity difference is calculated based on the following formula: in: DeltaH i,j Characterizing the first directional difference at position (i, j) in the image to be processed; deltaX i,j It represents the X-direction difference of the position (i, j) in the image to be processed, where the X-direction can be east or west. wxya i,j Represents the X-direction fusion reference information of the position (i, j) in the image to be processed, where the X-direction can be east or west; The second directivity difference is calculated based on the following formula: in: DeltaV i,j Characterizing the second directional difference at position (i, j) in the image to be processed; deltaX i,j Represents the X-direction difference of the position (i, j) in the image to be processed, where the X-direction can be south or north; wxya i,j Represents the X-direction fusion reference information of the position (i, j) in the image to be processed, where the X-direction can be south or north; The non-directional difference is calculated based on the following formula: in: DeltaHV i,j Characterizing the directionless difference at position (i, j) in the image to be processed; deltaX i,j It represents the X-direction difference of the position (i, j) in the image to be processed, where the X-direction can be east, west, south and north; wxya i,j The X-direction fusion reference information of the position (i, j) in the image to be processed is represented, where the X-direction can be east, west, south and north.

5. The Bayer image processing method according to claim 1, wherein: Determining difference fusion reference information of the designated pixel point based on the first color difference and the second color difference includes: Calculating a difference in first color difference between two adjacent pixels in a first direction of the designated pixel to obtain a first gradient, and calculating a difference in second color difference between two adjacent pixels in a second direction of the designated pixel to obtain a second gradient; Determining a first gradient upper threshold, a first gradient lower threshold, a second gradient upper threshold, and a second gradient lower threshold based on the first gradient and the second gradient; First fusion reference information and second fusion reference information are determined based on the first gradient threshold upper limit, the first gradient threshold lower limit, the second gradient threshold upper limit, and the second gradient threshold lower limit.

6. The Bayer image processing method according to claim 5, characterized in that: The first gradient and the second gradient are calculated based on the following formula: diffH i,j =|DH i,j-1 -DH i,j+1 | diffV i,j =|DV i-1,j -DV i+1,j | in: diffH i,j Characterizing the first gradient at position (i, j) in the image to be processed; diffV i,j Characterizing the second gradient at position (i, j) in the image to be processed; DH i,j-1 Characterizes the first color difference at position (i, j-1) in the image to be processed; DH i,j+1 Characterizes the first color difference at position (i, j+1) in the image to be processed; DV i-1,j Characterizes the second color difference at position (i-1, j) in the image to be processed; DV i+1,j Characterizes the second color difference at position (i+1, j) in the image to be processed; The first gradient threshold upper limit, the first gradient threshold lower limit, the second gradient threshold upper limit, and the second gradient threshold lower limit are calculated based on the following formula: thL1=diffV i,j ×coef1;thH1=diffV i,j ×coef2 thL2=diffV i,j ×coef3;thH2=diffV i,j ×coef4 in, thL1 represents the lower limit of the first gradient threshold in the image to be processed; thH1 represents the upper limit of the first gradient threshold in the image to be processed; thL2 represents the lower limit of the second gradient threshold in the image to be processed; thH2 represents the upper limit of the second gradient threshold in the image to be processed; coef1 and coef2 represent the evaluation coefficients of the first gradient threshold in the image to be processed; coef3 and coef4 represent the evaluation coefficients of the second gradient threshold in the image to be processed; Among them, coef1 <coef2,coef3<coef4, coef1≥1,coef2≥1,0≤coef3≤1,0≤coef4≤1; The first fusion reference information and the second fusion reference information are calculated based on the following formula: in: alphaH i,j Characterizing the first fusion reference information in the image to be processed; alphaV i,j Characterizing the second fused reference information in the image to be processed; diffH i,j Characterizing the first gradient at position (i, j) in the image to be processed; diffV i,j Characterizing the second gradient at position (i, j) in the image to be processed; thL1 represents the lower limit of the first gradient threshold in the image to be processed; thH1 represents the upper limit of the first gradient threshold of the designated pixel point in the image to be processed; thL2 represents the lower limit of the second gradient threshold in the image to be processed; thH2 represents the upper limit of the second gradient threshold in the image to be processed.

7. The Bayer image processing method according to claim 6, characterized in that: The method further comprises: fusing the first directional difference, the second directional difference, and the non-directional difference based on the fusion reference information of the designated pixel to obtain a fused target difference of the designated pixel. The method further comprises: The target difference after fusion of the specified pixel point is calculated based on the following formula: in: Delta i,j Characterizes the fused target difference at position (i, j) in the image to be processed; DeltaH i,j Characterizing the first directional difference at position (i, j) in the image to be processed; DeltaV i,j Characterizing the second directional difference at position (i, j) in the image to be processed; DeltaHV i,j Characterizing the directionless difference at position (i, j) in the image to be processed; alphaH i,j The first fusion reference information representing the position (i, j) in the image to be processed; alphaV i,j The second fusion reference information representing the position (i, j) in the image to be processed; diffH i,j Characterizing the first gradient at position (i, j) in the image to be processed; diffV i,j Characterizing the second gradient at position (i, j) in the image to be processed; thL1 represents the lower limit of the first gradient threshold in the image to be processed; thH1 represents the upper limit of the first gradient threshold in the image to be processed; thL2 represents the lower limit of the second gradient threshold in the image to be processed; thH2 represents the upper limit of the second gradient threshold in the image to be processed.

8. The Bayer image processing method according to claim 7, characterized in that: Determining the green pixel value of the designated pixel point based on the fused target difference includes: Calculating the sum of the fused target difference value of the designated pixel point and the original pixel value of the designated pixel point to obtain a green pixel value of the designated pixel point; G i,j =C i,j +Delta i,j ; in: G i,j Represents the green pixel value of the specified pixel at position (i, j) in the image to be processed; C i,j A pixel value representing a specified color of the specified pixel at position (i, j) in the image to be processed; Delta i,j The fused target difference value of the designated pixel point at the (i, j) position in the image to be processed is represented.

9. The Bayer image processing method according to claim 1, wherein: Filter the green pixel values ​​of all pixels, including: Determining a green pixel correction value for each pixel point based on the green pixel value and the original pixel value at the location of each pixel point in the image to be processed; Determining filtering fusion reference information for each pixel point based on the original pixel value at the position of each pixel point in the image to be processed; the filtering fusion reference information represents the influence of the green pixel value and the green pixel correction value of the corresponding pixel point on the fused green pixel value; Based on the filtering fusion reference information, the green pixel value and the green pixel correction value are fused to obtain a fused green pixel value of the corresponding pixel point, and the fused green pixel value is used as the filtered green pixel value.

10. The Bayer image processing method according to claim 9, characterized in that: Determining a green pixel correction value for each pixel point based on the green pixel value and the original pixel value at the position of each pixel point in the image to be processed includes: Based on the original pixel value of each pixel point in the image to be processed, the mean of the corresponding pixel points under the influence of the change of the green pixel value in the first direction is evaluated to obtain a first directional mean value; and the mean of the specified pixel points under the influence of the change of the green pixel value in the second direction is evaluated to obtain a second directional mean value; Determining first directional filtering fusion reference information of the first direction and second directional filtering fusion reference information of the second direction based on the original pixel value of each pixel point in the image to be processed; Determining first directional filtering reference information based on first filtering fusion reference information of one or more pixel points; Determining second directional filtering reference information based on second filtering fusion reference information of one or more pixels; A green pixel correction value for each pixel is determined based on the first directional mean value, the second directional mean value, the first directional filtering reference information, and the second directional filtering reference information.

11. The Bayer image processing method according to claim 10, characterized in that: The first directivity mean and the second directivity mean are calculated based on the following formula: in: GH i,j Characterizing the first directional mean value of the (i, j) position in the image to be processed; GV i,j Characterizing the second directional mean value of the (i, j) position in the image to be processed; G i,j-1 represents the green pixel value at position (i, j-1) in the image to be processed; G i,j represents the green pixel value at position (i, j) in the image to be processed; G i,j+1 represents the green pixel value at position (i, j+1) in the image to be processed; G i-1,j represents the green pixel value at position (i-1, j) in the image to be processed; G i+1,j represents the green pixel value at position (i+1, j) in the image to be processed; The first directional filtering fusion reference information and the second directional filtering fusion reference information are calculated based on the following formula: WH i,j The first directional filtering fusion reference information representing the position (i, j) in the image to be processed; WV i,j The second directional filtering fusion reference information representing the position (i, j) in the image to be processed; I i,j represents the original pixel value at position (i, j) in the image to be processed; I i-1,j-1 represents the original pixel value at position (i-1, j-1) in the image to be processed; I i-1,j Represents the original pixel value at position (i-1, j) in the image to be processed; I i-1,j+1 Represents the original pixel value at position (i-1, j+1) in the image to be processed; I i,j-1 Represents the original pixel value at position (i, j-1) in the image to be processed; I i,j+1 Represents the original pixel value at position (i, j+1) in the image to be processed; I i+1,j-1 represents the original pixel value at position (i+1, j-1) in the image to be processed; I i+1,j represents the original pixel value at position (i+1, j) in the image to be processed; I i+1,j+1 represents the original pixel value at position (i+1, j+1) in the image to be processed; The first directional filter reference information and the second directional filter reference information are calculated based on the following formula: in: The first directional filtering reference information representing the position (i, j) in the image to be processed; The second directional filtering reference information representing the position (i, j) in the image to be processed; r1 is the radius, which represents the calculation the number of pixels in the positive direction of the first direction, the negative direction of the first direction, the positive direction of the second direction, and the negative direction of the second direction selected when ; N represents the total number of pixels within the radius r; WH m,n The first direction filtering reference information representing the pixel points within the radius r1 in the image to be processed, wherein: -r1≤m≤r1, -r1≤n≤r1; WV m,n The second direction filtering reference information representing the pixel points within the radius r in the image to be processed, wherein: -r1≤m≤r1, -r1≤n≤r1; The green pixel correction value of the specified pixel is calculated based on the following formula: in: Gest i,j represents the green pixel correction value at position (i, j) in the image to be processed; The first directional filtering reference information representing the position (i, j) in the image to be processed; The second directional filtering reference information representing the position (i, j) in the image to be processed; GH i,j Characterizing the first directional mean value of the (i, j) position in the image to be processed; GV i,j The second directional mean value of the position (i, j) in the image to be processed is characterized.

12. The Bayer image processing method according to claim 11, characterized in that: Determining the filtering fusion reference information of the designated pixel point based on the green pixel correction value of the position of each pixel point in the image to be processed includes: Calculating the mean and variance of the green pixel correction values; Based on the mean and the variance, filtering fusion reference information of the designated pixel point is determined.

13. The Bayer image processing method according to claim 12, characterized in that: The mean and variance are calculated based on the following formula: in: represents the mean value of the green pixel correction value of the designated pixel at position (i, j) in the image to be processed; r1 is the radius, which represents the calculation the number of pixels in the positive direction of the first direction, the negative direction of the first direction, the positive direction of the second direction, and the negative direction of the second direction selected when ; N represents the total number of pixels within the radius r1; Gest m,n represents the green pixel correction value of the pixel points within the radius r1 in the image to be processed, wherein: -r1≤m≤r1, -r1≤n≤r1; var i,j Characterizes the variance of the green pixel correction value at position (i, j) in the image to be processed; The filtering fusion reference information is calculated based on the following formula: in: var i,j Characterizes the variance of the green pixel correction value at position (i, j) in the image to be processed; var n Characterizes the noise variance of the (i, j) position in the image to be processed, and the noise variance can be calculated based on the brightness or gradient of the image to be processed; alpha represents the filtering fusion reference information of the specified pixel point at position (i, j) in the image to be processed; The filtered green pixel value of the specified pixel is calculated based on the following formula: in: G' i,j represents the filtered green pixel value at position (i, j) in the image to be processed; G i,j represents the green pixel value at position (i, j) in the image to be processed; alpha represents the filtering fusion reference information of the (i, j) position in the image to be processed; It represents the mean of the green pixel correction values ​​at the (i, j) position in the image to be processed.

14. A Bayer image processing device, characterized in that: The device comprises: An acquisition module, used for acquiring an image to be processed; A green interpolation module is used to interpolate designated pixel points in the image to be processed to obtain green pixel values ​​of the designated pixel points, where the pixel values ​​of the designated pixel points are determined, wherein some designated pixel points are red and some designated pixel points are blue; The green interpolation module includes a color difference calculation submodule, a directional difference calculation submodule, a difference fusion reference information calculation submodule, a target difference calculation submodule and a green pixel fusion submodule; a color difference calculation submodule, configured to calculate a first color difference in a first direction and a second color difference in a second direction of the pixel value of each pixel point in the image to be processed; the pixel value of the designated color of the designated pixel point is determined, wherein some designated pixel points are red and some designated pixel points are blue; a directional difference calculation submodule for evaluating, based on the first color difference and the second color difference, a target difference for a specified pixel point under the influence of a change in pixel value in the first direction to obtain a first directional difference; evaluating the target difference for the specified pixel point under the influence of a change in pixel value in the second direction to obtain a second directional difference; and evaluating the target difference for the specified pixel point under the influence of a change in surrounding pixel values ​​to obtain a non-directional difference; the target difference representing the difference between an original pixel value and a green pixel value of the specified pixel point; a difference fusion reference information calculation submodule, configured to determine difference fusion reference information of the specified pixel point based on the first color difference and the second color difference, wherein the difference fusion reference information represents the influence of the first directional difference, the second directional difference, and the non-directional difference on a fused target difference; a target difference calculation submodule, which fuses the first directional difference, the second directional difference and the non-directional difference based on the fusion reference information of the specified pixel point to obtain a fused target difference value of the specified pixel point; A green pixel fusion submodule, which determines the green pixel value of the designated pixel point based on the fused target difference; A filtering module is used to filter the green pixel values ​​of all pixels; The red and blue interpolation module is used to interpolate the pixel values ​​of the remaining colors of each pixel point based on the filtered green pixel value and the pixel value of the specified pixel point to obtain the final red pixel value, green pixel value and blue pixel value of each pixel point.

15. An electronic device, characterized in that: Including memory and processor, The memory is used to store code; The processor is configured to execute the code in the memory to implement the method according to any one of claims 1 to 13.

16. A storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 13 is implemented.

Images