Bayer image denoising method

Abstract

The invention belongs to the technical field of image processing, and particularly relates to a Bayer image de-noising method, which comprises the following steps of: acquiring an A group of shooting gray scale card Bayer images of different ISO (International Standards Organization) through a sensor; performing noise calibration on the Bayer images of different ISOs in the A group to obtain a noise calibration parameter group; acquiring an input Bayer image as a first image, and acquiring exposure information of the first image; processing the first image exposure information and the noise calibration parameter group to obtain a noise model of the first image; denoising the first image according to the noise model of the first image to obtain a result image; according to the method, the noise parameter is calibrated, the noise model is generated by inputting the Bayer image exposure parameter, whether the area is detail is judged according to the noise model, and the non-detail area is denoised, so that the edge information of the image is effectively improved, the noise of the non-edge area is reduced, and an ideal Bayer image denoising effect is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a Bayer image denoising method. Background Art

[0002] With the rapid development of mobile terminal image sensor technology, as the mainstream color filter array solution, the denoising quality of the raw data of the Bayer pattern directly affects the final imaging effect of the ISP (Image Signal Processing) pipeline. However, traditional Bayer image denoising methods face the following technical bottlenecks:

[0003] (1) Noise diffusion and detail loss. Gaussian filtering or non-local means (NLM) algorithms with fixed intensity are difficult to adapt to the spatial characteristics of the image content, resulting in over-smoothing of high-frequency details (such as textures and edges) or residual low-frequency noise.

[0004] (2) Algorithm efficiency and resource consumption. Existing Bayer domain denoising algorithms, such as non-local means and wavelet transform, can partially suppress noise, but have high computational complexity and are difficult to meet the real-time requirements of embedded devices.

[0005] In summary, the existing technology urgently needs a Bayer image denoising method that takes into account detail preservation, noise suppression, and computational efficiency, especially needs to break through the technical barrier between the discrimination accuracy of noise details and the adaptability of embedded hardware. Summary of the Invention

[0006] The present invention provides a Bayer image denoising method to solve problems such as suppressing noise while retaining detail features. The specific technical solutions are as follows:

[0007] The present invention provides a Bayer image denoising method, based on the original Bayer image, comprising the following steps:

[0008] S1. Collect Bayer images of a set of gray scale cards with different ISOs through a sensor;

[0009] S2. Perform noise calibration on the set of Bayer images with different ISOs to obtain a set of noise calibration parameters;

[0010] S3. Collect an input Bayer image as the first image, and obtain the exposure information of the first image;

[0011] S4. Process the exposure information of the first image and the set of noise calibration parameters to obtain the noise model of the first image;

[0012] S5. Denoise the first image according to the noise model of the first image to obtain a result image.

[0013] Further, the step S1 further includes:

[0014] S11. Place the grayscale card in the shooting scene, ensure uniform light source illumination, fix the scene light source and the aperture of the imaging device;

[0015] S12. Set Group A ISO through the sensor, denoted as the ISO sequence {iso1, iso2... iso A};

[0016] S13. Adjust the exposure time of the sensor so that, under the ISO sequence, the average brightness value of the white color block in the Bayer image of the grayscale card collected is 0.8 times the saturation level; where the depth of the Bayer image is N, the saturation level value is 2 N -1;

[0017] S14. Save the Bayer image sequence under the ISO sequence, denoted

[0018] Furthermore, step S2 also includes:

[0019] S21. Obtain as the input calibrated Bayer image;

[0020] S22. Perform channel separation processing on the input calibrated Bayer image to obtain Bayer C , where C are the four color channels R, Gr, Gb, B of the Bayer image;

[0021] S23. Extract the regional coordinates of the color blocks, and denote the number of color blocks as Q;

[0022] S24. Calculate the brightness mean matrix and variance mean matrix for the Q color block regions; where the brightness mean matrix {μ1 C , μ2 C ... μ Q C}, variance matrix {Var1 C , Var2 C ,... Var Q C}, where C are the four color channels R, Gr, Gb, B of the Bayer image, that is, μ1 Gr represents the brightness mean of the first color block in the Gr channel;

[0023] S25. Solve through non-linear least squares to obtain the gain parameter a C and the offset parameter b C ; The formula is:

[0024]

[0025] where, where C are the four color channels R, Gr, Gb, B of the Bayer image, i CDenote the average brightness of the \(i\)-th color patch in the C color channel as \(\mu\), and denote the variance as Var i C Denote the variance of the brightness of the \(i\)-th color patch in the C color channel; Solve to obtain the brightness value \(\mu\) of each color channel C and the variance Var C The corresponding relationship is as follows:

[0026] Var C (Pull C ) = a C *\(\mu\) C +b C

[0027] where a C is the calibration gain parameter of the C color channel for the currently solved exposure parameter ISO, and b C is the calibration offset parameter of the C color channel for the currently solved exposure parameter ISO;

[0028] S26. Obtain the Bayer image sequence of step S14 as the input image, and repeat steps S22 - S25 to obtain a group of noise calibration parameter groups under A ISOs, that is, obtain:

[0029]

[0030] where iso k is the \(k\)-th ISO value collected, \(k\leq A\), is the gain parameter of the \(k\)-th ISO of the C color channel, is the offset parameter of the \(k\)-th ISO of the C color channel.

[0031] Furthermore, step S3 further includes:

[0032] Collect the input Bayer image as the first image, denoted as I1;

[0033] Obtain the exposure information of the first image, denoted as iso current .

[0034] Furthermore, step S4 further includes:

[0035] Obtain that the exposure information iso current of the first image is in the interval of the A-group data of the calibrated ISO, that is, it is necessary to satisfy iso current ≥iso k and iso current ≤iso k+1 , \(k\leq A - 1\); where k represents the \(k\)-th calibrated ISO;

[0036] Calculate the calibration gain parameter of the exposure parameter iso current of and the calibration offset parameter The formula is as follows:

[0037]

[0038] Among them, is the calibration gain parameter under the C color channel, is the calibration offset parameter under the C color channel;

[0039] Obtain the first image exposure parameter iso current The noise model of is given by the formula:

[0040]

[0041] Among them, μ C is the brightness value under the C color channel, is the exposure parameter iso current , and the brightness is μ C The variance of the value.

[0042] Furthermore, step S5 further includes:

[0043] S51. Obtain the first image, denoted as I1, with the image width being W, the image height being H, and the image depth being N;

[0044] S52. Perform color channel separation processing on the first image to obtain the first image color separation map, denoted as where C are the four color channels R, Gr, Gb, and B of the Bayer image;

[0045] S53. Traverse the first image color separation map with a first preset window, and perform denoising processing to obtain the denoised color separation map

[0046] S54. Perform channel merging processing on the denoised color separation map to obtain the resulting Bayer image.

[0047] Furthermore, step S53 further includes:

[0048] S531. Traverse the first image color separation map with the radius of the first preset window Perform Gaussian filtering on each window pixel matrix. The Gaussian kernel formula is as follows:

[0049]

[0050] Among them, u and v are the window coordinate values, H[u, v] is the Gaussian kernel, σ is the standard deviation of the Gaussian kernel, which controls the window denoising intensity, r is the radius of the first preset window, is the brightness value at the position (i, j) in the C channel, is the value after Gaussian filtering at the position (i, j) in the C channel;

[0051] S532. Traverse the first image separation color map with the first preset window radius Obtain the local mean and local variance information of each window pixel matrix. The formula is as follows:

[0052]

[0053] where is the brightness value at the position (i, j) in the C channel, is the mean of the pixel values within the window in the C channel, u and v are the window coordinate values, r is the first preset window radius, and Var ij C is the local variance;

[0054] S533. Calculate the noise model at the current brightness value to obtain the noise variance range at the current brightness. The formula is as follows:

[0055]

[0056] S534. Determine whether it is a detail area and denoise the non-detail area. The formula is:

[0057]

[0058] where It is considered that the pixel value is within the noise fluctuation range, which is a flat area. The output pixel value is the Gaussian denoising result, and the denoising intensity can be controlled by the standard deviation of the Gaussian kernel in step S531; It indicates that this area is a detail area, and the original pixel value is retained, thus retaining the image details.

[0059] The present invention provides a Bayer image denoising method. By collecting the Bayer image of a gray scale card through a sensor for noise calibration, a noise calibration parameter group is obtained; a noise model is obtained according to the exposure parameters of the input Bayer image, and it is judged whether the image area is a detail based on the noise model; then targeted denoising is carried out; this method solves the problem of detail blurring caused by conventional denoising, judges whether the area is a detail according to the noise model, denoises the non-detail area, effectively improves the edge information of the image, reduces the noise in the non-edge area, and has a low computational complexity, meeting the real-time requirements of embedded devices and achieving an ideal Bayer image denoising effect. Description of the Drawings

[0060] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments of the present application or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments recorded in the present application. For those of ordinary skill in the art, other accompanying drawings can also be obtained based on these accompanying drawings of the embodiments of the present application.

[0061] Figure 1 It is a schematic flowchart of a Bayer image denoising method in an implementation manner of the present application. Specific implementation manner

[0062] In order to make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions of the embodiments of the present invention with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the described embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0063] Unless otherwise defined, the technical terms or scientific terms used in the present invention shall have the ordinary meanings understood by those of ordinary skill in the art to which the present invention pertains. The "first", "second", and similar terms used in the present invention do not denote any order, quantity, or importance, but are only used to distinguish different components. The terms such as "comprising" or "including" mean that the elements or items appearing before this word cover the elements or items listed after this word and their equivalents, without excluding other elements or items. The terms such as "connected" or "coupled" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The terms such as "upper", "lower", "left", "right", etc. are only used to represent relative positional relationships, and when the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0064] To keep the following description of the embodiments of the present invention clear and concise, the detailed descriptions of some known functions and known components are omitted in the present invention.

[0065] To ensure that the denoising of Bayer images takes into account detail preservation, noise suppression, and computational efficiency. This example provides an image brightening method.

[0066] Referring to Figure 1 , this example designs a Bayer image denoising method, which is applied to an endoscopic system. Based on the endoscopic image of Bayer, this method specifically includes the following steps:

[0067] S1. Collect Bayer images of gray scale cards with different ISOs in Group A through sensors;

[0068] S2. Perform noise calibration on the Bayer images of different ISOs in Group A to obtain a set of noise calibration parameters;

[0069] S3. Collect the input Bayer image as the first image and obtain the exposure information of the first image;

[0070] S4. Process the exposure information of the first image and the set of noise calibration parameters to obtain the noise model of the first image;

[0071] S5. Denoise the first image according to the noise model of the first image to obtain the result image.

[0072] In step S1 of this embodiment, the sensor is an endoscope camera module, and its function is to collect raw Bayer image data. The X-Rite ColorChecker Classic standard 24-step calibration card is used as the shooting target. In a controlled optical environment with a fixed aperture value of f / 5.6 and a constant color temperature light source of 5500K for the endoscope camera module, calibration image acquisition is performed by the CMOS sensor according to the preset ISO sequence. Among them:

[0073] The ISO sequence starts from ISO 100 and increases by an arithmetic step of 200 to ISO 7900, forming a total of 40 groups of exposure parameters, that is, {iso1, iso2... iso 40}, iso1 = ISO_100, iso 40 = ISO_7900.

[0074] The shutter time is dynamically adjusted through the automatic exposure control module to ensure that the average brightness value of the white color block of the 24-step calibration card at each ISO level is 80% of the saturation level (corresponding to the brightness value 818 of 10-bit depth).

[0075] Store the calibration image set in the original Bayer format, with a data format of RGGB Bayer array, a resolution of 1920×1080, and a bit depth of 10 bits.

[0076] In step S2 of this embodiment:

[0077] S21. Obtain The input calibration Bayer image taken at ISO_100, with a Bayer format of RGGB.

[0078] S22. Perform channel separation processing on the input calibration Bayer image to obtain Bayer C, where C are the four color channels R, Gr, Gb, and B of the Bayer image. According to the RGGB Bayer array pattern, the original image is decomposed into four independent channel images: the red channel R, the green channels Gr / Gb, and the blue channel B. The pixels of the R channel are located at the odd rows and odd columns, the pixels of the Gr channel are located at the odd rows and even columns, the pixels of the Gb channel are located at the even rows and odd columns, and the pixels of the B channel are located at the even rows and even columns.

[0079] S23. Extract the regional coordinates of the color patches, and record the number of color patches as 24. For each color channel, extract the 50×50 pixel central region of 24 color patches as the region of interest (ROI).

[0080] S24. Calculate the brightness mean matrix and variance mean matrix of the color patches for the 24 regions of interest (ROIs); the brightness mean matrix {μ1 C , μ2 C ...μ Q C}, the variance matrix {Var1 C , Var2 C ,...Var 24 C}, where C are the four color channels R, Gr, Gb, and B of the Bayer image, that is, μ1 Gr represents the brightness mean of the first color patch in the Gr channel.

[0081] S25. Obtain the gain parameter a C and the offset parameter b C by solving through non-linear least squares, and the formula is:

[0082]

[0083] where, μ i C represents the brightness mean of the i-th color patch in the C color channel, and Var i C represents the brightness variance of the i-th color patch in the C color channel; the corresponding relationship between the brightness value μ C and the variance Var C of each color channel is obtained as follows:

[0084] Var C (μ C ) = a C * μ C + b C

[0085] where, a C is the calibration gain parameter of the C color channel for the currently obtained exposure parameter ISO, and b CThe calibration offset parameter of the C color channel for the currently obtained exposure parameter ISO.

[0086] S26. Obtain the Bayer image sequence in step S14 as the input image, and repeat steps S22 - S25 to obtain a set of noise calibration parameters for A groups of ISO, that is:

[0087]

[0088] where, iso k is the k-th ISO value collected, k ≤ 40, is the gain parameter of the k-th ISO for the C color channel, is the offset parameter of the k-th ISO for the C color channel.

[0089] In step S3, in this embodiment:

[0090] Collect the Bayer data of the real endoscope as the input Bayer image, denoted as the first image I1.

[0091] Obtain the exposure information of the first image, denoted as iso current , in this embodiment, the exposure information of the first image is ISO_200.

[0092] In step S4, in this embodiment:

[0093] Obtain the interval segment of 40 groups of data where the exposure information ISO 200 of the first image is at the calibrated ISO, that is, it is required to satisfy ISO_200 ≥ iso1 and ISO_200 ≤ iso2, k ≤ 40.

[0094] Calculate the calibrated gain parameter and the calibrated offset parameter The formula is as follows:

[0095]

[0096] where, is the calibrated gain parameter under the C color channel, is the calibrated offset parameter under the C color channel.

[0097] Obtain the noise model of the exposure parameter iso current of the first image, and the formula is:

[0098]

[0099] where, μ C is the brightness value under the C color channel, is the exposure parameter iso current , and the brightness is μC Variance of values.

[0100] In this embodiment, since the result after ISO 200 interpolation is:

[0101] R channel: The noise model is

[0102] Gr channel: The noise model is

[0103] Gb channel: The noise model is

[0104] B channel: The noise model is

[0105] where Y is the luminance value of the pixel.

[0106] In step S5, in this embodiment:

[0107] S51. Obtain the first image, denoted as I1, with the image width of 1920, the image height of 1080, and the image depth of 10 bits.

[0108] S52. Perform color channel separation processing on the first image to obtain the first color-separated image, denoted as where C are the four color channels R, Gr, Gb, and B of the Bayer image, and the sizes of the four color channels are 960x540.

[0109] S53. Traverse the first color-separated image with a 3x3 pixel window, and perform denoising processing to obtain the denoised color-separated image

[0110] Step 553 further includes:

[0111] S531. Traverse the first color-separated image with a 3x3 pixel window Perform Gaussian filtering on each window pixel matrix, and the Gaussian kernel formula is as follows:

[0112]

[0113] where u and v are the window coordinate values, H[u, v] is the Gaussian kernel, σ is the standard deviation of the Gaussian kernel, and in this embodiment, σ is 1.0 to control the window denoising intensity, and the first preset window radius is r = 1, is the luminance value at the position (i, j) in the C channel, is the value after Gaussian filtering at the position (i, j) in the C channel.

[0114] Traverse the first image separated color map with the first preset window radius Obtain the local mean and local variance information of each window pixel matrix. The formula is as follows:

[0115]

[0116] Where is the brightness value at position (i, j) in the C channel, is the mean of the pixel values within the window in the C channel, u and v are the window coordinate values, and the first preset window radius is r = 1, Var ij C is the local variance.

[0117] S533. Calculate the noise model at the current brightness value to obtain the noise variance range at the current brightness. The formula is as follows:

[0118]

[0119] S534. Determine whether it is a detail area and denoise the non-detail area. The formula is:

[0120]

[0121] Where It is considered that the pixel value is within the noise fluctuation range, which is a flat area, and the output pixel value is the result of Gaussian denoising. The denoising intensity can be controlled by the standard deviation of the Gaussian kernel in step S531; Then it indicates that this area is a detail area, and the original pixel value is retained, thus retaining the image details.

[0122] S54. Perform channel merging processing on the denoised color separation map to obtain the resulting Bayer image. In this embodiment, channel merging processing is performed on the denoised color separation map to obtain the resulting Bayer image, and the resulting Bayer image enters the subsequent ISP process for processing.

[0123] A Bayer image denoising method provided by an embodiment of the present invention is applied to an endoscopic system. Noise calibration is performed according to the gray scale card collected by the endoscopic system to form a noise calibration parameter group. The input first image is a Bayer image collected from the endoscopic working scene. A noise model is formed according to the exposure parameters, and it is determined whether it is a detail area based on the noise model for targeted denoising; the noise in the non-edge area is reduced, and the computational complexity is low, meeting the real-time requirements of endoscopic equipment, achieving an ideal Bayer image denoising effect, and can be applied to actual medical endoscopic projects.

[0124] The above are only the preferred embodiments of the present invention and are not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention are included within the protection scope of the present invention.

Claims

1. A Bayer image denoising method, characterized in that, It includes the following steps: S1. Collect Bayer images of a gray-scale card with different ISOs through a sensor; S2. Calibrate the noise of the Bayer images with different ISOs in group A to obtain a group of noise calibration parameters; S3. Collect an input Bayer image as the first image and obtain the exposure information of the first image; S4. Process the exposure information of the first image and the group of noise calibration parameters to obtain the noise model of the first image; S5. Denoise the first image according to the noise model of the first image to obtain a result image.

2. The Bayer image denoising method according to claim 1, wherein The step S1 further includes: S11. Place the gray-scale card in the shooting scene, ensure uniform light source illumination, fix the scene light source and the aperture of the imaging device; S12. Set group A ISO through the sensor, denoted as the ISO sequence {iso1, iso2... iso A}; S13. Adjust the exposure time of the sensor so that, under the ISO sequence, the average brightness value of the white color block in the Bayer image of the gray scale card collected is 0.8 times the saturation level; where the depth of the Bayer image is N, the saturation level value is 2 N -1; S14. Save the Bayer image sequence under the ISO sequence, denoted as 3. The Bayer image denoising method according to claim 2, wherein The step S2 further includes: S21. Obtain the input calibrated Bayer image; S22. Separating channels from the input calibrated Bayer image to obtain Bayer C , where C are the four color channels of R, Gr, Gb, and B of the Bayer image; S23. Extract the regional coordinates of the color blocks, and record the number of color blocks as Q; S24. Calculate the mean brightness matrix and the mean variance matrix of color patches for Q color patch regions; where the mean brightness matrix {μ1 C , μ2 c ... μ Q C}, and the variance matrix {Var1 C , Var2 C ,... Var Q C}, where C are the four color channels of the Bayer image, namely R, Gr, Gb, and B, i.e., μ1 Gr represents the mean brightness of the first color patch in the Gr channel; S25. The gain parameter a is obtained by solving through non - linear least - squares C and the offset parameter b C ; The formula is: Among them, C are the four color channels of R, Gr, Gb, and B of the Bayer image, and μ i C represents the average brightness of the i-th color block in the C color channel, and Var i C represents the brightness variance of the i-th color block in the C color channel; the average brightness value μ C and variance Var C corresponding relationship are as follows: Var C (Pull C ) = a C * Pull C + b C Among them, a C is the calibration gain parameter of the C color channel of the currently obtained exposure parameter ISO, and b C is the calibration offset parameter of the C color channel of the currently obtained exposure parameter ISO; S26. Obtain the Bayer image sequence in step S14 as the input image, and repeat steps S22 - S25 to obtain a group of noise calibration parameters under different ISOs in group A; that is where, iso k is the k-th ISO value collected, where k ≤ A, is the gain parameter of the k-th ISO for the C color channel, is the offset parameter of the k-th ISO for the C color channel.

4. A Bayer image denoising method according to claim 3, characterized in that, The step S3 further includes: Collect an input Bayer image as the first image, denoted as I1; Obtain the first image exposure information, denoted as iso current .

5. A Bayer image denoising method according to claim 4, characterized in that The step S4 further includes: Obtain the first image exposure information iso current In the interval of the A-group data at the calibrated ISO, that is, it is necessary to satisfy iso current ≥iso k And iso current ≤iso k+1 , k ≤ A - 1; where k represents the k-th calibrated ISO; Calculate the exposure parameter iso current Calibration gain parameter And calibration offset parameter The formula is as follows: Among them, is the calibration gain parameter under the C color channel, is the calibration offset parameter under the C color channel; Obtain the first image exposure parameter iso current The noise model of which is given by the formula: Among them, μ C is the brightness value under the C color channel, is the exposure parameter iso current , and the brightness is the variance of the μ C value.

6. A Bayer image denoising method according to claim 5, characterized in that The step S5 further includes: S51. Obtain the first image, denoted as I1, with the image width being W, the image height being H, and the image depth being N; S52. Perform color channel separation processing on the first image to obtain a first image color separation map, denoted as where C are the four color channels of R, Gr, Gb, and B of the Bayer image; Traverse the first image color separation map with a first preset window, and perform denoising processing to obtain a denoised color separation map S54. Merge the channels of the denoised color separation map to obtain a result Bayer image.

7. A Bayer image denoising method according to claim 6, characterized in that The step S53 further includes: Traverse the first image separation color map with the first preset window radius Perform Gaussian filtering on each window pixel matrix. The Gaussian kernel formula is as follows: Among them, u and v are window coordinate values, H[u, v] is the Gaussian kernel, σ is the standard deviation of the Gaussian kernel, which controls the denoising intensity of the window, and r is the first preset window radius. is the luminance value at position (i, j) in the C channel. is the value after Gaussian filtering at position (i, j) in the C channel. Traverse the first image separation color map with the first preset window radius Obtain the local mean and local variance information of each window pixel matrix, and the formula is as follows: Wherein, is the luminance value at position (i, j) in the C channel, is the mean of the pixel values within the window in the C channel, u and v are the window coordinate values, r is the first preset window radius, and Var ij C is the local variance; S533. Calculate the noise model at the current brightness value to obtain the noise variance range at the current brightness. The formula is as follows: S534. Determine whether it is a detail area and denoise the non-detail area. The formula is: Among them, If it is considered that the pixel value is within the noise fluctuation range, it is a flat area, and the output pixel value is the result of Gaussian denoising, and the denoising intensity can be controlled by the standard deviation of the Gaussian kernel in step S531; Then it indicates that this area is a detail area, and the original pixel value is retained, thereby retaining the image details.

Images