A Denoising Method for Low-Light Images

Abstract

The present invention discloses a noise reduction method for low-light images: obtaining a RAW domain noise reduction data set; obtaining a grayscale image of a low-light noise image, and judging whether the current low-light noise image is in a low-light scene according to the characteristics of the grayscale image; preprocessing the thermal noise points of the low-light noise image to obtain a low-light noise image after removing the thermal noise points; calculating the noise parameters of the low-light noise image after removing the thermal noise points, and removing the noise according to the noise parameters to obtain a denoised image; the noise parameters include shot noise parameters and readout noise parameters; inputting the low-light noise image after removing the thermal noise points, the shot noise parameters and the readout noise parameters into an encoder-decoder structure to obtain a denoised image after noise reduction; the beneficial effects achieved by the present invention are: effectively reducing the influence of noise on image restoration, enabling it to have better local texture restoration ability, retaining details, and having good anti-noise performance.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for reducing noise in low-light images. Background Art

[0002] Low-light image enhancement has received more and more extensive attention due to its practicality in cameras, and can effectively solve problems such as low brightness, low contrast, and serious loss of details in images obtained in low-light scenarios. At the same time, limited by the physical characteristics of sensors and the process level, after the camera completes photoelectric conversion under low-light conditions, a large amount of noise is contained in the original image data. If these noises cannot be effectively suppressed, it will seriously affect the quality of images under low light and subsequent processing and analysis.

[0003] Currently, for low-light image enhancement in low-light scenarios, noise reduction of images is mostly performed at a later stage in the ISP pipeline and processed on RGB or YUV. However, at the same time, noise will continuously generate, propagate, amplify, and change statistical characteristics in each module of the ISP pipeline, and the impact on image quality is increasing. The pressure of noise reduction is relatively high only on RGB or YUV. To solve this problem, a method of removing noise in the original RAW data is proposed to improve the signal-to-noise ratio of the data. Most of the existing technologies use traditional algorithms, such as non-local mean denoising, wavelet transform, etc. to denoise images in the RAW domain. Doing so can suppress noise, but the benefits brought are limited, and it is not possible to achieve a good balance between noise reduction and detail preservation. Removing noise forcibly under low light is likely to result in over-smoothing, affecting the overall quality of the image.

[0004] Using a neural network method to perform noise reduction in the RAW domain can achieve a higher-quality noise reduction effect, but there are also problems such as limited noise reduction effect and loss of sharpness, and the generality is poor. Therefore, it is necessary to make improvements on the existing basis to learn more complex image features and noise patterns to achieve a higher-quality noise reduction effect.

[0005] Aiming at the deficiencies of the existing technology, the present invention proposes a method for reducing noise in low-light images. Summary of the Invention

[0006] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide a method for reducing noise in low-light images with more ideal restoration of texture information and better noise reduction effect, realizing automatic restoration enhancement and noise reduction processing of 4K low-light images, and improving the visual effect and quality of low-light images.

[0007] The purpose of the present invention is achieved through the following technical solutions: A method for reducing noise in low-light images, including:

[0008] S1. Obtain a RAW domain noise reduction dataset; the RAW domain noise reduction dataset includes low-light noise images in RAW format and low-light noise-reduced images in RAW format;

[0009] S2. Obtain the grayscale image of the low-light noise image, and determine whether the current low-light noise image is in a low-light scene according to the grayscale image features;

[0010] If it is in a low-light scene, go to step S3;

[0011] If it is not in a low-light scene, end the noise reduction of the low-light image;

[0012] S3. Preprocess the thermal noise points of the low-light noise image to obtain a low-light noise image with thermal noise points removed;

[0013] S4. Calculate the noise parameters of the low-light noise image with thermal noise points removed;

[0014] S5. Input the low-light noise-reduced image in RAW format, the low-light noise image with thermal noise points removed, and the noise parameters into a symmetric encoder-decoder structure to obtain a noise-reduced image.

[0015] Further, the obtaining the grayscale image of the low-light noise image and determining whether the current low-light noise image is in a low-light scene according to the grayscale image includes:

[0016] S201. Downsample the low-light noise image to obtain a downsampled image, and perform grayscale calculation on the downsampled image to obtain a grayscale image;

[0017] S202. Obtain the grayscale value and grayscale histogram from the grayscale image, and calculate the cumulative distribution function;

[0018] S203. Obtain the value of m of the corresponding grayscale image when the value of the cumulative distribution function = 0.75;

[0019] When 0 < m ≤ the first preset value, the low-light noise image is in a low-light scene, and take the intensity parameter alpha for removing thermal noise points as 1;

[0020] When the first preset value < m < the second preset value, the low-light noise image is in a relatively low-light scene, and take 0 < alpha < 1;

[0021] When m ≥ the second preset value, the low-light noise image is in a normal bright light environment and not in a low-light scene, then alpha = 0, and end the noise reduction of the low-light image.

[0022] Further, the preprocessing the thermal noise points of the low-light noise image to obtain a low-light noise image with thermal noise points removed includes:

[0023] S301. Calculate the median of the gradient values in the horizontal, vertical, 45°, and 135° directions of the low-light noise image;

[0024] S302. Based on the minimum of the medians of the gradient values in the horizontal, vertical, 45°, and 135° directions, take the minimum value as the edge direction;

[0025] min_grad = min(median_Dh, median_Dv, median_D45, median_D135)

[0026] In the formula:

[0027] min_grad represents the edge direction;

[0028] median_Dh represents the minimum of the median in the horizontal direction;

[0029] median_Dv represents the minimum of the median in the vertical direction;

[0030] median_D45 represents the minimum of the median in the 45° direction;

[0031] median_D135 represents the minimum of the median in the 135° direction;

[0032] S303. Determine whether the central pixel Pc is a thermal noise point according to the edge direction;

[0033] If the central pixel point is not a thermal noise point, enter S4;

[0034] If the central pixel point is a thermal noise point, enter S304;

[0035] S304. Calculate the final correction result by the method of mixing with the intensity parameter alpha for removing thermal noise points;

[0036] S305. Obtain the low-light noise image after removing thermal noise points according to the correction result.

[0037] Furthermore, the calculation of the gradients in the horizontal, vertical, 45°, and 135° directions of the low-light noise image includes:

[0038] Obtain the central pixel Pc of the low-light noise image. P1, P2, P3, P4, P5, P6, P7, and P8 are the 8 pixel points adjacent to the central pixel Pc;

[0039] Calculate the three second-order gradients in the horizontal direction to obtain the absolute value of the horizontal direction gradient:

[0040] Dh1 = |P1 + P3 - 2 * P2|, Dh2 = |P4 + P5 - 2 * Pc|, Dh3 = |P6 + P8 - 2 * P7|;

[0041] Take the median of the absolute values of the horizontal gradients, and the calculation expression is as follows:

[0042] median_Dh = median(Dh1, Dh2, Dh3);

[0043] Calculate the three second-order gradients in the vertical direction to obtain the absolute value of the vertical gradient:

[0044] Dv1 = |P1 + P6 - 2 * P4|, Dv2 = |P2 + P7 - 2 * Pc|, Dv3 = |P3 + P8 - 2 * P5|;

[0045] Take the median of the absolute values of the vertical gradients, and the calculation expression is as follows:

[0046] median_Dv = median(Dv1, Dv2, Dv3);

[0047] The three second-order gradients in the 45° direction:

[0048] D45_1 = 2 * |P4 - P2|, D45_2 = |P3 + P6 - 2Pc|, D45_3 = 2 * |P7 - P5|;

[0049] Take the median of the absolute values of the vertical gradients, and the calculation expression is as follows:

[0050] median_D45 = median(D45_1, D45_2, D45_3);

[0051] The three second-order gradients in the 135° direction:

[0052] D135_1 = 2 * |P2 - P5|, D135_2 = |P1 + P8 - 2Pc|, D135_3 = 2 * |P7 - P4|;

[0053] Take the median of the absolute values of the vertical gradients, and the calculation expression is as follows:

[0054] median_D135 = median(D135_1, D135_2, D135_3).

[0055] Furthermore, determining whether the intermediate pixel Pc is a thermal noise point according to the edge direction includes;

[0056] If the edge direction is horizontal or vertical, determine whether the absolute value of the second-order gradient in the horizontal or vertical direction passing through the central pixel Pc is greater than 4 times the absolute values of the other two second-order gradients in the same direction;

[0057] When the edge direction is horizontal, if Dh2 > 4 * (Dh1 + Dh3), then Pc is a thermal noise point;

[0058] Otherwise, Pc is not a thermal noise point;

[0059] When the edge direction is vertical, if Dv2 > 4 * (Dv1 + Dv3), then Pc is a thermal noise point;

[0060] Otherwise, Pc is not a thermal noise point.

[0061] If the edge direction is 45°, calculate the sum of the absolute values of the differences between the absolute values of the second-order gradients in the 135° direction taken two by two. The formula is as follows:

[0062] D135_sum = |D135_1 - D135_2| + |D135_2 - D135_3| + |D135_3 - D135_1|;

[0063] If D135_sum < 400, and D45_2 > 3 * (D45_1 + D45_3) and D135_2 > 3 * (D135_1 + D135_3), then Pc is a thermal noise point;

[0064] If D135_sum >= 400, and D45_2 > 3 * (D45_1 + D45_3), then Pc is a thermal noise point at this time;

[0065] If the edge direction is 135°, calculate the sum of the absolute values of the differences between the absolute values of the second-order gradients in the 45° direction taken two by two. The formula is as follows:

[0066] D45_sum = |D45_1 - D45_2| + |D45_2 - D135_3| + |D45_3 - D45_1|;

[0067] If D45_sum < 400, and D45_2 > 3 * (D45_1 + D45_3) and D135_2 > 3 * (D135_1 + D135_3), then Pc is a thermal noise point;

[0068] If D45_sum >= 400, and D135_2 > 3 * (D135_1 + D135_3), then Pc is a thermal noise point at this time.

[0069] Furthermore, determining whether the intermediate pixel Pc is a thermal noise point according to the edge direction further includes;

[0070] When Pc < 60 and the values of the surrounding pixel points P1, P2, P3, P4, P5, P6, P7, and P8 are all more than 160 greater than the value of Pc, then Pc is a thermal noise point;

[0071] When Pc > 920 and the values of the surrounding pixel points P1, P2, P3, P4, P5, P6, P7, and P8 are all more than 120 less than the value of Pc, then Pc is a thermal noise point;

[0072] Furthermore, the method of detecting thermal noise points by using the gradient percentage, then preliminarily correcting the thermal noise points through median filtering, and finally calculating the final correction result by means of alpha blending includes:

[0073] If the edge direction is horizontal,

[0074] When |P4 - Pc| < |Pc - P5|, the value of the center pixel Pc after thermal noise point correction is obtained according to the gradual change of the brightness of the same color channel:

[0075] ouput = alpha * (P4 + (P2 + P7 - P1 - P6) / 2) + (1 - alpha) * Pc;

[0076] When |P4 - Pc| ≥ |Pc - P5|, the value of the center pixel Pc after thermal noise point correction is obtained according to the gradual change of the brightness of the same color channel:

[0077] ouput = alpha * (P5 + (P2 + P7 - P3 - P8) / 2) + (1 - alpha) * Pc;

[0078] If the edge direction is vertical,

[0079] When |P2 - Pc| < |Pc - P7|, the value of the center pixel Pc after thermal noise point correction is obtained according to the gradual change of the brightness of the same color channel:

[0080] ouput = alpha * (P2 + (P4 + P5 - P1 - P3) / 2) + (1 - alpha) * Pc;

[0081] When |P2 - Pc| ≥ |Pc - P7|, the value of the center pixel Pc after thermal noise point correction is obtained according to the gradual change of the brightness of the same color channel:

[0082] ouput = alpha * (P7 + (P4 + P5 - P6 - P8) / 2) + (1 - alpha) * Pc;

[0083] If the edge direction is 45°,

[0084] When |P3 - Pc| < |P6 - Pc|, the value of the central pixel c after thermal noise correction is obtained according to the gradual change of the brightness of the same color channel:

[0085] output = alpha * (P3 + (P4 + P7 - P2 - p5) / 2) + (1 - alpha) * Pc;

[0086] When |P3 - Pc| ≥ |P6 - Pc|, the value of the central pixel Pc after thermal noise correction is obtained according to the gradual change of the brightness of the same color channel:

[0087] output = alpha * (P6 + (P2 + P5 - P7 - p4) / 2) + (1 - alpha) * Pc;

[0088] If the edge direction is 135°,

[0089] When |P1 - Pc| < |P8 - Pc|, the value of the central pixel Pc after thermal noise correction is obtained according to the gradual change of the brightness of the same color channel:

[0090] output = alpha * (P1 + (P5 + P7 - P2 - p4) / 2) + (1 - alpha) * Pc;

[0091] When |P1 - Pc| ≥ |P8 - Pc|, the value of the central pixel Pc after thermal noise correction is obtained according to the gradual change of the brightness of the same color channel:

[0092] output = alpha * (P8 + (P2 + P4 - P5 - p7) / 2) + (1 - alpha) * Pc.

[0093] Furthermore, the noise parameters include shot noise parameters and readout noise parameters.

[0094] The calculation formula for the noise parameters of the shot noise parameters is as follows:

[0095] λ shot = g d g a

[0096] In the formula,

[0097] λ shot : Shot noise parameters;

[0098] g d : Digital gain;

[0099] g a : Analog gain;

[0100] The calculation formula for the noise parameters of the readout noise parameters is as follows:

[0101]

[0102] Wherein,

[0103] λ read : readout noise parameter;

[0104] Fixed readout variance.

[0105] Furthermore, the step of inputting the dark-light noise-reduced image in RAW format, the dark-light noise image after removing hot pixels, and the noise parameters into a symmetric encoder-decoder structure to obtain the noise-reduced image includes:

[0106] Input the dark-light noise-reduced image in RAW format into the decoder as a label image;

[0107] Input the dark-light noise image after removing hot pixels into the symmetric encoder-decoder;

[0108] Input the shot noise parameter and the readout noise parameter into the encoder;

[0109] Using the shot noise parameter and the readout noise parameter, the encoder adopts a contracting path to downsample the input dark-light noise image after removing hot pixels to obtain image feature maps of different sizes;

[0110] The decoder adopts an expanding path to upsample the input dark-light noise image after removing hot pixels to restore the size and texture details of the dark-light noise image after removing hot pixels;

[0111] Jump connect and splice the image feature maps on the path of the contracting path and the upsampled images on the path of the expanding path;

[0112] Continuously upsample the spliced image to the size of the original input dark-light noise image after removing hot pixels to obtain the spliced noise-reduced image;

[0113] Compare the difference between the noise-reduced image and the label image. When the difference between the noise-reduced image and the label image is reduced to a certain preset value, input the noise-reduced image.

[0114] The present invention has the following advantages:

[0115] (1) Before the low-light image is fed into the neural network, the present invention preprocesses the hot pixels of the low-light image, effectively reducing the image hot pixels introduced by the camera using a high ISO and long exposure mode in a low-light environment, and effectively improving the quality of the RAW image;

[0116] (2) The present invention comprehensively considers the characteristics of low-light images, which have a lot of noise and the thermal noise is sharper than the general noise. Using a general denoising network will remove too much image detail due to thermal noise. Therefore, a method of first identifying and then removing thermal noise is selected, and then a symmetric encoder-decoder structure is adopted to more accurately restore the texture and detail information of the original image;

[0117] (3) Adding noise parameters as additional inputs can effectively improve the denoising effect of low-light images and retain more details in the images, providing higher-quality RAW images for subsequent low-light enhancement. BRIEF DESCRIPTION OF THE DRAWINGS

[0118] Figure 1 is a flowchart of the method steps of the present invention;

[0119] Figure 2 is a graph of the relationship between m and alpha;

[0120] Figure 3 is a graph of the positional relationship between P1, P2, P3, P4, P5, P6, P7, P8 and Pc;

[0121] Figure 4 is a schematic diagram of a symmetric encoder-decoder;

[0122] Figure 5 is an experimental table of the comparison results between the present invention and other methods. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0123] The following further describes the present invention with reference to the accompanying drawings, but the protection scope of the present invention is not limited to the following.

[0124] It should be noted that the orientation or positional relationship indicated by "left", "right", etc. is based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the present invention is usually placed during use, or the orientation or positional relationship commonly understood by those skilled in the art. Such terms are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention.

[0125] It should be noted that, without conflict, the embodiments in the present invention and the features and technical solutions in the embodiments can be combined with each other.

[0126] Referring to Figure 1 , the present invention provides a method for denoising low-light images, including:

[0127] S1. Obtain a RAW domain denoising data set; the RAW domain denoising data set includes low-light noise images in RAW format and low-light denoised images in RAW format;

[0128] S2. Obtain the grayscale image of the low-light noise image, and determine whether the current low-light noise image is in a low-light scene according to the grayscale image;

[0129] If it is in a low-light scene, go to step S3;

[0130] If it is not in a low-light scene, end the noise reduction of the low-light image;

[0131] S3. Preprocess the thermal noise points of the low-light noise image to obtain the low-light noise image after removing the thermal noise points;

[0132] S4. Calculate the noise parameters of the low-light noise image after removing the thermal noise points;

[0133] S5. Input the RAW format low-light noise-reduced image, the low-light noise image after removing the thermal noise points, and the noise parameters into a symmetric encoder-decoder structure to obtain the noise-reduced image.

[0134] Further, the obtaining the grayscale image of the low-light noise image and determining whether the current low-light noise image is in a low-light scene according to the grayscale image includes:

[0135] S201. Downsample the low-light noise image to obtain a downsampled image, and perform grayscale binarization on the downsampled image to obtain a grayscale image;

[0136] S202. Obtain the grayscale value and grayscale histogram according to the grayscale image, and calculate the cumulative distribution function;

[0137] S203. Obtain the value of m corresponding to the grayscale image when the value of the cumulative distribution function = 0.75;

[0138] Refer to Figure 2 , Figure 2 is the relationship diagram between m and the intensity parameter alpha;

[0139] From Figure 2 it can be seen that

[0140] When 0 < m ≤ the first preset value th1, the low-light noise image is in a low-light scene, and take the intensity parameter alpha for removing the thermal noise points as 1;

[0141] When the first preset value th1 < m < the second preset value th2, the low-light noise image is in a relatively low-light scene, and take 0 < alpha < 1;

[0142] When m ≥ the second preset value th2, the low-light noise image is in a normal bright environment and not in a low-light scene, then alpha = 0, and end the noise reduction of the low-light image.

[0143] In some specific embodiments, the first preset value th1 is set to 50, and the second preset value th2 is set to 500.

[0144] Further, the preprocessing of the thermal noise points in the low-light noise image to obtain the low-light noise image after removing the thermal noise points includes:

[0145] S301. Calculate the median of the gradient values in the horizontal direction, vertical direction, 45° direction, and 135° direction of the low-light noise image;

[0146] S302. According to the minimum value of the medians of the gradient values in the horizontal direction, vertical direction, 45° direction, and 135° direction, use the minimum value as the edge direction;

[0147] min_grad = min(median_Dh, median_Dv, median_D45, median_D135)

[0148] In the formula:

[0149] min_grad represents the edge direction;

[0150] median_Dh represents the minimum value of the median in the horizontal direction;

[0151] median_Dv represents the minimum value of the median in the vertical direction;

[0152] median_D45 represents the minimum value of the median in the 45° direction;

[0153] median_D135 represents the minimum value of the median in the 135° direction;

[0154] S303. Determine whether the middle point is a thermal noise point according to the edge direction;

[0155] S304. Detect thermal noise points according to the gradient, perform preliminary correction on the thermal noise points through median filtering, and finally calculate the final correction result by the method of alpha blending of the intensity parameters for removing the thermal noise points;

[0156] S305. Obtain the low-light noise image after removing the thermal noise points according to the correction result.

[0157] Further, the calculation of the gradients in the horizontal direction, vertical direction, 45° direction, and 135° direction of the low-light noise image includes:

[0158] Obtain the central pixel Pc of the low-light noise image, and P1, P2, P3, P4, P5, P6, P7, and P8 are 8 pixel points adjacent to the central pixel Pc around it;

[0159] The positional relationships of P1, P2, P3, P4, P5, P6, P7, P8, and Pc are shown in Figure 3 ;

[0160] Calculate the three second-order gradients in the horizontal direction to obtain the absolute value of the horizontal gradient:

[0161] Dh1 = |P1 + P3 - 2 * P2|, Dh2 = |P4 + P5 - 2 * Pc|, Dh3 = |P6 + P8 - 2 * P7|;

[0162] Take the median of the absolute values of the horizontal gradients. The calculation expression is as follows:

[0163] median_Dh = median(Dh1, Dh2, Dh3);

[0164] Calculate the three second-order gradients in the vertical direction to obtain the absolute value of the vertical gradient:

[0165] Dv1 = |P1 + P6 - 2 * P4|, Dv2 = |P2 + P7 - 2 * Pc|, Dv3 = |P3 + P8 - 2 * P5|;

[0166] Take the median of the absolute values of the vertical gradients. The calculation expression is as follows:

[0167] median_Dv = median(Dv1, Dv2, Dv3);

[0168] The three second-order gradients in the 45° direction:

[0169] D45_1 = 2 * |P4 - P2|, D45_2 = |P3 + P6 - 2Pc|, D45_3 = 2 * |P7 - P5|;

[0170] Take the median of the absolute values of the vertical gradients. The calculation expression is as follows:

[0171] median_D45 = median(D45_1, D45_2, D45_3);

[0172] The three second-order gradients in the 135° direction:

[0173] D135_1 = 2 * |P2 - P5|, D135_2 = |P1 + P8 - 2Pc|, D135_3 = 2 * |P7 - P4|;

[0174] Take the median of the absolute values of the vertical gradients. The calculation expression is as follows:

[0175] median_D135 = median(D135_1, D135_2, D135_3).

[0176] Further, determining whether the intermediate pixel Pc is a thermal noise point according to the edge direction includes:

[0177] If the edge direction is horizontal or vertical, determine whether the absolute value of the second-order gradient in the horizontal or vertical direction passing through the central pixel Pc is greater than 4 times the absolute values of the other two second-order gradients in the same direction;

[0178] When the edge direction is horizontal, if Dh2 > 4 * (Dh1 + Dh3), then Pc is a thermal noise point;

[0179] Otherwise, Pc is not a thermal noise point;

[0180] When the edge direction is vertical, if Dv2 > 4 * (Dv1 + Dv3), then Pc is a thermal noise point;

[0181] Otherwise, Pc is not a thermal noise point.

[0182] If the edge direction is 45°, calculate the sum of the absolute values of the differences between the absolute values of the three second-order gradients in the 135° direction. The formula is as follows:

[0183] D135_sum = |D135_1 - D135_2| + |D135_2 - D135_3| + |D135_3 - D135_1|;

[0184] If D135_sum < 400, and D45_2 > 3 * (D45_1 + D45_3) and D135_2 > 3 * (D135_1 + D135_3), then Pc is a thermal noise point;

[0185] If D135_sum >= 400, and D45_2 > 3 * (D45_1 + D45_3), then Pc is a thermal noise point at this time;

[0186] If the edge direction is 135°, calculate the sum of the absolute values of the differences between the absolute values of the three second-order gradients in the 45° direction. The formula is as follows:

[0187] D45_sum = |D45_1 - D45_2| + |D45_2 - D135_3| + |D45_3 - D45_1|;

[0188] If D45_sum < 400, and D45_2 > 3 * (D45_1 + D45_3) and D135_2 > 3 * (D135_1 + D135_3), then Pc is a thermal noise point;

[0189] If D45_sum >= 400, and D135_2 > 3 * (D135_1 + D135_3), then Pc is a thermal noise point at this time.

[0190] Further, determining whether the middle pixel Pc is a thermal noise point according to the edge direction further includes:

[0191] When Pc < 60 and the values of the surrounding pixel points P1, P2, P3, P4, P5, P6, P7, and P8 are all more than 160 greater than the value of Pc, then Pc is a thermal noise point;

[0192] When Pc > 920 and the values of the surrounding pixel points P1, P2, P3, P4, P5, P6, P7, and P8 are all more than 120 less than the value of Pc, then Pc is a thermal noise point;

[0193] Further, detecting thermal noise points by means of gradient percentage, then preliminarily correcting the thermal noise points through median filtering, and finally calculating the final correction result through alpha blending, includes:

[0194] If the edge direction is horizontal,

[0195] When |P4 - Pc| < |Pc - P5|, the value of the center pixel Pc after thermal noise point correction is obtained according to the gradual change of the brightness of the same color channel:

[0196] ouput = alpha * (P4 + (P2 + P7 - P1 - P6) / 2) + (1 - alpha) * Pc;

[0197] When |P4 - Pc| ≥ |Pc - P5|, the value of the center pixel Pc after thermal noise point correction is obtained according to the gradual change of the brightness of the same color channel:

[0198] ouput = alpha * (P5 + (P2 + P7 - P3 - P8) / 2) + (1 - alpha) * Pc;

[0199] If the edge direction is vertical,

[0200] When |P2 - Pc| < |Pc - P7|, the value of the center pixel Pc after thermal noise point correction is obtained according to the gradual change of the brightness of the same color channel:

[0201] ouput = alpha * (P2 + (P4 + P5 - P1 - P3) / 2) + (1 - alpha) * Pc;

[0202] When |P2 - Pc| ≥ |Pc - P7|, the value of the center pixel Pc after thermal noise point correction is obtained according to the gradual change of the brightness of the same color channel:

[0203] ouput = alpha * (P7 + (P4 + P5 - P6 - P8) / 2) + (1 - alpha) * Pc;

[0204] If the edge direction is 45°,

[0205] When |P3 - Pc| < |P6 - Pc|, the value of the central pixel Pc after thermal noise correction is obtained according to the gradient of the brightness in the same color channel:

[0206] output = alpha * (P3 + (P4 + P7 - P2 - p5) / 2) + (1 - alpha) * Pc;

[0207] When |P3 - Pc| ≥ |P6 - Pc|, the value of the central pixel Pc after thermal noise correction is obtained according to the gradient of the brightness in the same color channel:

[0208] output = alpha * (P6 + (P2 + P5 - P7 - p4) / 2) + (1 - alpha) * Pc;

[0209] If the edge direction is 135°,

[0210] When |P1 - Pc| < |P8 - Pc|, the value of the central pixel Pc after thermal noise correction is obtained according to the gradient of the brightness in the same color channel:

[0211] output = alpha * (P1 + (P5 + P7 - P2 - p4) / 2) + (1 - alpha) * Pc.

[0212] In low - light environments, cameras usually use high - sensitivity and long - exposure modes for image capture. This can effectively brighten the overall brightness of the image, enabling the image to show more details. However, for images captured using high - sensitivity and long - exposure modes, due to the increase in the camera sensor gain and temperature rise, there will be some bright spots in the image where the pixel values are significantly higher than the surrounding pixels, namely thermal noise (hotpixel); the lower the environmental illumination of the image being captured, the higher the sensitivity of the camera sensor, the longer the exposure time, and the more obvious the image thermal noise; image thermal noise is significantly different from the overall noise of the image. If a neural - network - based method is used to filter out both thermal noise and other image noises simultaneously, the image will become much blurrier and a large amount of details will be lost; in the present invention, after judging the central pixel Pc, the thermal noise is corrected through the gradient of the brightness in the same color channel, thereby pre - processing the thermal noise and reducing the impact of thermal noise on the subsequent noise reduction process, mainly affecting the accuracy of the noise parameters.

[0213] Furthermore, the noise parameters include shot - noise parameters and read - out noise parameters.

[0214] The calculation formula for the noise parameters of the shot - noise parameters is as follows:

[0215] λ shot = gd g a

[0216] Wherein,

[0217] λ shot : Shot noise parameter;

[0218] g d : Digital gain;

[0219] g a : Analog gain;

[0220] The calculation formula of the noise parameter of the readout noise parameter is as follows:

[0221]

[0222] Wherein,

[0223] λ read : Readout noise parameter;

[0224] Fixed readout variance.

[0225] Furthermore, referring to Figure 4 , the process of inputting the RAW format low-light noise-reduced image, the low-light noise image after removing thermal noise, and the noise parameters into a symmetric encoder-decoder structure to obtain the noise-reduced image includes:

[0226] Input the RAW format low-light noise-reduced image into the decoder as the label image;

[0227] Input the low-light noise image after removing thermal noise into the symmetric encoder-decoder;

[0228] Input the shot noise parameter and the readout noise parameter into the encoder;

[0229] Using the shot noise parameter and the readout noise parameter, the encoder adopts a contraction path to downsample the input low-light noise image after removing thermal noise to obtain image feature maps of different sizes;

[0230] The decoder adopts an expansion path to upsample the input low-light noise image after removing thermal noise to restore the size and texture details of the low-light noise image after removing thermal noise;

[0231] Jump connect and splice the image feature maps on the path of the contraction path and the upsampled images on the path of the expansion path;

[0232] Continuously upsample the spliced image to the size of the original input low-light noise image after removing thermal noise to obtain the spliced noise-reduced image;

[0233] Compare the denoised image with the labeled image. After the difference between the denoised image and the labeled image is reduced to a certain preset value, input the denoised image.

[0234] Before sending the low-light image into the neural network, the present invention preprocesses the thermal noise points of the low-light image, effectively reducing the image thermal noise points introduced by the camera when shooting in a low-light environment using a high ISO and long exposure mode, and effectively improving the quality of the RAW image; moreover, the thermal noise points are sharper than the general noise points, and using a general denoising network will remove too much image detail due to the thermal noise points; therefore, the method of first confirming and then removing the thermal noise points is selected, and then a symmetric encoder-decoder structure is adopted to more accurately restore the texture and detail information of the original image; the present invention also adds the noise parameter as an additional input, which can effectively improve the denoising effect of the low-light image and retain more details in the image, providing a higher-quality RAW image for subsequent low-light enhancement.

[0235] For the method adopted by the present invention, the present invention uses the image effects before and after the improvement of the test set for comparative verification, and the test results are as Figure 5 shown. The peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) commonly used in the field of image enhancement are used as evaluation indicators. The larger the PSNR, the higher the image quality, and the higher the SSIM, the more similar the two images are.

[0236] The above embodiments only express relatively preferred implementation manners, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the patent of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several deformations and improvements can be made, and these all belong to the protection scope of the present invention.

Claims

1. A method for reducing noise in low-light images, characterized in that: Including: S1. Obtain a RAW domain noise reduction dataset; The RAW domain noise reduction dataset includes multiple pairs of low-light noise images in RAW format and low-light noise-reduced images in RAW format; S2. Obtain the grayscale image of the low-light noise image, and determine whether the current low-light noise image is in a low-light scene according to the grayscale image; If it is in a low-light scene, go to step S3; If it is not in a low-light scene, end the noise reduction of the low-light image; S3. Preprocess the thermal noise points of the low-light noise image to obtain a low-light noise image after removing thermal noise points; S4. Calculate the noise parameters of the low-light noise image after removing thermal noise points; S5. Input the low-light noise-reduced image in RAW format, the low-light noise image after removing thermal noise points, and the noise parameters into a symmetric encoder-decoder structure to obtain a noise-reduced image; S5 specifically includes the following: Input the low-light noise-reduced image in RAW format into the decoder as a label image; Input the low-light noise image after removing thermal noise points into the symmetric encoder-decoder; Input the shot noise parameter and the read noise parameter into the encoder; Through the shot noise parameter and the read noise parameter, use the encoder to downsample the input low-light noise image after removing thermal noise points by using a contracting path to obtain image feature maps of different sizes; Use the decoder to upsample the input low-light noise image after removing thermal noise points by using an expanding path to restore the size and texture details of the low-light noise image after removing thermal noise points; Jump-connect and splice the image feature maps on the path of the contracting path and the upsampled images on the path of the expanding path; Continuously upsample the spliced image to the size of the original input low-light noise image after removing thermal noise points to obtain a spliced noise-reduced image; Compare the difference between the noise-reduced image and the label image. When the difference comparison between the noise-reduced image and the label image is reduced to a certain preset value, output the noise-reduced image.

2. The method for reducing noise of a low-light image according to claim 1, characterized in that: The obtaining of the grayscale image of the low-light noise image and determining whether the current low-light noise image is in a low-light scene according to the grayscale image includes: S201. Downsample the low-light noise image to obtain a downsampled image, and calculate the grayscale of the downsampled image to obtain a grayscale image; S202. Statistically analyze the grayscale information of the grayscale image to obtain a cumulative distribution function; S203. Obtain the value m of the corresponding grayscale image when the value of the cumulative distribution function = 0.75; When 0 < m ≤ the first preset value, the low-light noise image is in a low-light scene, and take the intensity parameter alpha for removing thermal noise points as 1; When the first preset value < m < the second preset value, the low-light noise image is in a relatively low-light scene, and take 0 < alpha < 1; When m ≥ the second preset value, the low-light noise image is in a normal bright light environment and not in a low-light scene, then alpha = 0, and end the noise reduction of the low-light image.

3. The method for reducing noise of a low-light image according to claim 1, characterized in that: The preprocessing of the thermal noise points of the low-light noise image to obtain a low-light noise image after removing thermal noise points includes: S301. Calculate the median of the gradient values of the low-light noise image in the horizontal direction, vertical direction, 45° direction, and 135° direction; S302, according to the minimum value of the median of the gradient values ​​in the horizontal direction, the vertical direction, the 45° direction and the 135° direction, the minimum value is taken as the edge direction; the calculation formula is as follows: min_grad=min(median_Dh,median_Dv,median_D45,median_D135) Where: min_grad: edge direction; median_Dh: minimum value of the median in the horizontal direction; median_Dv: minimum value of the median in the vertical direction; median_D45: minimum value of the median in the 45° direction; median_D135: minimum value of the median in the 135° direction; S303, judging whether the central pixel Pc is a thermal noise point according to the edge direction; If the central pixel is not a thermal noise point, then enter S4; If the central pixel is a thermal noise point, proceed to S304; S304, calculating the final correction result by removing the intensity parameter alpha blending of the hot noise points; S305 , obtaining a dark light noise image after removing thermal noise points according to the correction result.

4. The method for reducing noise of a low-light image according to claim 3, characterized in that: The step of calculating the gradients of the dark light noise image in the horizontal direction, the vertical direction, the 45° direction and the 135° direction comprises: The central pixel Pc of the dark light noise image is obtained, and P1, P2, P3, P4, P5, P6, P7 and P8 are eight adjacent pixels around the central pixel Pc; Calculate the three second-order gradients in the horizontal direction to get the absolute value of the horizontal gradient: Dh1=|P1+P3-2*P2|, Dh2=|P4+P5-2*Pc|, Dh3=|P6+P8-2*P7|; Take the median of the absolute value of the horizontal gradient. The calculation expression is as follows: median_Dh=median(Dh1,Dh2,Dh3); Calculate the three second-order gradients in the vertical direction to get the absolute value of the vertical gradient: Dv1=|P1+P6-2*P4|, Dv2=|P2+P7-2*Pc|, Dv3=|P3+P8-2*P5|; Take the median of the absolute value of the vertical gradient. The calculation expression is as follows: median_Dv=median(Dv1,Dv2,Dv3); Three second-order gradients in the 45° direction: D45_1=2*|P4-P2|, D45_2=|P3+P6-2Pc|, D45_3=2*|P7-P5|; Take the median of the absolute value of the vertical gradient. The calculation expression is as follows: median_D45=median(D45_1,D45_2,D45_3); Three second-order gradients in the 135° direction: D135_1=2*|P2-P5|, D135_2=|P1+P8-2Pc|, D135_3=2*|P7-P4|; Take the median of the absolute value of the vertical gradient. The calculation expression is as follows: median_D135=median(D135_1,D135_2,D135_3).

5. The method for reducing noise of a low-light image according to claim 4, characterized in that: The step of judging whether the middle pixel Pc is a thermal noise point according to the edge direction includes: If the edge direction is horizontal or vertical, determine whether the absolute value of the second-order gradient in the horizontal or vertical direction where the center pixel Pc is located is greater than 4 times the absolute values ​​of the other two second-order gradients in the same direction; When the edge direction is horizontal, Dh2>4*(Dh1+Dh3), then Pc is a thermal noise point; Otherwise, Pc is not a thermal noise point; When the edge direction is vertical, Dv2>4*(Dv1+Dv3), then Pc is a thermal noise point; Otherwise, Pc is not a thermal noise point; If the edge direction is 45°, the sum of the absolute values ​​of the differences between the absolute values ​​of the three second-order gradients in the 135° direction is calculated. The formula is as follows: D135_sum=|D135_1-D135_2|+|D135_2-D135_3|+|D135_3-D135_1|; If D135_sum<400, and D45_2>3*(D45_1+D45_3) and D135_2>3*(D135_1+D135_3), then Pc is a thermal noise point; If D135_sum>=400, and D45_2>3*(D45_1+D45_3), then Pc is a thermal noise point; If the edge direction is 135°, the sum of the absolute values ​​of the differences between the absolute values ​​of the three second-order gradients in the 45° direction is calculated. The formula is as follows: D45_sum=|D45_1-D45_2|+|D45_2-D135_3|+|D45_3-D45_1|; If D45_sum<400, and D45_2>3*(D45_1+D45_3) and D135_2>3*(D135_1+D135_3), then Pc is a thermal noise point; If D45_sum>=400, and D135_2>3*(D135_1+D135_3), then Pc is a thermal noise point.

6. The method for reducing noise of a low-light image according to claim 5, characterized in that: The step of judging whether the middle pixel Pc is a thermal noise point according to the edge direction also includes: When Pc<60, and the values ​​of the surrounding pixels P1, P2, P3, P4, P5, P6, P7 and P8 are all greater than Pc by more than 160, then Pc is a thermal noise point; When Pc>920, and the values ​​of the surrounding pixels P1, P2, P3, P4, P5, P6, P7 and P8 are more than 120 less than the Pc value, Pc is a thermal noise point.

7. The method for reducing noise of a low-light image according to claim 4, characterized in that: The thermal noise is detected by using a gradient percentage method, and then the thermal noise is preliminarily corrected by a median filter, and finally the final correction result is calculated by an alpha blending method, including: If the edge direction is horizontal, When |P4-Pc|<|Pc-P5|, the value of the central pixel Pc after thermal noise correction is obtained according to the gradient of the brightness of the same color channel: ouput=alpha*(P4+(P2+P7-P1-P6) / 2)+(1-alpha)*Pc; When |P4-Pc|≧|Pc-P5|, the value of the central pixel Pc after thermal noise correction is obtained according to the gradient of the brightness of the same color channel: ouput=alpha*(P5+(P2+P7-P3-P8) / 2)+(1-alpha)*Pc; If the edge direction is vertical, When |P2-Pc|<|Pc-P7|, the value of the central pixel Pc after thermal noise correction is obtained according to the gradient of the brightness of the same color channel: ouput=alpha*(P2+(P4+P5-P1-P3) / 2)+(1-alpha)*Pc; When |P2-Pc|≥|Pc-P7|, the value of the central pixel Pc after thermal noise correction is obtained according to the gradient of the brightness of the same color channel: ouput=alpha*(P7+(P4+P5-P6-P8) / 2)+(1-alpha)*Pc; If the edge direction is 45°, When |P3-Pc|<|P6-Pc|, the value of the central pixel c after thermal noise correction is obtained according to the gradient of the brightness of the same color channel: output=alpha*(P3+(P4+P7-P2-p5) / 2)+(1-alpha)*Pc; When |P3-Pc|≥|P6-Pc|, the value of the central pixel Pc after thermal noise correction is obtained according to the gradient of the brightness of the same color channel: output=alpha*(P6+(P2+P5-P7-p4) / 2)+(1-alpha)*Pc; If the edge direction is 135°, When |P1-Pc|<|P8-Pc|, the value of the central pixel Pc after thermal noise correction is obtained according to the gradient of the brightness of the same color channel: output=alpha*(P1+(P5+P7-P2-p4) / 2)+(1-alpha)*Pc; When |P1-Pc|≥|P8-Pc|, the value of the central pixel Pc after thermal noise correction is obtained according to the gradient of the brightness of the same color channel: output=alpha*(P8+(P2+P4-P5-p7) / 2)+(1-alpha)*Pc.

8. The method for reducing noise of a low-light image according to claim 2, characterized in that: The noise parameters include shot noise parameters and readout noise parameters; The calculation formula of the noise parameter of the shot noise parameter is as follows: l shot =g d g a In the formula, λ shot : Shot noise parameter; g d : digital gain; g a : analog gain; The calculation formula of the noise parameter of the readout noise parameter is as follows: In the formula, λ read : Readout noise parameters; Fixed readout variance.

Images