Image denoising and enhancing method for low-illumination image in indoor and outdoor scenes

Abstract

The invention relates to the technical field of image acquisition and processing, in particular to an image denoising and enhancing method for a low-illumination image in indoor and outdoor scenes, which comprises the following steps of: step 1, image denoising: denoising the low-illumination image by adopting improved wavelet transform denoising, Gaussian filtering, bilateral filtering and non-local mean filtering in sequence; step 2, image enhancement: carrying out enhancement processing on the low-illumination image by adopting an adaptive histogram equalization algorithm, linear mixing and Gaussian low-pass filtering in sequence; and step 3, weighting and synthesizing the image: firstly carrying out image quality evaluation calculation on the denoised and enhanced low-illumination images, then distributing weights for the denoised and enhanced low-illumination images according to the calculated image quality index value, and finally synthesizing an image which is clearer and higher in illumination than the original image. According to the invention, through a method of combining a plurality of traditional algorithms, denoising and enhancement of an extremely low illumination image in indoor and outdoor scenes are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image acquisition and processing, and specifically to an image denoising and enhancement method for indoor and outdoor scenes of low-light images. Background Art

[0002] Currently, the demand for image acquisition pervades every corner of people's lives. However, image acquisition devices are greatly affected by light. Images acquired under harsh lighting conditions exhibit problems such as many noise points, uneven lighting, and information loss, seriously affecting the visual effect of the images and making them unidentifiable and unusable. Therefore, researching low-light image denoising algorithms has very important application value for various scenarios such as area security and traffic command. Traditional low-light image denoising methods mainly include methods based on filtering, transform domain processing, and statistical models.

[0003] Currently, traditional low-light image denoising methods mainly include the following categories:

[0004] 1. Filter-based methods: These methods suppress noise through spatial domain or frequency domain filters. Common filters include mean filtering, median filtering, and Gaussian filtering, etc. These methods are simple and easy to implement, but often lead to the loss of image details and blurring.

[0005] 2. Transform domain-based methods: These methods convert the image from the spatial domain to the frequency domain (such as Fourier transform, wavelet transform, etc.), process the noise in the frequency domain, and then transform back to the spatial domain. For example, wavelet transform can decompose the image into sub-bands of different scales, and remove noise by processing the high-frequency sub-bands.

[0006] 3. Statistical model-based methods: These methods utilize the statistical characteristics of the image to construct a noise model, and denoise through methods such as maximum likelihood estimation or Bayesian inference. For example, the Non-Local Means (NLM) method denoises by using the redundant information of similar blocks in the image.

[0007] The above traditional methods have disadvantages such as over-smoothing and lack of flexibility, while deep learning methods require a large amount of image data of different scenes to be collected in advance, and if the noise types in the actual usage scenario and the training scenario differ greatly, the denoising effect of the algorithm cannot be guaranteed. Summary of the Invention

[0008] To solve the above technical problems, the present invention proposes an image denoising and enhancement method for indoor and outdoor scenes of low-light images.

[0009] The technical problems to be solved by the present invention are realized by adopting the following technical solutions:

[0010] An image denoising and enhancement method for indoor and outdoor scenes of low-light images, comprising the following steps:

[0011] Step 1, image denoising:

[0012] Successively use improved wavelet transform denoising, Gaussian filtering, bilateral filtering, and non-local mean filtering to denoise the low-light image;

[0013] Step 2, image enhancement:

[0014] Successively use the adaptive histogram equalization algorithm, linear mixing, and Gaussian low-pass filtering to enhance the low-light image;

[0015] Step 3, weighted synthesis of images:

[0016] First, perform image quality assessment calculations on the denoised low-light image and the enhanced low-light image. Then, assign weights to the denoised low-light image and the enhanced low-light image according to the calculated image quality index values. Finally, add them according to the corresponding weight coefficients to synthesize an image that is clearer and has a higher illumination than the original image.

[0017] As a further improvement of the present invention, the improvement of the improved wavelet transform denoising in Step 1 lies in: a compromise threshold is set between the existing hard threshold and soft threshold, and the compromise threshold range is between the hard threshold and the soft threshold.

[0018] As a further improvement of the present invention, define an image quality evaluation index before the image quality assessment calculation in Step 3. The image quality evaluation index includes signal-to-noise ratio, contrast, sharpness, and edge retention.

[0019] As a further improvement of the present invention, the calculation formula of the signal-to-noise ratio is as follows:

[0020]

[0021] In the above formula, the specific calculation formulas of MSV and MSE are:

[0022]

[0023] In the formula, M and N respectively represent the number of rows and columns of the image, that is, the height and width of the image, I(i,j) is the pixel value of the original image at (i,j), and K(i,j) is the pixel value of the image after algorithm processing at (i,j).

[0024] As a further improvement of the present invention, the calculation formula for assigning weights in Step 3 is as follows:

[0025]

[0026] In the formula, wenhanced and w denoised are the weight coefficients of the enhanced and denoised images in the final fused image, respectively.

[0027] As a further improvement of the present invention, the calculation formula of the fused image in step three is as follows:

[0028] I final = w enhanced ·I enhanced + w denoised ·I denoised ,

[0029] wherein, I final is the fused image, and I enhanced and I denoised are the enhanced and denoised images, respectively.

[0030] The beneficial effects of the present invention are as follows:

[0031] The present invention provides a method for image denoising and enhancement in indoor and outdoor scenes of low-light images. Considering both image denoising and enhancement of low-light images, through a method of combining multiple traditional algorithms, the denoising and enhancement of extremely low-light images in indoor and outdoor scenes are achieved, and the feasibility of this method is verified by testing the images collected indoors and outdoors. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The present invention will be further described below with reference to the drawings and embodiments:

[0033] Figure 1 is a flowchart of the existing wavelet threshold denoising method;

[0034] Figure 2 is a flowchart of step one of the present invention;

[0035] Figure 3 is a flowchart of step three of the present invention;

[0036] Figure 4 is a schematic diagram of the comparison of the test effects of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0037] In order to make the technical means, creative features, achieved purposes and functions of the present invention easy to understand, the present invention will be further described below with reference to the drawings and embodiments.

[0038] A method for image denoising and enhancement in indoor and outdoor scenes of low-light images includes the following steps:

[0039] Step 1, Image denoising. Its flowchart is as Figure 2 shown.

[0040] First, an improved wavelet transform denoising method is adopted to perform multi-scale decomposition on the image, and threshold processing is performed on the high-frequency part to initially remove noise.

[0041] In this step, traditional denoising algorithms such as Gaussian filtering, mean filtering, bilateral filtering, and non-local mean filtering can usually only handle one or two specific types of noise. However, the types of image noise in the actual environment are complex and variable. Relying solely on a single filtering method often leads to various problems in the denoised image, such as over-smoothing, detail loss, edge blurring, and noise residue.

[0042] In this embodiment, the wavelet threshold denoising method is first adopted. The basic idea of the wavelet threshold denoising method is to first set a critical threshold a. If the wavelet coefficient is less than a, it is considered that this coefficient is mainly caused by noise, and this part of the coefficient is removed. If the wavelet coefficient is greater than a, it is considered that this coefficient is mainly caused by the signal, and this part of the coefficient is retained. Then, the inverse wavelet transform is performed on the processed wavelet coefficients to obtain the denoised signal. The flow chart of the method is as Figure 1 shown. The wavelet threshold selection methods include fixed threshold (Sqtwolog threshold), risk threshold (Rigrsure threshold), heuristic threshold (Heursure threshold), and minimax threshold. The variance of the noise is not involved in these threshold selection methods. Therefore, in the present invention, the variance of the noise is processed separately. Take the median of the absolute values of the wavelet coefficients at each scale, and then divide the median by a constant as the estimate of the noise intensity in the wavelet coefficients at this scale, that is:

[0043]

[0044] In formula (1), j is the scale of wavelet decomposition, d j (k) is the wavelet decomposition coefficient, and median is the command for calculating the median. It can be seen from formula (1) that its global threshold is:

[0045]

[0046] In the above formula (2), M and N are the scale sizes of the image.

[0047] The selection of the threshold function is mainly divided into two categories: hard threshold and soft threshold. The hard threshold is a simple method of setting to zero, while the soft threshold "shrinks" the wavelet coefficients greater than the threshold, that is, subtracts the threshold, so that the input-output curve becomes continuous. In the selection of the threshold, the soft threshold is commonly used.

[0048] In this embodiment, the threshold function is improved. The improved threshold is a compromise between the hard threshold and the soft threshold. That is, when the wavelet coefficient is less than the threshold, instead of simply setting it to zero, its value is smoothly reduced to zero. When it is greater than the threshold, the amplitude of the wavelet coefficient is subtracted by the threshold. In this way, both large wavelet coefficients are ensured, and the smooth transition of the coefficients after adding the threshold is also ensured. In this design, a compromise threshold is customized. When the wavelet coefficient is equal to the threshold, a compromise coefficient b is added to the threshold, thus realizing the smooth transition of the threshold from the hard threshold to the soft threshold. The range of the compromise coefficient is 0 - 100, that is, when b = 0, it is the hard threshold, when b = 100, it is the soft threshold, and when b is within the range of 0 to 100, it is the compromise threshold.

[0049] Secondly, the Gaussian filtering method is adopted. Gaussian filtering is a linear filtering technique that uses the Gaussian function as the weight kernel to smooth the image. In this embodiment, it is used to initially reduce the Gaussian noise in the input image while retaining the image details.

[0050] Subsequently, the bilateral filtering method is used. Bilateral filtering is a non - linear filtering technique that can reduce the noise in the image while keeping the edges of the input image clear. The bilateral filtering method is considered in terms of the spatial proximity between pixels and the similarity of pixel values.

[0051] Finally, the non - local means filtering method is applied. The non - local means filtering method is an advanced denoising technique that removes noise by comparing non - local similar blocks in the image while retaining the detailed parts of the image and avoiding over - smoothing after image denoising.

[0052] Compared with the prior art, in this embodiment, through the above - mentioned combined method, the noise in the low - illumination image can be effectively removed, while the important details and edge information in the low - illumination image are retained, improving the visual quality and usability of the image.

[0053] Step 2: Image enhancement.

[0054] First, the adaptive histogram equalization algorithm is adopted. By using the exposure module in the skimage library, the size of the region for using the equalized histogram (kernel_size) and the degree of its histogram equalization (clip_limit) are modified, thus outputting the image with enhanced contrast.

[0055] The adaptive histogram equalization algorithm has several advantages compared with other enhancement algorithms:

[0056] First: For low - illumination images that usually accompany increased noise and reduced contrast, CLAHE effectively improves the image contrast through local histogram equalization while limiting the noise amplification, which is crucial for improving the quality of low - illumination images.

[0057] Second: Under low - illumination conditions, the details and texture information of the image are often not obvious. The CLAHE algorithm makes these details and texture information more prominent by enhancing the local contrast, improving the visual clarity of the image.

[0058] Third: Low - illumination images are often accompanied by the problem of uneven illumination. CLAHE effectively reduces the impact caused by uneven illumination by processing local regions of the image, making the overall visual effect of the image more balanced.

[0059] Fourth: While maintaining high efficiency and real - time performance, the CLAHE algorithm provides good image enhancement effects, making it suitable for application scenarios that require real - time processing, such as video surveillance and real - time image analysis.

[0060] Fifth: The CLAHE algorithm provides parameters such as contrast limit and clipLimit, allowing users to adjust the behavior of the algorithm according to specific application requirements to achieve the best enhancement effect.

[0061] Sixth: This method can adapt to different illumination conditions and image contents, while deep - learning methods need to spend a lot of time training or set different models to achieve better enhancement effects in different scenarios.

[0062] Secondly, the enhanced image is combined with the original image through linear mixing. Through mixing, the details of the original image can be retained while enhancing the contrast of the image.

[0063] Subsequently, the mixed image is smoothed by applying Gaussian low - pass filtering to reduce image noise while retaining image details.

[0064] Through the enhancement method in Step 2 above, the present invention can improve the quality of low - illumination images, effectively enhance the image contrast through local histogram equalization while limiting noise amplification; enhance details and textures, enhance local contrast, make the details and texture information of the image more prominent, and improve the visual clarity of the image; reduce the impact of uneven illumination, process local regions of the image, effectively reduce the impact caused by uneven illumination, and make the overall visual effect of the image more balanced; adapt to different illumination conditions and image contents, and can adapt to different illumination conditions and image contents, while deep - learning methods need a lot of time to train or set different models to achieve better enhancement effects.

[0065] Step 3: Weighted synthesis of the image. Its flowchart is as Figure 3 shown. It includes the following steps:

[0066] First, image quality assessment:

[0067] Define image quality evaluation metrics. Common metrics include signal-to-noise ratio (SNR), contrast, sharpness, edge preservation, etc. Calculate the above quality metrics for the enhanced image and the denoised image respectively. Through comparison, the present invention decides to use SNR as the image quality evaluation metric.

[0068] The signal-to-noise ratio is a metric that measures the ratio of the signal intensity to the noise intensity in an image and is usually used to evaluate the effect of image denoising. The higher the signal-to-noise ratio, the less noise in the image and the better the image quality. In image processing, the power of the signal can usually be expressed as the mean squared value (MSV) of the image signal, and the power of the noise can be expressed as the mean squared error (MSE) of the image. Therefore, the calculation formula for the SNR metric is as follows:

[0069]

[0070] In the above formula (3), the specific calculation formulas for MSV and MSE are:

[0071]

[0072] In formulas (4) and (5), M and N respectively represent the number of rows and columns of the image, that is, the height and width of the image, I(i,j) is the pixel value of the original image at (i,j), and K(i,j) is the pixel value of the image after algorithm processing at (i,j).

[0073] Second, weight calculation.

[0074] According to the calculated SNR quality metric value, assign weights to the enhanced and denoised images. In the present invention, weight assignment is carried out based on the signal-to-noise ratio as the quality evaluation metric, and the goal is to make the image with a higher signal-to-noise ratio have a greater weight in the final fused image.

[0075] After the above enhancement and denoising, two images and their respective signal-to-noise ratios can be obtained. The weight assignment formula is as follows:

[0076]

[0077] In formulas (6) and (7), w enhanced and w denoised are the weight coefficients of the enhanced and denoised images in the final fused image respectively.

[0078] Third, the fusion process.

[0079] Using the above weight values, the final fused image I final can be calculated by the following formula:

[0080] Ifinal = w enhanced ·I enhanced + w denoised ·I denoised (8)

[0081] In formula (8), I final is the fused image, and I enhanced and I denoised are the enhanced and denoised images respectively.

[0082] Through the method in the above step three, the present invention can dynamically adjust the fusion weight according to the actual quality of the image, making the fusion effect more in line with the actual visual effect of the image, and by selectively retaining the best parts in the enhanced and denoised images, improving the visual quality of the final image. This weighted synthesis method can adapt to different images and different application scenarios, and has good flexibility and adaptability.

[0083] Compared with the existing traditional weighted average method that simply combines the results of these two steps but may not be able to fully utilize the image quality information, the fusion method based on image quality of the present invention can generate a higher-quality final image by evaluating and comparing the image quality of the enhanced and denoised images and dynamically adjusting the fusion weight. By controlling the weight values of the enhanced and denoised images in different processing steps in this way, the final image is generated, which not only retains the details of the image but also effectively reduces the noise in the image and improves the overall quality of the image.

[0084] Furthermore, in order to verify the effect of the present invention, low-light image experiments were carried out indoors and outdoors, and the experimental results are as Figure 4 shown. Among them Figure 4 the left side is the original image, and the right side is the image effect processed by the present invention. It can be seen from the above image comparison that for the image processed by the present invention, whether indoors or outdoors, both the denoising effect of the image and the overall brightness of the image have better denoising and enhancement effects compared with the original image.

[0085] The above has shown and described the basic principles, main features and advantages of the present invention. Those skilled in the art of this industry should understand that the present invention is not limited by the above embodiments. What is described in the above embodiments and the specification is only the principle of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for image denoising and enhancement in indoor and outdoor scenes of low-light images, characterized in that: The following steps are involved: Step 1: Image denoising: Improved wavelet transform denoising, Gaussian filtering, bilateral filtering and non-local mean filtering are used in turn to denoise the low-illumination image. Step 2: Image enhancement: Adaptive histogram equalization algorithm, linear mixing and Gaussian low-pass filtering are used in sequence to enhance the low-illumination image; Step 3: Weighted composite image: Firstly, the image quality evaluation calculation is performed on the denoised low-illuminance image and the enhanced low-illuminance image. Then, weights are assigned to the denoised low-illuminance image and the enhanced low-illuminance image according to the calculated image quality index values. Finally, the corresponding weight coefficients are added together to synthesize an image that is clearer and has a higher illumination than the original image.

2. The method for image denoising and enhancement in indoor and outdoor scenes of low-light images according to claim 1, characterized in that: The improvement of the improved wavelet transform denoising in step 1 is that a compromise threshold is set between the existing hard threshold and soft threshold, and the range of the compromise threshold is between the hard threshold and the soft threshold.

3. The method for image denoising and enhancement in indoor and outdoor scenes of low-light images according to claim 1, characterized in that: In step 3, image quality evaluation indicators are defined before image quality evaluation calculation. The image quality evaluation indicators include signal-to-noise ratio, contrast, sharpness, and edge preservation.

4. The method for image denoising and enhancement in indoor and outdoor scenes of low-light images according to claim 3, characterized in that: The signal-to-noise ratio is calculated as follows: In the above formula, the specific calculation formulas for MSV and MSE are: Where M and N represent the number of rows and columns of the image, i.e., the height and width of the image, respectively; I(i, j) is the pixel value of the original image at (i, j); and K(i, j) is the pixel value of the image at (i, j) after being processed by the algorithm.

5. The method for image denoising and enhancement in indoor and outdoor scenes of low-light images according to claim 1, characterized in that: The calculation formula for the allocation weight in step 3 is as follows: In the formula, w enhanced and w denoised are the weight coefficients of the enhanced and denoised images in the final fused image, respectively.

6. The method for image denoising and enhancement in indoor and outdoor scenes of low-light images according to claim 1, characterized in that: The calculation formula for the fused image in step 3 is as follows: AND final =in enhanced ·AND enhanced +in denoised ·AND denoised , In the formula, I final is the fused image, I enhanced and I denoised They are the enhanced and denoised images respectively.

Images