Low illumination image enhancement method and device

Abstract

The present invention provides a low-light image enhancement method and device, which relates to the field of computer vision technology, including: separating the hue component, lightness component, and saturation component of the original image in HSV space; using the BM3D filtering algorithm as the center-surround function of the Retinex algorithm to estimate the illumination component and obtain the corresponding reflection component; enhancing the lightness component based on the illumination component; sharpening the reflection component using a Laplacian convolution kernel; extracting the edge texture of the original image and recording the pixel coordinates corresponding to the edge texture; sharpening the edge position in the reflection component using a Laplacian convolution kernel; weighted fusion of the reflection component and the lightness component to form a new lightness component; channel-fusion of the hue component, lightness component, and saturation component into a new image, and converting it back to RGB space. This application achieves brightness enhancement of low-light images while effectively improving image detail loss and color distortion.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a low-illumination image enhancement method and device. Background Art

[0002] With the rapid development of computer vision technology, image-based applications are becoming increasingly widespread, encompassing areas such as facial recognition, surveillance, industrial production, and medical examinations. As the underlying data in visual research, high-definition, high-quality image data is crucial for implementing various algorithms. However, due to factors such as ambient illumination and image acquisition equipment, captured images inevitably suffer from quality issues such as blurred details, uneven illumination, and low contrast, which directly restrict subsequent vision-related computations, analysis, and applications. Therefore, research on low-light image enhancement methods is crucial to advancing the theory of image information mining. Enhanced low-light images can lead to loss of detail, color distortion, and increased noise. Summary of the Invention

[0003] The technical problem to be solved by this application is to propose a low-light image enhancement method and device in response to the above-mentioned deficiencies in the prior art.

[0004] A low-illumination image enhancement method, comprising:

[0005] Separate the hue component, lightness component, and saturation component of the original image in the HSV space;

[0006] The BM3D filtering algorithm is used as the center-surround function of the Retinex algorithm to estimate the illumination component and obtain the corresponding reflection component;

[0007] Enhance the brightness component according to the illumination component;

[0008] Use the Laplacian convolution kernel to sharpen the reflected component;

[0009] Extract the edge texture of the original image and record the pixel coordinates corresponding to the edge texture; use the Laplace convolution kernel to sharpen the edge position in the reflection component;

[0010] Perform weighted fusion on the reflection component and the brightness component to form a new brightness component;

[0011] The hue component, lightness component, and saturation component are channel-fused into a new image and converted back to RGB space;

[0012] Remove noise from the new image.

[0013] Optionally, before adopting the BM3D filtering algorithm as the center-surround function of the Retinex algorithm to estimate the illumination component, the method further includes the step of optimizing the luminance component using a contrast-limited adaptive histogram equalization (CLAHE) algorithm.

[0014] Optionally, a BM3D filtering algorithm is used as the center-surround function of the Retinex algorithm to estimate the illumination component, including the steps of:

[0015] In the first stage, if there is a same pixel x in multiple similar blocks, the repeated pixel points x are weighted averaged and then aggregated to obtain the basic estimated image required in the second stage. The basic estimation formula is as follows:

[0016]

[0017]

[0018] Where y basic (x) is the basic estimation image in the basic estimation filtering stage, is the similarity group estimation value, which can be obtained by a series of transformations of the similarity three-dimensional array obtained by the similarity block measurement formula; γ is the threshold filtering operation; and are three-dimensional transformation and three-dimensional inverse transformation respectively, is the characteristic function of similar blocks, ω h is the number of non-zero coefficients N of the similar group after hard threshold shrinkage h The estimated value weight is calculated as follows:

[0019]

[0020] In the second stage, the image obtained by the basic estimation is divided into blocks again and estimated block by block. Finally, all the reference blocks in the previous stage are weighted and aggregated to obtain the final estimated image y final (x), the specific formula is as follows:

[0021]

[0022]

[0023] Where, T' h and T h are the three-dimensional matrix of the basic estimate and the three-dimensional matrix of the original image, Similarity group estimates for Wiener filtering, and is a three-dimensional linear transformation and a three-dimensional inverse transformation, is the Wiener filter shrinkage coefficient, is the weight coefficient of the Wiener filter, and its value is obtained by the noise standard deviation σ and the Wiener filter shrinkage coefficient.

[0024] Optionally, enhancing the brightness component according to the illumination component comprises the steps of:

[0025] Determine whether there are bright areas and dark areas in the image based on the illumination component;

[0026] When there are both bright and dark areas in the image, the improved Gamma transformation function is used to enhance the dark part of the luminance component, and maintain or reduce the brightness of the bright part of the luminance component according to the bright information, so as to enhance the brightness of the dark part while preserving the details of the bright part. The improved Gamma transformation function is:

[0027]

[0028] In the above formula, δ is the value of x when f(x) = 0, and a and b are the adjustment parameters of the function, which can jointly participate in adjusting and improving the enhancement amplitude and range of the pixel points by the Gamma function. The solution formulas of a and b are as follows:

[0029]

[0030] Where m is the normalized mean of pixels with values ​​lower than 97 in the illumination component.

[0031] Optionally, the method further comprises the steps of:

[0032] When the image only has dark areas, the adaptive gamma transform is used to enhance the brightness component;

[0033] Among them, the enhancement formula of adaptive Gamma transform is: f(x)=x γ , where f(x) is the output image, x is the normalized image, and γ is the image enhancement parameter;

[0034] The formula for automatic calculation of γ parameters is as follows:

[0035]

[0036] Where N is the mean of the grayscale image to be enhanced, n is the normalized mean, and γ is the image enhancement parameter.

[0037] Optionally, the determining whether there are bright areas and dark areas in the image according to the illumination component is specifically as follows:

[0038] Divide the grayscale values ​​of all pixels of the illumination component into 16 levels;

[0039] When the proportion of the current four-level pixels is greater than 10%, it is determined that there is a dark area in the image;

[0040] When the proportion of the last four levels of pixels is greater than 10%, it is determined that there is a bright area in the image.

[0041] Optionally, the extracting the edge texture of the original image is specifically:

[0042] The Gabor filter and Canny algorithm are used to extract the texture of the original image.

[0043] Optionally, the noise of the new image is eliminated by:

[0044] The image sharpened by the Laplacian convolution kernel is used as a guidance map for guided filtering to remove noise from the new image.

[0045] Optionally, before performing channel fusion on the hue component, the lightness component, and the saturation component to form a new image, the method further includes:

[0046] Adaptive Gamma is used to correct the saturation component; the enhancement parameter γ value is γ=0.85+0.2g; the enhancement formula of adaptive Gamma transformation is: f(x)=x γ , where f(x) is the output image, x is the normalized image, γ is the image enhancement parameter, and g is the adjustment parameter of the gamma transform.

[0047] On the other hand, the present application also provides a low-light image enhancement device, comprising:

[0048] Separation module, used to separate the hue component, lightness component, and saturation component of the original image in HSV space;

[0049] The estimation module is used to estimate the illumination component and obtain the corresponding reflection component using the BM3D filtering algorithm as the center-surround function of the Retinex algorithm;

[0050] An enhancement module, used to enhance the brightness component according to the illumination component;

[0051] A sharpening module for sharpening the reflection component using a Laplacian convolution kernel;

[0052] The extraction module is used to extract the edge texture of the original image and record the pixel coordinates corresponding to the edge texture; the Laplace convolution kernel is used to sharpen the edge position in the reflection component;

[0053] A fusion module, used for performing weighted fusion of the reflection component and the brightness component to form a new brightness component;

[0054] The conversion module is used to fuse the hue component, lightness component, and saturation component into a new image and convert it back to RGB space;

[0055] The elimination module is used to eliminate the noise of the new image.

[0056] The low-light image enhancement method proposed in this application is based on the improved Retinex algorithm in the HSV color space. The BM3D filtering algorithm is used as the center-surround function of the Retinex algorithm to extract the illumination component of the low-light image and obtain the reflection component. The brightness component is enhanced according to the illumination component, and the reflection component and its texture are sharpened. The reflection component and the brightness component are then weightedly fused to achieve brightness enhancement of the low-light image while effectively improving the image detail loss and color distortion. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 This is one of the flow charts of the low-light image enhancement method in the embodiment of the present application.

[0058] Figure 2 This is the second flowchart of the low-light image enhancement method in the embodiment of the present application.

[0059] Figure 3 This is the third flowchart of the low-illumination image enhancement method in the embodiment of the present application.

[0060] Figure 4 It is an image component diagram in the embodiment of the present application.

[0061] Figure 5 It is an overall diagram of the Gamma transformation in the embodiment of the present application.

[0062] Figure 6 This is a partial diagram of the Gamma transformation in the embodiment of the present application.

[0063] Figure 7 This is a schematic diagram of the original image after the brightness component is enhanced in an embodiment of the present application.

[0064] Figure 8 It is the reflection component after sharpening in the embodiment of the present application.

[0065] Figure 9 It is a luminance component map after edge detection and weighted fusion in the embodiment of the present application.

[0066] Figure 10 This is the overall flow chart of the improved Retinex algorithm in the embodiment of this application.

[0067] Figure 11 3 is a comparison chart of four groups of experimental images in the examples of this application.

[0068] Figure 12 It is an indicator analysis chart of the experiment in the embodiment of this application.

[0069] Figure 13 It is a structural block diagram of the low-illumination image enhancement device in an embodiment of the present application. DETAILED DESCRIPTION

[0070] The following are specific embodiments of the present application and in conjunction with the accompanying drawings, the technical scheme of the present application is further described, but the application is not limited to these embodiments. In the following description, specific details such as specific configurations and components are provided only to help fully understand the embodiments of the present application. Therefore, it should be clear to those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. In addition, for clarity and brevity, the description of known functions and structures has been omitted.

[0071] It should be noted that, unless there is any conflict, the embodiments and features in the embodiments of this application can be combined with each other.

[0072] refer to Figure 1 The present invention provides a low-light image enhancement method for enhancing low-light images. The method is applied to low-light images to enhance the brightness of the low-light images while effectively reducing image detail loss and color distortion. The method includes steps S101 to S108, which are described in detail below with reference to the accompanying drawings.

[0073] Step S101 : Separate the hue component, lightness component, and saturation component of the original image in the HSV space.

[0074] Specifically, each color in the HSV space is represented by hue, saturation, and brightness. The color parameters in the HSV space are hue H, saturation S, and brightness V. In the HSV space, the hue H, saturation S, and brightness V channels of the original image are separated.

[0075] Step S102 : Using the BM3D filtering algorithm as the center-surround function of the Retinex algorithm, the illumination component is estimated and the corresponding reflection component is obtained.

[0076] Specifically, Retinex theory is based on the properties of color constancy and illumination invariance. It posits that the color and brightness of an object perceived by the human visual system (HVS) depend on the reflective properties of the object's surface. An image can be considered to consist of the object's own reflective component, L, and illumination component, R. The mathematical model based on this is shown below:

[0077] S(x,y)=R(x,y)×L(x,y)

[0078] Where S(x,y) represents the original image, R(x,y) is the reflection component, and L(x,y) is the illumination component.

[0079]

[0080] logR(x,y)=logS(x,y)-log[G(x,y)·S(x,y)]

[0081] In the above formula, G(x,y) is a Gaussian function and σ is a scale parameter. G(x,y) is used to perform a convolution operation on S(x,y) to estimate L(x,y). Then, the logarithm of both sides of the equation is taken to separate R(x,y) and L(x,y). The result is quantized to a pixel value in the range of [0,255] according to formula (3):

[0082] R(x,y)=(Value-Min) / (Max-Min)*255

[0083] Val ue is the current pixel value, Max is the maximum pixel value in the image, and Min is the minimum pixel value in the image. The Retinex algorithm uses R(x,y) as the enhanced image, and its enhancement effect cannot take into account both the details and color information of the image at the same time.

[0084] In one embodiment of the present application, the BM3D filtering algorithm is used as the center-surround function of the Retinex algorithm to estimate the illumination component, including a first stage and a second stage.

[0085] Specifically, the 3D block matching and 3D filtering algorithm (BM3D) is a 3D filtering algorithm based on the non-local mean filter (NLM). Images typically contain many similar and repetitive structures. The BM3D algorithm uses block matching to stack similar structural blocks into a 3D matrix, then performs collaborative filtering and aggregates the processed blocks to the original image block. The metric used to find similar blocks is as follows:

[0086]

[0087] Where Z x represents the sliding window during the search process, Z xR For a reference point X R The reference block at ,γ is a hard threshold filtering operation, which sets the value below the threshold to 0, is the two-dimensional discrete cosine transform, and M is the image block size.

[0088] The BM3D implementation process mainly includes two stages: basic estimation filtering and final estimation filtering. Both stages include several steps such as similar block matching estimation and grouping, 3D collaborative filtering, and aggregation weighting. The difference is that the first stage of collaborative filtering uses 3D collaborative hard threshold filtering, while the second stage uses 3D collaborative Wiener filtering.

[0089] In the first stage, if there is a same pixel x in multiple similar blocks, the repeated pixel points x are weighted averaged and then aggregated to obtain the basic estimated image required in the second stage. The basic estimation formula is as follows:

[0090]

[0091]

[0092] Where y basic (x) is the basic estimation image in the basic estimation filtering stage, is the similarity group estimation value, which can be obtained by a series of transformations of the similarity three-dimensional array obtained by the similarity block measurement formula; γ is the threshold filtering operation; and are three-dimensional transformation and three-dimensional inverse transformation respectively, is the characteristic function of similar blocks, ω h is the number of non-zero coefficients N of the similar group after hard threshold shrinkage h The estimated value weight is calculated as follows:

[0093]

[0094] In the second stage, the image obtained by the basic estimation is divided into blocks again and estimated block by block. Finally, all the reference blocks in the previous stage are weighted and aggregated to obtain the final estimated image y final (x), the specific formula is as follows:

[0095]

[0096]

[0097] Where, T' h and T h are the three-dimensional matrix of the basic estimate and the three-dimensional matrix of the original image, Similarity group estimates for Wiener filtering, and is a three-dimensional linear transformation and a three-dimensional inverse transformation, is the Wiener filter shrinkage coefficient, is the weight coefficient of the Wiener filter, and its value is obtained by the noise standard deviation σ and the Wiener filter shrinkage coefficient.

[0098] Compared with the Gaussian filter used by the Retinex algorithm, which only performs calculations in the image spatial domain, the BM3D algorithm also uses 3D orthogonal transformation in its implementation process to filter the image in the transform domain, making full use of the correlation within the image blocks and the correlation between image blocks, so that it can fully preserve the unique structure and detail information of the image while performing image denoising.

[0099] In one embodiment of the present application, before using the BM3D filtering algorithm as the center-surround function of the Retinex algorithm to estimate the illumination component, the step is further included: using the contrast-limited adaptive histogram equalization CLAHE algorithm to optimize the brightness component.

[0100] Specifically, the HSV color space model is a nonlinear transformation of the RGB model. The original RGB image is converted to the HSV space, and its H, S, and V channels are separated to obtain the brightness component V of the original image. CLAHE is used to equalize the brightness component to improve the contrast of the image, so that the BM3D algorithm can retain more detail information when extracting the illumination component. The brightness components before and after correction are as follows: Figure 4 b, Figure 4 c. Then, the noise standard deviation σ of the BM3D algorithm is set to 9, the hard threshold is set to 24.3, and the block similarity thresholds of the first and second stages are set to 1500 and 800 respectively. After extracting the illumination component, the reflection component is calculated in the logarithmic domain according to the principle of the Retinex algorithm, and then the reflection component is quantized to the [0,255] pixel range. The reflection component diagram is shown in FIG. Figure 4 (d) shown.

[0101] Step S103: enhance the brightness component according to the illumination component.

[0102] refer to Figure 2 In a specific implementation, step S103, enhancing the brightness component according to the illumination component, includes steps S1031 to S1033.

[0103] Step S1031 , determining whether there are bright areas and dark areas in the image according to the illumination component.

[0104] refer to Figure 3 In a specific implementation, step S1031 includes steps S1031a to S1031c.

[0105] Step S1031a: Divide the grayscale values ​​of all pixels of the illumination component into 16 levels.

[0106] Step S1031b: when the proportion of the current fourth-level pixels is greater than 10%, it is determined that there is a dark area in the image.

[0107] Step S1031c: when the proportion of the last four levels of pixels is greater than 10%, it is determined that there is a bright area in the image.

[0108] Specifically, the grayscale value of the illumination component L is divided into 16 levels. When the proportion of pixels in the first four levels of L is greater than 10%, it is determined that there is a dark area in the image, and when the proportion of pixels in the last four levels is greater than 10%, it is determined that there is a bright area in the image.

[0109] In step S1032, when both bright and dark areas exist in the image, an improved Gamma transformation function is used to enhance the dark areas of the luminance component, and the brightness of the bright areas of the luminance component is maintained or lowered based on the bright area information, thereby enhancing the brightness of the dark areas while preserving the details of the bright areas. The improved Gamma transformation function is:

[0110]

[0111] In the above formula, δ is the value of x when f(x) = 0, and a and b are the adjustment parameters of the function, which can jointly participate in adjusting and improving the enhancement amplitude and range of the pixel points by the Gamma function. The solution formulas of a and b are as follows:

[0112]

[0113] Where m is the normalized mean of pixels with values ​​lower than 97 in the illumination component.

[0114] Step S1033: When the image only has dark areas, the brightness component is enhanced using adaptive gamma transform;

[0115] Among them, the enhancement formula of adaptive Gamma transform is: f(x)=x γ , where f(x) is the output image, x is the normalized image, and γ is the image enhancement parameter;

[0116] The formula for automatic calculation of γ parameters is as follows:

[0117]

[0118] Where N is the mean of the grayscale image to be enhanced, n is the normalized mean, and γ is the image enhancement parameter.

[0119] Specifically, the image category is judged based on the light and dark information of L and the parameter g value is calculated. When the image only has dark parts, the algorithm uses adaptive Gamma to enhance the brightness component. At this time, the g value is the γ value; when the image has both light and dark information, the a and b values ​​are calculated based on g. At this time, if the image is not overexposed, the improved Gamma will be used to enhance the dark part of V and maintain the detail information of the bright part. If there is overexposure, the brightness of the bright part will be lowered.

[0120] Gamma transformation is effective for enhancing low-light images. However, when an image contains both highlights and lowlights due to backlighting, the traditional gamma transformation can only enhance the lowlights while sacrificing detail in the highlights, or reduce the highlights, making the lowlights darker and also causing loss of detail. Adaptive gamma transformation, based on the theory that the mean value of all normalized pixels in a reasonable image should be around 0.5, proposes a method for automatically calculating and selecting the gamma parameter. While adaptive gamma enables automatic selection of enhancement parameters, it does not fundamentally address the problems inherent in gamma transformation.

[0121] Furthermore, the improved Gamma transform method can simultaneously enhance the dark areas of an image while adjusting the highlights, potentially enhancing both low-light and high-light areas. When an image contains both light and dark information, the improved Gamma transform function can be used to enhance the dark areas of the image while maintaining or lowering the brightness of the luminance component based on the bright areas of the image. This enhances the brightness of the dark areas while preserving the details of the bright areas.

[0122] The backlit low-light images in the SYN dataset are enhanced using traditional Gamma transform and improved Gamma transform. The correction results and grayscale histograms are shown in Figure 2. Figure 5 As shown. Figure 5 It can be seen intuitively that the improved Gamma transform has a very significant effect on the enhancement of the dark part of the image, and the traditional Gamma transform has a very significant effect. Both methods distribute the pixels in the dark area of ​​the image approximately evenly to a higher pixel level, so that the main peak of the grayscale value in the dark area of ​​the image moves from about 10 to about 50. However, since the Gamma transform can only enhance the image in one direction, Figure 5 (e, f) It can be clearly seen that the bright pixel concentration area of ​​the original image is concentrated to a higher pixel level after being enhanced by the traditional Gamma transform, making the bright part of the image brighter.

[0123] In order to better see the changes in the details of the bright part of the image, a part of the bright part of the image is cut out. Figure 6 (a, b, c, d) and calculate the corresponding grayscale histogram Figure 6 (e, f, g, h), directly observing the intercepted bright part of the picture, we can see that the use of the traditional Gamma transformation method will cause some loss of details in the bright part of the image, which is reflected in the histogram that the grayscale value concentration area changes from 210-255 to 230-255. The improved Gamma transformation only makes a minor correction to the bright part of the image, and its grayscale value concentration area hardly changes, and the image details are preserved intact. Figure 5 (a, c, e, g) It can be seen that the adaptive gamma transformation only makes a very small enhancement to the original image. This is related to the automatic selection of the γ value. Since the pixels in the dark and bright parts of the image are relatively concentrated, and the pixel mean is relatively close to 0.5, the automatically calculated γ value is not suitable for this image. Figure 5 (d, h) and Figure 6 As can be seen in (d, h), the improved gamma transform enhances both dark and bright areas of the image, with the dark areas receiving a greater enhancement, significantly increasing their brightness. The pixels in the image are evenly distributed from 0-50 to 50-150 after enhancement, while the bright areas receive only a slight enhancement, resulting in a more balanced pixel distribution. The above analysis shows that compared to traditional gamma and adaptive gamma transform enhancement methods, the improved gamma transform achieves a more balanced enhancement effect for images with both dark and bright areas, preserving more complete details.

[0124] Furthermore, based on the g value calculated from the illumination component, the improved Gamma transform is used to enhance the brightness component of the original image. The enhanced result is as follows: Figure 7 shown.

[0125] Step S104: sharpen the reflection component using a Laplacian convolution kernel.

[0126] Specifically, image sharpening is a way to compensate for the image contour, which can enhance the edges, textures, certain linear target elements and grayscale jump parts in the image. Image sharpening can generally be divided into two steps: edge detection and edge enhancement. The result of edge detection directly affects the quality of image sharpening. If the edge detection result contains a large number of false edges, not only can the texture in the image not be enhanced, but it will also cause a large amount of noise, which reduces the image quality. Unsharp Masking (UM) is a commonly used image sharpening method. It combines Gaussian blur and Laplacian operators. It first uses a Gaussian smoother to filter the low-frequency part of the image, and then uses the Laplacian convolution kernel to enhance the high-frequency image to achieve the purpose of image sharpening. This application draws on the idea of ​​the UM algorithm, uses the BM3D algorithm to extract the low-frequency part of the image brightness component, and calculates the high-frequency image in the logarithmic domain, that is, the reflection component of the image obtained by the improved Retinex algorithm. Subsequently, the Laplacian convolution kernel is used to sharpen the reflection component. The reflection component after sharpening by the Laplacian convolution kernel is as follows. Figure 8 As shown. Figure 8 It can be clearly seen that the edges of the image have been greatly enhanced. However, since the Laplace operator sharpens the entire image, there are a large number of false edges in the edge detection results, resulting in more noise in the image. Therefore, this application adopts a multi-weight fusion method to reduce the negative effects of image sharpening.

[0127] Step S105 , extracting the edge texture of the original image and recording the pixel coordinates corresponding to the edge texture; and performing a sharpening operation on the edge position in the reflection component using a Laplacian convolution kernel.

[0128] Step S106: weighted fusion of the reflection component and the brightness component to form a new brightness component.

[0129] In one embodiment of the present application, step S105 , extracting the edge texture of the original image, specifically involves extracting the texture of the original image using a Gabor filter and a Canny algorithm.

[0130] Since low-weight sharpening images will weaken the enhancement effect of edge texture, this application combines Gabor filter and Canny edge detection algorithm to extract the edge of the original image. Gabor transform is a windowed short-time Fourier transform, which is extremely robust and can extract edge features of the image in the frequency domain; Canny edge detection algorithm is a multi-level detection algorithm, which is extremely effective in image edge detection. After extracting the edge texture of the image, the Laplace convolution kernel is used to sharpen only the extracted edge position, and finally, the reflection component and the brightness component are weightedly fused. The edge detection map and the enhanced reflection component are shown in Figure 2. Figure 9 shown.

[0131] Step S107: perform channel fusion on the hue component, the lightness component, and the saturation component to form a new image, and convert the new image back to the RGB space.

[0132] Step S108: Eliminate noise from the new image.

[0133] In one embodiment of the present application, the noise of the new image is eliminated, specifically by using an image sharpened by a Laplacian convolution kernel as a guide image for guided filtering to eliminate the noise of the new image.

[0134] Specifically, the enhanced reflection and illumination components of the image are recombined into a lightness component (V), and the saturation (S) and hue (H) channels are fused. The image is then converted back to RGB channels. Subsequently, the image, sharpened using a Laplacian convolution kernel, is used as a guide map for guided filtering to remove image noise. Guided filtering is an adaptive weighted filtering method that is particularly effective in smoothing images and preserving boundaries. It can effectively eliminate the noise effects of image enhancement and sharpening.

[0135] In one embodiment of the present application, before the hue component, the lightness component, and the saturation component are channel-fused into a new image, the method further includes: correcting the saturation component using adaptive Gamma; the enhancement parameter γ value is γ=0.85+0.2g; the enhancement formula of the adaptive Gamma transformation is: f(x)=x γ , where f(x) is the output image, x is the normalized image, γ is the image enhancement parameter, and g is the adjustment parameter of the gamma transform.

[0136] In the embodiment of this application, the Retinex algorithm has a good effect in maintaining color perception, so it is highly favored in low-quality image enhancement, and a variety of improved methods have been developed. The main direction of the Retinex algorithm improvement is to combine the S-curve function, color space conversion, and improved filtering. The process of the improved algorithm for low-light image enhancement is as follows: Figure 10As shown, the hue H, saturation S, and lightness V channels are separated in the HSV space; CLAHE is used to equalize V, and then the BM3D algorithm is used to estimate the illumination component L of the image and obtain the reflection component R. In order to give full play to the advantages of the improved algorithm, after extracting the L component, this application will analyze the image category. First, the grayscale value of L is divided into 16 levels. When the proportion of pixels in the first four levels of L is greater than 10%, it is determined that there is a dark area in the image, and when the proportion of pixels in the last four levels is greater than 10%, it is determined that there is a bright area in the image. Subsequently, the standard deviation of the number of grayscale values ​​in the last four levels is solved. When the standard deviation of the number of grayscale values ​​in the last level is twice that of the previous level, it is determined that the bright part of the image is overexposed. In the face of the above different situations, different methods will be selected to enhance the image. The image category is determined based on the light and dark information of L and the parameter g value is calculated. When the image only has dark parts, the algorithm uses adaptive Gamma to enhance the brightness component. At this time, the g value is the γ value. When the image has both light and dark information, the a and b values ​​are calculated based on g. At this time, if the image is not overexposed, the improved Gamma will be used to enhance the dark part of V and maintain the detail information of the bright part. If there is overexposure, the brightness of the bright part will be lowered. At the same time, the Gabor filter and Canny algorithm are used to extract the texture of the original image and record its texture pixel coordinates. The Laplacian operator is used to sharpen the reflection component. In addition, the pixels at the texture position in the reflection component are sharpened again. Then, the enhanced V and R are fused with weights of 0.7 and 0.3 to form a new V. For the saturation component S, this application designs an adaptive Gamma enhancement method. Its enhancement parameter γ value is 0.85 plus 0.2g. The lower the image brightness, the stronger the saturation enhancement. Finally, the hue component H, the enhanced lightness component V, and the corrected saturation component S are channel-fused and converted back to RGB space, and the image noise is eliminated using guided filtering.

[0137] Therefore, to address the problems of detail loss, color distortion, and increased noise after enhancing low-light images, this application proposes an improved Retinex algorithm based on the HSV color space. This algorithm uses the BM3D filtering algorithm as the center-surround function of the Retinex algorithm to extract the illumination component of the low-light image and obtain the reflection component. The correction parameters are calculated based on the illumination component, and the improved Gamma transform is used to correct the brightness and saturation components. The reflection component and its texture are sharpened, thereby enhancing the brightness of the low-light image while effectively improving the image's detail loss and color distortion.

[0138] To verify the effectiveness and practicality of our method, we conducted enhancement experiments using image data from the low-light image datasets SYN and LOL. These datasets contain a rich variety of real-world scene images, fully meeting the experimental requirements. The LOL dataset contains 500 low-light / normal-light image pairs. The original images were resized to 400×600 and converted to the Portable Network Graphics format, produced by Peking University. SYN is a synthetic dataset created by the Peking University team using 1,000 images randomly sampled from the RAISE dataset. 200 multi-scene low-light images were randomly selected from the dataset as experimental data. The image types include 100 images with only dark information and 100 images with coexistence of light and dark information. The image scenes include indoor, outdoor, natural and urban landscapes. The effectiveness of the improved method can be verified from different angles, and the ACE algorithm, LIME algorithm, MSRCR algorithm and the method of this application are used to enhance the extracted low-light images. The experiment was carried out on the Pycharm 2021, Windows 10, CPU Intel Core i9-10900, RAM 32G test platform. The experimental data enhancement effect is as follows: Figure 11 shown.

[0139] Specifically, the five algorithms all have a certain improvement on the image brightness of different scenes. The ACE algorithm performs well in scenes one and three, but performs poorly in scenes two and four. The brightness improvement of the dark part of the image is limited, and the bright part is over-enhanced, the image appears dark, and in scene four, the enhanced image color has a slight distortion phenomenon; the ALTM and LIME algorithms have similar enhancement effects, and have a good enhancement effect on the overall dark image. When there is both light and dark information in the enhanced image, the ALTM image enhancement effect is not obvious, such as in scenes two and four, while LIME will over-enhance the bright part information of the image, resulting in detail loss, such as in scenes two and four, and the algorithm will increase the image contrast, resulting in some dark part information enhancement is not obvious; the MSRCR algorithm has a good effect on improving the image brightness, and the image detail loss is small, but its color restoration is poor, and the image is overall gray. The enhancement effect of the algorithm of this application is relatively stable, and the overall brightness distribution is uniform. The image details are relatively complete and the image is clear. The image color is comfortable and natural, which is more in line with the characteristics of the human eye.

[0140] The advantage of using low-light dataset images in this application is that these datasets not only have a wealth of low-light images, but also have normal-light images that match the low-light images, which can provide strong support for the evaluation of enhanced images. In order to objectively evaluate the enhancement effect of each method on low-light images, the enhanced two hundred image data are calculated. This application will use information entropy (IE), peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), image quality index (UQI), average gradient (AG), and root mean square error (RMSE) as measurement indicators. Among them, the larger the IE, PSNR, SSIM, and AG values, the richer the enhanced image information and the more similar it is to the normal-light image; the larger the UQI, the higher the quality of the enhanced image; RMSE reflects the difference between the enhanced image and the normal-light image. The smaller the value, the smaller the difference. In order to clearly present each indicator, this application adjusts all their values ​​to between 0 and 10. The IE value will remain unchanged, the PSNR will be divided by 2, the SSIM and UQI values ​​will be multiplied by 10, and the AG and RMSE will be divided by 10 and 1000 respectively. At the same time, in order to verify the enhancement effect of the improved Gamma method and texture enhancement method on the image with coexistence of light and dark information, the experiment using the original Gamma and no texture enhancement was set as the control group. The experimental data was 100 extracted images with coexistence of light and dark information. The various index data are as follows Figure 12 shown.

[0141] Figure 12 In (a), the solid line with circular dots is the image evaluation data calculated by the method proposed in this application, and the LOW line is the data calculated from the low-light original image and its corresponding normal-light image. Figure 12 As can be seen in (a), the proposed method is significantly superior to other methods in terms of PSNR, SSIM, RMSE, and UQI, indicating that the enhanced image is highly similar to the normal illumination image, has good image quality, and is more consistent with human judgment of visual quality. IE and AG data are also at a relatively high level among all methods. The average IE value is as high as 7.285, the average PSNR value is 16.175dB, which is an average improvement of 6.475dB compared to the original image. The average SSIM value is 0.580, and the average QUI value is 0.8318, which is an improvement of 0.593 compared to the original image. This shows that the image quality has been greatly improved after enhancement, and the average AG value is as high as 91.407. Figure 12In (b), the circular points are the data calculated by using the improved method to enhance the image, and the square points are the data calculated by using the adaptive Gamma to enhance the image. It can be seen from the figure that the mean information entropy of the image enhanced by the improved algorithm reaches 7.462, which is 5.217% higher than that using the adaptive Gamma. The PSNR mean is 16.816dB, which is 0.567dB higher. The SSIM is slightly lower than the adaptive Gamma. The UQI mean reaches 0.839, which is 5.534% higher. The AG mean reaches 96.907, which is 36.911 higher than that using the adaptive Gamma. In terms of RMSE, it is also significantly better than the method using the adaptive Gamma, indicating that the image enhanced by the improved method is closer to the normal illumination image. In summary, the algorithm proposed in this application has a relatively excellent effect in improving the brightness of low-illuminance images, and can effectively suppress phenomena such as brightness oversaturation, preserve the edge details of the image intact, and restore comfortable and natural colors.

[0142] In summary, to address the low quality of feature information such as brightness, color, and detail in low-light images, this application proposes an improved Retinex low-light image enhancement algorithm. Based on the HSV color space, the algorithm corrects the saturation component S and the lightness component V, extracts the illumination component using the BM3D algorithm, and calculates the reflectance component. An improved Gamma transformation function is proposed to enhance the lightness component. An image texture enhancement method is proposed to sharpen the image and its texture. Finally, the image is multi-weighted fused and denoised using guided filtering. Experimental results show that the algorithm has a good enhancement effect on a variety of low-light scene images, with intact image edge details and natural color restoration. Compared with the original image, the mean IE value of the enhanced image is improved by 1.602, the mean PSNR is improved by 6.475dB, the mean SSIM is improved by 0.291, the mean UQI and AG are significantly improved, and the mean RMSE is significantly reduced, which is a significant improvement compared to other image enhancement algorithms.

[0143] refer to Figure 13 The embodiment of the present application also provides a low-illumination image enhancement device, including: a separation module 1301, an estimation module 1302, an enhancement module 1303, a sharpening module 1304, an extraction module 1305, a fusion module 1306, a conversion module 1307, and an elimination module 1308.

[0144] The separation module 1301 is used to separate the hue component, lightness component and saturation component of the original image in the HSV space.

[0145] The estimation module 1302 is configured to use the BM3D filtering algorithm as the center-surround function of the Retinex algorithm to estimate the illumination component and obtain the corresponding reflection component.

[0146] The enhancement module 1303 is configured to enhance the brightness component according to the illumination component.

[0147] The sharpening module 1304 is used to sharpen the reflection component using a Laplacian convolution kernel.

[0148] The extraction module 1305 is used to extract the edge texture of the original image and record the pixel coordinates corresponding to the edge texture; and use the Laplace convolution kernel to perform a sharpening operation on the edge position in the reflection component.

[0149] The fusion module 1306 is used to perform weighted fusion on the reflection component and the brightness component to form a new brightness component.

[0150] The conversion module 1307 is used to perform channel fusion on the hue component, the lightness component, and the saturation component to form a new image, and convert the new image back to the RGB space.

[0151] The elimination module 1308 is used to eliminate the noise of the new image.

[0152] In one embodiment, an optimization module is further included for optimizing the luminance component using a contrast-limited adaptive histogram equalization (CLAHE) algorithm before estimating the illumination component using the BM3D filtering algorithm as the center-surround function of the Retinex algorithm.

[0153] In one embodiment, the estimation module 1302 is further configured to use the BM3D filtering algorithm as the center-surround function of the Retinex algorithm to estimate the illumination component, including the steps of:

[0154] In the first stage, if there is a same pixel x in multiple similar blocks, the repeated pixel points x are weighted averaged and then aggregated to obtain the basic estimated image required in the second stage. The basic estimation formula is as follows:

[0155]

[0156]

[0157] Where y basic (x) is the basic estimation image in the basic estimation filtering stage, is the similarity group estimation value, which can be obtained by a series of transformations of the similarity three-dimensional array obtained by the similarity block measurement formula; γ is the threshold filtering operation; and are three-dimensional transformation and three-dimensional inverse transformation respectively, is the characteristic function of similar blocks, ω h is the number of non-zero coefficients N of the similar group after hard threshold shrinkage hThe estimated value weight is calculated as follows:

[0158]

[0159] In the second stage, the image obtained by the basic estimation is divided into blocks again and estimated block by block. Finally, all the reference blocks in the previous stage are weighted and aggregated to obtain the final estimated image y final (x), the specific formula is as follows:

[0160]

[0161]

[0162] Where, T' h and T h are the three-dimensional matrix of the basic estimate and the three-dimensional matrix of the original image, Similarity group estimates for Wiener filtering, and is a three-dimensional linear transformation and a three-dimensional inverse transformation, is the Wiener filter shrinkage coefficient, is the weight coefficient of the Wiener filter, and its value is obtained by the noise standard deviation σ and the Wiener filter shrinkage coefficient.

[0163] In one embodiment, the enhancement module 1303 is further configured to:

[0164] Determine whether there are bright areas and dark areas in the image based on the illumination component;

[0165] When there are both bright and dark areas in the image, the improved Gamma transformation function is used to enhance the dark part of the luminance component, and maintain or reduce the brightness of the bright part of the luminance component according to the bright information, so as to enhance the brightness of the dark part while preserving the details of the bright part. The improved Gamma transformation function is:

[0166]

[0167] In the above formula, δ is the value of x when f(x) = 0, and a and b are the adjustment parameters of the function, which can jointly participate in adjusting and improving the enhancement amplitude and range of the pixel points by the Gamma function. The solution formulas of a and b are as follows:

[0168]

[0169] Where m is the normalized mean of pixels with values ​​lower than 97 in the illumination component.

[0170] In one embodiment, the enhancement module 1303 is further configured to: when the image only has dark areas, enhance the brightness component using adaptive gamma transform;

[0171] Among them, the enhancement formula of adaptive Gamma transform is: f(x)=x γ , where f(x) is the output image, x is the normalized image, and γ is the image enhancement parameter;

[0172] The formula for automatic calculation of γ parameters is as follows:

[0173]

[0174] Where N is the mean of the grayscale image to be enhanced, n is the normalized mean, and γ is the image enhancement parameter.

[0175] In one embodiment, the enhancement module 1303 is further configured to:

[0176] Divide the grayscale values ​​of all pixels of the illumination component into 16 levels;

[0177] When the proportion of the current four-level pixels is greater than 10%, it is determined that there is a dark area in the image;

[0178] When the proportion of the last four levels of pixels is greater than 10%, it is determined that there is a bright area in the image.

[0179] In one embodiment, the extraction module 1305 is further configured to:

[0180] The Gabor filter and Canny algorithm are used to extract the texture of the original image.

[0181] In one embodiment, the elimination module 1308 is further configured to:

[0182] The image sharpened by the Laplacian convolution kernel is used as a guidance map for guided filtering to remove noise from the new image.

[0183] In one embodiment, a correction module is further included for correcting the saturation component using adaptive gamma before channel fusion of the hue component, the lightness component, and the saturation component into a new image; the enhancement parameter γ value is γ=0.85+0.2g; the enhancement formula of the adaptive gamma transformation is: f(x)=x γ , where f(x) is the output image, x is the normalized image, γ is the image enhancement parameter, and g is the adjustment parameter of the gamma transform.

[0184] The low-illumination image enhancement device provided in the embodiment of the present application can implement the technical solution shown in the above method embodiment. Its implementation principles and beneficial effects are similar and will not be repeated here.

[0185] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0186] The specific embodiments described in this application are merely illustrative of the spirit of this application. Those skilled in the art may make various modifications, additions, or substitute similar methods to the described specific embodiments without departing from the spirit of this application or exceeding the scope defined by the appended claims.

Claims

1. A low-light image enhancement method, characterized in that: include: Separate the hue component, lightness component, and saturation component of the original image in the HSV space; The BM3D filtering algorithm is used as the center-surround function of the Retinex algorithm to estimate the illumination component and obtain the corresponding reflection component; Enhance the brightness component according to the illumination component; Use the Laplacian convolution kernel to sharpen the reflected component; Extract the edge texture of the original image and record the pixel coordinates corresponding to the edge texture; Use the Laplacian convolution kernel to sharpen the edge positions in the reflection component; Perform weighted fusion on the reflection component and the brightness component to form a new brightness component; The hue component, lightness component, and saturation component are channel-fused into a new image and converted back to RGB space; Eliminate noise from the new image; The step of enhancing the brightness component according to the illumination component includes the following steps: Determine whether there are bright areas and dark areas in the image based on the illumination component; When there are both bright and dark areas in the image, the improved Gamma transformation function is used to enhance the dark part of the luminance component, and maintain or reduce the brightness of the bright part of the luminance component according to the bright information, so as to enhance the brightness of the dark part while preserving the details of the bright part. The improved Gamma transformation function is: In the above formula, δ is the value of x when f(x) = 0, and a and b are the adjustment parameters of the function, which can jointly participate in adjusting and improving the enhancement amplitude and range of the pixel points by the Gamma function. The solution formulas of a and b are as follows: Where m is the normalized mean of pixels with values ​​lower than 97 in the illumination component; When the image only has dark areas, the adaptive gamma transform is used to enhance the brightness component; Among them, the enhancement formula of adaptive Gamma transform is: f(x)=x γ , where f(x) is the output image, x is the normalized image, and γ is the image enhancement parameter; The formula for automatic calculation of γ parameters is as follows: Where N is the mean of the grayscale image to be enhanced, n is the normalized mean, and γ is the image enhancement parameter.

2. The low-light image enhancement method according to claim 1, wherein: Before adopting the BM3D filtering algorithm as the center-surround function of the Retinex algorithm to estimate the illumination component, the method further includes the steps of optimizing the brightness component using the contrast-limited adaptive histogram equalization (CLAHE) algorithm.

3. The low-light image enhancement method according to claim 1, wherein: The method of judging whether there are bright areas and dark areas in the image according to the illumination component is specifically as follows: Divide the grayscale values ​​of all pixels of the illumination component into 16 levels; When the proportion of the current four-level pixels is greater than 10%, it is determined that there is a dark area in the image; When the proportion of the last four levels of pixels is greater than 10%, it is determined that there is a bright area in the image.

4. The low-light image enhancement method according to claim 1, wherein: The edge texture of the original image is extracted, specifically: The Gabor filter and Canny algorithm are used to extract the texture of the original image.

5. The low-light image enhancement method according to claim 1, wherein: The noise of the new image is eliminated, specifically: The image sharpened by the Laplacian convolution kernel is used as a guidance map for guided filtering to remove noise from the new image.

6. The low-light image enhancement method according to claim 1, wherein: Before the hue component, lightness component, and saturation component are fused into a new image, the following steps are also included: Adaptive Gamma is used to correct the saturation component; the enhancement parameter γ value is γ=0.85+0.2g; the enhancement formula of adaptive Gamma transformation is: f(x)=x γ , where f(x) is the output image, x is the normalized image, γ is the image enhancement parameter, and g is the adjustment parameter of the gamma transform.

7. A low-light image enhancement device, characterized in that: include: Separation module, used to separate the hue component, lightness component, and saturation component of the original image in HSV space; The estimation module is used to estimate the illumination component and obtain the corresponding reflection component using the BM3D filtering algorithm as the center-surround function of the Retinex algorithm; An enhancement module, used to enhance the brightness component according to the illumination component; Specifically, the method includes: judging whether there are bright areas and dark areas in the image according to the illumination component; when both bright areas and dark areas exist in the image, the improved Gamma transformation function is used to enhance the dark area of ​​the luminance component, and the brightness of the bright area of ​​the luminance component is maintained or lowered according to the bright area information, thereby enhancing the brightness of the dark area while preserving the details of the bright area. The improved Gamma transformation function is: In the above formula, δ is the value of x when f(x) = 0, and a and b are the adjustment parameters of the function, which can jointly participate in adjusting and improving the enhancement amplitude and range of the pixel points by the Gamma function. The solution formulas of a and b are as follows: Where m is the normalized mean of pixels with values ​​lower than 97 in the illumination component; When the image only has dark areas, the adaptive gamma transform is used to enhance the brightness component; Among them, the enhancement formula of adaptive Gamma transform is: f(x)=x γ , where f(x) is the output image, x is the normalized image, and γ is the image enhancement parameter; The formula for automatic calculation of γ parameters is as follows: Where N is the mean of the grayscale image to be enhanced, n is the normalized mean, and γ is the image enhancement parameter; A sharpening module for sharpening the reflection component using a Laplacian convolution kernel; The extraction module is used to extract the edge texture of the original image and record the pixel coordinates corresponding to the edge texture; the Laplace convolution kernel is used to sharpen the edge position in the reflection component; A fusion module, used for performing weighted fusion of the reflection component and the brightness component to form a new brightness component; The conversion module is used to fuse the hue component, lightness component, and saturation component into a new image and convert it back to RGB space; The elimination module is used to eliminate the noise of the new image.

Images