Low-light image enhancement method based on multi-exposure and multi-scale recursive fusion

Abstract

The present invention discloses a low-light image enhancement method based on multi-exposure and multi-scale recursive fusion. It proposes a multi-exposure feature recursive fusion module block and a corresponding fusion strategy to retain the feature information in the original image and adaptively fuse the outputs of different exposure branches. The multi-exposure generation block is used to produce outputs of multiple exposures, and the brightness information of the normal light image is statistically analyzed to form a probability distribution. A global scaling factor is randomly extracted from the probability distribution to perform random exposure enhancement on the original input each time. The multi-exposure generation block is used to obtain outputs of different exposure levels of the original image. The outputs of different exposure levels are recursively enhanced from low resolution to full resolution using a multi-scale feature enhancement network. The enhanced results are fine-tuned to obtain high-quality enhanced output. It can also solve the problems of insufficient brightness enhancement, low contrast, light color, and noise amplification during the restoration process during image enhancement.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer digital image processing, and in particular relates to a low-light image enhancement method based on multi-exposure and multi-scale recursive fusion. Background Art

[0002] Images captured by cameras in low-light environments exhibit poor visual quality and low objective performance metrics. Directly enhancing the brightness and contrast of low-light images will also amplify noise. Furthermore, the single-brightness image input limits the spectral range of image enhancement, hindering the restoration of image brightness and color. To minimize this issue during image enhancement, we propose a multi-exposure, multi-scale fusion enhancement network (MEMS) for low-light image enhancement.

[0003] Numerous methods exist for low-light image enhancement. Traditional methods, such as histogram equalization, can directly enhance image brightness. However, this enhancement also amplifies noise and reduces image contrast. Many deep learning-based methods have demonstrated great potential in low-light image enhancement. However, existing methods all have limitations. Due to the low signal-to-noise ratio of low-light images, enhancing extremely dark areas can result in insufficient brightness enhancement, low contrast, and amplified noise during the restoration process. Summary of the Invention

[0004] The purpose of the present invention is to provide a low-light image enhancement method based on multi-exposure and multi-scale recursive fusion, which solves the problems of insufficient brightness enhancement, low contrast, pale color, and noise amplification during the image enhancement process.

[0005] The technical solution adopted by the present invention is a low-light image enhancement method based on multi-exposure and multi-scale recursive fusion, which is specifically implemented in the following steps: a low-light image I is subjected to uniform exposure and non-uniform exposure respectively to obtain a uniformly exposed image en1 and a non-uniformly exposed image en2; a multi-exposure and multi-scale image enhancement network for low-light image enhancement is designed, and image en1, image en2, and low-light image I are input for fusion, denoising, and fine-tuning; a loss function for the multi-exposure and multi-scale image enhancement network for low-light image enhancement is constructed, and the proportional weight of the loss function is adjusted to obtain a network for the optimal enhanced image; the network for the optimal enhanced image is trained to obtain a network for the optimal enhanced image after training, and the low-light image to be enhanced is input to obtain a low-light enhanced image.

[0006] The present invention is also characterized in that:

[0007] A low-light image I is subjected to uniform exposure and non-uniform exposure respectively to obtain uniformly exposed image en1 and non-uniformly exposed image en2. The specific process is as follows: normal light images are selected from the LOL dataset, and the brightness probability distribution D of the normal light images is statistically analyzed. The low-light image I enters the multi-exposure generation block, and the exposure adjustment brightness factor d is selected from the brightness probability distribution D. Uniform exposure and non-uniform exposure are performed on each input low-light image I to obtain uniformly exposed image en1 and non-uniformly exposed image en2.

[0008] The process of uniform exposure and non-uniform exposure in a multi-exposure generation block is shown in formula (1):

[0009] output=(random(D) / avg(input))*input (1)

[0010] Where input is the input low-light image I, avg() represents averaging the pixel values ​​of the input image, random(D) represents randomly extracting a value from the probability distribution D, random(D) / avg(input) is recorded as the exposure adjustment factor d, and output is the output of the uniform exposure image and uneven exposure image in the multi-exposure generation block, namely en1 and en2.

[0011] A multi-exposure and multi-scale image enhancement network is designed for low-light image enhancement. The input images en1, en2, and low-light image I are fused, denoised, and fine-tuned. The specific process is as follows:

[0012] Step 1: Design a multi-exposure and multi-scale image enhancement network for low-light image enhancement, including a multi-exposure feature recursive fusion module, a multi-exposure denoising and enhancement module, and a curve fine-tuning module; input image en1, image en2, and low-light image I into the multi-exposure feature recursive fusion module and the multi-exposure denoising and enhancement module respectively to obtain a 12-channel advantage feature map F fusion2 and the preliminary denoised image F enhance ;

[0013] Step 2: 12-channel advantage feature map F fusion2 And the preliminary denoising enhancement image F enhance Enter the curve fine-tuning module for fine-tuning to obtain the fine-tuned image

[0014] The specific process of step 1 is:

[0015] Step 1.1. Design a multi-exposure multi-scale image enhancement network for low-light image enhancement, including a multi-exposure feature recursive fusion module, a multi-exposure denoising enhancement module, and a curve fine-tuning module. The multi-exposure feature recursive fusion module includes two recursive fusion units, each of which includes 7 convolutional layers and corresponding batch normalization layers and annealing layers to extract feature information of the low-light image I. The extracted feature information is recorded as S feature ;

[0016] Step 1.2: The pre-trained vgg16 network performs quality assessment on the low-light image I and the uneven exposure image en2, and obtains the weights W of the low-light image I and the uneven exposure image en2 when performing the fusion process. I and W en2 , the low-light image I and the uneven exposure image en2 are input to the first recursive fusion unit, combined with the feature information S feature , generate the first fused image F through formula (2) fusion1 , the existing pre-trained vgg16 network is used to train F fusion1 And the uniform exposure image en1 is used for quality assessment to obtain F fusion1 and the weight W of the uniformly exposed image en1 when performing the fusion process Ffusion1 and W en1 ; The first fused image F fusion1 Together with the uniform exposure image en1, it is input into the second recursive fusion unit and combined with the feature information S feature , the second fused image F is generated by formula (3) fusion2 , the second fused image F fusion2 ;

[0017] F fusion1 =W I ×S feature ×I+(1-W en2 ×S feature )×en2 (2);

[0018] F fusion2 =W Ffusion1 ×S feature ×F fusion1 +(1-W en1 ×S feature )×en1 (3);

[0019] Step 1.3: The multi-exposure denoising and enhancement module includes three multi-scale image enhancement blocks. The low-light image I is input into one multi-scale image enhancement block for multi-scale denoising and enhancement, resulting in image add2. Similarly, the uniformly exposed image en1 and the non-uniformly exposed image en2 are input into the other two multi-scale image enhancement blocks for multi-scale denoising and enhancement, resulting in images add7 and add12.

[0020] Step 1.4: Use the existing pre-trained vgg16 network to perform quality assessment on image add2, image add7, and image add12, obtain their respective quality assessment scores, and convert the quality assessment scores into the fusion weight values ​​of image add2, image add7, and image add12, expressed as W add2 、W add7 and W add12 , use formula (4) to perform weighted fusion on image add2, image add7 and image add12 to obtain the preliminary denoised image F enhance ;

[0021] F enhance =W add2 ×add2+W add7 ×add7+W add12 ×add12 (5).

[0022] The specific process of step 2 is:

[0023] For the 12-channel advantage feature map F fusion2 Perform channel-level segmentation to obtain four three-channel adjustment factors α1, α2, α3, and α4. Substitute the adjustment factors into the four iterations of curve fine-tuning to adjust the initial denoising image F. enhance Perform auxiliary fine-tuning. The fine-tuning process is expressed as formula (6):

[0024]

[0025] in, Indicates the result of the last iteration; represents the result of the i-th iteration;

[0026] The fine-tuned image produced by the last iteration The number of iterations, n, is 4.

[0027] The loss function for constructing a multi-exposure and multi-scale image enhancement network for low-light image enhancement is as follows:

[0028] The loss function based on structural similarity loss, pixel gradient loss and color consistency loss is designed, including Lssim, Lgrad and L1. The weight ratio is selected to optimize the loss function. The loss function is expressed as follows:

[0029]

[0030]

[0031]

[0032]

[0033] Among them, GT represents the normal light image in the paired dataset used in the network training process, μ GT and Represents the normal light reference image GT and the fine-tuned image The average value, σ GT and represents the variance of the ground truth; similarly, Represents the preliminary denoising enhancement image F enhance The mean of Indicates F enhance Variance; take c1 = 0.0001, c2 = 0.0009 to prevent the denominator from being zero, and are the gradients of the image in the horizontal and vertical directions, respectively, enhancei It means F enhance The value of each pixel of GT i represents the value of each pixel in the normal light reference image, K is the number of pixels in the image, and α, β, and χ are the proportional weights corresponding to the three loss functions expressed in equations (7), (8), and (9).

[0034] The network for the optimal enhanced image is trained, and the network for the optimal enhanced image after training is specifically:

[0035] We selected a GPU with 12GB of video memory for training. We used the ADAM optimizer for the network, set the initial learning rate to 0.0003, and the learning rate annealing cosine scheduler for a total of 3,000 epochs.

[0036] The trained neural network parameters, the number of epochs being trained, the optimizer ADAM and the scheduler are saved to obtain the network with the optimal enhanced image.

[0037] The beneficial effects of the present invention are:

[0038] In order to better preserve the feature information in the original image and adaptively fuse the outputs of different exposure branches, the present invention proposes a multi-exposure feature recursive fusion module and a corresponding fusion strategy. In order to produce outputs of multiple different exposures, the present invention designs a multi-exposure generation block, which counts the brightness information of a large number of normal light images to form a probability distribution, and randomly extracts a global scaling factor from it each time to randomly enhance the exposure of each original input. The present invention obtains outputs of different exposure levels of the original image through a multi-exposure generation block, recursively enhances the outputs of different exposure levels from low resolution to full resolution using a multi-scale feature enhancement network, and fine-tunes the enhanced results to obtain high-quality enhanced output; thereby solving the problems of insufficient brightness enhancement, low contrast, light color, and noise amplification during the restoration process during image enhancement. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 Schematic diagram of the network structure of the network model used in the present invention;

[0040] Figure 2 It is the intermediate results of the three units in the multi-exposure multi-scale denoising and enhancement process of the present invention and their texture detail comparison map;

[0041] Figure 3 Schematic diagram of the structure of the multi-exposure feature recursive fusion module of the present invention;

[0042] Figure 4 The embodiment of the present invention generates output results of different exposure levels for low-light input based on a global scaling factor obtained by random sampling;

[0043] Figure 5 The following is a visual comparison of our method with other different low-light enhancement methods on the LOL dataset. DETAILED DESCRIPTION

[0044] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0045] The present invention is based on a low-light image enhancement method based on multi-exposure and multi-scale recursive fusion, and the network structure used is as follows: Figure 1 As shown, the specific steps are as follows:

[0046] A low-light image I is subjected to uniform exposure and non-uniform exposure, obtaining a uniformly exposed image en1 and a non-uniformly exposed image en2. To minimize problems such as insufficient image brightness enhancement, low contrast, faded colors, and noise amplification during the image enhancement process, the present invention proposes a multi-exposure multi-scale fusion enhancement network (MEMS) to enhance low-light images. Specifically, the network comprises:

[0047] Normal light images are selected from the LOL dataset, and the brightness probability distribution D of the normal light images is statistically analyzed. The low-light images I enter the multi-exposure generation block, and the exposure adjustment brightness factor d is selected from the brightness probability distribution D. Each input low-light image I is uniformly exposed and non-uniformly exposed to obtain the uniformly exposed image en1 and the non-uniformly exposed image en2.

[0048] The process of uniform exposure and non-uniform exposure in a multi-exposure generation block is shown in formula (1):

[0049] output=(random(D) / avg(input))*input (1)

[0050] Where input is the input low-light image I, avg() represents averaging the pixel values ​​of the input image, random(D) represents randomly extracting a value from the probability distribution D, random(D) / avg(input) is recorded as the exposure adjustment factor d, and output is the output of the uniform exposure image and uneven exposure image in the multi-exposure generation block, namely en1 and en2.

[0051] Since the brightness of daytime images is usually higher than that of nighttime images, in order to change the exposure of low-light image I, the present invention constructs a daytime brightness probability distribution D by statistically analyzing the brightness information of a set of real daytime images. Each time the multi-exposure generation block needs to generate an image with different exposures, it only needs to randomly sample from D to obtain a global scale factor d, and use d to adjust the exposure of the input image. Unlike simply taking the average, random sampling enables the multi-exposure generation block to generate multiple images with different exposure levels each time, which also enhances the generalization ability of the model to a certain extent.

[0052] The adjustment effect of the multi-exposure block is as follows Figure 2 As shown, from left to right are low-light images, non-uniform exposure images, and uniform exposure images. The non-uniform exposure images and uniform exposure images are generated by inputting the low-light image into the multi-exposure generation block. It can be seen that the adjusted image has a higher exposure than the original input.

[0053] A multi-exposure and multi-scale image enhancement network is designed for low-light image enhancement, including a multi-exposure feature recursive fusion module, a multi-exposure denoising enhancement module, and a curve fine-tuning module. Image en1, image en2, and low-light image I are input into the multi-exposure feature recursive fusion module and the multi-exposure denoising enhancement module respectively to obtain a 12-channel advantage feature map F. fusion2 and the preliminary denoised image F enhance Specifically:

[0054] Design the fusion process of the multi-exposure feature recursive fusion module. The fusion process is as follows: Figure 3 As shown in Figure 1, the multi-exposure feature recursive fusion module includes two recursive fusion units. Each recursive fusion unit includes 7 convolutional layers and corresponding batch normalization layers and annealing layers to enhance the generalization ability of the model. The multi-scale feature connection is used to better retain the feature information of the original image to extract the feature information of the low-light image I. The extracted feature information is recorded as S feature ;

[0055] The pre-trained vgg16 network performs quality assessment on the low-light image I and the uneven exposure image en2, and obtains the weight W of the low-light image I and the uneven exposure image en2 when performing the fusion process. I and W en2 , the low-light image I and the uneven exposure image en2 are input to the first recursive fusion unit, combined with the feature information S feature , generate the first fused image F through formula (2) fusion1 , the existing pre-trained vgg16 network is used to train F fusion1 And the uniform exposure image en1 is used for quality assessment to obtain F fusion1 and the weight W of the uniformly exposed image en1 when performing the fusion process Ffusion1 and W en1 ; The first fused image F fusion1 Together with the uniform exposure image en1, it is input into the second recursive fusion unit and combined with the feature information S feature , the second fused image F is generated by formula (3) fusion2 , the second fused image F fusion2 ;

[0056] F fusion1 =W I ×S feature ×I+(1-W en2 ×S feature )×en2 (2);

[0057] F fusion2 =W Ffusion1 ×S feature ×F fusion1 +(1-Wen1 ×S feature )×en1 (3);

[0058] The multi-exposure denoising and enhancement module includes three multi-scale image enhancement blocks. The low-light image I is input into one multi-scale image enhancement block for multi-scale denoising and enhancement, resulting in image add2. Similarly, the uniform exposure image en1 and the non-uniform exposure image en2 are input into the other two multi-scale image enhancement blocks for multi-scale denoising and enhancement, resulting in images add7 and add12.

[0059] Use the existing pre-trained vgg16 network to perform quality assessment on image add2, image add7, and image add12, and obtain their respective quality assessment scores. The quality assessment scores are converted into fusion weight values ​​of image add2, image add7, and image add12, expressed as W add2 、W add7 and W add12 After adjusting the value range of the quality assessment score to the (0,1) interval, the fusion weight of each small branch is expressed as wi = yi / (y1 + y2 + y3). The image add2, image add7 and image add12 are weighted fused using formula (4) to obtain the preliminary denoised image F enhance ;

[0060] F enhance =W add2 ×add2+W add7 ×add7+W add12 ×add12 (5).

[0061] The 12-channel advantage feature map F fusion2 And the preliminary denoising enhancement image F enhance Enter the curve fine-tuning module for fine-tuning to obtain the fine-tuned image Specifically:

[0062] For the 12-channel advantage feature map F fusion2 Perform channel-level segmentation to obtain four three-channel adjustment factors α1, α2, α3, and α4. Substitute the adjustment factors into the four iterations of curve fine-tuning to adjust the initial denoising image F. enhance Perform auxiliary fine-tuning. The fine-tuning process is expressed as formula (6):

[0063]

[0064] in, Indicates the result of the last iteration; represents the result of the i-th iteration;

[0065] The fine-tuned image produced by the last iteration The number of iterations, n, is 4.

[0066] For the 12-channel advantage feature map F fusion2 Use Canny or superpixel segmentation algorithms to perform region segmentation, brighten and darken different areas, and obtain uneven exposure output. Images generated in this way can improve the robustness of the model when dealing with images with uneven lighting.

[0067] Design a loss function based on structural similarity loss, pixel gradient loss and color consistency loss to fine-tune the image And the preliminary denoising enhancement image F enhance Constraints are imposed and the network with the optimal enhanced image is obtained by adjusting the proportional weight of the loss function; specifically:

[0068] The loss function based on structural similarity loss, pixel gradient loss and color consistency loss is designed, including Lssim, Lgrad and L1. The weight ratio is selected to optimize the loss function. The loss function is expressed as follows:

[0069]

[0070]

[0071]

[0072]

[0073] Among them, GT represents the normal light image in the paired dataset used in the network training process, μ GT and Represents the normal light reference image GT and the fine-tuned image The average value, σ GT and represents the variance of the ground truth; similarly, Represents the preliminary denoising enhancement image F enhance The mean of Indicates F enhance Variance; take c1 = 0.0001, c2 = 0.0009 to prevent the denominator from being zero, and are the gradients of the image in the horizontal and vertical directions, respectively, enhancei It means F enhance The value of each pixel of GT i represents the value of each pixel in the normal light reference image, K is the number of pixels in the image, and α, β, and χ are the proportional weights corresponding to the three loss functions expressed in equations (7), (8), and (9).

[0074] The network with the best enhanced image is trained for 3000 epochs to obtain the network with the best enhanced image after training. Specifically, a GPU with 12GB of video memory is selected for training, the network optimizer is the ADAM optimizer, and the initial learning rate is set to 0.0003. The scheduler strategy is the learning rate annealing cosine strategy, and a total of 3000 epochs are trained. The neural network parameters obtained from training, the number of epochs being trained, the ADAM optimizer, and the scheduler are saved to obtain the network with the best enhanced image.

[0075] The low-light image to be enhanced is input into the trained optimal enhancement image network to obtain a low-light enhanced image with higher brightness and less noise.

[0076] Images taken by cameras under low-light conditions often show poor visual quality and low objective performance indicators. Directly improving the brightness and contrast of low-light images often amplifies the noise present in the image, because when the image pixel value is increased, the noise is also amplified at the same time. In addition, using a single exposure as input limits the dynamic range of image enhancement and hinders the restoration of brightness and color. In order to solve these problems, the present invention proposes a multi-exposure multi-scale fusion enhancement method for enhancing low-light images. By enhancing the exposure of the low-light image, an exposed image with high contrast and more color information is obtained. In order to address the problem that the noise is also amplified when the exposure is increased, denoising and enhancement are performed on each scale of the exposed image. While removing the amplified noise, the contrast and color information of the image are retained to the greatest extent. Finally, the image is fine-tuned to obtain the final enhanced image.

[0077] like Figure 4 As shown, the output of the three multi-scale denoising enhancement blocks in the multi-exposure denoising enhancement module, where the first, second and third branches represent the output images of the multi-scale denoising enhancement block for low-light image input, the multi-scale denoising enhancement block for non-uniform exposure image input and the multi-scale denoising enhancement block for uniform exposure image input, respectively. It can be seen that the enhanced image obtained by directly using the low-light image as the input of the multi-scale denoising enhancement module is compared with the enhanced image obtained by using the multi-exposure generation block to increase the exposure of the low-light image as the input of the multi-scale denoising enhancement module (i.e. Figure 4 The multi-scale image enhancement block and the third multi-scale image enhancement block are used to obtain the image), and the multi-exposure generation block is used to increase the exposure of the low-light image as the input of the multi-scale denoising enhancement module. The enhanced image has a better restoration effect on the collar of the clothes, with better texture details and brighter colors.

[0078] Example

[0079] The proposed method is compared with several state-of-the-art methods, including LIME, RetinexNet, Zero-DCE, TBEFN, RUAS, Kind, SGRDR, EnlightenGAN, DRBN, Uretinex and SCI. Figure 5 The following is a visual comparison of different methods on the LOL-real dataset. These images were tested using the pre-trained models provided by the corresponding methods. [Images from our method] is the result obtained by our method, and Ground Truth is the reference. RetinexNet exhibits significant noise and color deviation. TBEFN exhibits noticeable white spots on black clothing. RUAS exhibits noticeable color deviation and uneven wall color. Zero-DCE, while still exhibiting noticeable noise, appears slightly better than RetinexNet. EnlightenGAN exhibits color deviation, noise, and uneven wall color. LIME achieves pleasant visual performance but lacks good restoration in dark areas. Compared to the ground truth, LIME achieves limited enhancement on edges and textures in black clothing. SGRDR slightly outperforms LIME in restoring dark areas but introduces artifacts on the wall. DRBN displays a noticeable shadow on the wall. Kind also introduces uneven artifacts in the restoration of the wall. In summary, our method achieves the most pleasant visual results and the most accurate color information, while also effectively removing noise and preserving texture details in low-light images.

[0080] Tables 1 and 2 show the objective metrics of our method and other methods on the LOL and SICE datasets, respectively. Our method achieves the highest PSNR and SSIM values. Therefore, our method outperforms other methods in both subjective and objective evaluations.

[0081] Table 1

[0082]

[0083] Table 2

[0084]

[0085] This invention is based on a low-light image enhancement method based on multi-exposure, multi-scale recursive fusion. It proposes a multi-exposure feature recursive fusion module and corresponding fusion strategy, which can better preserve the feature information in the original image and adaptively fuse the outputs of different exposure branches. A multi-exposure generation block is designed to produce outputs of multiple exposures. The brightness information of a large number of normal-light images is statistically analyzed to form a probability distribution, from which a global scaling factor is randomly extracted to randomly enhance the exposure of each original input. The invention uses the multi-exposure generation block to obtain outputs of different exposure levels of the original image. These outputs of different exposure levels are recursively enhanced from low resolution to full resolution using a multi-scale feature enhancement network. The enhanced results are then fine-tuned to obtain a high-quality enhanced output. By introducing the idea of ​​simulating multiple exposure images from a single low-light image and jointly processing their features and noise to achieve low-light image enhancement, combined with a multi-scale denoising enhancement module, the invention solves the problems of insufficient brightness enhancement, low contrast, faded colors, and noise amplification during the image enhancement process.

Claims

1. A low-light image enhancement method based on multi-exposure and multi-scale recursive fusion, characterized by: The specific implementation steps are as follows: a low-light image I is subjected to uniform exposure and non-uniform exposure, obtaining a uniformly exposed image en1 and a non-uniformly exposed image en2; a multi-exposure and multi-scale image enhancement network is designed for low-light image enhancement, and the input images en1, en2, and the low-light image I are fused, denoised, and fine-tuned; Construct a loss function for a multi-exposure and multi-scale image enhancement network for low-light image enhancement, adjust the proportional weights of the loss function, and obtain the network with the optimal enhanced image. The network for the optimal enhanced image is trained to obtain the trained optimal enhanced image, and the low-light image to be enhanced is input to obtain the low-light enhanced image; The multi-exposure and multi-scale image enhancement network designed for low-light image enhancement takes as input images en1, en2, and low-light image I and performs fusion, denoising, and fine-tuning as follows: Step 1: Design a multi-exposure and multi-scale image enhancement network for low-light image enhancement, including a multi-exposure feature recursive fusion module, a multi-exposure denoising and enhancement module, and a curve fine-tuning module; Input image en1, image en2, and low-light image I into the multi-exposure feature recursive fusion module and the multi-exposure denoising enhancement module respectively to obtain a 12-channel advantage feature map F fusion2 and the preliminary denoised image F enhance ; The specific process of step 1 is: Step 1.1: Design a multi-exposure and multi-scale image enhancement network for low-light image enhancement, including a multi-exposure feature recursive fusion module, a multi-exposure denoising and enhancement module, and a curve fine-tuning module. The multi-exposure feature recursive fusion module includes two recursive fusion units, each of which includes 7 convolutional layers and corresponding batch normalization layers and annealing layers, which are used to extract feature information of the low-light image I. The extracted feature information is recorded as S feature ; Step 1.2: The pre-trained vgg16 network performs quality assessment on the low-light image I and the uneven exposure image en2, and obtains the weights W of the low-light image I and the uneven exposure image en2 when performing the fusion process. I and W en2 , the low-light image I and the uneven exposure image en2 are input to the first recursive fusion unit, combined with the feature information S feature , the first fused image F is generated by formula (2) fusion1 , the existing pre-trained vgg16 network is used to train F fusion1 And the uniform exposure image en1 is used for quality assessment to obtain F fusion1 and the weight W of the uniformly exposed image en1 when performing the fusion process Ffusion1 and W en1 ; The first fused image F fusion1 Together with the uniform exposure image en1, it is input into the second recursive fusion unit and combined with the feature information S feature , the second fused image F is generated by formula (3) fusion2 : （2）； （3）； Step 1.3: The multi-exposure denoising and enhancement module includes three multi-scale image enhancement blocks. The low-light image I is input into one multi-scale image enhancement block for multi-scale denoising and enhancement to obtain image add2. Similarly, the uniform exposure image en1 and the non-uniform exposure image en2 are input into the other two multi-scale image enhancement blocks for multi-scale denoising and enhancement to obtain images add7 and add12. Step 1.4: Use the existing pre-trained vgg16 network to perform quality assessment on image add2, image add7, and image add12, obtain their respective quality assessment scores, and convert the quality assessment scores into the fusion weight values ​​of image add2, image add7, and image add12, expressed as W add2 、W add7 and W add12 , use formula (4) to perform weighted fusion on image add2, image add7 and image add12 to obtain the preliminary denoised image F enhance ; （5）； Step 2: 12-channel advantage feature map F fusion2 And the preliminary denoising enhancement image F enhance Enter the curve fine-tuning module for fine-tuning to obtain the fine-tuned image .

2. The low-light image enhancement method based on multi-exposure and multi-scale recursive fusion according to claim 1, characterized in that: The specific process of performing uniform exposure and non-uniform exposure on a low-light image I to obtain a uniformly exposed image en1 and a non-uniformly exposed image en2 is as follows: a normal-light image is selected from the LOL data set, a brightness probability distribution D of the normal-light image is statistically distributed, the low-light image I enters the multi-exposure generation block, an exposure adjustment brightness factor d is selected from the brightness probability distribution D, and each input low-light image I is subjected to uniform exposure and non-uniform exposure to obtain a uniformly exposed image en1 and a non-uniformly exposed image en2.

3. The low-light image enhancement method based on multi-exposure and multi-scale recursive fusion according to claim 2, characterized in that: The process of uniform exposure and non-uniform exposure in the multi-exposure generation block is shown in formula (1): （1） Where input is the input low-light image I, avg() represents averaging the pixel values ​​of the input image, random(D) represents randomly extracting a value from the probability distribution D, random(D) / avg(input) is recorded as the exposure adjustment factor d, and output is the output of the uniform exposure image and uneven exposure image in the multi-exposure generation block, namely en1 and en2.

4. The low-light image enhancement method based on multi-exposure and multi-scale recursive fusion according to claim 3, characterized in that: The specific process of step 2 is: For the 12-channel advantage feature map F fusion2 Perform channel-level segmentation to obtain four three-channel adjustment factors α1, α2, α3, and α4. Substitute the adjustment factors into the four iterations of curve fine-tuning to adjust the initial denoising image F. enhance Perform auxiliary fine-tuning. The fine-tuning process is expressed as formula (6): (6) in, Indicates the result of the last iteration; represents the result of the i-th iteration; The fine-tuned image produced by the last iteration , the number of iterations, that is, n, is 4.

5. The low-light image enhancement method based on multi-exposure and multi-scale recursive fusion according to claim 4, characterized in that: The loss function for constructing a multi-exposure multi-scale image enhancement network for low-light image enhancement is specifically: The loss function based on structural similarity loss, pixel gradient loss and color consistency loss is designed, including Lssim, Lgrad and L1. The weight ratio is selected to optimize the loss function. The loss function is expressed as follows: （7） （8） （9） (10) Among them, GT represents the normal light image in the paired dataset used in the network training process, and denote the average value of the normal light reference image GT and the fine-tuned image The average value of and denote the variance of the normal light reference image GT and the fine-tuned image respectively variance; Represents the preliminary denoising enhancement image F enhance The mean of Indicates F enhance Variance; take c1 = 0.0001, c2 = 0.0009 to prevent the denominator from being zero, and are the gradients of the image in the horizontal and vertical directions, It means F enhance The value of each pixel of GT i It represents the value of each pixel in the normal light reference image, K is the number of pixels in the image, , , They are the proportional weights corresponding to the three loss functions expressed in equations (7), (8) and (9).

6. The low-light image enhancement method based on multi-exposure and multi-scale recursive fusion according to claim 4, characterized in that: The network for the optimal enhanced image is trained to obtain the optimal enhanced image network after training: We selected a GPU with 12GB of video memory for training. We used the ADAM optimizer for the network, set the initial learning rate to 0.0003, and the learning rate annealing cosine scheduler for a total of 3,000 epochs. The trained neural network parameters, the number of epochs being trained, the optimizer ADAM and the scheduler are saved to obtain the network with the optimal enhanced image.

Images