A low-light image enhancement method based on a UNet network

Abstract

The application discloses a low-light image enhancement method based on a UNet network, and comprises the following steps: 1. collecting images under different illuminations and adding to LOLdataset to synthesize a new dataset; 2. pre-processing image data of the new dataset and dividing the image data into a training set, a verification set and a test set; 3. selecting low-illumination and normal-illumination RGB images from the training set in a self-supervised manner based on an image enhancement network of the UNet network, finding the maximum value in all channel dimensions of each RGB image and adding the maximum value to the original RGB image as a new channel to obtain an RGB-D image; 4. decomposing a weak-light image and a corresponding normal-illumination image in the RGB-D image by using a hierarchical decomposition network to obtain a reflection component R low(x,y) and an illumination component I low(x,y) ; 5. adjusting a recovery network to receive the reflection component R low(x,y) and the illumination component I low(x,y) , generate I low_out (x, y), and suppressing noise by using a BM3D algorithm to generate R low_out(x,y) ; 6. multiplying R low_out (x, y) and I low_out (x, y) element by element to generate an enhanced image. The application improves the brightness of the image and makes the image details and textures more abundant.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically a low-light image enhancement method based on the UNet network. Background Technology

[0002] With the rapid evolution of the digital age, acquiring effective information has become a critical task. Among various information carriers, images occupy an important position and play a vital role in our daily lives. They have been widely used in fields such as video surveillance, facial recognition systems, object recognition, and autonomous driving technology. These applications typically have very high requirements for image quality. However, due to unfavorable lighting conditions such as night shooting and backlighting, or due to limitations in shooting equipment and shooting techniques, the obtained images often face problems such as loss of scene details, reduced contrast, noise, and color distortion. This not only affects people's visual experience but also poses challenges to the application of computer vision systems.

[0003] Low-light image enhancement aims to significantly improve the brightness and contrast of images and reduce noise interference through technical means, thereby improving the display of image details and the realism of colors, and thus comprehensively improving image quality. This process is not only a key step in basic vision tasks but also frequently serves as an important step in the data preprocessing stage of advanced computer vision tasks. Through enhancement, images become more consistent with human visual perception and can be more effectively recognized and processed by computer vision systems. Currently, research and applications of this technology involve multiple fields, including but not limited to traffic monitoring, satellite remote sensing, medical image analysis, and military reconnaissance. For example, in traffic monitoring, under low-light conditions at night, backlighting caused by vehicle headlights can interfere with the judgment of violations, such as crossing lines or illegal parking. In this case, image enhancement technology can help clarify vehicle details, allowing drivers to accurately identify violations. In the field of medical imaging, due to the limitations of imaging conditions inside the human body, the resulting images often have low contrast. Enhanced images can more clearly display lesion areas, providing doctors with more accurate diagnostic information. Therefore, in-depth research into low-light image enhancement technology is of great value in improving image processing capabilities and application efficiency in these fields.

[0004] Given that images captured in low-light environments are often affected by various adverse factors, actual images frequently exhibit problems such as low brightness, poor contrast, uneven light distribution, lack of texture detail, and color distortion. These problems severely affect the accurate acquisition of image information, impacting not only people's subjective visual experience but also posing challenges to computer vision applications. In recent years, researchers have proposed various traditional and deep learning-based methods to improve this situation, including histogram equalization, LLNet, SID, and Retinex-Net. Among these, histogram equalization... Histogram equalization (HUE) is a simple and effective image enhancement technique that aims to improve contrast by adjusting the image's histogram. HUE stretches the image's histogram to distribute brightness evenly across the entire range, thus improving overall contrast. However, because it doesn't follow the rules of object imaging models, it can easily introduce unexpected noise, reduce color fidelity, and lead to image distortion. LLNet, SID, and Retinex-Net are trained using large amounts of data to learn how to automatically improve the quality of low-light images. While they have made some progress in improving the brightness of low-light images, these enhancement techniques are generally inefficient, and the processed image quality is also relatively low. Therefore, further research is needed on low-light image enhancement methods under different lighting conditions to improve image quality. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the purpose of this invention is to provide a low-light image enhancement method based on the UNet network, which not only improves the brightness of the image but also enriches the details and textures of the image.

[0006] To achieve the above objectives, the present invention employs the following technical solution:

[0007] A low-light image enhancement method based on UNet network includes the following steps:

[0008] Step 1: Collect image data under different lighting conditions and add them to the open-source low-light dataset LOLdataset to synthesize a new dataset;

[0009] Step 2: Preprocess the image data in the new dataset, and divide the preprocessed dataset into training set, validation set and test set;

[0010] Step 3: Import the training set into the UNet-based image enhancement network. The image enhancement network selects low-light RGB images and normal-light RGB images from the training set in a self-supervised manner, finds the maximum value in all channel dimensions of each RGB image, and adds the maximum value as a new channel to the original RGB image to obtain the RGB-D image.

[0011] The image enhancement network based on the UNet network includes a hierarchical decomposition network and a conditioning and restoration network. The hierarchical decomposition network consists of eight layers: the first layer includes two convolutional layers with LReLU activation; the second, third, and fourth layers each include a pooling layer, two convolutional layers with LReLU activation; the fifth and sixth layers each include an upsampling layer, a skip connection layer, and two convolutional layers with LReLU activation; the seventh layer includes an upsampling layer, a skip connection layer, two convolutional layers with LReLU activation, and an SE channel attention module layer; and the eighth layer is a pointwise convolutional layer.

[0012] The adjustment and recovery network is an encoder-decoder architecture and incorporates multi-scale connections;

[0013] Step 4: The hierarchical decomposition network decomposes the low-light image and its corresponding normal-light image in the RGB-D image to obtain the reflection component R. low(x,y) and illumination component I low(x,y) ;

[0014] Step 5: Adjust and restore the received reflection component R of the network. low(x,y) and illumination component I low(x,y) The BM3D algorithm is used to suppress amplified noise in the reflection image and generate R. low_out(x,y) ;

[0015] Step 6: Adjust the recovery network to generate I through encoding and decoding. low_out (x,y), then the reflection component R low_out (x,y) and illumination component I low_out The elements (x,y) are multiplied element by element and then fused to generate the enhanced image.

[0016] Furthermore, step 2 involves preprocessing the image data in the new dataset by cropping all images in the new dataset into images of size 600×400.

[0017] Furthermore, the specific process of adding the maximum value as a new channel to the RGB image in step 3 is as follows:

[0018] The maximum value of all channels in the RGB image and the original RGB image of size 3×600×400 are concatenated to generate an RGB-D image of size 4×600×400.

[0019] Furthermore, the hierarchical decomposition network in step 3 includes a reflection component extraction channel and an illumination component extraction channel, and outputs four images with resolutions of 600×400, 300×200, 150×100 and 75×50 during the training process.

[0020] Furthermore, the encoder of the adjustment and recovery network in step 3 includes four convolutional layers with ReLU activation functions, wherein the stride of three of the convolutional layers is 2.

[0021] Furthermore, the specific process of step 4 is as follows:

[0022] Step 4.1: The first layer extracts shallow features of the image through two 3×3 convolutional layers, using LRelu as the activation function;

[0023] Step 4.2: The second, third and fourth layers are successively reduced by using 2×2 max pooling layers with a stride of 2, and then two 3×3 convolutional layers are used to extract shallow features of the image. LRelu is used as the activation function to reduce the size of the image and increase the number of channels.

[0024] Step 4.3: The fifth layer upsamples the output of the fourth layer and performs a channel-dimension skip connection with the output of the third layer. Then, it goes through two 3×3 convolutional layers with LRelu as the activation function to increase the image size and reduce the number of channels.

[0025] Step 4.4: The sixth layer upsamples the output of the fifth layer and performs a channel-dimension skip connection with the output of the second layer. Then, it goes through two 3×3 convolutional layers with LRelu as the activation function to increase the image size and reduce the number of channels.

[0026] Step 4.5: The seventh layer upsamples the output of the sixth layer and makes a channel-dimension skip connection with the output of the first layer. Then, it goes through two 3×3 convolutional layers with LRelu as the activation function to increase the image size and reduce the number of channels, so that the image is restored to its original size. Then, the SE channel attention mechanism is used to enhance the network's learning of important channels.

[0027] Step 4.6: The eighth layer performs a 1×1 pointwise convolution on the output of the seventh layer. The reflection component extraction channel projects the original RGB image from the feature space into a reflection map, and then maps the result to the range of 0-1 using the Sigmoid function, outputting the reflection component Rlow(x,y). The illumination component extraction channel projects the original RGB image from the feature space into an illumination map, and then maps the result to the range of 0-1 using the Sigmoid function, outputting the illumination component I. low(x,y) .

[0028] Furthermore, the total loss function of the hierarchical decomposition network in step 4 is expressed as:

[0029] L DLP =L recon +αL ir +βL s

[0030] In the formula, L recon To reconstruct the loss function, L ir Let L be the constant reflectivity loss function. s The loss function is a smoothing function, where α and β are coefficients, and α = 0.1 and β = 0.01.

[0031] The reconstruction loss function is expressed as follows:

[0032]

[0033] In the formula, γ mn The reconstruction coefficient is 0.0001; I n R represents the illumination component. m S is the reflection component; n For normal illumination, low and high represent low-light and normal-light images, respectively;

[0034] The constant reflectivity loss function is expressed as:

[0035] L ir =|‖R low -R high ||1

[0036] In the formula, R low and R high These represent the reflection images under low light and the reflection images under normal light, respectively.

[0037] The smoothing loss function is expressed as:

[0038]

[0039] In the formula, I represents the first derivative operator in the horizontal and vertical directions; m λ represents the smoothing loss of illumination components. g This represents the structural strength balance coefficient, with a value of 10.

[0040] Furthermore, the specific process of step 6 is as follows:

[0041] Step 6.1: Convert the reflection component R low(x，y ) and illumination component I low(x,y ) splice along the channel dimension (dim=1);

[0042] Step 6.2: Adjust the encoder of the recovery network. During the encoding stage, downsample the feature map to reduce the spatial resolution of the feature map and increase the number of feature channels.

[0043] The encoder consists of four convolutional layers with ReLU activation functions, where the stride of three of the convolutional layers is 2;

[0044] Step 6.3: Adjust the decoder of the recovery network. In the decoding stage, the feature map is upsampled by the cubic interpolation function F.interpolate to increase the spatial resolution, and then concatenated with the feature map of the previous layer. Feature fusion is performed by convolutional layers and ReLU activation function until the size of the feature map is close to the size of the input.

[0045] Step 6.4: Convert the multi-channel feature map into a single-channel output through a convolutional layer to generate the enhanced image.

[0046] Furthermore, the total loss function of the adjustment and recovery network in step 6 is expressed as:

[0047] L adjust =L′ recon +L is

[0048] In the formula, L′ recon L represents the reconstruction loss function. is Represents the smoothing loss function;

[0049]

[0050] In the formula, R represents low Gradient mapping weighting, S normal This indicates normal illuminance.

[0051] Compared with the prior art, the present invention has the following technical effects:

[0052] This invention significantly improves the overall brightness of low-light images by adjusting the illumination components, while maintaining the naturalness and balance of the illumination distribution. It overcomes the limitations imposed on image brightness and detail texture by ambient light and hardware performance. Utilizing the powerful feature extraction capabilities of the UNet network, it effectively recovers lost or hidden details and textures in low-light images. Since the value of cultural relics images often lies in their details, this UNet-based low-light image enhancement network plays a positive role in the digital preservation and research of cultural relics images. Furthermore, by integrating the BM3D algorithm for denoising, it effectively reduces noise that may be introduced during image enhancement or that was already present in the low-light image. This not only improves the brightness and detail of the enhanced image but also does not affect image quality, effectively preventing common over-processing problems. In summary, this UNet-based image enhancement method not only has a complete network architecture but also exhibits excellent performance in image enhancement, enriching the details and textures of the image. Attached Figure Description

[0053] Figure 1 This is the structural framework of the enhanced network of the present invention;

[0054] Figure 2 Images acquired under low light and normal light conditions are for this invention.

[0055] Figure 3 This is a diagram of the architecture of the hierarchical decomposition network of the present invention;

[0056] Figure 4 This is a schematic diagram of the SE channel attention mechanism enhancement network of the present invention;

[0057] Figure 5 This is a diagram illustrating the overall architecture of the adjustment and recovery network of the present invention.

[0058] Figure 6 These are partial image examples from the dataset of this invention;

[0059] Figure 7 This presents a comparison of images before and after enhancement in the MITK5 dataset of this invention.

[0060] Figure 8 This is a comparison of images before and after enhancement in the LOL dataset of this invention;

[0061] Figure 9 for Figure 2 Comparison results of some images before and after enhancement. Detailed Implementation

[0062] The specific content of the present invention will be further explained in detail below with reference to the embodiments.

[0063] A low-light image enhancement method based on UNet network includes the following steps:

[0064] Step 1, collect such as Figure 2 The image data of cultural relics under different lighting conditions shown are added to the open-source low-light dataset LOLdataset to synthesize a new dataset;

[0065] Step 2: Preprocess the image data in the new dataset. Divide the preprocessed dataset into training set, validation set and test set for batch import into the UNet-based image enhancement network. The training set includes 500 pairs of low-light images and normal-light images, and the validation set and test set each include 15 pairs of low-light images and normal-light images.

[0066] The preprocessing involves uniformly cropping the images to a size of 600×400 to adapt to the model's input requirements, thereby standardizing the input of image data and improving the model's training efficiency.

[0067] Step 3: Import the training set into the UNet-based image enhancement network. The image enhancement network selects low-light RGB images and normal-light RGB images from the training set in a self-supervised manner, finds the maximum value in all channel dimensions of each RGB image, and adds the maximum value as a new channel to the RGB image of size 3×600×400. Channel concatenation is then performed to generate an RGB-D image of size 4×600×400. Specifically, the image enhancement network finds the maximum value in all channel dimensions of the RGB image, concatenates it with the original RGB image of size 3×600×400, and generates an RGB-D image of size 4×600×400.

[0068] like Figure 1 As shown, the image enhancement network based on the UNet network includes, as follows: Figure 3 The hierarchical decomposition network shown and such Figure 5 The adjustment and recovery network shown is as follows:

[0069] The hierarchical decomposition network consists of eight layers. The first layer includes two convolutional layers with LReLU activation. The second, third, and fourth layers each include a pooling layer, two convolutional layers with LReLU activation. The fifth and sixth layers each include an upsampling layer, a skip connection layer, and two convolutional layers with LReLU activation. The seventh layer includes an upsampling layer, a skip connection layer, two convolutional layers with LReLU activation, and an SE channel attention module layer. The eighth layer is a pointwise convolutional layer.

[0070] The hierarchical decomposition network includes a reflection component extraction channel and an illumination component extraction channel, and outputs four images with resolutions of 600×400, 300×200, 150×100 and 75×50 during training.

[0071] The adjustment and recovery network is an encoder-decoder architecture. The encoder includes four convolutional layers with ReLU activation functions. The stride of three of the convolutional layers is 2. The adjustment and recovery network introduces multi-scale connections, which can capture contextual information about the illumination distribution over a large range, thereby improving its adaptive adjustment capability.

[0072] Step 4: The hierarchical decomposition network receives the RGB-D image and decomposes the low-light image and its corresponding normal-light image in the RGB-D image to obtain the reflection component R. low(x,y) and illumination component I low(x,y) The specific process is as follows:

[0073] Step 4.1: The first layer extracts shallow features of the image through two 3×3 convolutional layers, using LRelu as the activation function;

[0074] Step 4.2: The second, third and fourth layers are successively reduced by using 2×2 max pooling layers with a stride of 2 to reduce the data dimensionality, thereby reducing the computational cost of the network. Then, two 3×3 convolutional layers are used to extract shallow features of the image, with LRelu as the activation function, to reduce the image size and increase the number of channels.

[0075] Step 4.3: The fifth layer upsamples the output of the fourth layer and makes a channel-dimension skip connection between the upsampled result and the output of the third layer to facilitate the network learning important features. Then, it goes through two 3×3 convolutional layers with LRelu as the activation function to increase the image size and reduce the number of channels.

[0076] Step 4.4: The sixth layer upsamples the output of the fifth layer, connects the upsampled result with the output of the second layer by skipping channels, and then passes it through two 3×3 convolutional layers with LRelu as the activation function to increase the image size and reduce the number of channels.

[0077] Step 4.5: The seventh layer upsamples the output of the sixth layer and performs a channel-dimension skip connection with the output of the first layer. Then, it passes through two 3×3 convolutional layers using LRelu as the activation function to increase the image size and reduce the number of channels, restoring the image to its original size of 600×400. Finally, the SE channel attention mechanism enhances the network's learning of important channels. The structure of the SE channel attention mechanism-enhanced network is as follows... Figure 4 As shown, it is a relatively deep part of the image enhancement network, and the feature map contains more complex and advanced information;

[0078] Step 4.6: The eighth layer performs a 1×1 pointwise convolution on the output of the seventh layer. The reflection component extraction channel projects the original RGB image from the feature space into a reflection map, and then maps the result to the range of 0-1 using the Sigmoid function, outputting the reflection component Rlow(x,y). The illumination component extraction channel projects the original RGB image from the feature space into an illumination map, and then maps the result to the range of 0-1 using the Sigmoid function, outputting the illumination component I. low(x,y) ;

[0079] The total loss function of the hierarchical decomposition network is expressed as:

[0080] L DLP =L recon +αL ir +βL s

[0081] In the formula, L recon To reconstruct the loss function, L ir Let L be the constant reflectivity loss function. sThe loss function is a smoothing function, where α and β are coefficients, and α = 0.1 and β = 0.01.

[0082] To reconstruct the original image from the reflection component R and the illumination component I, the reconstruction loss function is expressed as:

[0083]

[0084] In the formula, γ mn Reconstruction coefficient, with a value of 0.0001; I n R represents the illumination component. m The reflection component is represented by low and high, respectively, and ||1 represents the L1 loss function.

[0085] Based on the principle of consistent reflectivity components under different light intensities, the constant reflectivity loss function is expressed as:

[0086] L ir =||R low -R high ||1

[0087] In the formula, R low and R high These represent the reflection images under low light and the reflection images under normal light, respectively.

[0088] Ideally, the illumination component I should be as smooth as possible in terms of texture details while maintaining good preservation of the overall structure. The smoothing loss function of the illumination component can be expressed as:

[0089]

[0090] In the formula, I represents the first derivative operator in the horizontal and vertical directions; m λ represents the smoothing loss of illumination components. g This represents the structural strength balance coefficient, with a value of 10.

[0091] Step 5: Adjust and restore the received reflection component R of the network. low(x,y ) and illumination component I low(x，y) The BM3D algorithm is used to suppress amplified noise in the reflection image, achieving denoising and generating R... low_out(x,y) ;

[0092] Step 6: Adjust the recovery network to generate I through encoding and decoding. low_out (x, y), then the reflection component R low_out (x, y) and illumination component I low_out The elements-wise product of (x, y) is fused to generate the enhanced image. The specific process is as follows:

[0093] Step 6.1: Convert the reflection component R low(x,y ) and illumination component I low(x,y) Stitch along the channel dimension (dim=1);

[0094] Step 6.2: Adjust the encoder of the recovery network. During the encoding stage, the feature map is downsampled to gradually reduce the spatial resolution of the feature map and increase the depth of the feature (i.e., the number of channels).

[0095] Step 6.3: Adjust the decoder of the recovery network. In the decoding stage, the feature map is upsampled by the cubic interpolation function F.interpolate to increase the spatial resolution and is concatenated with the feature map of the previous layer. Feature fusion is performed by convolutional layers and ReLU activation function until the size of the feature map is restored to close to the size of the original input feature map.

[0096] Step 6.4: A convolutional layer is used to convert the multi-channel feature map into a single-channel output to generate an enhanced image, thus restoring the color and texture of the low-light image.

[0097] The total loss function of the adjustment and recovery network is expressed as:

[0098] L adjust =L′ recon +L is

[0099] In the formula, R represents low Gradient mapping weighting, L′ recon L represents the reconstruction loss function. is This indicates the smoothing loss of the illumination component;

[0100]

[0101] In the formula, S normal This indicates normal lighting conditions.

[0102] To verify the effectiveness of the method in this embodiment for enhancing images obtained in low-light environments, the experimental platform was a computer with an NVIDIA GeForce RTX 3080 GPU. The experimental parameters were set to a batch size of 16 in the image decomposition network and the illumination adjustment network, and 96×96 image patches were used for 100 iterations. In the optimization process of the model, the Adam algorithm was used as the optimizer, and the learning rate (lr) was set to lr[0] and the momentum parameter (betas) was set to (0.9, 0.999) to help the enhancement network model be effectively trained on the LOL dataset to improve the reflection recovery effect of the image.

[0103] To verify the generalization performance of the network, the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) were used to evaluate the performance of the low-light image enhancement network based on the UNet network proposed in this embodiment on the test set, as well as the quality difference between the enhanced image and the original image.

[0104] During the evaluation process, LOL, MITK5, and other software were used. Figure 2 The artifacts shown were collected by the researchers; some images are examples. Figure 6 As shown, the experimental results for the MITK5 dataset are as follows: Figure 7 As shown, the left side is the input low-light image, and the right side is the enhanced image, verifying that the present invention has good generalization performance and also performs well in outdoor scenes; the experimental results on the LOL dataset test set are as follows. Figure 8 As shown, the left side is the input low-light image, and the right side is the enhanced image, which also verifies that the invention has good generalization performance; the results of the artifact self-collection experiment are as follows. Figure 9 As shown, the left side is the input low-light image, and the right side is the image after image enhancement. This also verifies that the present invention has good image enhancement performance and generalization performance. Specific experimental results are shown in Table 1. Among them, LOL-real, LOL-Syn, and LOL-eval are three types of data in the LOL dataset, representing extremely low-light image data, low-light image data, and the eval validation set, respectively. As can be seen from Table 1, the SSIM values ​​of the images before and after enhancement are 0.7937 to 0.8854, and the PSNR is greater than 20. These two indicators show that the signal-to-noise ratio of the enhanced image is lower than that of the original image, and the similarity with the original image is higher, further proving that the image enhancement method of this embodiment has good image enhancement performance.

[0105] Table 1: Experimental results metrics of the image enhancement method in this embodiment on different datasets

[0106] LOL-Real 22.5938 0.8854 LOL-Syn 20.3805 0.8079 LOL-eval 21.5303 0.8548 MITK5 20.2876 0.7937

Claims

1. A low-light image enhancement method based on UNet network, characterized in that, Includes the following steps: Step 1: Collect image data under different lighting conditions and add them to the open-source low-light dataset LOLdataset to synthesize a new dataset; Step 2: Preprocess the image data in the new dataset, and divide the preprocessed dataset into training set, validation set and test set; Step 3: Import the training set into the UNet-based image enhancement network. The image enhancement network selects low-light RGB images and normal-light RGB images from the training set in a self-supervised manner, finds the maximum value in all channel dimensions of each RGB image, and adds the maximum value as a new channel to the original RGB image to obtain the RGB-D image. The image enhancement network based on the UNet network includes a hierarchical decomposition network and a conditioning and restoration network. The hierarchical decomposition network consists of eight layers: the first layer includes two convolutional layers with LReLU activation; the second, third, and fourth layers each include a pooling layer, two convolutional layers with LReLU activation; the fifth and sixth layers each include an upsampling layer, a skip connection layer, and two convolutional layers with LReLU activation; the seventh layer includes an upsampling layer, a skip connection layer, two convolutional layers with LReLU activation, and an SE channel attention module layer; and the eighth layer is a pointwise convolutional layer. The adjustment and recovery network is an encoder-decoder architecture and incorporates multi-scale connections; Step 4: The hierarchical decomposition network decomposes the low-light image and its corresponding normal-light image in the RGB-D image to obtain the reflection component R. low(x,y) and illumination component I low(x,y) ; Step 5: Adjust and restore the received reflection component R of the network. low(x,y) and illumination component I low(x,y) The BM3D algorithm is used to suppress amplified noise in the reflection image and generate R. low_out(x,y) ; Step 6: Adjust the recovery network to generate I through encoding and decoding. low_out (x,y), then the reflection component R low_out (x,y) and illumination component I low_out The elements (x,y) are multiplied element by element and then fused to generate the enhanced image.

2. The low-light image enhancement method based on UNet network according to claim 1, characterized in that, The image data preprocessing process in step 2 of the new dataset is as follows: all images in the new dataset are cropped to a size of 600×400.

3. The low-light image enhancement method based on UNet network according to claim 1, characterized in that, The specific process of adding the maximum value as a new channel to the RGB image in step 3 is as follows: The maximum value of all channels in the RGB image and the original RGB image of size 3×600×400 are concatenated to generate an RGB-D image of size 4×600×400.

4. The low-light image enhancement method based on UNet network according to claim 1, characterized in that, The hierarchical decomposition network in step 3 includes a reflection component extraction channel and an illumination component extraction channel, and outputs four images with resolutions of 600×400, 300×200, 150×100 and 75×50 during the training process.

5. The low-light image enhancement method based on UNet network according to claim 4, characterized in that, The encoder of the adjustment and recovery network in step 3 includes four convolutional layers with ReLU activation functions, where the stride of three convolutional layers is 2.

6. The low-light image enhancement method based on UNet network according to claim 4, characterized in that, The specific process of step 4 is as follows: Step 4.1: The first layer extracts shallow features of the image through two 3×3 convolutional layers, using LRelu as the activation function; Step 4.2: The second, third and fourth layers are successively reduced by using 2×2 max pooling layers with a stride of 2, and then two 3×3 convolutional layers are used to extract shallow features of the image. LRelu is used as the activation function to reduce the size of the image and increase the number of channels. Step 4.3: The fifth layer upsamples the output of the fourth layer and performs a channel-dimension skip connection with the output of the third layer. Then, it goes through two 3×3 convolutional layers with LRelu as the activation function to increase the image size and reduce the number of channels. Step 4.4: The sixth layer upsamples the output of the fifth layer and performs a channel-dimension skip connection with the output of the second layer. Then, it goes through two 3×3 convolutional layers with LRelu as the activation function to increase the image size and reduce the number of channels. Step 4.5: The seventh layer upsamples the output of the sixth layer and makes a channel-dimension skip connection with the output of the first layer. Then, it goes through two 3×3 convolutional layers with LRelu as the activation function to increase the image size and reduce the number of channels, so that the image is restored to its original size. Then, the SE channel attention mechanism is used to enhance the network's learning of important channels. Step 4.6: The eighth layer performs a 1×1 pointwise convolution on the output of the seventh layer. The reflection component extraction channel projects the original RGB image from the feature space into a reflection map, and then maps the result to the range of 0-1 using the Sigmoid function, outputting the reflection component Rlow(x,y). The illumination component extraction channel projects the original RGB image from the feature space into an illumination map, and then maps the result to the range of 0-1 using the Sigmoid function, outputting the illumination component I. low(x,y) .

7. The low-light image enhancement method based on UNet network according to claim 1, characterized in that, The total loss function of the hierarchical decomposition network in step 4 is expressed as: L DLP =L recon +αL ir +βL s In the formula, L recon To reconstruct the loss function, L ir Let L be the constant reflectivity loss function. s The loss function is a smoothing function, where α and β are coefficients, and α = 0.1 and β = 0.

01. The reconstruction loss function is expressed as follows: In the formula, γ mn The reconstruction coefficient is 0.0001; I n R represents the illumination component. m S is the reflection component; n For normal illumination, low and high represent low-light and normal-light images, respectively; The constant reflectivity loss function is expressed as: L ir =||R low -R high ||1 In the formula, R low and R high These represent the reflection images under low light and the reflection images under normal light, respectively. The smoothing loss function is expressed as: In the formula, I represents the first derivative operator in the horizontal and vertical directions; m λ represents the smoothing loss of illumination components. g This represents the structural strength balance coefficient, with a value of 10.

8. The low-light image enhancement method based on UNet network according to claim 5, characterized in that, The specific process of step 6 is as follows: Step 6.1: Convert the reflection component R low(x,y) and illumination component I low(x,y) Stitch along the channel dimension (dim=1); Step 6.2: Adjust the encoder of the recovery network. During the encoding stage, downsample the feature map to reduce the spatial resolution of the feature map and increase the number of feature channels. The encoder consists of four convolutional layers with ReLU activation functions, where the stride of three of the convolutional layers is 2; Step 6.3: Adjust the decoder of the recovery network. In the decoding stage, the feature map is upsampled by the cubic interpolation function F.interpolate to increase the spatial resolution, and then concatenated with the feature map of the previous layer. Feature fusion is performed by convolutional layers and ReLU activation function until the size of the feature map is close to the size of the input. Step 6.4: Convert the multi-channel feature map into a single-channel output through a convolutional layer to generate the enhanced image.

9. The low-light image enhancement method based on UNet network according to claim 1, characterized in that, The total loss function of the adjustment and recovery network in step 6 is expressed as: L adjust L′ recon +L is In the formula, L′ recon L represents the reconstruction loss function. is Represents the smoothing loss function; In the formula, R represents low Gradient mapping weighting, S normal This indicates normal illuminance.

Images