Low-light image enhancement method based on deep convolutional attention and multi-scale feature fusion

Abstract

The present invention relates to the field of image processing technology, and in particular to a low-light image enhancement method of deep convolutional attention and multi-scale feature fusion. The present invention designs a new low-light attention module and a multi-scale feature compensation module. LLAB consists of a low-light multi-head self-attention module, a dual-branch equalization module and two normalization layers. The low-light multi-head self-attention module is designed to extract semantic information from different channels, and balance the feature weights between different channels by calculating the attention map between channels, thereby improving the visibility and contrast of the image; the dual-branch equalization module further improves the contrast of the image; a multi-scale feature compensation module is proposed to compensate for the loss of detail information of the image in the low-light attention module and the downsampling stage, and to fuse the deep spatial information of images of different scales. Finally, after experiments, the method of the present invention can obtain images with better visual effects.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a low-light image enhancement method combining deep convolutional attention and multi-scale feature fusion. Background Art

[0002] High-quality images can improve the accuracy of models for some advanced computer vision tasks (such as object detection, image classification, semantic segmentation, etc.). In order to obtain images of the highest quality possible, a large number of researchers have made many different attempts. For example, increasing the aperture and extending the exposure time when taking images. However, this not only requires the photographer to have professional photography skills, but also it is difficult to completely avoid the presence of noise in the captured images.

[0003] Based on the above situation, some researchers have tried to use algorithms to solve the problems of low-light images. However, these methods still have some problems. For example, grayscale transformation method: This method is a method that directly converts the pixel value of an image into other numerical values ​​through a mathematical function. This method changes the pixel value and range of the image by mapping, thereby achieving the purpose of enhancing the image. This method is fast and easy to implement. However, this method does not take into account the overall distribution of the image pixel value, so the enhancement effect of this type of method is limited, and it performs poorly in some low-light images. Histogram equalization method: The HE method uses a cumulative distribution function to adjust the pixel value of the low-light image to output the grayscale. This type of method can extract details in the image and improve the clarity of the image. And with the development, a large number of existing algorithms have integrated the HE method, which allows the algorithm based on the HE method to better improve the contrast and details of the image. However, this type of algorithm will cause the image fidelity to decrease, generate a lot of noise and cause image distortion. Method based on Retinex theory: Based on this theory, the low-light image enhancement method decomposes the low-light image into a reflection component and an illumination component, and often assumes the illumination component as the result of low-light enhancement. This type of method can not only improve the contrast and brightness of the image, but also has obvious advantages in image detail processing. However, the Retinex-based model often ignores noise, so the enhanced image will have a lot of noise. And the enhanced image will have artifacts and color distortion.

[0004] In recent years, with the development of computer technology, deep learning has achieved remarkable results in the fields of image denoising, image detection, image super-resolution and image dehazing. At the same time, in the field of low-light image enhancement, researchers have also proposed a large number of deep learning algorithms. Among them, the use of attention mechanisms and multi-stage methods are popular. Some of these methods have good visual effects after the network models are enhanced. However, more algorithms have shortcomings. Among them: (1) Pooling operations are performed before using the attention mechanism, which reduces the features of the image and has defects in image detail recovery; (2) Although the multi-stage method can extract deeper features of the image, the multi-stage structure will also lead to the loss of image detail information and inhibit image restoration. In addition, these methods will cause problems such as artifacts, halos and color deviations in the image.

[0005] To this end, we propose a low-light image enhancement method based on deep convolutional attention and multi-scale feature fusion. Summary of the invention

[0006] The purpose of the present invention is to provide a low-light image enhancement method based on deep convolutional attention and multi-scale feature fusion, which is used to solve the problems of noise, halo and distortion existing in existing low-light image enhancement methods.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] The present invention provides a low-light image enhancement method based on deep convolutional attention and multi-scale feature fusion, characterized in that the method comprises the following steps:

[0009] Constructing a paired data set, the data set comprising low-light images and normal-light images, each low-light image corresponding to a normal-light image of the same scene;

[0010] Using the normal light image as the reference image, the shallow features of the low light image are extracted through the Embedding module;

[0011] Input the shallow features into a DAMFFN encoder, wherein the DAMFFN encoder includes a LLAB module, a downsampling module, and an MSFCB module; the LLAB module includes a LL-MSAB module and a DBWEB module;

[0012] Through the LL-MSAB module, the semantic information of low-light images is extracted and the feature weights between different channels are balanced;

[0013] Improve the contrast of low-light images through the DBWEB module;

[0014] Through the MSFCB module, the image quality is restored and the lost detail information is compensated;

[0015] In the DAMFFN decoder, the difference between the enhanced image and the reference image in the frequency domain space is reduced by a multi-scale frequency domain loss function;

[0016] Outputs the final enhanced low-light image.

[0017] Furthermore, the embedding module uses a convolution module to increase the number of channels of low-light images from 3 to 48.

[0018] Further, Figure 2 The LL-MSAB module shown can extract semantic information of low-light images and balance the feature weights between different channels to improve the visibility of the image. It features:

[0019] Use deep convolution to extract semantic information and local features of different channels;

[0020] The extracted feature matrix is ​​directly assigned to Q, K and V. The channel attention mechanism obtains the channel attention map by extracting the global features, that is, the channel attention map is extracted using Q and K.

[0021] Finally, V is multiplied by the channel attention map to obtain the output result of the LL-MSAB module.

[0022] Figure 2 The detailed processing of the LL-MSAB module shown includes:

[0023] (1)X input It is input to the 3x3 deep convolution layer to implicitly model the local relationship between pixels in the channel, and is processed by the 3x3 deep convolution layer to obtain X, which is directly assigned to X Q , X K and X V Then, X Q , X K and X V Reshape into Q, K and V respectively. So actually Q, K and V are three identical feature matrices. They are expressed as follows:

[0024] X Q =X input W Q ,X K =X input W K ,X V =X input W V

[0025] W Q =W K =WV , X Q =X K =X V =X

[0026] Q = Reshape(X Q ),K=Reshape(X K ),V=Reshape(X V )

[0027] Q=K=V

[0028] X input represents the input feature matrix of the LL-MSAB module, W Q , W K and W V represents the weight matrix of the 3x3 depthwise convolutional layer, and Reshape(·) represents the reshaping operation.

[0029] (2) Due to the limitation of computing power, the present invention converts Q (Q = [Q1, ..., Q N ]), K(K=[K1,...,K N ]) and V(V=[V1,...,V N ]) is divided into N heads j , where each head j The channel dimension is dim h =C / N (C represents the total number of channels in the feature matrix). LL-MSAB needs to calculate each head j Channel attention map Atten j . It is expressed as follows:

[0030]

[0031] head j =V j ·Atten j

[0032] K j The transposed matrix of j represents a learnable parameter.

[0033] (3) Splicing N heads j , and connected into a linear projection. The obtained feature matrix is ​​reshaped, and finally, the reshaped result is input into the 1x1 convolution layer to obtain the output result X of LL-MSAB output .

[0034]

[0035] Conv1(·) represents a 1x1 convolutional layer, Concat(·) represents a concatenation operation, W is a learnable parameter, and Reshape(·) represents a reshaping operation.

[0036] Furthermore, the method of improving the contrast of low-light images by using the DBWEB module includes:

[0037] The input image is copied into two copies, passing through a 1x1 convolution layer and a 3x3 depth convolution layer respectively;

[0038] Perform a dot multiplication on the result of one branch processed by the Sigmoid activation function and the result of the other branch;

[0039] The result of the dot product is passed through a 1x1 convolutional layer to obtain the output of DBWEB, which is expressed as follows:

[0040]

[0041] Among them, φ represents the Sigmoid activation function, represents a 3x3 depth convolutional layer, represents a 1x1 depth convolutional layer, ⊙ represents an element-wise multiplication operation, Concat(·) represents a concatenation operation, and w d Represents a 1x1 depthwise convolutional layer.

[0042] Furthermore, the MSFCB module includes three MSC modules, and the input feature matrices of the three MSC modules are all outputs of the first LLAB module of the DAMFFN encoder; the output feature matrices of the three MSC modules have different sizes, and the output feature matrices of the MSC modules are also added to the corresponding downsampling results of the DAMFFN encoder to complete the fusion of features of different scales.

[0043] Furthermore, the multi-scale frequency domain loss function (Mult-MSE) is expressed as follows:

[0044]

[0045] Among them, Nums=4 represents the 4 outputs of the decoder, H represents the height of the feature matrix, and W represents the width of the feature matrix. and Represent the pixel values ​​at the corresponding coordinates of the enhanced image and the reference image respectively, and ∑ represents the summation symbol.

[0046] The calculation method of the multi-scale frequency domain loss function (Mult-SFD) is as follows:

[0047]

[0048] Among them, x irepresents the total number of pixels, f(·) represents the fast Fourier transform, ||·||1 represents the L1 norm, and ∑ represents the summation symbol.

[0049] Finally, the total loss function Loss of the present invention is as follows:

[0050] Loss=Loss Mult-MSE +λLoss Mult-SFD

[0051] Wherein, λ represents the weight hyperparameter of Mult-SFD (λ=0.1).

[0052] The present invention has at least the following beneficial effects:

[0053] (1) We design DAMFFN, a deep convolutional attention multi-scale feature fusion network for low-light image enhancement. The network is used to learn multi-scale local-global representations on low-light images without decomposing the image into local windows, thereby leveraging long-range image context information.

[0054] (2) We design a low-light multi-head self-attention module (LL-MSAB), which can aggregate local and global features and balance the feature weights between different channels.

[0055] (3) The present invention adopts a dual-branch equalization module (DBWEB), which suppresses features with less information and promotes the forward transmission of features with rich information by increasing the gradient between different pixels.

[0056] (4) A multi-scale feature compensation module (MSFCB) is designed to reduce the loss of image detail information by fusing the deep spatial information of images of different scales.

[0057] (5) The present invention adopts a multi-scale frequency domain loss function (MSFD) to reduce the difference between the enhanced low-light image and the reference image in the frequency domain space. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0059] Figure 1 It is the overall network framework of the present invention;

[0060] Figure 2 This is a structural diagram of the low-light attention module invented by the present invention;

[0061] Figure 3 This is a structural diagram of a dual-branch equalization module used in the present invention;

[0062] Figure 4 This is a structural diagram of the multi-scale feature compensation module of the present invention;

[0063] Figure 5 A diagram showing the structure of the multi-scale loss function invented by the present invention;

[0064] Figure 6 This is a comparison chart of images restored by the present invention and other algorithms. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0066] The present invention is achieved by Figure 1 The DAMFN model shown performs enhancement processing on low-light images. Since the low-light image restoration task is more complicated than other image restoration tasks, the present invention designs different modules to restore different aspects of low-light images.

[0067] First, the LL-MSAB module balances the weights between different channels and cooperates with the loss function to perform overall restoration and enhancement of low-light images; secondly, since the restoration of local features in the overall restoration process is weak and LL-MSAB is insufficient in restoring the image contrast, the present invention designs a DBWEB module to assign different weights to each pixel, thereby repairing the local area. DEWEB also uses the Sigmoid activation function to improve the contrast of the image. Finally, since the low-light image restoration task is a pixel-level restoration task, and with the repeated use of LLAB and downsampling, a large amount of image detail information is lost. In order to improve the restoration effect of the final enhanced image, the present invention designs an MSFCB module to compensate for the loss of low-light image details and fuse image features of different scales. In summary, the present invention designs a low-light image enhancement method with deep convolutional attention and multi-scale feature fusion.

[0068] The overall process of the present invention is as follows:

[0069] 1. Build a dataset of low-light / normal-light image pairs. The dataset should consist of low-light images and normal-light images, where each low-light image I low The normal lighting image I of the same scene ref Correspondingly, the dataset is then divided into training set and test set.

[0070] 2. Input a low-light image from the training set and use the corresponding normal-light image as the reference image.

[0071] 3. Use the Embedding module (a 3×3 convolutional layer) to extract the shallow features of the input image. It is expressed as follows:

[0072] F shalow =f Conv (I low )

[0073] 4. Input the extracted shallow features into the encoder of DAMFFN, which mainly consists of Figure 2 The LLAB module and downsampling operation shown in the figure are composed of. In order to extract the semantic information of different channels of the image and balance the feature weights between different channels, the present invention designs a low-light multi-head self-attention module (LL-MSAB). First, the module uses deep convolution to extract the semantic information and local features of different channels, increase the local contrast of the image, and make the texture of the image clearer; secondly, the channel attention mechanism in the LL-MSAB module calculates the channel attention map by extracting global features; finally, the channel attention map is used to balance the feature weights between different channels of the image matrix, thereby improving the visibility of the image.

[0074] LL-MSAB module such as Figure 2 As shown, X input ∈R H×W×C As the input of LL-MSAB, it is fed into a 3X3 deep convolutional layer to extract image semantic information and implicitly model the local relationship between pixels within the channel, and finally outputs X∈R H×W×C ; Then X∈R H×W×C Make 3 copies and reshape them into Q(query)∈R HW×C , K(key)∈R HW×C and V(value)∈R HW×C , the above process is defined as:

[0075] X Q =X input W Q ,X K =X input W K ,X V =X input W V

[0076] W Q =W K =W V , X Q =X K =XV =X

[0077] Q = Reshape(X Q ),K=Reshape(X K ),V=Reshape(X V )

[0078] Q=K=V

[0079] X input represents the input feature matrix of the LL-MSAB module, W Q , W K and W V Reshape(·) represents the weight matrix of the 3x3 deep convolutional layer. The overall pixel value of low-light images is low, and the difference between local pixels is small. The weight has a great influence on the pixel value. A small fluctuation in the weight of local pixels may cause the image to appear blurred or over-smoothed. Therefore, in the LL-MSAB module, W Q , W K and W V The actual data is exactly the same, so it also leads to X Q , X K and X V All three are exactly the same, Q, K and V are exactly the same.

[0080] (2) Due to the limitation of computing power, the present invention converts Q (Q = [Q1, ..., Q N ]), K(K=[K1,...,K N ]) and V(V=[V1,...,V N ]) is divided into N heads j , where each head j The channel dimension is dim h =C / N (C represents the total number of channels in the feature matrix). LL-MSAB needs to calculate each head j Channel attention map Atten j . It is expressed as follows:

[0081]

[0082] head j =V j ·Atten j

[0083] K j The transposed matrix of j represents a learnable parameter.

[0084] (3) Splicing N headsj , and connected into a linear projection. The obtained feature matrix is ​​reshaped, and finally, the reshaped result is input into the 1x1 convolution layer to obtain the output result X of LL-MSAB output .

[0085]

[0086] Conv1(·) represents a 1x1 convolutional layer, Concat(·) represents a concatenation operation, W is a learnable parameter, and Reshape(·) represents a reshaping operation.

[0087] 5. After analyzing the low-light image data, the present invention has made the following findings: (1) The pixel values ​​of low-light images are relatively low, and after normalization, the values ​​are generally distributed around 0; (2) The pixel difference between local pixels in low-light images is small, and the image contrast is low. However, the characteristics of normal-light images and low-light image data are exactly the opposite. The pixel values ​​of normal-light images are generally large, and the pixel difference between local pixels is large, and the image contrast is high. Therefore, in order to improve the contrast of low-light images, the present invention uses a dual-branch equalization module (DBWEB), such as Figure 3 As shown. First, the input image Duplicate it into two copies and pass them through a 1x1 convolutional layer and a 3x3 depth convolutional layer respectively; secondly, perform dot multiplication on the result of the left branch of DBWEB processed by the Sigmoid activation function and the result of the right branch of DBWEB; finally, pass the result of the dot multiplication through a 1x1 convolutional layer to get the output of DBWEB The slope near 0 in the Sigmoid activation function is large, and the slope is almost kept at the same level, which can increase the pixel value of the entire image by approximately the same proportion, and the overall brightness of the image will be improved; and the larger slope is conducive to improving the contrast between local pixels in low-light images. The DBWEB process is defined as:

[0088]

[0089] Among them, φ represents the Sigmoid activation function, represents a 3x3 depth convolutional layer, represents a 1x1 depth convolutional layer, ⊙ represents an element-wise multiplication operation, Concat(·) represents a concatenation operation, and w d Represents a 1x1 depthwise convolutional layer.

[0090] 6. After analyzing the structure of the network model and the data of low-light images, it was found that: in the encoder stage, after the low-light images have passed through multiple LLAB modules and downsampling, a large amount of detail information of the low-light images is lost, and the encoder structure of the model is not conducive to the fusion of multi-scale information. Therefore, the present invention designs a multi-scale feature compensation module (MSFCB) to ensure the quality of the final restored image. Its structure is as follows Figure 4 As shown. MSFCB consists of three MSC modules, and their three input feature matrices are the outputs of the first LLAB module of the DAMFFN encoder. The output feature matrices of the three MSC modules have different sizes. The output feature matrices of the MSC modules need to be added to the corresponding downsampling results of the DAMFFN encoder to complete the fusion of features of different scales. It is expressed as follows:

[0091]

[0092] 7. The network model designed by the present invention belongs to a multi-scale low-light image enhancement network, so the present invention designs a multi-scale loss function (Mult-Loss). Mult-Loss consists of a multi-scale MSE loss function and a multi-scale frequency domain loss function. Among them, the multi-scale MSE loss function (Mult-MSE) is to calculate the mean square error (MeanSquare Error) between the output image of each decoding layer and the corresponding reference image. The multi-scale frequency domain loss function (MSFD) is to reduce the difference in frequency domain space between the output image of each decoding layer and the corresponding reference image.

[0093] The calculation method of the multi-scale MSE loss function (Mult-MSE) is as follows:

[0094]

[0095] Among them, Nums=4 represents the 4 outputs of the decoder, H represents the height of the feature matrix, and W represents the width of the feature matrix. and Represent the pixel values ​​at the corresponding coordinates of the enhanced image and the reference image respectively, and ∑ represents the summation symbol.

[0096] The calculation method of the multi-scale frequency domain loss function (Mult-SFD) is as follows:

[0097]

[0098] Among them, x i represents the total number of pixels, f(·) represents the fast Fourier transform, ||·||1 represents the L1 norm, and ∑ represents the summation symbol.

[0099] Finally, the total loss function Loss of the present invention is as follows:

[0100] Loss=Loss Mult-MSE +λLoss Mult-SFD

[0101] Wherein, λ represents the weight hyperparameter of Mult-SFD (λ=0.1).

[0102] In a specific embodiment:

[0103] DAMFFN is implemented based on Python 3.9 and Pytorch 1.21.1 environment. The present invention performs data augmentation operations such as folding and rotating the training data (image rotation 90°, 180° or 270°). During the training process, the batch size is 1, and the selected image is randomly cropped into a slice size of 256×256×3. Gradient optimization uses the AdamW optimizer, with momentum terms β1=0.9, β2=0.999. The initial learning rate is set to 3e-4, the weight decay is set to 0.5e-4, and the learning rate is gradually reduced to 1e-6 using the cosine decay strategy. The present invention uses peak signal-to-noise ratio (PSNR), structural similarity (SSIM) and natural image quality evaluator (NIQE) to evaluate the performance of the model, and conducts qualitative and quantitative experimental comparisons with many existing low-light image enhancement algorithms on LOL, MIT-Adobe FiveK, LIME, MEF, and DICM low-light image datasets. PSNR evaluates the quality of an image by calculating the mean square error between the low-light image enhanced by the network model and the reference image (ground-truth image). The higher the PSNR value, the better the quality of the enhanced image. SSIM evaluates the similarity between the enhanced image and the reference image (ground-truth image) in terms of brightness, contrast, and structure. The larger the SSI M value, the more similar the enhanced image is to the reference image (ground-truth image), and the better the network model effect. NIQE is a distance metric between the model statistics and the enhanced low-light image based on the spatial domain natural scene statistical model. The lower the NIQE value, the closer the image is to the natural image and the higher the image quality. In the implementation of DAMFFN, the experimental equipment of the present invention is configured as a 16GB NVIDIA Quadro RTX5000 GPU, C=48, N1=2, N2=N3=4, N4=8.

[0104] The LOL dataset is a commonly used real dataset in the field of low-light image enhancement. The dataset contains a large number of indoor and outdoor low-light scene images, with a total of 500 pairs of low-light / normal-light images. As shown in Table 1 below, our method achieved the best results in both PSNR and SSIM. The results of the present invention are PSNR=24.87dB, SSIM=0.856, the PSNR value exceeds MIRNetv2 by 0.13dB, and the SSIM value exceeds 0.005.

[0105] Table 1 Quantitative comparison of the PSNR and SSIM indicators of the LOL test set (the best result is marked with ※, and the suboptimal result is marked with #)

[0106]

[0107] The MIT-Adobe FiveK dataset contains 5,000 captured images, and images retouched by 5 experts can be used as label images. Like other low-light image enhancement algorithms trained on this dataset, the present invention uses the retouching results of expert C as the ground truth, and uses the first 4,500 pairs as the training set and the remaining 500 pairs as the test set. As shown in Table 2, by comparing with existing advanced algorithms, the present invention achieved the best results at PSNR = 25.78dB (surpassing MIRNetv2 1.73dB), and achieved good results at SSIM = 0.912.

[0108] Table 2 Quantitative comparison of the PSNR and SSIM indicators of the MIT-Adobe FiveK test set (the best result is marked with ※, and the suboptimal result is marked with #)

[0109]

[0110] See also Figure 6 It can be seen that: through multiple qualitative comparisons, it is found that the algorithm of the present invention can achieve better visual effects. There is still a lot of noise in the images enhanced by most algorithms, such as KinD and MIRNetv2. The overall brightness of the enhancement results of some algorithms is not enough, such as Zero-DCE and EnlightenGAN. Among them, although the Retinex-Net method improves the brightness of the overall image, it also adds a lot of noise to the image, making the image very blurry. Relatively speaking, the algorithm of the present invention can improve the overall brightness of the image while ensuring the image quality, and remove the noise in the enhanced image. The restored image has good visibility, contrast and clear texture.

[0111] By comparing the experimental table 3, it can be seen that the present invention conducted 6 ablation experiments to determine the effectiveness and importance of each module. Finally, it is proved by experiments that when all modules are added to DAMFFN, the performance of DAMFFN can reach the best. As shown in Table 3, on the LOL data set, when there is only LL-MSAB or DBWEB, the performance of the network model has not reached the best, and the gap with the optimal performance is large. When the LL-MSAB module is missing, the performance of the network model reaches the worst, which is enough to show that the LL-MSAB module can improve the performance of the entire network model. When there is only the LL-MSAB module, the PSNR of the network model is 23.25dB and SSIM is 0.812. After the DBWEB module is added, the PSNR value of the network model has hardly changed, but the SSIM value has increased by 0.003. This is because the sigmoid function in DBWEB can increase the difference between pixels and the contrast of local areas, so DBWEB can increase the structural similarity of the entire image. When only the MSFCB module and the LL-MSAB module are added, the performance of DAMFFN is greatly reduced, which further illustrates the effectiveness of the DBWEB module. After adding the MSFCB module, the PSNR value of the network model increased from 23.22dB to 24.87dB, and the SSIM increased from 0.842 to 0.856, of which the PSNR increased by 1.65dB and the SSIM increased by 0.014. This is because the MSFCB module can fully compensate for the loss of low-light image detail information in the LLAB and downsampling stages, and can integrate the deep spatial information of images of different scales. At the same time, it also proves that the MSFCB module can greatly improve the performance of the model.

[0112] Table 3 Quantitative results of ablation experiments on PSNR and SSIM indicators of each component of the DAMFFN network model on the LOL test set

[0113]

[0114] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A low-light image enhancement method based on deep convolutional attention and multi-scale feature fusion, characterized by: The method comprises the following steps: Constructing a paired data set, the data set comprising low-light images and normal-light images, each low-light image corresponding to a normal-light image of the same scene; Using the normal light image as the reference image, the shallow features of the low light image are extracted through the Embedding module; Input the shallow features into a DAMFFN encoder, wherein the DAMFFN encoder includes a LLAB module, a downsampling module, and an MSFCB module; the LLAB module includes a LL-MSAB module and a DBWEB module; Through the LL-MSAB module, the semantic information of the low-light image is extracted, and the feature weights between different channels are balanced to improve the visibility of the image; the LL-MSAB module includes deep convolution, channel attention mechanism, and channel attention map. The LL-MSAB module uses deep convolution to extract the semantic information and local features of different channels, increase the local contrast of the image, and make the texture of the image clearer; secondly, the channel attention mechanism in the LL-MSAB module calculates the channel attention map by extracting global features; finally, the channel attention map is used to balance the feature weights between different channels of the image matrix; The contrast of low-light images is improved through the DBWEB module; the DBWEB module includes two 1x1 convolutional layers and a 3x3 deep convolutional layer, a Sigmoid activation function, and a 1x1 convolutional layer. The input image is copied into two copies, and the two 1x1 convolutional layers and the 3x3 deep convolutional layers are respectively passed through. The result of one branch processed by the Sigmoid activation function is dot-multiplied with the result of the other branch, and the result of the dot-multiplication is passed through a 1x1 convolutional layer to obtain the output of DBWEB; The image quality is restored and the lost detail information is compensated by the MSFCB module; the MSFCB module includes three MSC modules, and the input feature matrices of the three MSC modules are all outputs of the first LLAB module of the DAMFFN encoder; the output feature matrices of the three MSC modules have different sizes, and the output feature matrices of the MSC modules are also added to the corresponding downsampling results of the DAMFFN encoder to complete the fusion of features of different scales; In the DAMFFN decoder, the difference between the enhanced image and the reference image in the frequency domain space is reduced by a multi-scale frequency domain loss function; Outputs the final enhanced low-light image.

2. According to claim 1, a low-light image enhancement method using deep convolutional attention and multi-scale feature fusion is characterized in that: The embedding module uses a convolutional module to increase the number of channels of low-light images from 3 to 48.

3. According to claim 1, a low-light image enhancement method using deep convolutional attention and multi-scale feature fusion is characterized in that: The LL-MSAB module is used to extract semantic information of low-light images and balance feature weights between different channels to improve image visibility, including: Shallow feature X input It is input into a 3x3 deep convolutional layer to extract image semantic information to implicitly model the local relationship between pixels in the channel, and obtain X, which is directly assigned to X Q , X K and X V ; X Q , X K and X V Reshape into three identical feature matrices Q, K and V, expressed as follows: X Q =X input W Q ,X K =X input W K ,X V =X input W V W Q =W K =W V ,X Q =X K =X V =X Q=Reshape(X Q ),K=Reshape(X K ),V=Reshape(X V ) Q=K=V X input represents the input feature matrix of the LL-MSAB module, W Q , W K and W V represents the weight matrix of the 3x3 depth convolutional layer, and Reshape(·) represents the reshaping operation; Q(Q=[Q1,...,Q N ]), K(K=[K1,...,K N ]) and V(V=[V1,...,V N ]) Divided into N heads j , where each head j The channel dimension is C represents the total number of channels of the feature matrix; LL-MSAB calculates each head j Channel attention map Atten j , which is expressed as follows: head j =V j ·Atten j K j The transposed matrix of j represents a learnable parameter; Splicing N heads j， And connected into a linear projection, the obtained feature matrix is ​​reshaped, and finally, the reshaped result is input into the 1x1 convolution layer to obtain the output result X of LL-MSAB output , which is expressed as follows: Conv1(·) represents a 1x1 convolutional layer, Concat(·) represents a concatenation operation, W is a learnable parameter, and Reshape(·) represents a reshaping operation.

4. The method for low-light image enhancement based on deep convolutional attention and multi-scale feature fusion according to claim 1, characterized in that: The method of improving the contrast of low-light images by using the DBWEB module includes: The input image is copied into two copies, passing through a 1x1 convolution layer and a 3x3 depth convolution layer respectively; Perform a dot multiplication on the result of one branch processed by the Sigmoid activation function and the result of the other branch; The result of the dot product is passed through a 1x1 convolutional layer to obtain the output of DBWEB, which is expressed as follows: Among them, φ represents the Sigmoid activation function, represents a 3x3 depth convolutional layer, represents a 1x1 depth convolutional layer, ⊙ represents an element-wise multiplication operation, Concat(·) represents a concatenation operation, and w d Represents a 1x1 depthwise convolutional layer.

5. The method for low-light image enhancement based on deep convolutional attention and multi-scale feature fusion according to claim 1, characterized in that: The multi-scale frequency domain loss function is expressed as follows: Among them, Nums=4 represents the 4 outputs of the decoder, H represents the height of the feature matrix, and W represents the width of the feature matrix. and Respectively represent the pixel values ​​at the corresponding coordinates of the enhanced image and the reference image, ∑ represents the summation symbol, The calculation method of the multi-scale frequency domain loss function is as follows: Among them, x i represents the total number of pixels, f(·) represents the fast Fourier transform, ∥·∥1 represents the L1 norm, ∑ represents the summation symbol, The total loss function Loss is as follows: Loss=Loss Mult-MSE +λLoss Mult-SFD Wherein, λ represents the weight hyperparameter of Mult-SFD, λ=0.1.

Images