Low-light image enhancement method integrating cross-hybrid attention and receptive field expansion

By combining cross-mixed attention and receptive field expansion, the problems of regional degradation and low color saturation in low-light image enhancement are solved, and the image brightness and details are improved, which significantly improves visual quality.

CN118212163BActive Publication Date: 2024-12-03SHAANXI NUOYIFEITE OPTOELECTRONICS TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410097222.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-01-24
Publication Date
2024-12-03
Estimated Expiration
2044-01-24

AI Technical Summary

Technical Problem

Existing low-light image enhancement methods have problems such as area degradation, low color saturation, low contrast and loss of details, resulting in the image not achieving satisfactory visual quality.

Method used

Using the method of fusion cross-mixed attention and receptive field expansion, the images are decomposed into reflectivity and illumination through U-shaped network, and the light and noise are adjusted using a multi-scale lighting mechanism, combining the cross-mixed attention mechanism and receptive field expansion mechanism to reduce negative aspects. Interfere and strengthen details and use the color consistency loss function to restore color.

Benefits of technology

It effectively improves the brightness and detailed information of low-light images, reduces negative impacts, realizes color recovery and details enhancement, and significantly improves the visual quality of the image.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118212163B_ABST
    Figure CN118212163B_ABST
Patent Text Reader

Abstract

The invention relates to the field of computer vision technology, and specifically discloses a low-light image enhancement method integrating cross-mixed attention and receptive field expansion, comprising: an initial module, an optimization module, and detail recovery three parts; in the initial module, two different U-shaped networks are used to decompose the image into a reflection component and an illumination component through multi-layer convolution; in the optimization network module, a multi-scale illumination mechanism is used to enhance the image brightness and perform denoising at the same time; multi-layer convolution is used to integrate the cross-mixed attention mechanism to filter the information of four aspects, namely, channel, space, vertical and horizontal attention; finally, in the detail recovery module, a receptive field expansion mechanism is used to enhance the receptive field and strengthen the detail information; at the same time, a joint loss function is used to enhance the image information, wherein a color consistency loss function is used to perform color recovery; the invention can solve the problems of low color saturation, low contrast and detail loss after low-light image enhancement.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a low-light image enhancement method integrating cross-hybrid attention and receptive field expansion. Background Art

[0002] Currently, low-light image enhancement is an important field in the field of computer vision. Its main purpose is to enhance the visual quality of images in low-light environments. Low-light image enhancement has a wide range of applications, such as nighttime aerial photography, nighttime self-driving cars, etc. Low-light image enhancement methods are mainly divided into three categories: distribution mapping-based methods, model optimization-based methods, and deep learning-based methods.

[0003] At present, deep learning-based methods have become the mainstream method for low-light image enhancement. In 2020, the TBEFN method adopted a two-branch average estimation scheme to fuse images end-to-end and refine the results, but the performance needs to be optimized. In 2021, Jiang et al. proposed the EnlightenGAN method for self-supervised low-light image enhancement, which trains the generator and discriminator, and adds self-attention mechanism and self-regularization perceptual loss for enhancement. In 2022, the URetinex-Net method added new constraints and advanced network design on the basis of the Retinex-Net method, thereby achieving better performance. In 2023, Fu et al. proposed improvements to the Retinex theory, enhancing low-light images by learning adaptive prior knowledge and removing inappropriate features of the original image. However, existing models still have problems with regional degradation, such as low color saturation, low contrast, and loss of details, which makes it impossible for the entire image to achieve satisfactory visual quality.

[0004] Therefore, the present invention proposes a low-light image enhancement method that integrates cross-hybrid attention and receptive field expansion. Summary of the invention

[0005] The present invention provides a low-light image enhancement method that integrates cross-hybrid attention and receptive field expansion, which uses a U-shaped network and Retinex theory to decompose an image into reflectance and illumination rate. Under the action of multi-layer convolution, the normal light image shares parameters with the low-light image; a multi-scale illumination mechanism is used to adjust illumination and noise, and a cross-hybrid attention mechanism is integrated after convolution to remove negative interference from four aspects: channel, space, vertical and horizontal attention. Using the receptive field expansion mechanism, hole convolutions of different sizes form a multi-branch structure to enhance the perception of the image, and the color is restored using the color consistency loss function.

[0006] The present invention provides a low-light image enhancement method integrating cross-hybrid attention and receptive field expansion, comprising:

[0007] S1: Get low-light image dataset;

[0008] S2: Based on two different U-shaped networks in the initial module and after multiple layers of convolution, the low-light images in the low-light image dataset are decomposed into reflection components and illumination components;

[0009] S3: Using the multi-scale illumination mechanism in the optimization network module and the reflection component and illumination component in the low-light image, the image brightness of the low-light image in the low-light image dataset is enhanced, and denoising is performed at the same time to obtain the final output feature map;

[0010] S4: Based on the multi-layer convolution fusion cross-mixed attention mechanism, the information of channel, space, vertical and horizontal attention is filtered to reduce the negative impact of the final output feature map in the enhancement process and obtain the cross-mixed attention feature map;

[0011] S5: The receptive field expansion mechanism in the detail restoration module is used to enhance the receptive field of the cross-mixed attention feature map, strengthen the detail information, and obtain the detail feature map. At the same time, the color consistency loss function is used to restore the color of the cross-mixed attention feature map to obtain the image enhancement result of the low-light image dataset.

[0012] Preferably, in step 1,

[0013] Obtain the LOL dataset, which consists of 500 pairs of low-light images and normal-light images taken in real-world environments.

[0014] Filter out the low-light image dataset in the LOL dataset.

[0015] Preferably, in step S2,

[0016] The U-shaped network of the reflection component and the U-shaped network of the illumination component are both composed of 3*3 convolution, rule function and maximum pooling;

[0017] Based on two different U-shaped networks, feature maps with output channels of 32 and 64 are extracted from the input low-light image, and then the important features of the input low-light image are retained through the bottleneck layer feature map with an output channel of 128;

[0018] All feature maps are upsampled using concat and deconvolution to restore the image information of the input low-light image, thus forming a U-shaped network for the illumination component;

[0019] The reflection component U-type network, based on the illumination component U-type network, uses a 3*3 convolution with rule activation function for feature extraction to form the final reflection component U-type network, and then feeds the sigmoid function to constrain the reflection component and extract the illumination component.

[0020] Preferably, S3: utilizing the multi-scale illumination mechanism in the optimization network module and the reflection component and illumination component in the low-light image to enhance the image brightness of the low-light image in the low-light image dataset, and simultaneously performing denoising to obtain a final output feature map, including:

[0021] The multi-scale illumination mechanism is performed by the illumination adjustment module and the multi-scale sampling module;

[0022] The illumination component is sent into the network, and the illumination adjustment is performed through a 3×3 convolution and a sigmoid function to obtain the illumination information, and the input feature map and the illumination information are multiplied;

[0023] The multi-scale sampling module obtains the first-layer feature map from the illumination adjustment module;

[0024] Use 3×3 convolution, normalization and rule function to obtain the second layer feature map;

[0025] The third layer of feature maps consists of maximum pooling, 3×3 convolution to obtain the number of channels, normalization, rule function, and 2×2 deconvolution to obtain the number of channels;

[0026] The fourth convolution layer consists of maximum pooling, 1×1 convolution to obtain the number of channels, normalization, relu function, and two 2×2 deconvolutions to obtain the number of channels;

[0027] Get the number of channels for convolution and use the channel information to get the details;

[0028] The four layers of feature maps are stacked and the final output feature map is obtained through a 1×1 convolution.

[0029] Preferably, S4: Based on a multi-layer convolution fusion cross-mixed attention mechanism, information in four aspects, namely, channel, space, vertical and horizontal attention, is filtered to reduce the negative impact of the final output feature map in the enhancement process, and a cross-mixed attention feature map is obtained, including:

[0030] The size of the adaptive convolution kernel is calculated based on the channel attention mechanism and the number of channels of the input feature map. After global average pooling, one-dimensional convolution is used to replace the fully connected layer.

[0031] The channel attention mechanism does not have a large number of convolutional structures and uses fewer parameters to achieve cross-channel information interaction;

[0032] The spatial attention mechanism processes the feature map in the spatial domain. After maximum pooling and average pooling, the maps are stacked, two-dimensional convolution is used to fuse the spatial feature information, and the sigmoid function is used for normalization.

[0033] Both the channel attention mechanism and the spatial attention mechanism multiply the input feature map with the relevant weights and output it;

[0034] The channel attention mechanism and the spatial attention mechanism are combined to send their respective output feature maps into the row-column attention mechanism to obtain a cross-mixed attention feature map.

[0035] Preferably, the channel attention mechanism and the spatial attention mechanism are combined to send their respective output feature maps into the row-column attention mechanism to obtain a cross-mixed attention feature map, including:

[0036] The image average of the feature map row is calculated by the reduce_mean function, and then the row attention and column attention are calculated using the two-dimensional convolution and softmax functions. When the row and column attention maps are obtained, the row and column attention information in the row and column attention maps are shared and stacked, and finally the cross-mixed attention feature map is output.

[0037] Preferably, S5: using the receptive field expansion mechanism in the detail restoration module to enhance the receptive field of the cross-mixed attention feature map, strengthening the detail information, obtaining the detail feature map, and using the color consistency loss function to perform color restoration on the cross-mixed attention feature map to obtain the image enhancement result of the low-light image dataset, including:

[0038] By stacking the nearest neighbor interpolation, rich feature information is obtained, and the context information is obtained through the receptive field expansion mechanism to enhance the feature details;

[0039] The receptive field expansion mechanism consists of five branches, four of which use 1×1 convolution, 1×3 convolution, and 3×1 convolution to obtain feature maps of different sizes;

[0040] Use dilated convolution with dilation rates of 1, 3, and 5 to expand the receptive field, extract features of different scales and positions in the image, and enhance the response of neurons to input data;

[0041] The feature maps of four different expansion rates are stacked, fused through 1×1 convolution, and finally added to the feature map of the previous layer after 1×1 convolution to output the detail feature map;

[0042] The color information is corrected and restored using the color consistency loss function to obtain the image enhancement results of the low-light image dataset.

[0043] Preferably, the color consistency loss function includes:

[0044]

[0045]

[0046]

[0047]

[0048]

[0049] L e =MSE(R l ,R h )+0.5SSIM(R l ,R h )+0.5Grad(R l ,R h )+color

[0050]

[0051] Among them, the color consistency loss function is composed of the reflection component loss Light loss Consistency loss and error loss The total loss function of the initial module is L d ; The loss function of the optimization module is composed of the mean square error loss function MSE, the structural similarity loss function SSIM and the gradient loss function Grad. The total loss function of the optimization module is L e ; The loss function of the detail recovery module is composed of the color consistency loss function color; ||·||1 is the paradigm L1; R l is the reflection component of the low-light image, R h is the reflection component of the normal light image; I l is the illumination component of the low-light image, I h is the component of the normal light image; L l For low-light images, L h is a normal light image; is the first-order derivative operator in the horizontal and vertical directions; ε is 0.01 to avoid the denominator being zero; |·| is the absolute value; and c is the number of color channels.

[0052] Preferably, the low-light image dataset is selected from the LOL dataset, including:

[0053] Calculate the sum of the brightness values ​​of all pixels in each image in the LOL dataset as the sum of the brightness values ​​of each image;

[0054] Determine the maximum and minimum brightness values ​​in each image in the LOL dataset;

[0055] Calculate the product of the difference between the maximum brightness value and the minimum brightness value in each image and the preset ratio as the brightness screening threshold accumulation value of each image;

[0056] The sum of the minimum brightness value of each image and the accumulated value of the brightness screening threshold is used as the brightness screening threshold of each image;

[0057] Based on the brightness screening threshold of each image, the total number of pixels that do not exceed the brightness screening threshold is screened out in each image as the total number of low-light pixels of each image;

[0058] Based on the sum of the brightness values ​​of each image in the LOL dataset and the total number of low-light pixels, a low-light image dataset is screened out from the LOL dataset.

[0059] Preferably, based on the sum of the brightness values ​​of each image in the LOL dataset and the total number of low-light pixels, a low-light image dataset is screened out from the LOL dataset, including:

[0060] Based on the total number of pixels contained in each image in the LOL dataset and the preset ratio, the low-light pixel total threshold is calculated;

[0061] Based on the total number of pixels contained in each image in the LOL dataset and the preset low-light brightness value per pixel, a low-light brightness value sum threshold of each image is calculated;

[0062] In the LOL dataset, images whose total brightness value does not exceed the low-light illumination value total threshold and whose total number of low-light pixels is not less than the total number of low-light pixels threshold are selected and summarized to obtain a low-light image dataset.

[0063] The beneficial effects of the present invention compared with the prior art are as follows: In the present invention, a low-light image enhancement method integrating cross-mixed attention and receptive field expansion is proposed, which is divided into three parts: an initial module, an optimization module, and detail restoration; the multi-scale illumination mechanism in the initial module solves the problems of low illumination and low contrast to adapt to more application environments; the cross-mixed attention mechanism is utilized to remove negative information generated in the feature fusion process from four aspects: channel, space, horizontal and vertical attention; in view of the problems of low details and color saturation, the receptive field expansion mechanism is utilized to enhance the receptive field, strengthen the detail information, and the color loss function is used for color restoration; the method of the present invention is evaluated by objective evaluation indicators on the LOL dataset, and the results show that the method of the present invention has a good effect on low-light image enhancement.

[0064] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures specifically pointed out in this application document.

[0065] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0067] Figure 1 It is the overall framework diagram of CE-Retinex in the embodiment of the present invention;

[0068] Figure 2 A module diagram of a multi-scale illumination mechanism in an embodiment of the present invention;

[0069] Figure 3 Module diagram of the cross-hybrid attention mechanism in an embodiment of the present invention;

[0070] Figure 4 Schematic diagram of the receptive field expansion mechanism module in an embodiment of the present invention;

[0071] Figure 5 It is a comparison diagram of various method experiments in the embodiments of the present invention;

[0072] Figure 6 It is a comparison diagram of RGB histogram in an embodiment of the present invention. DETAILED DESCRIPTION

[0073] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0074] Embodiment 1:

[0075] like Figure 1 As shown, the low-light image enhancement algorithm integrating cross-mixed attention and receptive field expansion includes the following steps: In order to better understand the technical content of the present invention, an embodiment is specifically cited and described as follows in conjunction with the attached drawings.

[0076] S1. Obtain a low-light image dataset. The LOL dataset is selected. The LOL dataset consists of low-light images and normal-light images taken in real-world environments, with a total of 500 pairs of low-light images and normal-light images.

[0077] S2. Low-light image enhancement Obtaining reflectivity and illumination information is the key. The reflectivity of an object is determined by the surface properties of the object itself, while the ambient light component is the illumination from an external light source. Therefore, the initial module of this paper uses the Retinex theory to simulate the way the human visual system perceives image details and brightness. The Retinex theory considers an image as two components, namely illumination and reflectivity. The initial module structure diagram is as follows: Figure 1 (a) is shown. Retinex theory can preserve the information of the image. The initial module obtains the illumination and reflectance information from the input normal light image. The illumination is used to restore the image brightness, and the reflectance carries the details and possible degradation. When the low-light image is input to the initial module, the low-light image learns the illumination and reflectance information of the normal light image in the training data set; the U-type network of the reflectance map and the U-type network of the illumination map are both composed of 3*3 convolution, rule function and maximum pooling. They extract feature maps with output channels of 32 and 64 respectively from the input image, and then pass through the bottleneck layer feature map with an output channel of 128 to retain important features, and use concat and deconvolution to upsample the feature map to restore the image resolution. Thus, the illumination map U-type network is formed. However, in the reflection map U-type network module, more detailed information is required, so after the above convolution, a 3*3 convolution with the activation function of rule is used for feature extraction to form the final reflection map U-type network, and then the sigmoid function is fed into the reflection map and illumination map to constrain the reflection map. Finally, the illumination map and the reflection map are fused and sent to the optimization module. The initial module retains sufficient feature information and lighting information, which helps the optimization module and detail recovery module to remove negative interference and restore features of low-light images.

[0078] S3. Reference Figure 2As shown in the figure, the multi-scale illumination mechanism consists of an illumination adjustment module and a multi-scale sampling module. First, the illumination map is sent to the network, and the illumination is adjusted and the illumination information is obtained through a 3×3 convolution and a sigmoid function. The input feature map and the illumination information are multiplied to guide the network to focus on the area with the most severe illumination degradation. The multi-scale sampling module obtains the feature map from the illumination adjustment module and uses four branches to obtain different feature information. First, the first layer of feature maps is obtained from the illumination adjustment module. The second layer of feature maps is obtained by using 3×3 convolution, normalization and rule function to obtain the number of channels. The third layer of feature maps consists of maximum pooling, 3×3 convolution, normalization, rule function and 2×2 deconvolution. The fourth layer of convolution consists of maximum pooling, 1×1 convolution, normalization, relu function and two 2×2 deconvolutions. Convolution of the number of channels is obtained, and the channel information is used to obtain detail information. Finally, the four layers of feature maps are stacked and a 1×1 convolution is performed to obtain the final output feature map. The multi-scale sampling module extracts rich features, thereby achieving the effect of removing uneven spots and noise.

[0079] S4. References Figure 3 As shown in Figure 1, in order to address the impact of negative information generated during the feature fusion process, the cross-hybrid attention mechanism is used to remove negative interference from four aspects: channel, space, vertical and horizontal attention. Figure 3 As shown in the figure. First, the channel attention mechanism calculates the size of the adaptive convolution kernel according to the number of channels of the input feature map. After global average pooling, one-dimensional convolution is used to replace the fully connected layer. The channel attention mechanism does not have a large number of convolution structures and uses fewer parameters to achieve cross-channel information interaction. The spatial attention mechanism processes the feature map in the spatial domain. After maximum pooling and average pooling, it is stacked, and two-dimensional convolution is used to fuse the spatial feature information, and the sigmoid function is used for normalization. Both the channel attention mechanism and the spatial attention mechanism multiply the input feature map with the relevant weights and output them to prevent the feature details from being ignored. The channel attention mechanism and the spatial attention mechanism merge and send their respective output feature maps to the row-column attention mechanism. The row-column attention mechanism focuses on the vertical and horizontal feature information of the input feature map. First, the image average of the feature map row is calculated by the reduce_mean function, and then the row attention is calculated by two-dimensional convolution and softmax function, and the row attention is applied to the column. The column attention is the same operation. When the row and column attention maps are obtained respectively, the row and column attention information is shared and stacked, and finally the cross-mixed attention feature map is output.

[0080] S5. Reference Figure 4As shown in Figure 1, the receptive field expansion mechanism enables neurons to receive information from a wider area, not just information from a local area. The receptive field expansion mechanism helps improve the network's understanding of the overall structure and relevance of the input data, thereby achieving the effect of strengthening detailed information. The structure of the receptive field expansion mechanism is shown in Figure 1. Figure 4 As shown in the figure. The receptive field expansion mechanism consists of five branches, four of which use 1×1 convolution, 1×3 convolution, and 3×1 convolution to obtain feature maps of different sizes, and use dilated convolutions with dilation rates of 1, 3, and 5 to expand the receptive field, extract features of different scales and positions in the image, and enhance the response of neurons to input data. The feature maps of four different dilation rates are stacked, fused through 1×1 convolution, and finally added to the feature map of the previous layer after 1×1 convolution to output the detail feature map.

[0081] refer to Figure 5 As shown in Figure 1, three images are selected for testing. The first column is the original input image, the second column is the Retinex-Net algorithm experimental result graph, the third column is the KinD algorithm experimental result graph, the fourth column is the KinD++ algorithm experimental result graph, and the fifth column is the experimental result graph of the present invention. Figure 5 It can be seen that the Retinex-Net algorithm, KinD algorithm and KinD++ algorithm have the problems of blur and color distortion compared with the present invention; in summary, it can be seen that the algorithm of the present invention has achieved better visual effects.

[0082] refer to Figure 6 As shown, the first line is the RGB color distribution of the original low-light input image, and the second line is the RGB color distribution of the low-light image enhanced by the present invention. From the figure, we can see that the color distribution of the enhanced low-light image is more uniform, and color restoration is achieved.

[0083] The environment used in the training experiment is NVIDIA GeForce RTX 3060, based on the TensorFlow-GPU2.4 framework, Python version 3.7, and Adam is used as the optimizer during model training. The specific parameters are shown in Table 1.

[0084] Table 1 Experimental parameter settings

[0085]

[0086] In order to verify the effectiveness of each module of the algorithm in this paper, an ablation experiment is performed on the LOL dataset. As shown in Table 2, the reference indicator PSNR and the non-reference indicator NIQE are used for comparison. Peak signal-to-noise ratio (PSNR) is a commonly used image quality evaluation indicator used to compare image noise. The natural image quality evaluator (NIQE) measures the differences in the multivariate distribution of natural images. On the basis of the basic modules of the algorithm, the receptive field expansion mechanism, the cross-hybrid attention mechanism and the multi-scale illumination mechanism are added one by one, and the effectiveness of each module of the CE-Retinex algorithm is analyzed by comparing the experimental results. The bold ones are the best results.

[0087] Table 2 Ablation experiment analysis table

[0088]

[0089]

[0090] Embodiment 2:

[0091] Based on Example 1, a low-light image dataset is selected from the LOL dataset, including:

[0092] Calculate the sum of the brightness values ​​of all pixels in each image in the LOL dataset as the sum of the brightness values ​​of each image;

[0093] Determine the maximum and minimum brightness values ​​in each image in the LOL dataset;

[0094] Calculate the product of the difference between the maximum brightness value and the minimum brightness value in each image and the preset ratio as the brightness screening threshold accumulation value of each image;

[0095] The sum of the minimum brightness value of each image and the accumulated value of the brightness screening threshold is used as the brightness screening threshold of each image;

[0096] Based on the brightness screening threshold of each image, the total number of pixels that do not exceed the brightness screening threshold is screened out in each image as the total number of low-light pixels of each image;

[0097] Based on the sum of the brightness values ​​of each image in the LOL dataset and the total number of low-light pixels, a low-light image dataset is screened out from the LOL dataset.

[0098] In this embodiment, the preset ratio is a preset ratio for calculating the accumulated value of the brightness screening threshold value, and the preset ratio is greater than 0.5 and less than 1, for example, it can be 0.7.

[0099] In this embodiment, the brightness screening threshold accumulated value is a value that needs to be accumulated based on the minimum brightness value of the corresponding image when determining the brightness screening threshold.

[0100] In this embodiment, the brightness screening threshold is a screening threshold for screening out the brightness of low-light pixels in the corresponding image.

[0101] The beneficial effect of the above technology is: it is used to determine the total number of low-light pixels and the sum of the brightness values ​​of each image in the LOL dataset by using the sum of the brightness values, the maximum brightness value, and the minimum brightness value, and based on this, further filter out the low-light image dataset in the LOL dataset.

[0102] Embodiment 3:

[0103] On the basis of Example 2, based on the sum of the brightness values ​​of each image in the LOL dataset and the total number of low-light pixels, a low-light image dataset is screened out from the LOL dataset, including:

[0104] Based on the total number of pixels contained in each image in the LOL dataset and the preset ratio, the low-light pixel total threshold is calculated;

[0105] Based on the total number of pixels contained in each image in the LOL dataset and the preset low-light brightness value per pixel, a low-light brightness value sum threshold of each image is calculated;

[0106] In the LOL dataset, images whose total brightness value does not exceed the low-light illumination value total threshold and whose total number of low-light pixels is not less than the total number of low-light pixels threshold are selected and summarized to obtain a low-light image dataset.

[0107] In this embodiment, the low-light pixel total threshold is the minimum value that the total number of low-light pixel points included in the low-light image selected as the low-light image data set needs to reach.

[0108] In this embodiment, the average of the quotient of the brightness value of all pixels of each low-light image in the multiple low-light images that meet the requirements and the total number of all pixels of the corresponding low-light image is used as the preset low-light brightness value of the unit pixel.

[0109] In this embodiment, the low-light illumination value sum threshold is a value that the brightness values ​​of all pixels in the low-light image selected as the low-light image data set cannot exceed.

[0110] The beneficial effects of the above technology are: using the sum of the brightness values ​​of each image in the LOL dataset and the total number of low-light pixels, the low-light brightness value sum threshold and the low-light pixel total threshold of each image are calculated, and using the low-light brightness value sum threshold and the low-light pixel total threshold of each image, the low-light image dataset that needs to perform image enhancement operations in the LOL dataset is screened out.

[0111] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A low-light image enhancement method integrating cross-hybrid attention and receptive field expansion, characterized in that: include: S1: Get low-light image dataset; S2: Based on two different U-shaped networks in the initial module and after multiple layers of convolution, the low-light images in the low-light image dataset are decomposed into reflection components and illumination components; S3: Using the multi-scale illumination mechanism in the optimization network module and the reflection component and illumination component in the low-light image, the image brightness of the low-light image in the low-light image dataset is enhanced, and denoising is performed at the same time to obtain the final output feature map; S4: Based on the multi-layer convolution fusion cross-mixed attention mechanism, the information of channel, space, vertical and horizontal attention is filtered to reduce the negative impact of the final output feature map in the enhancement process and obtain the cross-mixed attention feature map; S5: Use the receptive field expansion mechanism in the detail restoration module to enhance the receptive field of the cross-mixed attention feature map, strengthen the detail information, and obtain the detail feature map. At the same time, use the color consistency loss function to restore the color of the cross-mixed attention feature map to obtain the image enhancement result of the low-light image dataset; Among them, in step S2, The U-shaped network of the reflection component and the U-shaped network of the illumination component are both composed of 3*3 convolution, rule function and maximum pooling; Based on two different U-shaped networks, feature maps with output channels of 32 and 64 are extracted from the input low-light image, and then the important features of the input low-light image are retained through the bottleneck layer feature map with an output channel of 128; All feature maps are upsampled using concat and deconvolution to restore the image information of the input low-light image, thus forming a U-shaped network for the illumination component; The reflection component U-type network, based on the illumination component U-type network, uses a 3*3 convolution with the activation function as rule for feature extraction to form the final reflection component U-type network, and then feeds the sigmoid function to constrain the reflection component and extract the illumination component; Among them, S3: utilizes the multi-scale illumination mechanism in the optimization network module and the reflection component and illumination component in the low-light image to enhance the image brightness of the low-light image in the low-light image dataset, and performs denoising at the same time to obtain the final output feature map, including: The multi-scale illumination mechanism is performed by the illumination adjustment module and the multi-scale sampling module; The illumination component is sent into the network, and the illumination adjustment is performed through a 3×3 convolution and a sigmoid function to obtain the illumination information, and the input feature map and the illumination information are multiplied; The multi-scale sampling module obtains the first-layer feature map from the illumination adjustment module; Use 3×3 convolution, normalization and rule function to obtain the second layer feature map; The third layer of feature maps consists of maximum pooling, 3×3 convolution to obtain the number of channels, normalization, rule function, and 2×2 deconvolution to obtain the number of channels; The fourth convolution layer consists of maximum pooling, 1×1 convolution to obtain the number of channels, normalization, relu function, and two 2×2 deconvolutions to obtain the number of channels; Get the number of channels for convolution and use the channel information to get the details; The four layers of feature maps are stacked and the final output feature map is obtained through a 1×1 convolution; Among them, S4: Based on the multi-layer convolution fusion cross-mixed attention mechanism, the information of channel, space, vertical and horizontal attention is filtered to reduce the negative impact of the final output feature map in the enhancement process, and obtain the cross-mixed attention feature map, including: The size of the adaptive convolution kernel is calculated based on the channel attention mechanism and the number of channels of the input feature map. After global average pooling, one-dimensional convolution is used to replace the fully connected layer. The channel attention mechanism does not have a large number of convolutional structures and uses fewer parameters to achieve cross-channel information interaction; The spatial attention mechanism processes the feature map in the spatial domain. After maximum pooling and average pooling, the maps are stacked, two-dimensional convolution is used to fuse the spatial feature information, and the sigmoid function is used for normalization. Both the channel attention mechanism and the spatial attention mechanism multiply the input feature map with the relevant weights and output it; The channel attention mechanism and the spatial attention mechanism are combined to send their respective output feature maps into the row-column attention mechanism to obtain a cross-mixed attention feature map; Among them, the channel attention mechanism and the spatial attention mechanism merge and send their respective output feature maps into the row-column attention mechanism to obtain a cross-mixed attention feature map, including: The image average of the feature map row is calculated by the reduce_mean function, and then the row attention and column attention are calculated by using the two-dimensional convolution and softmax function. When the row and column attention map is obtained, the row and column attention information in the row and column attention map is shared and stacked, and finally the cross-mixed attention feature map is output; Among them, S5: using the receptive field expansion mechanism in the detail restoration module to enhance the receptive field of the cross-mixed attention feature map, strengthen the detail information, obtain the detail feature map, and use the color consistency loss function to restore the color of the cross-mixed attention feature map to obtain the image enhancement results of the low-light image dataset, including: By stacking the nearest neighbor interpolation, rich feature information is obtained, and the context information is obtained through the receptive field expansion mechanism to enhance the feature details; The receptive field expansion mechanism consists of five branches, four of which use 1×1 convolution, 1×3 convolution, and 3×1 convolution to obtain feature maps of different sizes; Use dilated convolution with dilation rates of 1, 3, and 5 to expand the receptive field, extract features of different scales and positions in the image, and enhance the response of neurons to input data; The feature maps of four different expansion rates are stacked, fused through 1×1 convolution, and finally added to the feature map of the previous layer after 1×1 convolution to output the detail feature map; The color information is corrected and restored using the color consistency loss function to obtain the image enhancement results of the low-light image dataset; Among them, the color consistency loss function includes: Among them, the color consistency loss function is composed of the reflection component loss Light loss Consistency loss and error loss The total loss function of the initial module is L d ; The loss function of the optimization module is composed of the mean square error loss function MSE, the structural similarity loss function SSIM and the gradient loss function Grad. The total loss function of the optimization module is L e ; The loss function of the detail recovery module is composed of the color consistency loss function color; ||·||1 is the paradigm L1; R l is the reflection component of the low-light image, R h is the reflection component of the normal light image; I l is the illumination component of the low-light image, I h is the component of the normal light image; L l For low-light images, L h is a normal light image; is the first-order derivative operator in the horizontal and vertical directions; ε is 0.01 to avoid the denominator being zero; |·| is the absolute value; and c is the number of color channels.

2. The low-light image enhancement method integrating cross-hybrid attention and receptive field expansion according to claim 1, characterized in that: In step 1, Obtain the LOL dataset, which consists of 500 pairs of low-light images and normal-light images taken in real-world environments. Filter out the low-light image dataset in the LOL dataset.

3. The low-light image enhancement method integrating cross-hybrid attention and receptive field expansion according to claim 2, characterized in that: The low-light image datasets are selected from the LOL dataset, including: Calculate the sum of the brightness values ​​of all pixels in each image in the LOL dataset as the sum of the brightness values ​​of each image; Determine the maximum and minimum brightness values ​​in each image in the LOL dataset; Calculate the product of the difference between the maximum brightness value and the minimum brightness value in each image and the preset ratio as the brightness screening threshold accumulation value of each image; The sum of the minimum brightness value of each image and the accumulated value of the brightness screening threshold is used as the brightness screening threshold of each image; Based on the brightness screening threshold of each image, the total number of pixels that do not exceed the brightness screening threshold is screened out in each image as the total number of low-light pixels of each image; Based on the sum of the brightness values ​​of each image in the LOL dataset and the total number of low-light pixels, a low-light image dataset is screened out from the LOL dataset.

4. The low-light image enhancement method integrating cross-hybrid attention and receptive field expansion according to claim 3, characterized in that: Based on the sum of the brightness values ​​of each image in the LOL dataset and the total number of low-light pixels, a low-light image dataset is screened out from the LOL dataset, including: Based on the total number of pixels contained in each image in the LOL dataset and the preset ratio, the low-light pixel total threshold is calculated; Based on the total number of pixels contained in each image in the LOL dataset and the preset low-light brightness value per pixel, a low-light brightness value sum threshold of each image is calculated; In the LOL dataset, images whose total brightness value does not exceed the low-light illumination value total threshold and whose total number of low-light pixels is not less than the total number of low-light pixels threshold are selected and summarized to obtain a low-light image dataset.