Method and device for enhancing low-light image, electronic equipment and storage medium

Abstract

The application provides a low-light image enhancement method and device, electronic equipment and storage medium. The method comprises the following steps: obtaining a low-light image to be enhanced; inputting the low-light image into an image decomposition network to obtain a first reflectivity component map and a first illumination component map corresponding to the low-light image; inputting the first reflectivity component map and the first illumination component map into a reflectivity adjustment network to obtain a second reflectivity component map, and inputting the first illumination component map into an illumination adjustment network to obtain a second illumination component map; and obtaining an enhanced image corresponding to the low-light image according to the second reflectivity component map and the second illumination component map. The method can accurately decompose the low-light image by using the image decomposition network, and can adjust the decomposed low-light image from coarse to fine by using the reflectivity adjustment network and the illumination adjustment network, so as to obtain the corresponding reflectivity component and illumination component, thereby effectively improving the accuracy of obtaining the enhanced image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision image, and particularly relates to a low-light image enhancement method and device, electronic equipment and storage medium. BACKGROUND

[0002] With the sustained and rapid development of economy in recent years, the popularity rate of electronic equipment is getting higher and higher, and gradually becomes an irreplaceable part in people's daily life and work. An image is one of important carriers of information, and people can collect image information through the above-mentioned electronic equipment. However, since the light condition in the image collection environment will significantly affect the quality of the image, especially the image collected in the low-light environment will have many degradation problems such as low brightness, low contrast, color distortion and content information loss, which will produce serious quality degradation and affect the subsequent intelligent analysis, therefore, the electronic equipment needs to enhance the collected low-light image to ensure that the image details corresponding to the low-light image can be recovered, the brightness and contrast can be improved, and the noise can be reduced.

[0003] The existing image enhancement methods can be divided into traditional image enhancement methods and deep learning-based image enhancement methods. The former often produces unnatural results and does not consider the removal of noise in the low-light image, so that the quality of the enhanced image is poor, that is, the enhanced image corresponding to the low-light image cannot be accurately obtained; the latter lacks the support of physical models, and has poor interpretability and generalization, and the quality of the enhanced image depends on the design of the network model, so the enhanced image corresponding to the low-light image cannot be accurately obtained.

[0004] In summary, the existing image enhancement methods have certain limitations, which leads to that the electronic equipment cannot accurately obtain the enhanced image corresponding to the low-light image. SUMMARY

[0005] The present application provides a low-light image enhancement method, device, electronic equipment and storage medium, to solve the defects that the existing image enhancement methods have certain limitations, which leads to that the electronic equipment cannot accurately obtain the enhanced image corresponding to the low-light image, and realize that the image decomposition network can pay attention to the connection and change between the reflectance component and the illumination component in the decomposition process of the low-light image, through the interaction between the features and the expansion of the reflectance component, the low-light image can be better decomposed, and then the reflectance adjustment network and the illumination adjustment network are used, the multi-scale information is fully utilized, the reflectance component and the illumination component are adjusted from coarse to fine, and then the accuracy of obtaining the enhanced image can be effectively improved.

[0006] The present application provides a low-light image enhancement method, comprising:

[0007] obtaining a low-light image to be enhanced;

[0008] input the low-light image into an image decomposition network to obtain a first reflectance component map and a first illumination component map corresponding to the low-light image;

[0009] input the first reflectance component map and the first illumination component map into a reflectance adjustment network to obtain a second reflectance component map, and input the first illumination component map into an illumination adjustment network to obtain a second illumination component map;

[0010] obtain an enhanced image corresponding to the low-light image according to the second reflectance component map and the second illumination component map.

[0011] The application further provides a low-light image enhancement device, comprising:

[0012] an acquisition module configured to acquire a low-light image to be enhanced;

[0013] a processing module configured to input the low-light image into an image decomposition network to obtain a first reflectance component map and a first illumination component map corresponding to the low-light image; input the first reflectance component map and the first illumination component map into a reflectance adjustment network to obtain a second reflectance component map, and input the first illumination component map into an illumination adjustment network to obtain a second illumination component map; and obtain an enhanced image corresponding to the low-light image according to the second reflectance component map and the second illumination component map.

[0014] The application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the low-light image enhancement method according to any one of the above embodiments when executing the program.

[0015] The application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program is executable on a processor to implement the low-light image enhancement method according to any one of the above embodiments.

[0016] The application further provides a computer program product comprising a computer program, wherein the computer program is executable on a processor to implement the low-light image enhancement method according to any one of the above embodiments.

[0017] The application provides a low-light image enhancement method and device, electronic equipment and a storage medium. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can also be obtained by those skilled in the art without any creative effort.

[0019] Figure 1 FIG. 1 is a flowchart of a low-light image enhancement method provided by the application;

[0020] Figure 2 FIG. 2 is a structural diagram of an image decomposition network and a feature interaction module FIM provided by the application;

[0021] Figure 3 FIG. 3 is a structural diagram of a feature fusion module FFM provided by the application;

[0022] Figure 4 FIG. 4 is a structural diagram of a reflectivity adjustment network provided by the application;

[0023] Figure 5 FIG. 5 is a structural diagram of an illumination adjustment network provided by the application;

[0024] Figure 6is a structural schematic diagram of the low-light image enhancement device provided by the present application

[0025] Figure 7 is a structural schematic diagram of the electronic device provided by the present application. DETAILED DESCRIPTION

[0026] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0027] It should be noted that the execution subject involved in the embodiments of the present application can be the low-light image enhancement device, or the electronic device. Optionally, the electronic device can include a computer, a mobile terminal, a wearable device, etc.

[0028] The embodiments of the present application will be further described below taking the electronic device as an example.

[0029] As shown in Figure 1 is a flowchart of the low-light image enhancement method provided by the present application, which can include the following steps.

[0030] 101. Obtain a low-light image to be enhanced.

[0031] The low-light image refers to an image with a luminance value less than a preset luminance threshold.

[0032] Optionally, the number of low-light images is not limited.

[0033] Optionally, the preset luminance threshold can be set before the electronic device is shipped, or can be customized by a user, which is not limited here.

[0034] After the electronic device obtains the image, the luminance value of the image can be obtained first. Then, the electronic device determines the relationship between the luminance value and the preset luminance threshold, and determines that the luminance value of the image is low in the case that the luminance value is less than the preset luminance threshold. At this time, the electronic device determines the image as the low-light image to be enhanced.

[0035] 102. Input the low-light image to an image decomposition network to obtain a first reflectance component map and a first illumination component map corresponding to the low-light image.

[0036] The image decomposition network can include a reflectance branch, an illumination branch, a feature interaction branch, etc.

[0037] Optionally, the reflectance branch can include: 1 U-Net structure and 2 feature fusion modules (FFM); the illumination branch can include: 1 U-Net structure and a convolution (Conv) composed of 4 3*3 convolution operations; and the feature interaction branch can include: 4 feature interaction modules (FIM).

[0038] The feature interaction module FIM can include: a channel concatenation operation and two convolution operations. The electronic device can perform interaction between component features (for example: reflectance component and illumination component) in different levels in the low-light image, learn useful information in each component feature, thereby improving the representation ability of the component feature and promoting the feature extraction ability.

[0039] For example, as shown in Table 1, the network parameters of the image decomposition network provided by the present application are shown.

[0040] Table 1

[0041] Cell Down Up Conv1 Conv3 Convolutional / pooling kernel 2*2 2*2 1*1 3*3 Stride 2 2 1 1 Activation function LReLU LReLU LReLU LReLU

[0042] In the formula, Conv1 represents a convolution operation with a convolution kernel size of 1*1; and Conv3 represents a convolution operation with a convolution kernel size of 3*3.

[0043] For example, as shown in Table 1, the network parameters of the image decomposition network provided by the present application are shown. Figure 2 As shown in Table 1, the structure diagram of the image decomposition network and the feature interaction module FIM provided by the present application is shown. Figure 2 In the image decomposition network, the reflectance component and the illumination component change simultaneously in the process of decomposing the low-light image, and the reflectance component and the illumination component are essentially interrelated and interdependent. Therefore, at least one feature interaction module FIM is added between the reflectance branch and the illumination branch in the image decomposition network to effectively improve the interaction learning ability between the reflectance component and the illumination component. In the formula, Sigmoid represents an activation Sigmoid function.

[0044] For example, as shown in Table 1, the network parameters of the image decomposition network provided by the present application are shown. Figure 3 As shown in Table 1, the structure diagram of the feature fusion module FFM provided by the present application is shown. Figure 3 In the image decomposition network, the reflectance component and the illumination component change simultaneously in the process of decomposing the low-light image, and the reflectance component and the illumination component are essentially interrelated and interdependent. Therefore, at least one feature interaction module FIM is added between the reflectance branch and the illumination branch in the image decomposition network to effectively improve the interaction learning ability between the reflectance component and the illumination component. In the formula, Sigmoid represents an activation Sigmoid function. Figure 3In some embodiments, the input of the feature fusion module FFM can include three different scale encoder features. The electronic device can first adjust the different scale features to the same resolution and channel number, the up-sampling unit uses the deconvolution operation, the convolution kernel size is 2*2, and the step is 2; the down-sampling unit uses the max-pooling operation, the pooling kernel size is 2*2, and the step is 2; the convolution unit can include a convolution layer and an LReLU activation function layer, wherein the convolution kernel size of the convolution layer is 1*1; then, the electronic device adds the three features, and then improves the feature representation ability through two convolution units. Figure 3 In some embodiments, the first convolution unit can include a convolution layer and an LReLU activation function layer, wherein the convolution kernel size of the convolution layer is 3*3; the second convolution unit can include a convolution layer and a non-activation function layer, wherein the convolution kernel size of the convolution layer is 1*1.

[0045] In some embodiments, the electronic device inputs the low-light image into the image decomposition network to obtain the first reflectance component map and the first illumination component map corresponding to the low-light image can include: the electronic device inputs the low-light image into the reflectance branch in the image decomposition network to obtain multi-layer reflectance features; the electronic device inputs the low-light image into the illumination branch in the image decomposition network to obtain multi-layer illumination features; and the electronic device interacts the multi-layer reflectance features and the multi-layer illumination features to obtain the first reflectance component map and the first illumination component map corresponding to the low-light image.

[0046] After obtaining the low-light image to be enhanced, the electronic device can input the low-light image into the image decomposition network; then, in step 1, the electronic device performs a convolution operation with a convolution kernel size of 3*3 and a number of 32 on the low-light image in the reflectance branch of the image decomposition network to obtain the first layer reflectance feature F R1 ; then, the electronic device performs a convolution operation with a convolution kernel size of 3*3 and a number of 32 on the low-light image in the illumination branch of the image decomposition network to obtain the first layer illumination feature F I1 .

[0047] In step 2, the electronic device interacts the first layer reflectance feature F R1 and the first layer illumination feature F I1 in the feature interaction branch, that is, the two features are concatenated in the channel dimension in the feature interaction branch, and a convolution operation with a convolution kernel size of 1*1 and a number of 32 is performed on each of the two features to obtain the first interaction feature F R1-f and the second interaction feature F I1-f after the first feature interaction.

[0048] In step 3, the electronic device inputs the first interaction feature F R1-fPerform max pooling with a kernel size of 2*2 and a stride of 2, and then downsample to obtain the first sampled feature F. R1-D Then, the electronic device will use the second interaction feature F I1-f Perform max pooling with a kernel size of 2*2 and a stride of 2, and then downsample to obtain the second sampled feature F. I1-D Next, the electronic device will use the first sampling feature F R1-D and the second sampling feature F I1-D Perform convolution operations with a kernel size of 3*3 and a number of 64 kernels to obtain the second layer reflectance feature F. R2 and the second layer of illumination characteristics F I2 .

[0049] In step 4, the electronic device will transfer the first layer reflectivity feature F R2 and the second layer of illumination feature F I2 Concatenate along the channel dimension and perform convolution operations with a kernel size of 1*1 and a number of 64 kernels to obtain the third interactive feature F after the second feature interaction. R2-f and the fourth interaction feature F I2-f .

[0050] In step 5, the electronic device will use the third interaction feature F R2-f and the fourth interaction feature F I2-f Perform max pooling operations with a kernel size of 2*2 and a stride of 2, and then downsample to obtain the third sampled feature F. R2-D and the fourth sampling feature F I2-D Next, the electronic device will then transfer the third sampling feature F R2-D and the fourth sampling feature F I2-D Perform convolution operations with a kernel size of 3*3 and a number of kernels of 128 to obtain the third layer reflectance feature F. R3 and the third layer of illumination characteristics F I3 .

[0051] In step 6, the electronic device will transfer the third layer reflectivity feature F R3 and the third layer of illumination characteristics F I3 Concatenate along the channel dimension and perform convolution operations with a kernel size of 1*1 and a number of 128 kernels to obtain the fifth interactive feature F after the third feature interaction. R3-f and the sixth interaction feature F I3-f .

[0052] In step 7, the electronic device will transfer the first layer reflectivity feature F R1 Perform max pooling with a kernel size of 2*2 and a stride of 2 to obtain feature F. ffm-d1The electronic device will display the second layer reflectivity characteristic F R2 Perform a convolution operation with a kernel size of 1*1 and a number of 64 kernels to obtain the feature F. ffm-c1 The electronic device will display the third layer reflectivity characteristic F R3 Perform a deconvolution operation with a kernel size of 2*2 to obtain the feature F. ffm-u1 Then, the electronic device will assign the feature F ffm-d1 This feature F ffm-c1 and the feature F ffm-u1 After summing, perform a convolution operation with a kernel size of 1*1 and 64 kernels, then perform a convolution operation with a kernel size of 3*3 and 64 kernels to obtain feature F. ffm1 .

[0053] In step 8, the electronic device will use the fifth interaction feature F R3-f After performing a deconvolution operation with a kernel size of 2*2, and then combining it with the feature F... ffm1 Concatenate along the channel dimension, then perform convolution operations with a kernel size of 3*3 and a number of 64 kernels to obtain the fourth layer of reflectivity features F. R4 The electronic device will use the sixth interaction feature F I3-f After performing a deconvolution operation with a kernel size of 2*2, it is combined with the second layer illumination feature F. I2 Concatenate along the channel dimension, then perform convolution operations with a kernel size of 3*3 and a number of 64 kernels to obtain the fourth layer of illumination features F. I4 .

[0054] In step 9, the electronic device will transfer the fourth layer reflectivity feature F R4 With the fourth layer of illumination feature F I4 Concatenate along the channel dimension and perform convolution operations with a kernel size of 1*1 and a number of 64 kernels to obtain the seventh interactive feature F after the fourth feature interaction. R4-f and the eighth interaction feature F I4-f .

[0055] In step 10, the electronic device applies the first layer reflectivity feature F R1 Perform a convolution operation with a kernel size of 1*1 and a number of 32 kernels to obtain the feature F. ffm-c2 The electronic device will display the second layer reflectivity characteristic F R2 Perform a deconvolution operation with a kernel size of 2*2 to obtain the feature F. ffm-c2 The electronic device will display the third layer reflectivity characteristic F R3 Perform two consecutive deconvolution operations with a kernel size of 2*2 to obtain the feature F. ffm-u2 Then, the electronic device will assign the feature F ffm-c2 This feature Fffm-c2 and the feature F ffm-u2 After addition, a convolution operation with a kernel size of 1*1 and a number of 32 is performed, and then a convolution operation with a kernel size of 3*3 and a number of 32 is performed to obtain the feature F ffm2 .

[0056] In step 11, the electronic device performs a convolution operation with a kernel size of 2*2 on the seventh interaction feature F R4-f , concatenates the features F ffm2 in the channel dimension, and then performs a convolution operation with a kernel size of 3*3 and a number of 32 to obtain a fifth reflectance feature F R5 ; the electronic device performs a convolution operation with a kernel size of 2*2 on the eighth interaction feature F I4-f , concatenates the features F I1 in the channel dimension, and then performs a convolution operation with a kernel size of 3*3 and a number of 32 to obtain a fifth layer of illumination feature F I5 .

[0057] In step 12, the electronic device performs a convolution operation with a kernel size of 3*3 and a number of 32 on the low-light image for four consecutive times to obtain a feature F I1+ ; the electronic device concatenates the features F I1+ , the fifth layer of illumination feature F I5 in the channel dimension, and then performs a convolution operation with a kernel size of 1*1 and a number of 1, and after passing through a Sigmoid function, obtains a first reflectance component map I corresponding to the low-light image; the electronic device performs a convolution operation with a kernel size of 3*3 and a number of 3 on the fifth layer of reflectance feature F R5 , and after passing through a Sigmoid function, obtains a first illumination component map R corresponding to the low-light image.

[0058] In summary, the electronic device inputs the obtained low-light image into a reflectance branch in the image decomposition network to obtain five layers of reflectance features; the electronic device inputs the low-light image into an illumination branch in the image decomposition network to obtain five layers of illumination features; the electronic device performs feature interaction on the five layers of reflectance features and the five layers of illumination features respectively to obtain a first reflectance component map I and a first illumination component map R corresponding to the low-light image.

[0059] It should be noted that the timing of the electronic device obtaining the multiple layers of reflectance features and the timing of the electronic device obtaining the multiple layers of illumination features are not limited; the timing of the electronic device obtaining the first reflectance component map I and the timing of the electronic device obtaining the first illumination component map R are not limited.

[0060] In some embodiments, the image decomposition network can be trained according to the following steps: the electronic device obtains a low-light image sample and a normal-light image sample corresponding to the same scene by changing the exposure time and the sensitivity of the camera; the electronic device determines a first reflectance component map sample and a first illumination component map sample corresponding to the low-light image sample, and a second reflectance component map sample and a second illumination component map sample corresponding to the normal-light image sample; the electronic device determines a total loss corresponding to the image decomposition network to be trained according to the first reflectance component map sample, the first illumination component map sample, the second reflectance component map sample, and the second illumination component map sample; and the electronic device adjusts the network parameters of the image decomposition network to be trained according to the total loss to obtain the trained image decomposition network.

[0061] The exposure time refers to the time for which the shutter is opened in order to project light onto the light-sensitive surface of the photographic light-sensitive material.

[0062] The sensitivity can be used to measure the sensitivity of the film to light.

[0063] Optionally, the scene can include shooting locations, shooting scenes, and shooting content.

[0064] The low-light image sample has a lower luminance value than the normal-light image sample, and the resolution of the low-light image sample is the same as that of the normal-light image sample.

[0065] For example, the resolution of the low-light image sample and the normal-light image sample is 48*48.

[0066] Optionally, the low-light image sample and the normal-light image sample can be image samples in a data set (Low-Light dataset, LOL), which are obtained by the electronic device by changing the exposure time and the sensitivity of the camera to shoot a real scene.

[0067] For example, the number of low-light image samples is 485, and the number of normal-light image samples is 485.

[0068] Optionally, during the training of the image decomposition network to be trained, the electronic device can use an adaptive moment estimation (Adam) optimizer, wherein the electronic device can set the learning rate of the Adam optimizer to 0.0001 and the number of iterations to 2500.

[0069] Optionally, the electronic device determines the total loss corresponding to the image decomposition network to be trained based on the first reflectance component map sample, the first illumination component map sample, the second reflectance component map sample, and the second illumination component map sample. This may include: the electronic device determining the image reconstruction loss according to the first loss formula; the electronic device determining the reflectance consistency loss according to the second loss formula; the electronic device determining the illumination smoothing loss according to the third loss formula; the electronic device determining the mutual consistency loss according to the fourth loss formula; and the electronic device obtaining the total loss corresponding to the image decomposition network to be trained according to the fifth loss formula.

[0070] The first loss formula is:

[0071] The second loss formula is: L rc =||R high -R low ||1;

[0072] The third loss formula is:

[0073] The fourth loss formula is:

[0074]

[0075] The fifth loss formula is: L ID-Net =λ ir L ir +λ rc L rc +λ is L is +λ mc L mc +λ wd L wd ;

[0076] L ir S represents the image reconstruction loss; low Represents a low-light image sample; R low Indicates the first reflectivity component map sample; I low S represents the first illumination component image sample; high Represents a normal light image sample; R high Indicates the second reflectivity component map sample; I high This represents a sample of the second illumination component image; This indicates element-wise multiplication between image samples, that is, multiplying the pixels of one image sample by the pixels of another image sample; ||·|1 represents the L1 norm, which refers to the sum of the absolute values ​​of each element (e.g., pixel) between image samples; L rc Indicates the loss of reflectivity uniformity; L is Indicates the loss of illumination smoothness; denotes a horizontal and vertical gradient calculation symbol; ∈1 denotes a first constant for preventing the denominator from being zero, i.e., ∈1≠0; ∈2 denotes a second constant for preventing the denominator from being zero, i.e., ∈2≠0; L mc denotes a mutual consistency loss; L ID-Net denotes a total loss corresponding to the image decomposition network to be trained; L wd denotes an L2 weight decay loss; λ ir denotes a first weight parameter; λ rc denotes a second weight parameter; λ is denotes a third weight parameter; λ mc denotes a fourth weight parameter; λ wd denotes a fifth weight parameter.

[0077] It should be noted that the electronic device uses the image reconstruction loss L ir to ensure the consistency of the reflectance component sample and the illumination component sample before and after the image sample is decomposed; according to the image enhancement (Retinex) theory, since the color of the object in the image sample is determined by the reflection ability and is irrelevant to the reflection intensity, the reflectance components between the low-light image sample and the normal-light image sample should be the same, which can be realized through the reflectance consistency loss L rc ; the electronic device uses the illumination smoothing loss L is to retain the low-frequency information of the illumination in the image sample, so that the illumination image sample is smooth and consistent; the electronic device uses the mutual consistency loss L mc to retain the strong edges of the image sample and suppress the weak edges of the image sample; the electronic device uses the L2 weight decay loss L wd to prevent the trained image decomposition network from overfitting.

[0078] 103、input the first reflectance component map and the first illumination component map into a reflectance adjustment network to obtain a second reflectance component map, and input the first illumination component map into an illumination adjustment network to obtain a second illumination component map.

[0079] Among them, the first reflectance component map and the second reflectance component map are different, and the first illumination component map and the second illumination component map are different.

[0080] The reflectance adjustment network can process the reflectance of the first reflectance component map in a coarse-to-fine manner, and the reflectance adjustment network can introduce the illumination component when adjusting the reflectance, i.e., introducing the first illumination component map, and taking the first illumination component map as a kind of attention mechanism to assist the adjustment process of the reflectance.

[0081] The illumination adjustment network can process the illumination of the image in a coarse-to-fine manner.

[0082] Optionally, the reflectivity adjustment network can include 3 U-Net structures and 2 residual channels and spatial attention (RCSA) modules, etc.

[0083] Optionally, the illumination adjustment network can include 3 U-Net structures and 2 RCSA modules, etc.

[0084] Exemplarily, as shown in Table 2, are structure parameters of the U-Net structure provided by the present application.

[0085] Table 2

[0086]

[0087]

[0088] wherein C1 represents a first convolutional layer; D1 represents a first down-sampling layer; C2 represents a second convolutional layer; D2 represents a second down-sampling layer; C3 represents a third convolutional layer; U1 represents a first up-sampling layer; C4 represents a fourth convolutional layer; U2 represents a second up-sampling layer; and C5 represents a fifth convolutional layer.

[0089] Exemplarily, as shown in Table 2, are structure parameters of the U-Net structure provided by the present application. Figure 4 Exemplarily, as shown in Table 2, are structure parameters of the U-Net structure provided by the present application. Figure 4 In the reflectivity adjustment network, the residual channels and spatial attention RCSA module can include residual connection, channel attention and spatial attention. The input of the reflectivity adjustment network is the feature after the channel concatenation of different scale features and dimension reduction, then, after the convolution operation, channel attention operation and spatial attention operation, the residual connection with the original feature is performed. In the reflectivity adjustment network, the channel and spatial importance of the feature can be learned, and the useful information can be selectively enhanced and the other information can be inhibited. In addition, from the perspective of the network structure, the reflectivity adjustment network can be regarded as a residual network, which can effectively alleviate the problem of vanishing gradient in the training process. Figure 4As can be seen, the RCSA module can selectively enhance features through the channel attention mechanism and the spatial attention mechanism. Specifically, first, the electronic device performs a convolution operation on the first reflectance component map with a 2x2 convolution kernel to obtain a feature F; then, the electronic device performs a global average pooling operation (GAP) on the feature F to extract a channel feature, further improves the representation ability of the feature F through two convolution operations with a 1x1 convolution kernel, and then generates a channel attention map F_CA corresponding to the feature F using a Sigmoid function; then, the electronic device multiplies the feature F and the channel attention map F_CA to enhance useful channel information and suppress useless channel information, and obtains a feature F_C; thereafter, the electronic device performs a global average pooling GAP and a global maximum pooling operation (GMP) on the feature F_C, respectively, concatenates them in the channel dimension, and then performs a convolution operation with a 1x1 convolution kernel and a Sigmoid function to obtain a spatial attention map F_SA; then, the electronic device multiplies the feature F_C and the spatial attention map F_SA to obtain a feature F_S with spatial importance; finally, the electronic device performs a residual connection between the original feature and the feature F_S to obtain a second illumination component map.

[0090] As shown in an example, Figure 5 The structure of the illumination adjustment network provided by the present application is shown in the structure diagram. Figure 5 In the structure diagram, the illumination adjustment network can include three U-Net structures and two RCSA modules.

[0091] In some embodiments, the electronic device inputs the first reflectance component map and the first illumination component map into the reflectance adjustment network to obtain a second reflectance component map can include: the electronic device inputs the first reflectance component map into the reflectance adjustment network to obtain a third reflectance component map, the resolution corresponding to the third reflectance component map is different from the resolution corresponding to the first reflectance component map; the electronic device inputs the first illumination component map into the reflectance adjustment network to obtain a third illumination component map, the resolution corresponding to the third illumination component map is different from the resolution corresponding to the first illumination component map; the electronic device determines a first feature map corresponding to the first reflectance component map, a second feature map corresponding to the third reflectance component map, a first illumination attention map corresponding to the first illumination component map, and a second illumination attention map corresponding to the third illumination component map; and the electronic device determines the second reflectance component map according to the first feature map, the second feature map, the first illumination attention map, and the second illumination attention map.

[0092] Among them, there is at least one third reflectivity component map, and when there are multiple third reflectivity component maps, the resolution of each third reflectivity component map is also different; there is at least one third illumination component map, and when there are multiple third illumination component maps, the resolution of each third illumination component map is also different.

[0093] For example, the electronic device inputs a first reflectivity component map R1 into the reflectivity adjustment network. After performing max pooling with a kernel size of 2*2 and a stride of 2 on R1, it downsamples the map to obtain two third reflectivity component maps, denoted as R2 and R4, respectively. The resolution of R2 is half that of R1, and the resolution of R4 is one-quarter that of R1. The electronic device then inputs a first illumination component map I1 into the reflectivity adjustment network. After performing max pooling with a kernel size of 2*2 and a stride of 2 on I1, it downsamples the map to obtain two third illumination component maps, denoted as I2 and I4, respectively. The resolution of I2 is half that of I1, and the resolution of I4 is one-quarter that of I1. Finally, the electronic device performs convolution operations on R1, R2, and R4 with a kernel size of 3*3 and a convolution count of 32 to obtain their respective feature maps F. R1 F R2 and F R4 The electronic device performs convolution operations on I1, I2, and I4 with a kernel size of 3*3 and a number of 32 kernels to obtain their respective illumination attention features F. I1 F I2 and F I4 Next, the electronic device, based on the light attention characteristics F... I1 F I2 and F I4 Using the Sigmoid function, an illumination attention map F' is generated. I1 F' I2 and F' I4 Then, the electronic device, based on the feature map F R1 F R2 F R4 Illumination attention diagram F' I1 F' I2 and F' I4 Determine the second reflectivity component map R adjusted .

[0094] In some embodiments, the electronic device can determine the second reflectance component map according to the first feature map, the second feature map, the first illumination attention map and the second illumination attention map, which can include: the electronic device determines a third feature map according to the first feature map and the first illumination attention map; the electronic device determines a fourth feature map according to the second feature map and the second illumination attention map; and the electronic device concatenates the third feature map and the fourth feature map to obtain the second reflectance component map.

[0095] In the process of determining the second reflectance component map according to the first feature map, the second feature map, the first illumination attention map and the second illumination attention map, the electronic device: in step 1, the electronic device multiplies the feature map F R1 and the illumination attention map F I1 to obtain a third feature map, i.e., a feature F1 with illumination attention, multiplies the feature map F R2 and the illumination attention map F I2 to obtain a fourth feature map, i.e., a feature F2 with illumination attention, multiplies the feature map F R4 and the illumination attention map F I4 to obtain a fifth feature map, i.e., a feature F4 with illumination attention, and finally, the electronic device determines the second reflectance component map according to the feature F1, the feature F2 and the feature F4.

[0096] In step 2, the electronic device enhances the feature, restores the color and removes the noise by the U-Net structure of the reflectance adjustment network according to the feature F4 to obtain a feature F 4-out .

[0097] In step 3, the electronic device performs a deconvolution operation with a kernel size of 2*2 on the feature F 4-out , concatenates the feature F2 in the channel dimension, and then performs a convolution operation with a kernel size of 3*3 and a number of 32, and finally, obtains a feature F RCSA1 through the RCSA module of the reflectance adjustment network.

[0098] In step 4, the electronic device further enhances the feature, restores the color and removes the noise by the U-Net structure according to the feature F RCSA1 to obtain a feature F 2-out .

[0099] In step 5, the electronic device performs a deconvolution operation with a kernel size of 2*2 on the feature F 2-out , concatenates the feature F1 in the channel dimension, and then performs a convolution operation with a kernel size of 3*3 and a number of 32, and finally, obtains a feature F RCSA2 through the above RCSA module.

[0100] In step 6, the electronic device, based on feature F RCSA2 By further enhancing features, restoring colors, and removing noise through the U-Net structure, feature F is obtained. 1-out .

[0101] In step 7, the electronic device will assign the feature F 1-out Perform a convolution operation with a kernel size of 3*3 and a number of kernels of 3, and then use the Sigmoid function to obtain the second reflectance component map R. adjusted .

[0102] It should be noted that the timing of the electronic device acquiring the third feature map and the electronic device acquiring the fourth feature map are not limited.

[0103] Optionally, the reflectivity adjustment network can be trained according to the following steps: the electronic device determines the reflectivity adjustment loss according to the first formula; the electronic device determines the structural similarity index (SSIM) loss according to the second formula; the electronic device determines the visual geometry group (VGG) loss according to the third formula; and the electronic device determines the total loss corresponding to the reflectivity adjustment network according to the fourth formula.

[0104] The first formula is:

[0105] The second formula is: L ssim =1-SSIM(R) high R adjusted );

[0106] The third formula is:

[0107] The fourth formula is: L RA-Net =λ ra L ra +λ ssim L ssim +λ vgg L vgg ;

[0108] L ra Represents reflectance adjustment loss; C represents the number of channels in the image sample; H represents the height of the image sample; W represents the width of the image sample; ∈3 represents a preset constant; L ssim Indicates SSIM loss; L vgg Represents VGG loss; ||·||2 represents the L2 norm; Ф(·) represents the Conv4-4 feature of the VGG19 pre-trained network; L RA-Net λ represents the total loss corresponding to the reflectivity adjustment network.ra denotes a sixth weight parameter; λ ssim denotes a seventh weight parameter; λ vgg denotes an eighth weight parameter.

[0109] It should be noted that the electronic device adjusts the reflectivity by using the reflectivity adjustment loss L ra to ensure that the reflectivity of the normal light image sample is the same as the reflectivity of the second reflectivity component map; the electronic device maximizes the structural similarity by minimizing the SSIM loss L ssim to improve the structural quality of the second reflectivity component map; the electronic device improves the image quality of the second reflectivity component map by using the VGG loss L vgg .

[0110] In some embodiments, the electronic device inputs the first illumination component map into the illumination adjustment network to obtain the second illumination component map can include: the electronic device inputs the first illumination component map into the illumination adjustment network to obtain a fourth illumination component map, the resolution corresponding to the fourth illumination component map is different from the resolution corresponding to the first illumination component map; the electronic device adjusts the resolution corresponding to the first illumination component map and the resolution corresponding to the fourth illumination component map by using an illumination adjustment factor; the electronic device determines a fifth feature map corresponding to the adjusted first illumination component map and a sixth feature map corresponding to the adjusted fourth illumination component map; and the electronic device determines the second illumination component map according to the fifth feature map and the sixth feature map.

[0111] Wherein, the number of the fourth illumination component maps is at least one, and in the case that the number of the fourth illumination component maps is multiple, the resolutions of the respective fourth illumination component maps are also different.

[0112] In the process of inputting the first illumination component map into the illumination adjustment network by the electronic device to obtain the second illumination component map: in step 1, the electronic device inputs the first illumination component map I1 into the illumination adjustment network, and performs a maximum pooling operation with a pooling kernel size of 2*2 and a step of 2 on the first illumination component map I1 to obtain two fourth illumination component maps, denoted as I2 and I4, respectively, the resolution corresponding to I2 is 1 / 2 of the resolution corresponding to I1, and the resolution corresponding to I4 is 1 / 4 of the resolution corresponding to I1; then, the electronic device expands I1, I2 and I4 into illumination component maps with the same resolution by using an illumination adjustment factor a; then, the electronic device concatenates the three illumination component maps with the same resolution according to the channel dimension, and respectively performs a convolution operation with a convolution kernel size of 3*3 and a number of 32 to obtain features T1, T2 and T4.

[0113] In step 2, the electronic device adjusts the brightness according to the feature T4 through the U-Net structure in the illumination adjustment network to obtain a feature T 4-out .

[0114] In step 3, the electronic device adjusts the feature T 4-out The deconvolution operation with a kernel size of 2*2 is performed, and the feature T2 is concatenated with the feature T in the channel dimension, and then the convolution operation with a kernel size of 3*3 and a number of 32 is performed. Finally, the feature T RCSA1 .

[0115] In step 4, the electronic device adjusts the brightness of the feature T RCSA1 by the U-Net structure to obtain the feature T 2-out .

[0116] In step 5, the electronic device adjusts the feature T 2-out by the deconvolution operation with a kernel size of 2*2, and concatenates the feature T1 with the feature T in the channel dimension, and then performs the convolution operation with a kernel size of 3*3 and a number of 32. Finally, the feature T RCSA2 is obtained through the above RCSA module.

[0117] In step 6, the electronic device further adjusts the brightness of the feature T RCSA2 by the U-Net structure to obtain the feature T 1-out .

[0118] In step 7, the electronic device adjusts the feature T 1-out by the convolution operation with a kernel size of 3*3 and a number of 1, and then utilizes the Sigmoid function to obtain the second illumination component map I adjusted .

[0119] Optionally, the illumination adjustment network can be obtained by training according to the following steps: the electronic device determines the illumination consistency loss according to the fifth formula; the electronic device determines the smooth consistency loss according to the sixth formula; and the electronic device determines the total loss corresponding to the illumination adjustment network according to the seventh formula.

[0120] The fifth formula is:

[0121] The sixth formula is:

[0122] The seventh formula is: L IA-Net = λ ic L ic + λ sc L sc ;

[0123] L ic represents the illumination consistency loss; L sc represents the smooth consistency loss; and L IA-Netdenotes a total loss corresponding to the illumination adjustment network; λ ic denotes a ninth weight parameter; λ sc denotes a tenth weight parameter.

[0124] It should be noted that the electronic device uses the illumination consistency loss L ic to ensure that the illumination map corresponding to the normal light image sample is the same as the second illumination component map I adjusted . The electronic device uses the smoothing consistency loss L sc to ensure that the smoothness of the normal light image sample is the same as the smoothness of the second illumination component map I adjusted .

[0125] In some embodiments, the electronic device adjusts the resolution corresponding to the first illumination component map and the resolution corresponding to the fourth illumination component map using the illumination adjustment factor, which can include: the electronic device obtains a low light image sample and a normal light image sample corresponding to the same scene by changing the exposure time and the sensitivity of the camera; the electronic device obtains a first brightness value corresponding to the low light image sample and a second brightness value corresponding to the normal light image sample; the electronic device determines the illumination adjustment factor according to the first brightness value and the second brightness value; and the electronic device adjusts the resolution corresponding to the first illumination component map and the resolution corresponding to the fourth illumination component map using the illumination adjustment factor.

[0126] The illumination adjustment factor can solve the non-uniqueness of the solution to the low light image enhancement problem caused by the inherent subjectivity of brightness perception.

[0127] In the process of obtaining the illumination condition factor, the electronic device can first obtain a low light image sample and a normal light image sample corresponding to the same scene by changing the exposure time and the sensitivity of the camera, that is, the shooting picture of the low light image sample is the same as the shooting picture of the normal light image sample; then, the electronic device obtains a first brightness value corresponding to the low light image sample and a second brightness value corresponding to the normal light image sample; then, the electronic device obtains a brightness ratio of the second brightness value and the first brightness value, and determines the brightness ratio as the illumination adjustment factor; finally, the electronic device can adjust the resolution corresponding to the first illumination component map and the resolution corresponding to the fourth illumination component map using the illumination adjustment factor.

[0128] It should be noted that the timing of the electronic device obtaining the first brightness value and the electronic device obtaining the second brightness value is not limited; the timing of the electronic device adjusting the resolution corresponding to the first illumination component map and the electronic device adjusting the resolution corresponding to the fourth illumination component map is not limited.

[0129] 104. Obtain an enhanced image corresponding to the low light image according to the second reflectance component map and the second illumination component map.

[0130] After acquiring the second reflectivity component map and the second illumination component map, the electronic device can directly multiply the second reflectivity component map and the second illumination component map to obtain the enhanced image corresponding to the low-light image.

[0131] In other words, this invention constructs a supervised low-light image enhancement network model based on Retinex decomposition and multi-scale adjustment. This model may include an image decomposition network, a reflectivity adjustment network, and an illumination adjustment network. This model ensures that it can simultaneously restore the image details corresponding to the low-light image, improve brightness and contrast, and reduce noise, resulting in a relatively accurate enhanced image.

[0132] In this embodiment of the invention, a low-light image to be enhanced is acquired; the low-light image is input into an image decomposition network to obtain a first reflectance component map and a first illumination component map corresponding to the low-light image; the first reflectance component map and the first illumination component map are input into a reflectance adjustment network to obtain a second reflectance component map, and the first illumination component map is input into an illumination adjustment network to obtain a second illumination component map; based on the second reflectance component map and the second illumination component map, an enhanced image corresponding to the low-light image is obtained. This method addresses the limitations of existing image enhancement methods, which prevent electronic devices from accurately acquiring the enhanced image corresponding to the low-light image. By utilizing an image decomposition network, the relationship and changes between the reflectance component and the illumination component can be monitored during the decomposition of the low-light image. Through the interaction between features and the expansion of the reflectance component, the low-light image can be better decomposed. Then, by utilizing the reflectance adjustment network and the illumination adjustment network, multi-scale information is fully utilized to adjust the reflectance component and the illumination component in a coarse-to-fine manner, thereby effectively improving the accuracy of acquiring the enhanced image.

[0133] The low-light image enhancement apparatus provided by the present invention will be described below. The low-light image enhancement apparatus described below can be referred to in correspondence with the low-light image enhancement method described above.

[0134] like Figure 6 The diagram shown is a structural schematic of the low-light image enhancement device provided by the present invention, which may include:

[0135] Acquisition module 601 is used to acquire the low-light image to be enhanced;

[0136] The processing module 602 is configured to input the low-light image into an image decomposition network to obtain a first reflectance component map and a first illumination component map corresponding to the low-light image; input the first reflectance component map and the first illumination component map into a reflectance adjustment network to obtain a second reflectance component map, and input the first illumination component map into an illumination adjustment network to obtain a second illumination component map; and obtain an enhanced image corresponding to the low-light image according to the second reflectance component map and the second illumination component map.

[0137] Optionally, the processing module 602 is specifically configured to input the low-light image into a reflectance branch in the image decomposition network to obtain multi-layer reflectance features; input the low-light image into an illumination branch in the image decomposition network to obtain multi-layer illumination features; and obtain the first reflectance component map and the first illumination component map corresponding to the low-light image by interacting the multi-layer reflectance features and the multi-layer illumination features.

[0138] Optionally, the acquisition module 601 is specifically configured to acquire a low-light image sample and a normal-light image sample corresponding to a same scene by changing exposure time and sensitivity of a camera.

[0139] The processing module 602 is specifically configured to determine a first reflectance component map sample and a first illumination component map sample corresponding to the low-light image sample, and a second reflectance component map sample and a second illumination component map sample corresponding to the normal-light image sample; determine a total loss corresponding to a to-be-trained image decomposition network according to the first reflectance component map sample, the first illumination component map sample, the second reflectance component map sample and the second illumination component map sample; and adjust network parameters of the to-be-trained image decomposition network according to the total loss to obtain a trained image decomposition network.

[0140] Optionally, the processing module 602 is specifically configured to input the first reflectance component map into the reflectance adjustment network to obtain a third reflectance component map, the third reflectance component map corresponding to a resolution different from that of the first reflectance component map; input the first illumination component map into the reflectance adjustment network to obtain a third illumination component map, the third illumination component map corresponding to a resolution different from that of the first illumination component map; determine a first feature map corresponding to the first reflectance component map, a second feature map corresponding to the third reflectance component map, a first illumination attention map corresponding to the first illumination component map, and a second illumination attention map corresponding to the third illumination component map; and determine a second reflectance component map according to the first feature map, the second feature map, the first illumination attention map and the second illumination attention map.

[0141] Optionally, the processing module 602 is specifically configured to determine a third feature map according to the first feature map and the first illumination attention map; determine a fourth feature map according to the second feature map and the second illumination attention map; and concatenate the third feature map and the fourth feature map to obtain a second reflectivity component map.

[0142] Optionally, the processing module 602 is specifically configured to input the first illumination component map into the illumination adjustment network to obtain a fourth illumination component map, the fourth illumination component map corresponding to a resolution different from that of the first illumination component map; adjust the resolution corresponding to the first illumination component map and the resolution corresponding to the fourth illumination component map by using an illumination adjustment factor; determine a fifth feature map corresponding to the adjusted first illumination component map and a sixth feature map corresponding to the adjusted fourth illumination component map; and determine a second illumination component map according to the fifth feature map and the sixth feature map.

[0143] Optionally, the acquisition module 601 is specifically configured to obtain a low-light image sample and a normal-light image sample corresponding to the same scene by changing the exposure time and the light sensitivity of a camera; obtain a first luminance value corresponding to the low-light image sample and a second luminance value corresponding to the normal-light image sample.

[0144] The processing module 602 is specifically configured to determine the illumination adjustment factor according to the first luminance value and the second luminance value; and adjust the resolution corresponding to the first illumination component map and the resolution corresponding to the fourth illumination component map by using the illumination adjustment factor.

[0145] As shown in Figure 7 Fig. 1 is a structural schematic diagram of an electronic device provided by the present application, which can include a processor 710, a communications interface 720, a memory 730 and a communications bus 740, wherein the processor 710, the communications interface 720 and the memory 730 complete mutual communication through the communications bus 740. The processor 710 can invoke a logical instruction in the memory 730 to execute a low-light image enhancement method, which includes: acquiring a low-light image to be enhanced; inputting the low-light image into an image decomposition network to obtain a first reflectivity component map and a first illumination component map corresponding to the low-light image; inputting the first reflectivity component map and the first illumination component map into a reflectivity adjustment network to obtain a second reflectivity component map, and inputting the first illumination component map into an illumination adjustment network to obtain a second illumination component map; and obtaining an enhanced image corresponding to the low-light image according to the second reflectivity component map and the second illumination component map.

[0146] In addition, the logic instructions in the memory 730 described above can be implemented in the form of software function units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0147] In another aspect, the present application also provides a computer program product, which includes a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program is executed by a processor, so that the computer can execute the low-light image enhancement method provided by the above-mentioned methods.

[0148] In another aspect, the present application also provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the low-light image enhancement method provided by the above-mentioned methods.

[0149] The device embodiments described above are only schematic, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0150] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus necessary universal hardware platforms, and of course, can also be realized by hardware. Based on such understanding, the above technical solutions essentially or the parts that contribute to the prior art can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments.

[0151] It should be pointed out finally that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit the same; and although the present application has been described in detail with reference to the foregoing embodiments, it should be appreciated by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for enhancing low-light images, characterized in that, include: Acquire the low-light image to be enhanced; The low-light image is input into an image decomposition network to obtain the first reflectance component map and the first illumination component map corresponding to the low-light image. The first reflectance component map and the first illumination component map are input into the reflectance adjustment network to obtain the second reflectance component map, and the first illumination component map is input into the illumination adjustment network to obtain the second illumination component map. Based on the second reflectance component map and the second illumination component map, the enhanced image corresponding to the low-light image is obtained; The step of inputting the low-light image into an image decomposition network to obtain a first reflectance component map and a first illumination component map corresponding to the low-light image includes: The low-light image is input into the reflectance branch of the image decomposition network to obtain multi-layer reflectance features; The low-light image is input into the illumination branch of the image decomposition network to obtain multi-layer illumination features; By interacting the multilayer reflectivity features and the multilayer illumination features, the first reflectivity component map and the first illumination component map corresponding to the low-light image are obtained. The step of inputting the first reflectivity component map and the first illumination component map into the reflectivity adjustment network to obtain the second reflectivity component map includes: The first reflectance component map is input into the reflectance adjustment network to obtain a third reflectance component map, the resolution of which is different from the resolution of the first reflectance component map. The first illumination component map is input into the reflectivity adjustment network to obtain the third illumination component map, and the resolution corresponding to the third illumination component map is different from the resolution corresponding to the first illumination component map. Determine the first feature map corresponding to the first reflectance component map, the second feature map corresponding to the third reflectance component map, the first illumination attention map corresponding to the first illumination component map, and the second illumination attention map corresponding to the third illumination component map. Based on the first feature map, the second feature map, the first illumination attention map, and the second illumination attention map, a second reflectivity component map is determined. The step of inputting the first illumination component map into the illumination adjustment network to obtain the second illumination component map includes: The first illumination component map is input into the illumination adjustment network to obtain a fourth illumination component map, the resolution of which is different from the resolution of the first illumination component map. The resolution corresponding to the first illumination component map and the resolution corresponding to the fourth illumination component map are adjusted using the illumination adjustment factor. Determine the fifth feature map corresponding to the adjusted first illumination component map and the sixth feature map corresponding to the adjusted fourth illumination component map; The second illumination component map is determined based on the fifth feature map and the sixth feature map.

2. The method according to claim 1, characterized in that, The image decomposition network is trained according to the following steps: By changing the camera's exposure time and ISO, low-light image samples and normal-light image samples corresponding to the same scene are obtained. Determine the first reflectance component map sample and the first illumination component map sample corresponding to the low-light image sample, and the second reflectance component map sample and the second illumination component map sample corresponding to the normal light image sample; Based on the first reflectance component map sample, the first illumination component map sample, the second reflectance component map sample, and the second illumination component map sample, determine the total loss corresponding to the image decomposition network to be trained. Based on the total loss, the network parameters of the image decomposition network to be trained are adjusted to obtain the trained image decomposition network.

3. The method according to claim 1, characterized in that, The step of determining the second reflectance component map based on the first feature map, the second feature map, the first illumination attention map, and the second illumination attention map includes: Based on the first feature map and the first illumination attention map, a third feature map is determined; The fourth feature map is determined based on the second feature map and the second illumination attention map; The third feature map and the fourth feature map are cascaded to obtain the second reflectivity component map.

4. The method according to claim 1, characterized in that, The step of adjusting the resolution corresponding to the first illumination component map and the resolution corresponding to the fourth illumination component map using an illumination adjustment factor includes: By changing the camera's exposure time and ISO, low-light image samples and normal-light image samples corresponding to the same scene are obtained. Obtain the first brightness value corresponding to the low-light image sample and the second brightness value corresponding to the normal-light image sample; The illumination adjustment factor is determined based on the first brightness value and the second brightness value; The resolution corresponding to the first illumination component map and the resolution corresponding to the fourth illumination component map are adjusted using the illumination adjustment factor.

5. A device for enhancing low-light images, characterized in that, include: The acquisition module is used to acquire the low-light image to be enhanced; The processing module is configured to input the low-light image into an image decomposition network to obtain a first reflectance component map and a first illumination component map corresponding to the low-light image; input the first reflectance component map and the first illumination component map into a reflectance adjustment network to obtain a second reflectance component map, and input the first illumination component map into an illumination adjustment network to obtain a second illumination component map; and obtain an enhanced image corresponding to the low-light image based on the second reflectance component map and the second illumination component map. Specifically, the processing module is used to input the low-light image into an image decomposition network to obtain a first reflectance component map and a first illumination component map corresponding to the low-light image, including: The low-light image is input into the reflectance branch of the image decomposition network to obtain multi-layer reflectance features; The low-light image is input into the illumination branch of the image decomposition network to obtain multi-layer illumination features; By interacting the multilayer reflectivity features and the multilayer illumination features, the first reflectivity component map and the first illumination component map corresponding to the low-light image are obtained. The processing module is specifically used to input the first reflectivity component map and the first illumination component map into the reflectivity adjustment network to obtain the second reflectivity component map, including: The first reflectance component map is input into the reflectance adjustment network to obtain a third reflectance component map, the resolution of which is different from the resolution of the first reflectance component map. The first illumination component map is input into the reflectivity adjustment network to obtain the third illumination component map, and the resolution corresponding to the third illumination component map is different from the resolution corresponding to the first illumination component map. Determine the first feature map corresponding to the first reflectance component map, the second feature map corresponding to the third reflectance component map, the first illumination attention map corresponding to the first illumination component map, and the second illumination attention map corresponding to the third illumination component map. Based on the first feature map, the second feature map, the first illumination attention map, and the second illumination attention map, a second reflectivity component map is determined. The processing module is specifically used to input the first illumination component map into the illumination adjustment network to obtain the second illumination component map, including: The first illumination component map is input into the illumination adjustment network to obtain a fourth illumination component map, the resolution of which is different from the resolution of the first illumination component map. The resolution corresponding to the first illumination component map and the resolution corresponding to the fourth illumination component map are adjusted using the illumination adjustment factor. Determine the fifth feature map corresponding to the adjusted first illumination component map and the sixth feature map corresponding to the adjusted fourth illumination component map; The second illumination component map is determined based on the fifth feature map and the sixth feature map.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the low-light image enhancement method as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the low-light image enhancement method as described in any one of claims 1 to 4.

Images