Low-illumination image enhancement model training method, image enhancement method and device

Abstract

The invention discloses a training method for a low-illumination image enhancement model, and the method comprises the steps: inputting a low-illumination image and a normal image into an image decomposition module of the low-illumination image enhancement model, and obtaining a corresponding low-illumination image reflectivity graph and a normal image reflectivity graph; respectively carrying out degradation perception processing on the low-illumination image reflectivity image and the normal image reflectivity image to obtain a corresponding low-illumination image reflection characteristic image and a corresponding normal image reflection characteristic image; determining a reflectivity degradation loss function according to the obtained image; and determining degradation perception processing parameters according to the reflectivity degradation loss function, and using the reflectivity degradation loss function to train an image decomposition module. According to the method, noise amplification and image distortion can be avoided when the low-illumination image is enhanced. The invention further discloses a low-illumination image enhancement method, a model training device, an image enhancement device, computer equipment and a readable storage medium.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and image processing, and in particular to a low-illumination image enhancement model training method, a low-illumination image enhancement method, a low-illumination image enhancement model training device, a low-illumination image enhancement device, equipment and a computer storage medium. Background Art

[0002] Images captured in low-light environments suffer from low quality and numerous issues, severely impacting the practical application of computer vision technology. For example, faces in uneven lighting are more difficult to recognize, and underexposed nighttime video significantly complicates vehicle detection. Therefore, enhancing low-light images to provide universal, high-quality, and stable input for subsequent advanced computer vision tasks is crucial in computer vision.

[0003] Currently, there are two mainstream approaches to compensate for the degradation of image quality caused by insufficient ambient light: a method-driven traditional algorithm approach and a data-driven deep learning approach.

[0004] Traditional algorithms, such as histogram equalization and image decomposition, are method-driven, with clear algorithmic design logic and strong interpretability. However, their enhancement effects still lag behind those of deep learning algorithms. In recent years, significant progress in this area has included: more efficient edge-preserving filtering methods, probability estimation methods based on prior information, l1-l0 hybrid decomposition, and the introduction of weighted variational models to optimize the decomposition of light sources and scene object reflectance; the introduction of multi-scale information, YUV color space, gradient domain weighting, and structural block decomposition to enhance the detail and spatial smoothness of the fused image; the introduction of brightness-dependent adaptive color restoration to ensure unchanged image color saturation; and the introduction of constrained optimization to amplify grayscale differences between adjacent pixels.

[0005] Data-driven deep learning methods represented by RetinexNet and Kind have developed rapidly in recent years. These methods use large amounts of data to drive neural networks to automatically learn relevant features, and often have better enhancement effects than traditional methods. However, their algorithms are poorly interpretable and computationally intensive, requiring specialized hardware support.

[0006] At present, whether method-driven or data-driven, current low-light image enhancement methods often focus on adjusting image brightness, and to some extent ignore other distortions of images captured in low-light environments, such as noise pollution, color distortion, and halo artifacts, resulting in these distortions being synchronously amplified during the brightness enhancement process.

[0007] Even when noise pollution and other factors are taken into account, existing technologies often do not perform denoising or halo artifact avoidance during the enhancement process. Instead, they implement additional neural networks or use traditional algorithms to perform denoising and other processing on the already enhanced image after enhancement is complete. Furthermore, even when existing technologies do take noise and other issues into account during the enhancement process, hoping to directly obtain the decomposed image after denoising and other processing during image decomposition, the neural networks and their associated loss functions are overly complex, computationally intensive, and inefficient. Furthermore, the resulting images still require further improvement in terms of noise, color distortion, halo artifacts, and other aspects. Summary of the Invention

[0008] The present invention proposes a training method for a low-light image enhancement model to solve the technical problems that the loss function used in the current low-light image enhancement method is too complex, the computational complexity of the related neural network training is large, the resulting image still contains noise and distortion, and the resulting image needs further improvement.

[0009] In a first aspect, an embodiment of the present invention discloses a method for training a low-light image enhancement model. The low-light image enhancement model includes an image decomposition module, including:

[0010] Input the low-light image and the normal image into the image decomposition module respectively to obtain a low-light image reflectance map corresponding to the low-light image and a normal image reflectance map corresponding to the normal image;

[0011] Performing degradation perception processing on the low-illumination image reflectance map and the normal image reflectance map respectively to obtain a low-illumination image reflection feature map corresponding to the low-illumination image reflectance map and a normal image reflection feature map corresponding to the normal image reflectance map;

[0012] Calculating a reflectance degradation loss function corresponding to degradation perception processing according to the low-illumination image reflectance map, the normal image reflectance map, the low-illumination image reflection feature map, and the normal image reflection feature map, including: determining a reflectance similarity loss function according to the low-illumination image reflectance map and the normal image reflectance map; determining a reflection feature invariance loss function according to the low-illumination image reflection feature map and the normal image reflection feature map; determining a reflection feature smoothness loss function according to the low-illumination image reflection feature map; determining a reflectance degradation loss function according to the reflectance similarity loss function, the reflection feature invariance loss function, and the reflection feature smoothness loss function;

[0013] The parameters of the degradation perception processing are updated according to the calculated reflectivity degradation loss function until the calculation result of the reflectivity degradation loss function meets the first preset condition. The degradation perception processing after the parameter update is determined as the final degradation perception processing, and the reflectivity degradation loss function corresponding to the final degradation perception processing is determined as the final reflectivity degradation loss function. The final reflectivity degradation loss function is used to train the image decomposition module.

[0014] Using this technical solution, the low-light image enhancement model training method introduces degradation-aware processing and designs a reflectance degradation loss function. This simple reflectance degradation loss function is then used to train the degradation-aware processing. This extracts the image's global noise feature map and color information, integrating brightness adjustment and noise suppression to avoid reflectance map degradation while retaining more detail. This also reduces computational effort and ensures training efficiency. The image decomposition module is then trained using the final reflectance degradation loss function corresponding to the trained degradation-aware processing. This allows the trained image decomposition module to directly decompose and generate a scene reflectance map free of noise amplification, color distortion, and halo artifacts.

[0015] According to another specific embodiment of the present invention, the reflectivity degradation loss function is:

[0016]

[0017] Among them, L DA represents the reflectivity degradation loss function, R in Represents the low-light image reflectivity map, F(R in ) represents the low-light image reflection feature map, R ref Represents the normal image reflectivity map, F(R ref ) represents the normal image reflection feature map, Represents the gradient in the horizontal direction, represents the gradient in the vertical direction, ||·||1 represents the 1-norm, ||·||2 represents the 2-norm, and C1 and C2 represent weights.

[0018] According to another specific embodiment of the present invention, the final reflectivity degradation loss function is:

[0019]

[0020] Among them, L DA Represents the final reflectivity degradation loss function, R in Represents the low-light image reflectivity map, F(R in ) represents the low-light image reflection feature map, R ref Represents the normal image reflectivity map, F(R ref ) represents the normal image reflection feature map, Represents the gradient in the horizontal direction, represents the gradient in the vertical direction, ||·||1 represents the 1-norm, and ||·||2 represents the 2-norm.

[0021] According to another specific embodiment of the present invention, a five-layer convolutional neural network is used in degradation perception processing; wherein the first four layers are 32-output channel convolution layers, and the fifth layer is a 3-output channel convolution layer.

[0022] According to another specific embodiment of the present invention, it also includes:

[0023] The low-illumination image and the normal image are input into the image decomposition module respectively, and a low-illumination image illumination map corresponding to the low-illumination image and a normal image illumination map corresponding to the normal image are obtained;

[0024] calculating a decomposition loss function according to the final reflectance degradation loss function, the low illumination image, the low illumination image reflectance map, the low illumination image illumination map, the normal image, the normal image reflectance map, and the normal image illumination map;

[0025] Update the parameters of the image decomposition module according to the calculated decomposition loss function until the calculation result of the decomposition loss function meets the second preset condition, and determine that the image decomposition module with completed parameter update is the trained image decomposition module, and the decomposition loss function corresponding to the trained image decomposition module is the final decomposition loss function.

[0026] According to another specific embodiment of the present invention, the decomposition loss function is calculated based on the final reflectance degradation loss function, the low-light image, the low-light image reflectance map, the low-light image illumination map, the normal image, the normal image reflectance map, and the normal image illumination map, specifically including:

[0027] Determine the reflectivity invariance loss function according to the low-light image reflectivity map and the normal image reflectivity map;

[0028] Determining a decomposition and reconstruction loss function according to the normal image, the normal image reflectance map, the normal image illumination map, the low illumination image, the low illumination image reflectance map, and the low illumination image illumination map;

[0029] determining an illumination smoothness loss function according to a normal image reflectance map, a normal image illumination map, a low illumination image reflectance map, and a low illumination image illumination map;

[0030] The decomposition loss function is determined according to the final reflectance degradation loss function, reflectance invariance loss function, decomposition and reconstruction loss function, and illumination smoothness loss function.

[0031] According to another specific embodiment of the present invention, the decomposition loss function is:

[0032]

[0033] Among them, L DE represents the decomposition loss function, L DA represents the final reflectivity degradation loss function, S in represents low-light image, R in Represents the low-light image reflectance map, I in Represents the illumination map of the low-light image, S ref represents a normal image, R ref Represents the normal image reflectivity map, I ref represents the normal image illumination map, Represents the gradient in the horizontal direction, represents the gradient in the vertical direction, ||·||1 represents the 1-norm, and C3, C4, C5, C6, C7, C8, and C9 all represent weights.

[0034] According to another specific embodiment of the present invention, the final decomposition loss function is:

[0035]

[0036] Among them, L DE Represents the final decomposition loss function, L DA represents the final reflectivity degradation loss function, S in represents low-light image, R in Represents the low-light image reflectance map, I in Represents the illumination map of the low-light image, S ref represents a normal image, R ref Represents the normal image reflectivity map, I ref represents the normal image illumination map, Represents the gradient in the horizontal direction, represents the gradient in the vertical direction, and ||·||1 represents the 1-norm.

[0037] According to another specific embodiment of the present invention, the image decomposition module includes a reflection branch neural network and an illumination branch neural network. The reflection branch neural network is used to obtain a reflectance map of the low-illumination image to be processed, and the illumination branch is used to obtain an illumination map of the low-illumination image to be processed.

[0038] According to another specific embodiment of the present invention, the reflection branch neural network is a convolutional neural network with an upsampling structure and a downsampling structure.

[0039] According to another specific embodiment of the present invention, the reflection branch neural network has eleven layers, the first to fifth layers are downsampling layers, the sixth to tenth layers are upsampling layers, and the eleventh layer is a 3-output channel convolution layer; wherein the first downsampling layer has a 32-output channel convolution layer.

[0040] According to another specific embodiment of the present invention, the illumination branch neural network is a three-layer convolutional neural network; wherein, the first two layers are 32-output channel convolution layers, and the third layer is a 3-output channel convolution layer.

[0041] According to another specific embodiment of the present invention, it also includes:

[0042] The low-light image enhancement model also includes a brightness adaptation module;

[0043] Inputting the illumination map of the low illumination image into the brightness adaptation module to obtain a plurality of illumination maps with different brightness adaptation levels corresponding to the illumination map of the low illumination image;

[0044] Multiple illumination maps with different brightness adaptation levels are stitched together to obtain a multi-channel illumination map.

[0045] According to another specific embodiment of the present invention, the number of illumination maps with different brightness adaptation levels is 32, and the multi-channel illumination map is a 32-channel illumination map.

[0046] According to another specific embodiment of the present invention, 32 illumination maps with different brightness adaptation levels are obtained by the following formula:

[0047]

[0048] Among them, I in Represents the illumination map of low-light image, I n ,n∈{1,2,3,...,32} represents multiple illumination maps with different brightness adaptation levels.

[0049] According to another specific embodiment of the present invention, it also includes:

[0050] The low-light image enhancement model also includes an image enhancement module;

[0051] Input the 32-channel illumination map into the image enhancement module to obtain the corresponding enhanced illumination map;

[0052] Calculate the enhancement loss function according to the enhanced illumination map, the normal image, the normal image reflectance map, the normal image illumination map, and the low illumination image reflectance map;

[0053] The parameters of the image enhancement module are updated according to the calculated enhancement loss function until the calculation result of the enhancement loss function meets the third preset condition, and the image enhancement module whose parameters are updated is determined to be the trained image enhancement module, and the enhancement loss function corresponding to the trained image enhancement module is the final enhancement loss function.

[0054] According to another specific embodiment of the present invention, the enhancement loss function is calculated according to the enhanced illumination map, the normal image, the normal image reflectance map, the normal image illumination map, and the low illumination image reflectance map, specifically including:

[0055] Determining an enhanced reconstruction loss function according to the enhanced illumination map, the normal image, the normal image reflectance map, and the low illumination image reflectance map;

[0056] Determine the brightness difference loss function according to the enhanced illumination map and the normal image illumination map;

[0057] Determine the gradient reconstruction loss function according to the enhanced illumination map, the normal image and the reflectance map of the low illumination image;

[0058] The enhancement loss function is determined according to the enhancement reconstruction loss function, the brightness difference loss function and the gradient reconstruction loss function.

[0059] According to another specific embodiment of the present invention, the enhancement loss function is:

[0060]

[0061] Among them, L EN represents the enhanced loss function, I out represents the enhanced illumination map, R ref Represents the normal image reflectivity map, S ref represents a normal image, R in Represents the low-light image reflectance map, I ref represents the normal image illumination map, Represents the gradient in the horizontal direction, represents the gradient in the vertical direction, ||·||1 represents the 1-norm, and C10 represents the weight.

[0062] According to another specific embodiment of the present invention, the final enhancement loss function is:

[0063]

[0064] Among them, L EN Represents the final enhanced loss function, I out represents the enhanced illumination map, R ref Represents the normal image reflectivity map, S ref represents a normal image, R in Represents the low-light image reflectance map, I ref represents the normal image illumination map, Represents the gradient in the horizontal direction, represents the gradient in the vertical direction, and ||·||1 represents the 1-norm.

[0065] According to another specific embodiment of the present invention, the image enhancement module includes an image enhancement neural network, and the image enhancement neural network has a convolutional neural network with an upsampling structure and a downsampling structure.

[0066] According to another specific embodiment of the present invention, the image enhancement neural network has eleven layers, the first to fifth layers are downsampling layers, the sixth to tenth layers are upsampling layers, and the eleventh layer is a single-output channel convolution layer; wherein the first downsampling layer has a 32-output channel convolution layer.

[0067] According to another specific embodiment of the present invention, it also includes:

[0068] A trained low-illumination image enhancement model is obtained according to the trained image decomposition module, the brightness adaptation module and the trained image enhancement module.

[0069] In a second aspect, an embodiment of the present invention further discloses a low-light image enhancement method, wherein a low-light image is enhanced by a low-light image enhancement model, wherein the low-light image enhancement model includes an image decomposition module, and the low-light image enhancement method includes:

[0070] Inputting the low-illumination image to be processed into the image decomposition module to obtain a reflectance map of the low-illumination image to be processed and an illumination map of the low-illumination image to be processed;

[0071] The image decomposition module is trained by the final reflectivity degradation function, which is obtained by the following method:

[0072] Input the low-light image and the normal image into the image decomposition module respectively to obtain a low-light image reflectance map corresponding to the low-light image and a normal image reflectance map corresponding to the normal image;

[0073] Performing degradation perception processing on the low-illumination image reflectance map and the normal image reflectance map respectively to obtain a low-illumination image reflection feature map corresponding to the low-illumination image reflectance map and a normal image reflection feature map corresponding to the normal image reflectance map;

[0074] Calculating a reflectance degradation loss function corresponding to degradation perception processing according to the low-illumination image reflectance map, the normal image reflectance map, the low-illumination image reflection feature map, and the normal image reflection feature map, including: determining a reflectance similarity loss function according to the low-illumination image reflectance map and the normal image reflectance map; determining a reflection feature invariance loss function according to the low-illumination image reflection feature map and the normal image reflection feature map; determining a reflection feature smoothness loss function according to the low-illumination image reflection feature map; determining a reflectance degradation loss function according to the reflectance similarity loss function, the reflection feature invariance loss function, and the reflection feature smoothness loss function;

[0075] The parameters of the degradation perception processing are updated according to the calculated reflectivity degradation loss function until the calculation result of the reflectivity degradation loss function meets the first preset condition, and the degradation perception processing after the parameter update is determined as the final degradation perception processing, and the reflectivity degradation loss function corresponding to the final degradation perception processing is determined as the final reflectivity degradation loss function.

[0076] By adopting the above-mentioned technical solution, the low-illumination image enhancement method provided by the present invention enhances the low-illumination image through a low-illumination image enhancement model, wherein the image decomposition module of the low-illumination image enhancement model is trained by the final reflectivity degradation loss function, so that the low-illumination image enhancement model can directly decompose the reflectivity map of the image to be processed and the illumination map of the low-illumination image to be processed without noise amplification, color distortion and halo artifacts through the image decomposition module, which is conducive to achieving better results of subsequent image enhancement.

[0077] According to another specific embodiment of the present invention, the low-light image enhancement model further includes a brightness adaptation module, and the low-light image enhancement method further includes:

[0078] The illumination map of the low-illumination image to be processed is input into the brightness adaptation module to obtain a multi-channel illumination map of the low-illumination image to be processed.

[0079] According to another specific embodiment of the present invention, the low-light image enhancement model further includes an image enhancement module, and the low-light image enhancement method further includes:

[0080] The multi-channel low-illumination image illumination map to be processed is input into the image enhancement module to obtain the enhanced low-illumination image illumination map.

[0081] According to another specific embodiment of the present invention, it also includes:

[0082] The dot product of the illumination map of the enhanced low-illumination image and the reflectance map of the low-illumination image to be processed is calculated to obtain an enhanced image corresponding to the low-illumination image to be processed after enhancement.

[0083] In a third aspect, an embodiment of the present invention further discloses a low-light image enhancement model training device for training a low-light image enhancement model, comprising:

[0084] An image input module is used to input the low-light image and the normal image into the image decomposition module of the low-light image enhancement model respectively, to obtain a low-light image reflectance map corresponding to the low-light image and a normal image reflectance map corresponding to the normal image;

[0085] A degradation perception design module is used to perform degradation perception processing on the low-illumination image reflectance map and the normal image reflectance map, respectively, to obtain a low-illumination image reflection feature map corresponding to the low-illumination image reflectance map and a normal image reflection feature map corresponding to the normal image reflectance map;

[0086] A reflectivity degradation loss function design module is used to calculate the reflectivity degradation loss function corresponding to the degradation perception processing based on the low-light image reflectivity map, the normal image reflectivity map, the low-light image reflection feature map and the normal image reflection feature map, including: determining the reflectivity similarity loss function based on the low-light image reflectivity map and the normal image reflectivity map; determining the reflectivity feature invariance loss function based on the low-light image reflection feature map and the normal image reflection feature map; determining the reflectivity feature smoothness loss function based on the low-light image reflection feature map; determining the reflectivity degradation loss function based on the reflectivity similarity loss function, the reflectivity feature invariance loss function and the reflectivity feature smoothness loss function;

[0087] The degradation perception training module is used to update the parameters of the degradation perception processing according to the calculated reflectivity degradation loss function until the calculation result of the reflectivity degradation loss function meets the first preset condition, and the degradation perception processing after the parameter update is determined as the final degradation perception processing, and the reflectivity degradation loss function corresponding to the final degradation perception processing is determined as the final reflectivity degradation loss function.

[0088] By adopting the above technical solution, the low-illumination image enhancement model training device combines the image input module, the degradation perception design module, the reflectance degradation loss function design module and the degradation perception training module to obtain the final reflectance degradation loss function, and uses the final reflectance degradation loss function to train the image decomposition module of the low-illumination image enhancement model, so that the trained image decomposition module directly decomposes and generates a scene reflectance map without noise amplification, color distortion and halo artifacts. At the same time, the loss function structure is simple, the amount of calculation is reduced, the training speed is fast and the efficiency is high.

[0089] According to another specific embodiment of the present invention, it also includes:

[0090] a decomposition input module, configured to input the low-illumination image and the normal image into the image decomposition module respectively, to obtain a low-illumination image illumination map corresponding to the low-illumination image, and a normal image illumination map corresponding to the normal image;

[0091] a decomposition loss function design module, for calculating a decomposition loss function according to a final reflectance degradation loss function, a low-light image, a low-light image reflectance map, a low-light image illumination map, a normal image, a normal image reflectance map, and a normal image illumination map;

[0092] A decomposition training module is used to update the parameters of the image decomposition module based on the calculated decomposition loss function until the calculation result of the decomposition loss function meets the second preset condition, and determine that the image decomposition module with completed parameter update is the trained image decomposition module, and the decomposition loss function corresponding to the trained image decomposition module is the final decomposition loss function.

[0093] According to another specific embodiment of the present invention, it also includes:

[0094] A brightness input module is used to input the low-light image illumination map into the brightness adaptation module of the low-light image enhancement model to obtain multiple illumination maps with different brightness adaptation levels corresponding to the low-light image illumination map;

[0095] The brightness stitching module is used to stitch multiple illumination maps with different brightness adaptation levels to obtain a multi-channel illumination map.

[0096] According to another specific embodiment of the present invention, it also includes:

[0097] An enhancement input module is used to input the multi-channel illumination map into the image enhancement module of the low illumination image enhancement model to obtain the corresponding enhanced illumination map;

[0098] An enhancement loss function design module is used to calculate the enhancement loss function according to the enhanced illumination map, the normal image, the normal image reflectance map, the normal image illumination map, and the low illumination image reflectance map;

[0099] An enhancement training module is used to update the parameters of the image enhancement module based on the calculated enhancement loss function until the calculation result of the enhancement loss function meets the third preset condition, and determine that the image enhancement module with completed parameter update is the trained image enhancement module, and the enhancement loss function corresponding to the trained image enhancement module is the final enhancement loss function.

[0100] In a fourth aspect, an embodiment of the present invention further discloses a low-light image enhancement device, which enhances a low-light image using a low-light image enhancement model, comprising:

[0101] An image input module inputs the low-illumination image to be processed into the image decomposition module of the low-illumination image enhancement model to obtain a reflectance map of the low-illumination image to be processed and an illumination map of the low-illumination image to be processed;

[0102] The image decomposition module is trained by the final reflectivity degradation function, which is obtained by the following method:

[0103] Input the low-light image and the normal image into the image decomposition module respectively to obtain a low-light image reflectance map corresponding to the low-light image and a normal image reflectance map corresponding to the normal image;

[0104] Performing degradation perception processing on the low-illumination image reflectance map and the normal image reflectance map respectively to obtain a low-illumination image reflection feature map corresponding to the low-illumination image reflectance map and a normal image reflection feature map corresponding to the normal image reflectance map;

[0105] Calculating a reflectance degradation loss function corresponding to degradation perception processing according to the low-illumination image reflectance map, the normal image reflectance map, the low-illumination image reflection feature map, and the normal image reflection feature map, including: determining a reflectance similarity loss function according to the low-illumination image reflectance map and the normal image reflectance map; determining a reflection feature invariance loss function according to the low-illumination image reflection feature map and the normal image reflection feature map; determining a reflection feature smoothness loss function according to the low-illumination image reflection feature map; determining a reflectance degradation loss function according to the reflectance similarity loss function, the reflection feature invariance loss function, and the reflection feature smoothness loss function;

[0106] The parameters of the degradation perception processing are updated according to the calculated reflectivity degradation loss function until the calculation result of the reflectivity degradation loss function meets the first preset condition, and the degradation perception processing after the parameter update is determined as the final degradation perception processing, and the reflectivity degradation loss function corresponding to the final degradation perception processing is determined as the final reflectivity degradation loss function.

[0107] Using this technical solution, a low-light image enhancement device enhances low-light images using a low-light image enhancement model. The model's image decomposition module is trained using degradation-aware processing. By decomposing the image to be processed using the image input module, the device directly generates a reflectance map and an illumination map for the processed low-light image, free of noise amplification, color distortion, and halo artifacts. This facilitates achieving better results in subsequent image enhancement.

[0108] According to another specific embodiment of the present invention, it also includes:

[0109] The brightness adaptation adjustment module is used to input the illumination map of the low-illumination image to be processed into the brightness adaptation module of the low-illumination image enhancement model to obtain the illumination map of the multi-channel low-illumination image to be processed.

[0110] According to another specific embodiment of the present invention, it also includes:

[0111] The image enhancement integration module is used to input the illumination map of the multi-channel low-illumination image to be processed into the image enhancement module of the low-illumination image enhancement model to obtain the enhanced low-illumination image illumination map.

[0112] According to another specific embodiment of the present invention, it also includes:

[0113] The image output module is used to calculate the dot product of the illumination map of the enhanced low-light image and the reflectance map of the low-light image to be processed, so as to obtain the enhanced image corresponding to the low-light image to be processed after enhancement.

[0114] In a fifth aspect, an embodiment of the present invention further discloses a computer device comprising a processor and a memory, wherein the memory stores at least one instruction, and when the at least one instruction is executed by the processor, implements the training method of the low-light image enhancement model in any of the aforementioned embodiments, and / or implements the low-light image enhancement method in any of the aforementioned embodiments.

[0115] By adopting the above technical solution, the computer device obtains the final reflectivity degradation loss function by adjusting the parameters of the degradation perception processing, and then uses the final reflectivity degradation loss function to train the image decomposition module of the low-light image enhancement model. The trained image decomposition module can directly decompose the scene reflectivity map without noise amplification, color distortion and halo artifacts. The low-light image enhancement model finally trained also has a better enhancement effect on low-light images.

[0116] In a sixth aspect, an embodiment of the present invention further discloses a computer-readable storage medium, in which at least one instruction is stored. When the at least one instruction is executed, the training method of the low-light image enhancement model in any of the aforementioned embodiments is implemented, and / or the low-light image enhancement method in any of the aforementioned embodiments is implemented.

[0117] By adopting the above technical solution, the computer-readable storage medium obtains the final reflectivity degradation loss function by adjusting the parameters of the degradation perception processing, and then uses the final reflectivity degradation loss function to train the image decomposition module of the low-light image enhancement model, so that the trained image decomposition module can directly decompose the scene reflectivity map without noise amplification, color distortion and halo artifacts. The low-light image enhancement model finally trained also has a better enhancement effect on low-light images. BRIEF DESCRIPTION OF THE DRAWINGS

[0118] Figure 1 The process of the low-light image enhancement model training method in the embodiment of the present invention is shown as follows Figure 1 ;

[0119] Figure 2 The process of the low-light image enhancement model training method in the embodiment of the present invention is shown as follows Figure 2 ;

[0120] Figure 3 The process of the low-light image enhancement model training method in the embodiment of the present invention is shown as follows Figure 3 ;

[0121] Figure 4 The process of the low-light image enhancement model training method in the embodiment of the present invention is shown as follows Figure 4 ;

[0122] Figure 5 The process of the low-light image enhancement model training method in the embodiment of the present invention is shown as follows Figure 5 ;

[0123] Figure 6 The process of the low-light image enhancement model training method in the embodiment of the present invention is shown as follows Figure 6 ;

[0124] Figure 7 The process of the low-light image enhancement model training method in the embodiment of the present invention is shown as follows Figure 7 ;

[0125] Figure 8 The process of the low-light image enhancement model training method in the embodiment of the present invention is shown as follows Figure 8 ;

[0126] Figure 9 The process of the low-light image enhancement model training method in the embodiment of the present invention is shown as follows Figure 9 ;

[0127] Figure 10 A flow chart of a low-illumination image enhancement method according to an embodiment of the present invention is shown;

[0128] Figure 11 A schematic diagram showing the structure of a low-illumination image enhancement model training device according to an embodiment of the present invention is shown;

[0129] Figure 12 A schematic structural diagram of a low-illumination image enhancement device according to an embodiment of the present invention is shown;

[0130] Figure 13 A schematic diagram showing the structure of a computer device according to an embodiment of the present invention is shown;

[0131] Figure 14 shows a low-light image used in an embodiment of the present invention;

[0132] Figure 15 shows a normal image used in an embodiment of the present invention;

[0133] Figure 16 FIG4 shows an enhanced image of a low-illumination image used in an embodiment of the present invention after being enhanced by the method of an embodiment of the present invention;

[0134] Figure 17 A schematic diagram showing a comparison between a low-light image reflectance map and a low-light image illumination map in the prior art and an embodiment of the present invention;

[0135] Figure 18 A schematic diagram showing a comparison between a low-light image illumination map and a 32-channel low-light image illumination map in an embodiment of the present invention;

[0136] Figure 19 A schematic diagram showing a comparison of enhanced images in the prior art and in an embodiment of the present invention;

[0137] Figure 20 A schematic diagram showing a test comparison between the prior art and an embodiment of the present invention when the image to be processed comes from the LOL dataset;

[0138] Figure 21 A schematic diagram showing a test comparison between the prior art and an embodiment of the present invention when the image to be processed is an image taken in an actual low-light environment;

[0139] Figure 22 A schematic diagram showing the structures of a trained degradation perception module and a low-illumination image enhancement model in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0140] The following is an explanation of the embodiments of the present invention by specific specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Although the description of the present invention will be introduced in conjunction with the preferred embodiment, this does not mean that the features of this invention are limited to this embodiment. On the contrary, the purpose of introducing the invention in conjunction with the embodiment is to cover other options or modifications that may be extended based on the claims of the present invention. In order to provide a deep understanding of the present invention, the following description will contain many specific details. The present invention can also be implemented without using these details. In addition, in order to avoid confusion or blurring the focus of the present invention, some specific details will be omitted in the description. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0141] It should be noted that in this specification, similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0142] The terms “first”, “second”, etc. are only used for distinguishing descriptions and should not be understood as indicating or implying relative importance.

[0143] In the description of this embodiment, it should be noted that, unless otherwise specified or limited, the terms "disposed," "connected," and "connected" should be understood broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal connections between two components. Those skilled in the art will understand the specific meanings of the above terms in this embodiment based on specific circumstances.

[0144] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0145] First, reference Figure 1 The present invention provides a training method for a low-light image enhancement model. The low-light image enhancement model includes an image decomposition module. The training method includes:

[0146] S1: Input the low-light image and the normal image into the image decomposition module respectively to obtain a low-light image reflectance map corresponding to the low-light image and a normal image reflectance map corresponding to the normal image.

[0147] Among them, the normal image and the low-light image come from a training data set. In the training data set, a plurality of image pairs are included, each image pair consists of a low-light image and its corresponding normal image, and the low-light image and its corresponding normal image are two pictures taken in the same scene with different lighting conditions. The training data set can be a self-set data set, or it can be an existing set data set, such as the LOL data set. Preferably, the images used for training can be selected from the LOL data set. The LOL data set (Low-Light Enhancement data set) is an existing and specially used to train models related to deep learning methods involving enhanced low-light images, and the data set contains scene images of multiple scenes and different exposure levels. By using this data set to train the model, it is beneficial to improve the generalization ability of the model. The reflectance map refers to the image after removing the highlights from the original image.

[0148] S2: performing degradation perception processing on the low-illumination image reflectance map and the normal image reflectance map respectively to obtain a low-illumination image reflection feature map corresponding to the low-illumination image reflectance map and a normal image reflection feature map corresponding to the normal image reflectance map.

[0149] The reflectance feature map is the global image noise and color information extracted from the reflectance map. Degradation-aware processing can be performed, for example, by a degradation-aware module. The following uses the degradation-aware module as an example to describe the subsequent steps of a low-light image enhancement model training method, a low-light image enhancement method, a low-light image enhancement model training device, and a low-light image enhancement device.

[0150] S3: Calculate the reflectivity degradation loss function corresponding to the degradation perception module according to the low-light image reflectivity map, the normal image reflectivity map, the low-light image reflectivity feature map, and the normal image reflectivity feature map.

[0151] Among them, the reflectivity degradation loss function is designed to ensure that the reflectance feature map of the low-light image is as close as possible to the reflectance feature map of the normal image in terms of noise information, color information, etc.

[0152] S4: updating the parameters of the degradation perception module according to the calculated reflectivity degradation loss function until the calculation result of the reflectivity degradation loss function meets the first preset condition, and determining the degradation perception module with the updated parameters as the final degradation perception module.

[0153] The reflectivity degradation loss function corresponding to the final degradation perception module is the final reflectivity degradation loss function, and the final reflectivity degradation loss function is used to train the image decomposition module.

[0154] The degradation-aware module is used to extract the image's global noise feature map and color information. In this embodiment, the aforementioned technical solution is employed to pre-train the degradation-aware module and incorporate the trained degradation-aware module's final reflectivity degradation loss function into the decomposition loss function of the image decomposition module. This allows the trained image decomposition module to directly decompose and generate a reflectivity map free of noise amplification, color distortion, and halo artifacts. This prevents noise amplification or distortion during subsequent image enhancement, which could compromise the quality of the enhanced image.

[0155] In some possible embodiments provided by the present invention, reference is made to Figure 2 , step S3 specifically includes the following steps:

[0156] S31: Determine a reflectivity similarity loss function according to the low-illumination image reflectivity map and the normal image reflectivity map.

[0157] The reflectivity similarity loss function is used to make the reflectivity of the low-light image reflectivity map and the normal image reflectivity map as consistent as possible. The specific formula is:

[0158]

[0159] Among them, L1 represents the reflectivity similarity loss function, R in Represents the low-light image reflectance map, R ref represents the normal image reflectance map, ||·||2 represents the 2-norm, and C1 represents the weight.

[0160] S32: Determine a reflection feature invariance loss function according to the low-illumination image reflection feature map and the normal image reflection feature map.

[0161] The reflection feature invariance loss function is used to make the reflection features of the low-light image reflection feature map and the normal image reflection feature map as consistent as possible. The specific formula is:

[0162] L2=C2×||F(R in )-F(R ref )||1 (Formula 2)

[0163] Among them, L2 represents the reflection feature invariance loss function, F(R in) represents the low-light image reflection feature map, F(R ref ) represents the normal image reflection feature map, ||·||1 represents the 1-norm, and C2 represents the weight.

[0164] S33: Determine a reflection feature smoothing loss function according to the low-illumination image reflection feature map.

[0165] The reflection feature smoothing loss function is used to make the gradient of the reflection feature change smoothly. The specific formula is:

[0166]

[0167] Among them, L3 represents the reflection feature smoothing loss function, F(R in ) represents the low-light image reflection feature map, F(R ref ) represents the normal image reflection feature map, Represents the gradient in the horizontal direction, represents the gradient in the vertical direction, and ||·||2 represents the 2-norm.

[0168] S34: Determine a reflectivity degradation loss function according to the reflectivity similarity loss function, the reflectivity feature invariance loss function, and the reflectivity feature smoothness loss function.

[0169] Combining the formulas in the above steps, the initial reflectivity degradation loss function formula obtained in step S34 is:

[0170]

[0171] Among them, L DA represents the reflectivity degradation loss function, R in Represents the low-light image reflectivity map, F(R in ) represents the low-light image reflection feature map, R ref Represents the normal image reflectivity map, F(R ref ) represents the normal image reflection feature map, Represents the gradient in the horizontal direction, represents the gradient in the vertical direction, ||·||1 represents the 1-norm, ||·||2 represents the 2-norm, and C1 and C2 represent weights.

[0172] Similar loss function formulas in the prior art are complex and have many terms. Consequently, model training is computationally intensive and time-consuming, impacting model training speed, making optimization difficult and potentially causing training to fail to converge. Generally speaking, the large number of terms and complex formulas in a loss function indicate that a wide range of loss types are considered, potentially more comprehensive. Models trained using such loss functions should be able to produce better enhanced images when processing low-light images. However, complex loss functions are computationally intensive, impacting model training efficiency. The inventors sought to simplify the loss function and determine the weights of the different loss terms in the simplified loss function, while also ensuring the quality of the model trained using the simplified loss function. However, simplifying the loss function presents numerous challenges. Considering the role of different loss terms in the overall loss function and their impact on model training ensures that the loss function, even after removing some loss terms, still produces a good model. Furthermore, the weights of the different loss terms must be readjusted. The relationship between the weights of the different loss terms in the simplified loss function can potentially lead to unexpected new impacts on model training.

[0173] Specifically, in this embodiment, the inventors have found through careful research that the reflectivity degradation loss function L DA Only the reflectance similarity loss function L1, the reflectance feature invariance loss function L2 and the reflectance feature smoothness loss function L3 can be introduced, while the calculation of the 2-norm of the difference between the low-light image reflectance feature map and the normal image reflectance feature map, the 1-norm of the difference between the low-light image reflectance map and the normal image reflectance map and other related loss terms are omitted. At this time, the results of the model training will not deteriorate, and the performance of the model can even be improved, thereby improving the degradation perception module in removing image noise and extracting image color features. It has a better effect. Among them, the 1-norm can achieve sparseness and is the optimal convex approximation of the 0-norm (directly solving the 0-norm is an NP-hard problem and cannot be solved). The reflectance degradation loss function L of this embodiment is used DA It is conducive to the automatic selection of features, the removal of useless features, and better interpretability. It can not only ensure better training results in the subsequent training of the neural network used in the low-light image enhancement model and obtain enhanced images with better quality, but also simplify calculations and improve model training efficiency.

[0174] In some possible embodiments provided by the present invention, reference is made to Figure 1 , step S4 is specifically as follows: first, the low-light image reflectance map and the normal image reflectance map generated by the initial decomposition of the image decomposition module are respectively input into the initially set degradation perception module to obtain the corresponding low-light image reflectance feature map and normal image reflectance feature map.

[0175] Substitute the image information of the low-illumination image reflectance map, the normal image reflectance map, the low-illumination image reflection feature map, and the normal image reflection feature map into the reflectance degradation loss function (Formula 4) to calculate the reflectance degradation loss function value.

[0176] Then, the parameters of the neural network in the degradation perception module are updated according to the value of the reflectivity degradation loss function. The parameters of the neural network can be, for example, the weights of each node in the neural network. Specifically, the parameters of the neural network are updated by the back propagation algorithm. The back propagation algorithm is an optimization algorithm based on gradient descent. By calculating the gradient of the loss function with respect to the parameters of the neural network model, the parameters are slightly updated along the gradient direction, thereby updating the parameters of the neural network model to minimize the loss function. In this embodiment, the Adam method is adopted to adjust the parameters of the neural network by maintaining the first-order momentum and second-order momentum of the neural network gradient and the square of the gradient until the first preset condition is met. Among them, the first preset condition is that the reflectivity degradation loss function has been minimized, which can specifically be when a preset number of training times is reached. At this time, it is considered that the reflectivity degradation loss function has been minimized. The preset number of training times can be, for example, 200 times.

[0177] In the preferred embodiment provided by the present invention, C1 is 20, C2 is 10, and the final reflectivity degradation loss function is:

[0178]

[0179] Among them, L DA Represents the final reflectivity degradation loss function, R in Represents the low-light image reflectivity map, F(R in ) represents the low-light image reflection feature map, R ref Represents the normal image reflectivity map, F(R ref ) represents the normal image reflection feature map, Represents the gradient in the horizontal direction, represents the gradient in the vertical direction, ||·||1 represents the 1-norm, and ||·||2 represents the 2-norm.

[0180] C1 and C2 are hyperparameters that adapt to varying noise levels and degradation. The inventors found that a higher C1 / C2 ratio preserves better detail but reduces denoising capabilities, while a lower C1 / C2 ratio enhances denoising capabilities but reduces detail. In this embodiment, C1 is set to 10 and C2 is set to 20, which achieves both good denoising capabilities and minimal detail loss.

[0181] The final reflectance degradation loss function is added to the decomposition loss function of the image decomposition module, and the image decomposition module is subsequently trained. The trained image decomposition module directly decomposes the reflectance and illumination maps without noise amplification, color distortion, and halo artifacts. This avoids noise amplification and distortion amplification after image enhancement, and eliminates the need for additional processing steps such as denoising after image enhancement. This can save the running time of the image processing algorithm and reduce the complexity of the code. In addition, the use of the degradation perception module to integrate brightness adjustment and noise suppression can also retain more details while avoiding degradation of the reflectance map, so that the model can achieve better generalization and robustness.

[0182] In some possible embodiments provided by the present invention, the degradation perception module has a five-layer convolutional neural network; wherein, the first four layers are 32-output channel convolutional layers, and the fifth layer is a 3-output channel convolutional layer. The final reflectivity degradation function obtained by training the degradation perception module has a better training effect on the image decomposition module, and has a better effect in avoiding noise amplification, color distortion and halo artifacts. In the degradation perception module, a convolutional neural network with fewer layers than five layers has insufficient representation capabilities for image features and is difficult to effectively extract image degradation patterns; while a convolutional neural network with more layers than five layers will bring more parameters, resulting in a decrease in the training speed and inference speed of the model, an increase in the model running time, and low efficiency. Selecting a five-layer convolutional neural network in the degradation perception module can not only ensure the effective extraction of image degradation patterns, but also ensure high operating efficiency.

[0183] In some possible embodiments provided by the present invention, reference is made to Figure 3 , the training method of the low-light image enhancement model also includes the following steps:

[0184] S5: Inputting the low-illumination image and the normal image into the image decomposition module respectively, and obtaining the corresponding low-illumination image illumination map and normal image illumination map respectively.

[0185] For example, Figure 14 The low-light image shown, Figure 15 The normal image shown is input to the image decomposition module.

[0186] S6: Calculate a decomposition loss function according to the final reflectance degradation loss function, the low illumination image, the low illumination image reflectance map, the low illumination image illumination map, the normal image, the normal image reflectance map, and the normal image illumination map.

[0187] Among them, the decomposition loss function is designed to ensure that the image generated after the decomposition of the low-light image is close to the normal image in terms of reflectivity and illumination.

[0188] S7: Update the parameters of the image decomposition module according to the calculated decomposition loss function until the calculation result of the decomposition loss function meets the second preset condition, and determine that the image decomposition module with the updated parameters is the trained image decomposition module.

[0189] The image decomposition module is trained under the guidance of the degradation perception module. The trained image decomposition module generates scene reflectance and illumination maps without noise amplification, color distortion, or halo artifacts. The decomposition loss function corresponding to the trained image decomposition module is the final decomposition loss function.

[0190] In some possible embodiments provided by the present invention, reference is made to Figure 4 Combined with Figure 3 , step S6 specifically includes the following steps:

[0191] S61: Determine a reflectivity invariance loss function according to the low-illumination image reflectivity map and the normal image reflectivity map.

[0192] In this embodiment, the reflectivity invariance loss function of step S61 is used to make the reflectivity of the low-light image reflectivity map and the normal image reflectivity map as consistent as possible. The specific formula is:

[0193] L4=C3×||R in -R ref ||1 (Formula 6)

[0194] Among them, L4 represents the reflectivity invariance loss function, R in Represents the low-light image reflectance map, R ref represents the normal image reflectance map, ||·||1 represents the 1-norm, and C3 represents the weight.

[0195] S62: Determine a decomposition and reconstruction loss function according to the normal image, the normal image reflectance map, the normal image illumination map, the low illumination image, the low illumination image reflectance map, and the low illumination image illumination map.

[0196] In this embodiment, the specific formula of the decomposition and reconstruction loss function in step S62 is:

[0197] L5=||I in ×R in -S in ||1+||I ref ×R ref -S ref ||1+C4×||I in ×R ref -S in ||1+C5×||I ref ×R in -S ref||1 (Formula 7)

[0198] Among them, L5 represents the decomposition and reconstruction loss function, R in Represents the low-light image reflectance map, I in Represents the illumination map of the low-light image, S ref represents a normal image, R ref Represents the normal image reflectivity map, I ref represents the illumination map of a normal image, ||·||1 represents the 1-norm, and C4 and C5 represent weights.

[0199] S63: Determine an illumination smoothness loss function according to the normal image reflectance map, the normal image illumination map, the low illumination image reflectance map, and the low illumination image illumination map.

[0200] In this embodiment, the illumination smoothness loss function in step S63 is used to constrain the situation where the illumination changes greatly so that the gradient changes smoothly. The specific formula is:

[0201]

[0202] Wherein, L6 represents the lighting smoothness loss function, I in Represents the illumination map of the low-light image, I ref represents the normal image illumination map, Represents the gradient in the horizontal direction, represents the gradient in the vertical direction, ||·||1 represents the 1-norm, and C6, C7, C8, and C9 all represent weights.

[0203] S64: Determine the decomposition loss function according to the final reflectivity degradation loss function, the reflectivity invariance loss function, the decomposition and reconstruction loss function, and the illumination smoothness loss function.

[0204] Combining the formulas in the above steps, the decomposition loss function formula of the initial setting obtained in step S64 is:

[0205]

[0206] Among them, L DE represents the decomposition loss function, L DA represents the final reflectivity degradation loss function, S in represents low-light image, R in Represents the low-light image reflectance map, I in Represents the illumination map of the low-light image, S ref represents a normal image, R ref Represents the normal image reflectivity map, I ref represents the normal image illumination map, Represents the gradient in the horizontal direction, represents the gradient in the vertical direction, ||·||1 represents the 1-norm, and C3, C4, C5, C6, C7, C8, and C9 all represent weights.

[0207] The inventors have simplified the design of the decomposition loss function and re-determined the weights. Specifically, in this embodiment, the decomposition loss function L DE Only the reflectivity invariance loss function L4, the decomposition and reconstruction loss function L5, and the illumination smoothness loss function L6 are introduced, which eliminates the need to calculate the 2-norm or other losses of the differences between different images, which is conducive to obtaining sparse results. At the same time, the Laplace prior is introduced to enhance the generalization ability, which can not only ensure that a better image decomposition module is obtained in subsequent training, and obtain better quality decomposed illumination maps and reflectance maps, but also simplify the calculation and improve the efficiency of model training.

[0208] In some other possible embodiments provided by the present invention, refer to Figure 3 , step S7 specifically includes: firstly, inputting the low illumination image and the normal image into the initially set image decomposition module respectively, and obtaining the corresponding low illumination image reflectance map and normal image reflectance map.

[0209] Substitute the final reflectivity degradation loss function, the low-light image, the low-light image reflectivity map, the low-light image illumination map, the normal image, the normal image reflectivity map, and the normal image illumination map into the decomposition loss function (Formula 9) to calculate the decomposition loss function value.

[0210] Then, the parameters of the neural network in the image decomposition module are updated according to the value of the decomposition loss function. The parameters of the neural network can be, for example, the weights of each node in the neural network. Specifically, the parameters of the neural network are updated by the back propagation algorithm. The back propagation algorithm is an optimization algorithm based on gradient descent. By calculating the gradient of the loss function with respect to the parameters of the neural network model, the parameters are slightly updated along the gradient direction, thereby updating the parameters of the neural network model to minimize the loss function. In this embodiment, the Adam method is adopted to adjust the parameters of the neural network by maintaining the first-order momentum and second-order momentum of the neural network gradient and the square of the gradient until the second preset condition is met. Among them, the second preset condition is that the decomposition loss function has been minimized, which can be specifically achieved when a preset number of training times is reached. At this time, it is considered that the decomposition loss function has been minimized. The preset number of training times can be, for example, 200 times.

[0211] In the preferred embodiment provided by the present invention, C3, C4 and C5 are all set to 0.01, and C6, C7, C8 and C9 are all set to 0.1. The final decomposition loss function is:

[0212]

[0213] Among them, L DE Represents the final decomposition loss function, L DA represents the final reflectivity degradation loss function, S in represents low-light image, R in Represents the low-light image reflectance map, I in Represents the illumination map of the low-light image, S ref represents a normal image, R ref Represents the normal image reflectivity map, I ref represents the normal image illumination map, Represents the gradient in the horizontal direction, represents the gradient in the vertical direction, and ||·||1 represents the 1-norm.

[0214] Among them, C3, C4, and C5 are all set to 0.01, and C6, C7, C8, and C9 are all set to 0.1, which can ensure that details of the final enhanced image are not lost. After careful research, the inventors found that if the corresponding weights of C3 to C9 are set to larger values, the final enhanced image will lose a lot of image details, affecting the quality of the enhanced image.

[0215] Using the above technical solution, since the final reflectance degradation loss function is added to the final decomposition loss function, the image decomposition module obtained after training can directly decompose the reflectance map and illumination map without noise amplification, color distortion and halo artifacts, avoiding the problem of noise amplification and distortion in the subsequent enhancement process.

[0216] In some possible embodiments provided by the present invention, the neural network used by the image decomposition module includes a reflection branch and an illumination branch.

[0217] The reflection branch is used to obtain the reflectance map of the low-light image to be processed. The reflection branch neural network is a convolutional neural network with upsampling and downsampling structures. The reflection branch neural network has eleven layers: the first five layers are downsampling layers, the sixth to tenth layers are upsampling layers, and the eleventh layer is a three-output channel convolution layer.

[0218] Specifically, the reflection branch neural network is a U-Net type network, in which the first layer consists of a 32-output channel convolution layer activated by two cascaded linear rectification functions and a maximum pooling layer, the second layer consists of a 64-output channel convolution layer activated by two cascaded linear rectification functions and a maximum pooling layer, the third layer consists of a 128-output channel convolution layer activated by two cascaded linear rectification functions and a maximum pooling layer, the fourth layer consists of a 256-output channel convolution layer activated by two cascaded linear rectification functions and a maximum pooling layer, and the fifth layer consists of a 512-output channel convolution layer activated by two cascaded linear rectification functions. and a maximum pooling layer, the sixth layer is a 256-output channel deconvolution upsampling layer, the seventh layer is composed of two 256-output channel convolution layers and a 128-output channel deconvolution upsampling layer in cascade, the eighth layer is composed of two 128-output channel convolution layers and a 64-output channel deconvolution upsampling layer in cascade, the ninth layer is composed of two 64-output channel convolution layers and a 32-output channel deconvolution upsampling layer in cascade, the tenth layer is composed of two 32-output channel convolution layers and a 4-output channel deconvolution upsampling layer in cascade, and the eleventh layer is a three-output channel convolution layer activated by a Sigmoid function.

[0219] The output of the first layer is spliced ​​with the output of the ninth layer via a jumper connection and then input into the tenth layer. The output of the second layer is spliced ​​with the output of the eighth layer via a jumper connection and then input into the ninth layer. The output of the third layer is spliced ​​with the output of the seventh layer via a jumper connection and then input into the eighth layer. The output of the fourth layer is spliced ​​with the output of the sixth layer via a jumper connection and then input into the seventh layer. The reflectance branch requires more image details than the illumination branch. Therefore, a U-Net network is used in the reflectance branch to extract more details through upsampling and downsampling.

[0220] In some possible embodiments provided by the present invention, the illumination branch neural network is a three-layer convolutional neural network. The first two layers are 32-output channel convolution layers activated by a linear rectifier function, and the third layer is a 3-output channel convolution layer activated by a sigmoid function. The input of the first layer is the output of the first layer of the reflection branch without maximum pooling, and the input of the second layer is the concatenation of the output of the first layer of this branch and the output of the tenth layer of the reflection branch.

[0221] Only a three-layer convolutional neural network is used in the illumination branch, which can not only achieve a good decomposition effect, but also reduce the complexity of the algorithm, which is conducive to speeding up the model training process.

[0222] In some other possible embodiments provided by the present invention, refer to Figure 5 , the training method of the low-light image enhancement model also includes the following steps:

[0223] S8: inputting the illumination map of the low illumination image into the brightness adaptation module of the low illumination image enhancement model to obtain a plurality of illumination maps with different brightness adaptation levels corresponding to the illumination map of the low illumination image;

[0224] S9: stitching multiple illumination maps with different brightness adaptation levels to obtain a multi-channel illumination map.

[0225] Among them, in the brightness adaptation module, the retinal mechanism of the organism is simulated, and based on this, multiple illumination maps with different brightness adaptation levels are generated based on the illumination map of the input low-light image.

[0226] Preferably, reference Figure 6 Combined with Figure 5 , steps S8 and S9 are specifically the following steps:

[0227] S81: Inputting the illumination map of the low illumination image into the brightness adaptation module of the low illumination image enhancement model to obtain 32 illumination maps with different brightness adaptation levels corresponding to the illumination map of the low illumination image.

[0228] Exemplarily, in step S81, 32 illumination images with different brightness adaptation levels are obtained by the following formula:

[0229]

[0230] Among them, I in Represents the illumination map of low-light image, I n ,n∈{1,2,3,...,32} represents 32 illumination maps with different brightness adaptation levels.

[0231] S91: stitch 32 illumination maps with different brightness adaptation levels to obtain a 32-channel illumination map.

[0232] In this embodiment, the advantage of selecting to generate 32 illumination maps with different brightness adaptation levels and splicing them by channel to generate a 32-channel illumination map is that it can ensure that most of the brightness level information of the original low-light image illumination map is extracted, and it is beneficial to reduce the amount of calculation and speed up the training process.

[0233] In the above embodiments, the network design is based on the biological retinal brightness adaptation mechanism, which not only adopts multiple neural networks but also combines the brightness adaptation method, taking into account the advantages of method-driven and data-driven. The algorithm model has stronger interpretability and is easier for people to understand.

[0234] In some possible embodiments provided by the present invention, reference is made to Figure 7 , the low-light image enhancement model training method also includes the following steps:

[0235] S10: Input the 32-channel illumination map into the image enhancement module of the low illumination image enhancement model to obtain a corresponding enhanced illumination map;

[0236] S11: Calculating an enhancement loss function according to the enhanced illumination map, the normal image, the normal image reflectance map, the normal image illumination map, and the low illumination image reflectance map;

[0237] Among them, the enhancement loss function is designed to ensure that the enhanced illumination map obtained from the low-illumination image is as close as possible to the illumination map of the normal image in terms of illumination level.

[0238] S12: Update the parameters of the image enhancement module according to the calculated enhancement loss function until the calculation result of the enhancement loss function meets the third preset condition, and determine that the image enhancement module with the updated parameters is the trained image enhancement module.

[0239] The image enhancement module is used to integrate the scene reflectance map and multiple illumination maps with different brightness adaptation levels to obtain an enhanced image. The enhancement loss function corresponding to the trained image enhancement module is the final enhancement loss function.

[0240] In some possible embodiments provided by the present invention, reference is made to Figure 8 Combined with Figure 7 , step S11 specifically includes the following steps:

[0241] S111: Determine an enhanced reconstruction loss function according to the enhanced illumination map, the normal image, the normal image reflectance map, and the low illumination image reflectance map.

[0242] For example, the specific formula of the enhanced reconstruction loss function in step S111 is:

[0243] L7=C10×||I out ×R ref -S ref ||1+||I out ×R in -S ref ||1 (Formula 12)

[0244] Among them, L7 represents the enhanced reconstruction loss function, I out represents the enhanced illumination map, R ref Represents the normal image reflectivity map, S ref represents a normal image, R in represents the reflectance map of the low-light image, ||·||1 represents the 1-norm, and C10 represents the weight.

[0245] S112: Determine a brightness difference loss function according to the enhanced illumination map and the normal image illumination map.

[0246] For example, the brightness difference loss function in step S112 is used to represent the brightness difference between the enhanced brightness map of the low-light image and the brightness map of the normal image. The specific formula is:

[0247] L8=||I out -I ref ||1 (Formula 13)

[0248] Among them, L8 represents the brightness difference loss function, I out represents the enhanced illumination map, R ref represents the normal image reflectance map, and ||·||1 represents the 1-norm.

[0249] S113: Determine a gradient reconstruction loss function according to the enhanced illumination map, the normal image, and the low illumination image reflectance map.

[0250] The gradient reconstruction loss function in step S113 is used to make the gradient change smoothly. The specific formula is:

[0251]

[0252] Among them, I out represents the enhanced illumination map, R in Represents the low-light image reflectance map, S ref represents a normal image, Represents the gradient in the horizontal direction, represents the gradient in the vertical direction, and ||·||1 represents the 1-norm.

[0253] S114: Determine an enhancement loss function according to the enhancement reconstruction loss function, the brightness difference loss function, and the gradient reconstruction loss function.

[0254] Combining the formulas in the above steps, the enhanced loss function formula of the initial setting obtained in step S114 is:

[0255]

[0256] Among them, L EN Represents the final enhanced loss function, I out represents the enhanced illumination map, R ref Represents the normal image reflectivity map, S ref represents a normal image, R in Represents the low-light image reflectance map, I ref represents the normal image illumination map, Represents the gradient in the horizontal direction, represents the gradient in the vertical direction, ||·||1 represents the 1-norm, and C10 represents the weight.

[0257] In this embodiment, the enhanced loss function LEN Only the reflectivity invariance loss function L7, the decomposition and reconstruction loss function L8 and the illumination smoothness loss function L9 are introduced. Only the enhanced illumination map, the reflectivity maps corresponding to the low-illumination image and the normal image, the normal image and its illumination map are used, without involving the reflection feature maps corresponding to the low-illumination image and the normal image. The calculation of other losses is also omitted, which is conducive to obtaining sparse results. At the same time, the Laplace prior is introduced to enhance the generalization ability and help the network reconstruct details and texture information. It can not only ensure that a better image enhancement module is obtained in subsequent training, and the final enhanced image with less noise and obvious brightness improvement is obtained, but also simplify the calculation and improve the model training efficiency.

[0258] In some possible embodiments provided by the present invention, reference is made to Figure 7 Step S12 specifically includes the following steps: During the training of the image enhancement module by the enhancement training module 07, the 32-channel illumination map is first input into the initially configured image enhancement module to obtain a corresponding enhanced illumination map. The enhanced illumination map, the normal image, the normal image reflectance map, the normal image illumination map, and the low-illumination image reflectance map are substituted into the enhancement loss function (Formula 15) to calculate the enhancement loss function value.

[0259] Then, the parameters of the neural network in the image enhancement module are updated according to the value of the enhanced loss function. The parameters of the neural network can be, for example, the weights of each node in the neural network. Specifically, the parameters of the neural network are updated by the back propagation algorithm. The back propagation algorithm is an optimization algorithm based on gradient descent. By calculating the gradient of the loss function with respect to the parameters of the neural network model, the parameters are slightly updated along the gradient direction, thereby updating the parameters of the neural network model to minimize the loss function. In this embodiment, the Adam method is adopted to adjust the parameters of the neural network by maintaining the first-order momentum and second-order momentum of the neural network gradient and the square of the gradient until the third preset condition is met. Among them, the third preset condition is that the enhanced loss function has been minimized, which can specifically be achieved by reaching a preset number of training times. At this time, it is considered that the enhanced loss function has been minimized. The preset number of training times can be, for example, 200 times.

[0260] In order to reduce the relative weight of the reference image loss term and ensure that the network learning focus is on optimizing the input image, the inventors found through careful research that C10 is preferably set to 0.1, and the final enhancement loss function obtained is:

[0261]

[0262] Among them, L EN Represents the final enhanced loss function, I out represents the enhanced illumination map, R ref Represents the normal image reflectivity map, S refrepresents a normal image, R in Represents the low-light image reflectance map, I ref represents the normal image illumination map, Represents the gradient in the horizontal direction, represents the gradient in the vertical direction, and ||·||1 represents the 1-norm.

[0263] In some possible embodiments provided by the present invention, the image enhancement module includes an image enhancement neural network, which is a convolutional neural network with upsampling and downsampling structures. The image enhancement neural network has eleven layers, with the first to fifth layers being downsampling layers, the sixth to tenth layers being upsampling layers, and the eleventh layer being a single-output channel convolutional layer.

[0264] Exemplarily, the image enhancement neural network is a U-Net type network, in which the first layer consists of a 32-output channel convolution layer activated by two cascaded linear rectification functions and a maximum pooling layer, the second layer consists of a 64-output channel convolution layer activated by two cascaded linear rectification functions and a maximum pooling layer, the third layer consists of a 128-output channel convolution layer activated by two cascaded linear rectification functions and a maximum pooling layer, the fourth layer consists of a 256-output channel convolution layer activated by two cascaded linear rectification functions and a maximum pooling layer, and the fifth layer consists of a 512-output channel convolution layer activated by two cascaded linear rectification functions. and a maximum pooling layer, the sixth layer is a 256-output channel deconvolution upsampling layer, the seventh layer is composed of two 256-output channel convolution layers and a 128-output channel deconvolution upsampling layer in cascade, the eighth layer is composed of two 128-output channel convolution layers and a 64-output channel deconvolution upsampling layer in cascade, the ninth layer is composed of two 64-output channel convolution layers and a 32-output channel deconvolution upsampling layer in cascade, the tenth layer is composed of two 32-output channel convolution layers and a 4-output channel deconvolution upsampling layer in cascade, and the eleventh layer is a single-output channel convolution layer activated by a Sigmoid function.

[0265] The output of the first layer is spliced ​​with the output of the ninth layer through a jumper connection and then input into the tenth layer. The output of the second layer is spliced ​​with the output of the eighth layer through a jumper connection and then input into the ninth layer. The output of the third layer is spliced ​​with the output of the seventh layer through a jumper connection and then input into the eighth layer. The output of the fourth layer is spliced ​​with the output of the sixth layer through a jumper connection and then input into the seventh layer.

[0266] In the image enhancement module, an eleven-layer U-Net network is used to extract more details through up and down sampling structures.

[0267] In some other possible embodiments provided by the present invention, refer to Figure 9 , the training method of the low-light image enhancement model also includes the following steps:

[0268] S13: Obtain a trained low-illumination image enhancement model according to the trained image decomposition module, the brightness adaptation module, and the trained image enhancement module.

[0269] Specifically, refer to Figure 22 ,The trained low-light image enhancement model includes three modules, namely, image ,decomposition module, brightness adaptation module and image enhancement module.

[0270] When using the low-light image enhancement model, the low-light image to be processed (such as Figure 14 As shown in FIG ), the low-light image reflectance image and the low-light image illumination image (as shown in FIG ) are first decomposed into a low-light image reflectance image and a low-light image illumination image (as shown in FIG ) without noise amplification, color distortion and halo artifacts by the image decomposition module. Figure 17 The low-light image illumination map is then input into the brightness adaptation module to generate a 32-channel low-light image illumination map (as shown in Figure 18 As shown in the figure, the image brightness adaptation level information is retained and unnecessary fine structure information is omitted. Subsequently, the 32-channel low-light image illumination map is input into the image enhancement module for enhancement to obtain an enhanced illumination map. Finally, the dot product of the enhanced illumination map and the low-light image reflectivity map is calculated and output to obtain the desired enhanced image after low-light image enhancement (refer to Figure 16 ).

[0271] For example, in any of the above embodiments, the low-light image enhancement model training method uses the LOL dataset to train the low-light image enhancement model, and uses the Adam method to optimize each loss function (including the degradation loss function, the decomposition loss function, and the enhancement loss function). The training batch operation number is set to 4, the learning rate is set to 0.0001, and after 200 rounds of iterative training, it is considered that the neural network model parameters have been trained. Among them, the Adam method is a stochastic optimization method, an adaptive moment estimation gradient descent algorithm, which combines the idea of ​​momentum on the basis of the RMSprop algorithm and performs bias correction for the exponential weighted average. It is a common method for optimizing loss functions.

[0272] The low-light image enhancement model training method presented in this paper can effectively enhance low-light images captured in low-light or uneven-light environments. Unlike previous existing technologies, this method utilizes the biological retinal brightness adaptation mechanism for network design, combining the advantages of both method-driven and data-driven approaches. This method offers strong interpretability and ease of understanding for the algorithmic model. Furthermore, it utilizes a degradation-aware module to integrate brightness adjustment and noise suppression, preventing reflectance image degradation while retaining more detail, resulting in a model with improved generalization and robustness.

[0273] To better demonstrate the image enhancement effects of this application and the generalization and robustness of the model, we collected images from the LOL dataset and in real low-light environments, and conducted comparative experiments using several existing technologies and the technical solution of this application. The existing technologies used were the traditional algorithms LIME and MSRCR, as well as the deep learning algorithms KinD and RetinexNet. We also selected multiple metrics to evaluate image quality, including PSNR, FSIM, UQI, IL-NIQE, PIQE, and NoiseLevel.

[0274] Among the conventional algorithms in the prior art, the commonly used and representative algorithms for processing low-light images include the LIME algorithm and the MSRCR algorithm. Among the existing deep learning-related algorithms, the commonly used and representative algorithms include the KinD algorithm and the RetinexNet algorithm. Therefore, the above four algorithms were selected for comparison with this application to illustrate the superiority of the low-light image enhancement model of this application in processing low-light images.

[0275] Exemplarily, in this embodiment, the degradation perception module, image decomposition module and image enhancement module are trained by the low-light image enhancement model training method in the aforementioned embodiment, and then combined with the brightness adaptation module in the aforementioned embodiment to obtain a trained low-light image enhancement model. Among them, the neural network structure of the degradation perception module is the same as the degradation perception module in the aforementioned embodiment, and is obtained by training with the aforementioned formula 5; the neural network structure of the image decomposition module is the same as the image decomposition module in the aforementioned embodiment, and is obtained by training with the aforementioned formula 10; the formula involved in the brightness adaptation module is the aforementioned formula 11; the neural network structure of the image enhancement module is the same as the image enhancement module in the aforementioned embodiment, and is obtained by training with the aforementioned formula 16. The low-light image enhancement model of this embodiment and other existing technologies are used to process a low-light image from the LOL dataset and a low-light image taken in a real scene, respectively, and the results obtained are respectively Figure 20 and Figure 21 .

[0276] refer to Figure 20The low-light image to be processed is a test image in the LOL dataset, and the image has large contrast differences between different parts. From a subjective visual perspective: the results obtained by the LIME algorithm still have too low brightness in some areas; the results obtained by the MSRCR algorithm improve the brightness of the image, but the image color is distorted and there are many noise points; the image brightness obtained by the KinD algorithm can be further improved; the RetinexNet algorithm enhances the brightness of the low-light image, but there is image color deviation and the image has noise points; the low-light image enhancement method adopted in this embodiment significantly improves the brightness of the image, restores the image features, avoids image color distortion, removes noise, and achieves better enhancement effects on low-light images than the existing technology.

[0277] Table 1 shows the evaluation indicators of the test images in the LOL dataset after being enhanced by different low-light image enhancement methods. Figure 20 correspond.

[0278] Table 1:

[0279]

[0280] PSNR, FSIM, and UQI are all full-reference image quality evaluation metrics used to compare the quality of natural images (e.g., normal images) and distorted images (e.g., enhanced images obtained after algorithmic processing) with identical content. PSNR stands for Peak Signal-to-Noise Ratio; a higher PSNR value indicates less image distortion. FSIM stands for Feature Similarity, and UQI is a universal image quality indicator. Both can be used to compare the degree of similarity between an algorithm-generated enhanced image and a theoretical reference image (i.e., a normal image). A higher FSIM value indicates a higher degree of similarity. Similarly, a higher UQI value also indicates a higher degree of similarity.

[0281] According to Table 1, compared with the four existing technologies, the PSNR, FSIM, and UQI of the enhanced image obtained by this embodiment are significantly different from those of the existing technologies. The enhanced image obtained by this embodiment has the largest PSNR value, indicating that the enhanced image obtained by the low-light image enhancement method adopted by this embodiment has less distortion, is more realistic, and has better image quality. The enhanced image obtained by this embodiment has the largest FSIM value and the largest UQI value, indicating that the enhanced image obtained after processing the low-light image by this embodiment is closer to the normal image used in model training, and the enhanced image is more similar to the normal image, which can be used to demonstrate that the low-light image enhancement model of this embodiment has a superior enhancement effect.

[0282] For example, refer to Figure 21The low-light image to be processed is a low-light test image taken in an actual low-light environment, and the contrast differences between different parts of the image are large. From a subjective visual perspective: the result obtained by the LIME algorithm has an obvious enhancement effect, but there are problems with noise and color distortion; the result obtained by the MSRCR algorithm also has color deviations; the result obtained by the KinD algorithm has highlights in some areas, and there is a problem of over-enhancement; the result obtained by the RetinexNet algorithm still has low illumination in some areas, and the brightness needs to be further improved; and in the low-light image enhancement method adopted in this embodiment, combined with Figure 20 For both the LOL dataset and actual captured images, the brightness of low-light images can be significantly improved, image features can be better restored, and the problem of image color distortion can be avoided. The enhancement effect on low-light images from different sources is better than the existing technology, and it has significant advantages.

[0283] Table 2 shows the evaluation indicators of the test images after the images taken in the actual low-light environment are enhanced by different low-light image enhancement methods. Figure 21 correspond.

[0284] Table 2:

[0285]

[0286] Both IL-NIQE and PIQE are non-reference image evaluation metrics that can assess the quality of distorted images (such as enhanced images obtained after algorithmic processing). NIQE is a naturalness image quality evaluator, and IL-NIQE is derived based on NIQE. Smaller IL-NIQE values ​​indicate better image quality. PIQE is a perceptually based image quality evaluator. Smaller PIQE values ​​indicate better image quality. NoiseLevel can also indicate the quality of a distorted image; smaller NoiseLevel values ​​indicate better image quality.

[0287] According to Table 2, compared with the four existing technologies, the IL-NIQE value, PIQE value and NoiseLevel value of this embodiment are all the smallest, and there are relatively obvious differences with the existing technologies, indicating that in terms of enhancing the low-light images taken in the actual environment, the enhanced image obtained by the low-light image enhancement model of this embodiment has better quality and better enhancement effect. The low-light image enhancement method of this embodiment has significant superiority.

[0288] Second, reference Figure 10 The present invention provides a low-light image enhancement method, which uses the low-light image enhancement model trained by the low-light image enhancement model training method in the above embodiment to enhance the low-light image, and specifically includes the following steps:

[0289] S01: Inputting the low-illumination image to be processed into the image decomposition module of the trained low-illumination image enhancement model to obtain a reflectance map of the low-illumination image to be processed and an illumination map of the low-illumination image to be processed.

[0290] In step S01, the low-light image to be processed is, for example, Figure 14 As shown in the low-light image, since the image decomposition module of the low-light image enhancement model is obtained by training the degradation perception module in the aforementioned embodiment, the reflectance map of the low-light image to be processed and the illumination map of the low-light image to be processed obtained by the image decomposition module are directly maps without noise amplification, color distortion and halo artifacts. Figure 17 Therefore, when the decomposed image is enhanced in the subsequent steps, there will be no problem of amplified image noise or amplified image distortion, which is conducive to obtaining a better enhanced image.

[0291] S02: Inputting the illumination map of the low-illumination image to be processed into the brightness adaptation module of the trained low-illumination image enhancement model to obtain a multi-channel illumination map of the low-illumination image to be processed.

[0292] like Figure 10 In step S02 shown, the brightness adaptation module of the low-light image enhancement model first generates a plurality of illumination maps with different brightness adaptation levels based on the illumination map of the low-light image to be processed. In this embodiment, there are 32 illumination maps, and the reference formula is the formula 11 in step S81 of the aforementioned low-light image enhancement model training method embodiment. Then, the brightness adaptation module integrates the generated 32 illumination maps with different brightness adaptation levels into a 32-channel illumination map of the low-light image to be processed, and the reference formula is the formula 11 in step S81 of the aforementioned low-light image enhancement model training method embodiment. Figure 18 .

[0293] In this way, the brightness level information in the illumination map of the low-illumination image to be processed is extracted through 32 channels and then synthesized into a new 32-channel illumination map of the low-illumination image to be processed. This can not only retain most of the brightness level information in the original illumination map of the low-illumination image to be processed, but also reduce the computational amount of image processing while ensuring the image enhancement effect.

[0294] S03: Inputting the multi-channel low-illumination image illumination map to be processed into the image enhancement module of the trained low-illumination image enhancement model to obtain an enhanced low-illumination image illumination map.

[0295] For example, the illumination map of the multi-channel low-light image to be processed is Figure 18 32-channel illumination map of the low-light image to be processed in .

[0296] S04: Calculating the dot product of the illumination map of the enhanced low-illumination image and the reflectance map of the low-illumination image to be processed to obtain an enhanced image corresponding to the low-illumination image to be processed after enhancement.

[0297] refer to Figure 19 , Figure 19 A schematic diagram comparing the enhanced images of the prior art and this embodiment is shown. The prior art, for example, is RetinexNet. Compared to the prior art, this embodiment achieves better enhancement effects on low-light images, increasing image brightness while avoiding image distortion and preserving image features.

[0298] Thirdly, reference Figure 11 The present invention provides a low-illumination image enhancement model training device 1 for training a low-illumination image enhancement model, comprising an image input module 01, a degradation perception design module 02, a reflectivity degradation loss function design module 03 and a degradation perception training module 04.

[0299] Among them, the image input module 01 is used to input the low-light image and the normal image into the image decomposition module of the low-light image enhancement model respectively, to obtain the low-light image reflectivity map corresponding to the low-light image and the normal image reflectivity map corresponding to the normal image.

[0300] The degradation perception design module 02 is used to obtain the degradation perception module, and input the low-light image reflectance map and the normal image reflectance map obtained by the image input module 01 into the degradation perception module respectively, to obtain the corresponding low-light image reflection feature map and normal image reflection feature map respectively.

[0301] The reflectivity degradation loss function design module 03 is configured to calculate a reflectivity degradation loss function corresponding to the degradation perception module based on the low-light image reflectivity map, normal image reflectivity map, low-light image reflectivity feature map, and normal image reflectivity feature map obtained by the image input module 01 and the degradation perception design module 02. The reflectivity degradation loss function may be, for example, Formula 5 in the aforementioned embodiment of the low-light image enhancement model training method.

[0302] The degradation perception training module 04 is used to update the parameters of the degradation perception module based on the calculated reflectivity degradation loss function until the calculation result of the reflectivity degradation loss function meets the first preset condition, and the degradation perception module with the updated parameters is determined as the final degradation perception module, and the reflectivity degradation loss function corresponding to the final degradation perception module is determined as the final reflectivity degradation loss function. The final reflectivity degradation loss function is used to train the image decomposition module.

[0303] The data output end of the image input module 01 is connected to the data input end of the degradation perception design module 02, the data of the degradation perception design module 02 is connected to the data input end of the reflectivity degradation loss function design module 03, and the data output end of the reflectivity degradation loss function design module 03 is connected to the data input end of the degradation perception training module 04.

[0304] Specifically, during the training of the degradation perception module by the degradation perception training module 04, the image information of the low-light image reflectance map, the normal image reflectance map, the low-light image reflectance feature map, and the normal image reflectance feature map is substituted into the reflectance degradation loss function designed by the reflectance degradation loss function design module 03 to calculate the reflectance degradation loss function value. The reflectance degradation loss function may be the final reflectance degradation loss function of Formula 5.

[0305] Then, the parameters of the neural network in the degradation perception module are updated according to the value of the reflectivity degradation loss function. The parameters of the neural network can be, for example, the weights of each node in the neural network. Specifically, the parameters of the neural network are updated by the back propagation algorithm. The back propagation algorithm is an optimization algorithm based on gradient descent. By calculating the gradient of the loss function with respect to the parameters of the neural network model, the parameters are slightly updated along the gradient direction, thereby updating the parameters of the neural network model to minimize the loss function. In this embodiment, the Adam method is adopted to adjust the parameters of the neural network by maintaining the first-order momentum and second-order momentum of the neural network gradient and the square of the gradient until the first preset condition is met. Among them, the first preset condition is that the reflectivity degradation loss function has been minimized, which can specifically be when a preset number of training times is reached. At this time, it is considered that the reflectivity degradation loss function has been minimized. The preset number of training times can be, for example, 200 times.

[0306] After training is completed, refer to Figure 22 , and the final degradation perception module 001 is obtained. Subsequently, the final reflectivity degradation loss function of Formula 5 and the degradation perception module 001 are used to train the image decomposition module of the low-light image enhancement model.

[0307] In some possible embodiments provided by the present invention, continue to refer to Figure 11 The low-illumination image enhancement model training device 1 also includes: a decomposition input module 05, a decomposition loss function design module 06 and a decomposition training module 07.

[0308] The decomposition input module 05 is used to input the low-illumination image and the normal image into the image decomposition module respectively, and obtain the corresponding low-illumination image illumination map and normal image illumination map respectively.

[0309] Preferably, taking a normal image as an example, the image decomposition module decomposes the normal image to obtain a normal image illumination map and a normal image reflectance map at the same time, or obtains the normal image reflectance map first and then obtains the normal image illumination map.

[0310] The decomposition loss function design module 06 is used to calculate the decomposition loss function according to the final reflectivity degradation loss function, the low illumination image, the low illumination image reflectivity map, the low illumination image illumination map, the normal image, the normal image reflectivity map and the normal image illumination map.

[0311] Among them, the decomposition loss function is the decomposition loss function formula initially set. For the specific formula, refer to Formula 10 in the aforementioned low-illumination image enhancement model training method embodiment.

[0312] The decomposition training module 07 is used to update the parameters of the image decomposition module based on the calculated decomposition loss function until the calculation result of the decomposition loss function meets the second preset condition, and determine that the image decomposition module with completed parameter update is the trained image decomposition module, and the decomposition loss function corresponding to the trained image decomposition module is the final decomposition loss function.

[0313] The data output end of the decomposition input module 05 is connected to the data input end of the decomposition loss function design module 06 , and the data output end of the decomposition loss function design module 06 is connected to the data input end of the decomposition training module 07 .

[0314] Specifically, during the training of the image decomposition module by the decomposition training module 07, the final reflectivity degradation loss function, the low-light image, the low-light image reflectivity map, the low-light image illumination map, the normal image, the normal image reflectivity map, and the normal image illumination map are substituted into the decomposition loss function designed by the decomposition loss function design module 06 to calculate the decomposition loss function value. The decomposition loss function may be the final decomposition loss function of Formula 10.

[0315] Then, the parameters of the neural network in the image decomposition module are updated according to the value of the decomposition loss function. The parameters of the neural network can be, for example, the weights of each node in the neural network. Specifically, the parameters of the neural network are updated by the back propagation algorithm. The back propagation algorithm is an optimization algorithm based on gradient descent. By calculating the gradient of the loss function with respect to the parameters of the neural network model, the parameters are slightly updated along the gradient direction, thereby updating the parameters of the neural network model to minimize the loss function. In this embodiment, the Adam method is adopted to adjust the parameters of the neural network by maintaining the first-order momentum and second-order momentum of the neural network gradient and the square of the gradient until the second preset condition is met. Among them, the second preset condition is that the decomposition loss function has been minimized, which can be specifically achieved when a preset number of training times is reached. At this time, it is considered that the decomposition loss function has been minimized. The preset number of training times can be, for example, 200 times.

[0316] After training is completed, refer to Figure 22 , and obtain the final image decomposition module 002 in the low-illumination image enhancement model.

[0317] In some possible embodiments provided by the present invention, continue to refer to Figure 11 The low-illumination image enhancement model training device 1 further includes: a brightness input module 08 and a brightness splicing module 09. The data output end of the brightness input module 08 is connected to the data input end of the brightness splicing module 09.

[0318] Among them, the brightness input module 08 is used to input the low-light image illumination map into the brightness adaptation module 003 of the low-light image enhancement model (refer to Figure 22 ), obtain corresponding multiple illumination maps with different brightness adaptation levels, preferably 32. Specifically, the brightness adaptation module 003 generates 32 illumination maps with different brightness adaptation levels according to Formula 11 used in step S81 of the embodiment of the low-light image enhancement model training method.

[0319] The brightness splicing module 09 is used to splice the 32 illumination maps with different brightness adaptation levels generated by the brightness input module 08 to obtain a 32-channel illumination map.

[0320] Among them, the advantage of selecting to generate 32 illumination maps with different brightness adaptation levels and splicing them by channel to generate a 32-channel illumination map is that it can not only ensure the extraction of most of the brightness level information of the original low-light image illumination map, but also help to reduce the amount of calculation and speed up the training process.

[0321] In some possible embodiments provided by the present invention, continue to refer to Figure 11 The low-illumination image enhancement model training device 1 also includes: an enhancement input module 10, an enhancement loss function design module 11 and an enhancement training module 12.

[0322] The enhancement input module 10 is used to input the 32-channel illumination map into the image enhancement module of the low illumination image enhancement model to obtain a corresponding enhanced illumination map.

[0323] The enhancement loss function design module 11 is used to calculate the enhancement loss function based on the enhanced illumination map, the normal image, the normal image reflectance map, the normal image illumination map, and the low-illumination image reflectance map. The enhancement loss function is the initial enhancement loss function. The specific formula is referenced from Formula 15 in step S114 of the aforementioned low-illumination image enhancement model training method embodiment.

[0324] The enhancement training module 12 is used to update the parameters of the image enhancement module based on the calculated enhancement loss function until the calculation result of the enhancement loss function meets the third preset condition, and determine that the image enhancement module with completed parameter update is the trained image enhancement module, and the enhancement loss function corresponding to the trained image enhancement module is the final enhancement loss function.

[0325] The data output end of the enhanced input module 10 is connected to the data input end of the enhanced loss function design module 11 , and the data output end of the enhanced loss function design module 11 is connected to the data input end of the enhanced training module 12 .

[0326] Specifically, during the training of the image enhancement module by the enhancement training module 12, information about the enhanced illumination map, the normal image, the normal image reflectance map, the normal image illumination map, and the low-illumination image reflectance map is substituted into the enhancement loss function designed by the enhancement loss function design module 11 to calculate the enhancement loss function value. The enhancement loss function may be the final enhancement loss function of Formula 16.

[0327] Then, the parameters of the neural network in the image enhancement module are updated according to the value of the enhanced loss function. The parameters of the neural network can be, for example, the weights of each node in the neural network. Specifically, the parameters of the neural network are updated by the back propagation algorithm. The back propagation algorithm is an optimization algorithm based on gradient descent. By calculating the gradient of the loss function with respect to the parameters of the neural network model, the parameters are slightly updated along the gradient direction, thereby updating the parameters of the neural network model to minimize the loss function. In this embodiment, the Adam method is adopted to adjust the parameters of the neural network by maintaining the first-order momentum and second-order momentum of the neural network gradient and the square of the gradient until the third preset condition is met. Among them, the third preset condition is that the enhanced loss function has been minimized, which can specifically be achieved by reaching a preset number of training times. At this time, it is considered that the enhanced loss function has been minimized. The preset number of training times can be, for example, 200 times.

[0328] After training is completed, refer to Figure 22, and obtain the final image enhancement module 004 in the low-illumination image enhancement model.

[0329] After all modules have completed the corresponding steps, the image decomposition module 002 and the image enhancement module 004 of the low-light enhancement model can be considered to have been trained, and the low-light image enhancement model training device 1 can obtain a trained low-light enhancement model. At this point, the image decomposition module of the low-light enhancement model can directly decompose the reflectivity map and illuminance map without noise amplification, color distortion, and halo artifacts, and the possibility of amplified image distortion and noise is also avoided during subsequent enhancement, which is conducive to obtaining a better enhanced image.

[0330] Fourthly, reference Figure 12 The present invention provides a low-light image enhancement device 2 for enhancing low-light images using a low-light image enhancement model trained using any of the low-light image enhancement model training methods or training devices described in any of the aforementioned embodiments. The low-light image enhancement device 2 includes an image input module 21, a brightness adaptation adjustment module 22, an image enhancement integration module 23, and an image output module 24.

[0331] Among them, the data output end of the image input module 21 is connected to the data input end of the brightness adaptation adjustment module 22, the data output end of the brightness adaptation adjustment module 22 is connected to the data input end of the image enhancement integration module 23, and the data output end of the image enhancement integration module 23 is connected to the data input end of the image output module 24.

[0332] Continue to refer Figure 12 The image input module 21 is used to input the low-illumination image to be processed into the image decomposition module of the trained low-illumination image enhancement model to obtain a reflectance map of the low-illumination image to be processed and an illumination map of the low-illumination image to be processed, and the reflectance map of the low-illumination image to be processed and the illumination map of the low-illumination image to be processed obtained after decomposition are maps without noise amplification, color distortion and halo artifacts.

[0333] The brightness adaptation adjustment module 22 is used to input the illumination map of the low-light image to be processed into the brightness adaptation module of the low-light image enhancement model to obtain a multi-channel illumination map of the low-light image to be processed. The multi-channel may be, for example, 32 channels. The brightness adaptation adjustment module 22 first generates 32 illumination maps with different brightness adaptation levels from the illumination map of the low-light image to be processed, and then synthesizes them into a 32-channel illumination map of the low-light image to be processed. Using 32 channels can extract multi-level brightness level features, which also helps reduce the amount of calculation and speed up the image enhancement process.

[0334] The image enhancement integration module 23 is used to input the illumination map of the multi-channel low-illumination image to be processed into the image enhancement module of the low-illumination image enhancement model to obtain the enhanced low-illumination image illumination map.

[0335] The image output module 24 is used to calculate the dot product of the illuminance map of the enhanced low-light image and the reflectance map of the low-light image to be processed. The result of the dot product is the enhanced image corresponding to the low-light image to be processed after enhancement. Finally, the enhanced image is output to complete the enhancement of the low-light image to be processed.

[0336] Since the image decomposition module of the low-light image enhancement model trained by the degradation perception module in the aforementioned embodiment can perform denoising and color distortion reduction processing during decomposition, the enhanced image output by the image output module 24 of the low-light image enhancement device 2 is also an image without noise amplification, color distortion, and halo artifacts, and no subsequent image processing such as denoising is required. In contrast, the prior art still requires denoising after obtaining the enhanced image, and the algorithm is more complex. The algorithm in the low-light image enhancement device 2 of the present invention is simpler than that of the prior art.

[0337] Fifth, reference Figure 13 Embodiments of the present invention further provide a computer device 3, comprising a processor 31 and a memory 32. The memory 32 stores at least one instruction, which, when executed by the processor 31, implements any of the aforementioned methods for training a low-light image enhancement model and / or a method for low-light image enhancement. The memory 32 may include, for example, a system memory, a fixed non-volatile storage medium, or the like. The system memory may store, for example, an operating system, an application program, a boot loader, and other programs.

[0338] In this embodiment, the computer device 3 obtains the final reflectivity degradation loss function by training the degradation perception module, and then uses the final reflectivity degradation loss function to train the image decomposition module of the low-light image enhancement model, so that the trained image decomposition module can directly decompose the scene reflectivity map without noise amplification, color distortion and halo artifacts. The low-light image enhancement model finally trained has a better enhancement effect on low-light images.

[0339] Aspect 6. An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores at least one instruction, which, when executed, implements the training method of the low-light image enhancement model in any of the aforementioned embodiments and / or implements the low-light image enhancement method.

[0340] By adopting the above technical solution, the computer-readable storage medium obtains the final reflectivity degradation loss function by training the degradation perception module, and then uses the final reflectivity degradation loss function to train the image decomposition module of the low-light image enhancement model, so that the trained image decomposition module can directly decompose the scene reflectivity map without noise amplification, color distortion and halo artifacts. The low-light image enhancement model finally trained also has a better enhancement effect on low-light images.

[0341] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable non-transitory storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0342] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0343] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0344] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0345] Although the present invention has been illustrated and described with reference to certain preferred embodiments thereof, it should be understood by those skilled in the art that the above description is provided as a further detailed description of the present invention in conjunction with specific embodiments thereof, and that the specific implementation of the present invention is not limited to these descriptions. Those skilled in the art may make various changes in form and details, including simple deductions or substitutions, without departing from the spirit and scope of the present invention.

Claims

1. A training method for a low-light image enhancement model, characterized in that: The low-light image enhancement model includes an image decomposition module, including: Inputting the low-light image and the normal image into the image decomposition module respectively to obtain a low-light image reflectance map corresponding to the low-light image and a normal image reflectance map corresponding to the normal image; performing degradation perception processing on the low-illumination image reflectance map and the normal image reflectance map, respectively, to obtain a low-illumination image reflection feature map corresponding to the low-illumination image reflectance map and a normal image reflection feature map corresponding to the normal image reflectance map; Calculating the reflectivity degradation loss function corresponding to the degradation perception processing according to the low-illumination image reflectivity map, the normal image reflectivity map, the low-illumination image reflection feature map, and the normal image reflection feature map, including: determining a reflectivity similarity loss function according to the low-illumination image reflectivity map and the normal image reflectivity map; determining a reflection feature invariance loss function according to the low-illumination image reflection feature map and the normal image reflection feature map; determining a reflection feature smoothness loss function according to the low-illumination image reflection feature map; determining the reflectivity degradation loss function according to the reflectivity similarity loss function, the reflection feature invariance loss function, and the reflection feature smoothness loss function; The parameters of the degradation perception processing are updated according to the calculated reflectivity degradation loss function until the calculation result of the reflectivity degradation loss function meets the first preset condition, the degradation perception processing after the parameter update is determined as the final degradation perception processing, and the reflectivity degradation loss function corresponding to the final degradation perception processing is determined as the final reflectivity degradation loss function, and the final reflectivity degradation loss function is used to train the image decomposition module.

2. The training method for a low-light image enhancement model according to claim 1, wherein: The reflectivity degradation loss function is: Among them, L DA represents the reflectivity degradation loss function, R in Represents the low-light image reflectivity map, F(R in ) represents the low-light image reflection feature map, R ref Represents the normal image reflectivity map, F(R ref ) represents the normal image reflection feature map, ▽ h represents the gradient in the horizontal direction, ▽ v represents the gradient in the vertical direction, ||·||1 represents the 1-norm, ||·||2 represents the 2-norm, and C1 and C2 represent weights.

3. The training method for a low-light image enhancement model according to claim 2, wherein: The final reflectivity degradation loss function is: Among them, L DA represents the final reflectivity degradation loss function, R in Represents the low-light image reflectivity map, F(R in ) represents the low-light image reflection feature map, R ref Represents the normal image reflectivity map, F(R ref ) represents the normal image reflection feature map, ▽ h Represents the gradient in the horizontal direction, ▽ v represents the gradient in the vertical direction, ||·||1 represents the 1-norm, and ||·||2 represents the 2-norm.

4. The training method for a low-light image enhancement model according to claim 2, wherein: A five-layer convolutional neural network is used in the degradation perception processing; among them, the first four layers are 32-output channel convolution layers, and the fifth layer is a 3-output channel convolution layer.

5. The low-light image enhancement model training method according to claim 1, wherein: Also includes: Inputting the low-illumination image and the normal image into the image decomposition module respectively, and obtaining a low-illumination image illumination map corresponding to the low-illumination image and a normal image illumination map corresponding to the normal image; Calculating a decomposition loss function according to the final reflectivity degradation loss function, the low-light image, the low-light image reflectivity map, the low-light image illumination map, the normal image, the normal image reflectivity map, and the normal image illumination map; Update the parameters of the image decomposition module according to the calculated decomposition loss function until the calculation result of the decomposition loss function meets the second preset condition, determine that the image decomposition module after parameter update is the trained image decomposition module, and the decomposition loss function corresponding to the trained image decomposition module is the final decomposition loss function.

6. The training method for a low-light image enhancement model according to claim 5, wherein: The calculating of the decomposition loss function according to the final reflectivity degradation loss function, the low illumination image, the low illumination image reflectivity map, the low illumination image illumination map, the normal image, the normal image reflectivity map, and the normal image illumination map specifically includes: determining a reflectivity invariance loss function according to the low-illumination image reflectivity map and the normal image reflectivity map; Determining a decomposition and reconstruction loss function according to the normal image, the normal image reflectance map, the normal image illumination map, the low illumination image, the low illumination image reflectance map, and the low illumination image illumination map; determining an illumination smoothness loss function according to the normal image reflectance map, the normal image illumination map, the low illumination image reflectance map, and the low illumination image illumination map; The decomposition loss function is determined according to the final reflectivity degradation loss function, the reflectivity invariance loss function, the decomposition and reconstruction loss function, and the illumination smoothness loss function.

7. The training method for a low-light image enhancement model according to claim 6, wherein: The decomposition loss function is: Among them, L DE Represents the decomposition loss function, L DA represents the final reflectivity degradation loss function, S in represents the low-light image, R in Represents the low-light image reflectivity map, I in represents the illumination map of the low illumination image, S ref represents the normal image, R ref Represents the normal image reflectivity map, I ref represents the normal image illumination map, ▽ h Represents the gradient in the horizontal direction, ▽ v represents the gradient in the vertical direction, ||·||1 represents the 1-norm, and C3, C4, C5, C6, C7, C8, and C9 all represent weights.

8. The method for training a low-light image enhancement model according to claim 7, wherein: The final decomposition loss function is: Among them, L DE Represents the final decomposition loss function, L DA represents the final reflectivity degradation loss function, S in represents the low-light image, R in Represents the low-light image reflectivity map, I in represents the illumination map of the low illumination image, S ref represents the normal image, R ref Represents the normal image reflectivity map, I ref represents the normal image illumination map, ▽ h represents the gradient in the horizontal direction, ▽ v represents the gradient in the vertical direction, and ||·||1 represents the 1-norm.

9. The method for training a low-light image enhancement model according to claim 7, wherein: The image decomposition module includes a reflection branch neural network and an illumination branch neural network. The reflection branch neural network is used to obtain a reflectivity map of the low-illumination image to be processed, and the illumination branch is used to obtain an illumination map of the low-illumination image to be processed.

10. The low-light image enhancement model training method according to claim 9, wherein: The reflection branch neural network is a convolutional neural network with an upsampling structure and a downsampling structure.

11. The method for training a low-light image enhancement model according to claim 10, wherein: The reflection branch neural network has eleven layers, the first to fifth layers are downsampling layers, the sixth to tenth layers are upsampling layers, and the eleventh layer is a 3-output channel convolution layer; wherein the first downsampling layer has a 32-output channel convolution layer.

12. The method for training a low-light image enhancement model according to claim 10, wherein: The illumination branch neural network is a three-layer convolutional neural network; wherein the first two layers are 32-output channel convolution layers, and the third layer is a 3-output channel convolution layer.

13. The method for training a low-light image enhancement model according to claim 5, wherein: Also includes: The low-light image enhancement model also includes a brightness adaptation module; Inputting the low-light image illumination map into the brightness adaptation module to obtain a plurality of illumination maps with different brightness adaptation levels corresponding to the low-light image illumination map; The multiple illumination maps with different brightness adaptation levels are stitched together to obtain a multi-channel illumination map.

14. The method for training a low-light image enhancement model according to claim 13, wherein: The number of illumination maps with different brightness adaptation levels is 32, and the multi-channel illumination map is a 32-channel illumination map.

15. The method for training a low-light image enhancement model according to claim 14, wherein: The 32 illumination images with different brightness adaptation levels are obtained by the following formula: Among them, I in represents the illumination map of the low illumination image, I n ,n∈{1,2,3,...,32} represents the multiple illumination maps with different brightness adaptation levels.

16. The method for training a low-light image enhancement model according to claim 15, wherein: Also includes: The low-light image enhancement model also includes an image enhancement module; Inputting the 32-channel illumination map into the image enhancement module to obtain a corresponding enhanced illumination map; Calculating an enhancement loss function according to the enhanced illumination map, the normal image, the normal image reflectance map, the normal image illumination map, and the low illumination image reflectance map; The parameters of the image enhancement module are updated according to the calculated enhancement loss function until the calculation result of the enhancement loss function meets the third preset condition, and the image enhancement module after the parameter update is determined to be the trained image enhancement module, and the enhancement loss function corresponding to the trained image enhancement module is the final enhancement loss function.

17. The method for training a low-light image enhancement model according to claim 16, wherein: The calculating of the enhancement loss function according to the enhanced illumination map, the normal image, the normal image reflectance map, the normal image illumination map, and the low illumination image reflectance map specifically includes: determining an enhanced reconstruction loss function according to the enhanced illumination map, the normal image, the normal image reflectance map, and the low illumination image reflectance map; Determine a brightness difference loss function according to the enhanced illumination map and the normal image illumination map; Determining a gradient reconstruction loss function according to the enhanced illumination map, the normal image, and the low illumination image reflectance map; The enhancement loss function is determined according to the enhancement reconstruction loss function, the brightness difference loss function and the gradient reconstruction loss function.

18. The method for training a low-light image enhancement model according to claim 17, wherein: The enhancement loss function is: L EN =C10×||I out ×R ref -S ref ||1+||I out ×R in -S ref ||1+||I out -I ref ||1+||▽ v (I out ×R in )-▽ v S ref ||1+||▽ h (I out ×R in )-▽ h S ref ||1 Among them, L EN Represents the final enhanced loss function, I out represents the enhanced illumination map, R ref Represents the normal image reflectivity map, S ref represents the normal image, R in Represents the low-light image reflectivity map, I ref represents the normal image illumination map, ▽ h represents the gradient in the horizontal direction, ▽ v represents the gradient in the vertical direction, ||·||1 represents the 1-norm, and C10 represents the weight.

19. The method for training a low-light image enhancement model according to claim 18, wherein: The final enhancement loss function is: L EN =0.1×||I out ×R ref -S ref ||1+||I out ×R in -S ref ||1+||I out -I ref ||1+||▽ v (I out ×R in )-▽ v S ref ||1+||▽ h (I out ×R in )-▽ h S ref ||1 Among them, L EN Represents the final enhanced loss function, I out represents the enhanced illumination map, R ref Represents the normal image reflectivity map, S ref represents the normal image, R in Represents the low-light image reflectivity map, I ref represents the normal image illumination map, ▽ h Represents the gradient in the horizontal direction, ▽ v represents the gradient in the vertical direction, and ||·||1 represents the 1-norm.

20. The low-light image enhancement model training method according to claim 18, wherein: The image enhancement module includes an image enhancement neural network, and the image enhancement neural network has a convolutional neural network with an upsampling structure and a downsampling structure.

21. The method for training a low-light image enhancement model according to claim 20, wherein: The image enhancement neural network has eleven layers, the first to fifth layers are all downsampling layers, the sixth to tenth layers are all upsampling layers, and the eleventh layer is a single-output channel convolution layer; wherein the first downsampling layer has a 32-output channel convolution layer.

22. The method for training a low-light image enhancement model according to claim 17, wherein: Also includes: A trained low-illumination image enhancement model is obtained according to the trained image decomposition module, the brightness adaptation module and the trained image enhancement module.

23. A low-light image enhancement method, characterized in that: The low-illumination image is enhanced by a low-illumination image enhancement model, wherein the low-illumination image enhancement model includes an image decomposition module, and the low-illumination image enhancement method includes: Inputting the low-illumination image to be processed into the image decomposition module to obtain a reflectance map of the low-illumination image to be processed and an illumination map of the low-illumination image to be processed; The image decomposition module is obtained by training the final reflectivity degradation function, and the final reflectivity degradation function is obtained by the following method: Inputting the low-light image and the normal image into the image decomposition module respectively to obtain a low-light image reflectance map corresponding to the low-light image and a normal image reflectance map corresponding to the normal image; performing degradation perception processing on the low-illumination image reflectance map and the normal image reflectance map, respectively, to obtain a low-illumination image reflection feature map corresponding to the low-illumination image reflectance map and a normal image reflection feature map corresponding to the normal image reflectance map; Calculating the reflectivity degradation loss function corresponding to the degradation perception processing according to the low-illumination image reflectivity map, the normal image reflectivity map, the low-illumination image reflection feature map, and the normal image reflection feature map, including: determining a reflectivity similarity loss function according to the low-illumination image reflectivity map and the normal image reflectivity map; determining a reflection feature invariance loss function according to the low-illumination image reflection feature map and the normal image reflection feature map; determining a reflection feature smoothness loss function according to the low-illumination image reflection feature map; determining the reflectivity degradation loss function according to the reflectivity similarity loss function, the reflection feature invariance loss function, and the reflection feature smoothness loss function; The parameters of the degradation perception processing are updated according to the calculated reflectivity degradation loss function until the calculation result of the reflectivity degradation loss function meets the first preset condition, and the degradation perception processing after the parameter update is determined as the final degradation perception processing, and the reflectivity degradation loss function corresponding to the final degradation perception processing is determined as the final reflectivity degradation loss function.

24. The low-light image enhancement method according to claim 23, wherein: The low-light image enhancement model further includes a brightness adaptation module, and the low-light image enhancement method further includes: The illumination map of the low-illumination image to be processed is input into the brightness adaptation module to obtain a multi-channel illumination map of the low-illumination image to be processed.

25. The low-light image enhancement method according to claim 24, wherein: The low-light image enhancement model further includes an image enhancement module, and the low-light image enhancement method further includes: The multi-channel low-illumination image illumination map to be processed is input into the image enhancement module to obtain an enhanced low-illumination image illumination map.

26. The low-light image enhancement method according to claim 25, wherein: Also includes: The dot product of the illumination map of the enhanced low-illumination image and the reflectance map of the low-illumination image to be processed is calculated to obtain an enhanced image corresponding to the low-illumination image to be processed after enhancement.

27. A low-light image enhancement model training device for training a low-light image enhancement model, characterized in that: include: An image input module, configured to input a low-light image and a normal image into the image decomposition module of the low-light image enhancement model, respectively, to obtain a low-light image reflectance map corresponding to the low-light image and a normal image reflectance map corresponding to the normal image; a degradation-aware design module, configured to perform degradation-aware processing on the low-illumination image reflectance map and the normal image reflectance map, respectively, to obtain a low-illumination image reflection feature map corresponding to the low-illumination image reflectance map and a normal image reflection feature map corresponding to the normal image reflectance map; a reflectivity degradation loss function design module, configured to calculate a reflectivity degradation loss function corresponding to the degradation perception processing based on the low-illumination image reflectivity map, the normal image reflectivity map, the low-illumination image reflectivity feature map, and the normal image reflectivity feature map, including: determining a reflectivity similarity loss function based on the low-illumination image reflectivity map and the normal image reflectivity map; Determining a reflection feature invariance loss function according to the low-light image reflection feature map and the normal image reflection feature map; determining a reflection feature smoothness loss function according to the low-light image reflection feature map; determining the reflectivity degradation loss function according to the reflectivity similarity loss function, the reflection feature invariance loss function, and the reflection feature smoothness loss function; A degradation perception training module is used to update the parameters of the degradation perception processing based on the calculated reflectivity degradation loss function until the calculation result of the reflectivity degradation loss function meets a first preset condition, determine the degradation perception processing after the parameter update as the final degradation perception processing, and determine the reflectivity degradation loss function corresponding to the final degradation perception processing as the final reflectivity degradation loss function.

28. The low-light image enhancement model training device according to claim 27, wherein: Also includes: a decomposition input module, configured to input the low-illumination image and the normal image into the image decomposition module respectively, to obtain a low-illumination image illumination map corresponding to the low-illumination image, and a normal image illumination map corresponding to the normal image; a decomposition loss function design module, configured to calculate a decomposition loss function based on the final reflectivity degradation loss function, the low-light image, the low-light image reflectivity map, the low-light image illumination map, the normal image, the normal image reflectivity map, and the normal image illumination map; A decomposition training module is used to update the parameters of the image decomposition module according to the calculated decomposition loss function until the calculation result of the decomposition loss function meets a second preset condition, and determine that the image decomposition module after the parameter update is the trained image decomposition module, and the decomposition loss function corresponding to the trained image decomposition module is the final decomposition loss function.

29. The low-light image enhancement model training device according to claim 28, wherein: Also includes: a brightness input module, configured to input the low-illumination image illumination map into the brightness adaptation module of the low-illumination image enhancement model, and obtain a plurality of illumination maps with different brightness adaptation levels corresponding to the low-illumination image illumination map; The brightness splicing module is used to splice the multiple illumination images with different brightness adaptation levels to obtain a multi-channel illumination image.

30. The low-light image enhancement model training device according to claim 29, wherein: Also includes: an enhancement input module, configured to input the multi-channel illumination map into the image enhancement module of the low illumination image enhancement model to obtain a corresponding enhanced illumination map; an enhancement loss function design module, configured to calculate an enhancement loss function based on the enhanced illumination map, the normal image, the normal image reflectivity map, the normal image illumination map, and the low illumination image reflectivity map; An enhancement training module is used to update the parameters of the image enhancement module based on the calculated enhancement loss function until the calculation result of the enhancement loss function meets a third preset condition, and determine that the image enhancement module after the parameter update is the trained image enhancement module, and the enhancement loss function corresponding to the trained image enhancement module is the final enhancement loss function.

31. A low-light image enhancement device, which enhances low-light images using a low-light image enhancement model, characterized in that: include: An image input module inputs the low-illumination image to be processed into the image decomposition module of the low-illumination image enhancement model to obtain a reflectance map of the low-illumination image to be processed and an illumination map of the low-illumination image to be processed; The image decomposition module is obtained by training the final reflectivity degradation function, and the final reflectivity degradation function is obtained by the following method: Inputting the low-light image and the normal image into the image decomposition module respectively to obtain a low-light image reflectance map corresponding to the low-light image and a normal image reflectance map corresponding to the normal image; performing degradation perception processing on the low-illumination image reflectance map and the normal image reflectance map, respectively, to obtain a low-illumination image reflection feature map corresponding to the low-illumination image reflectance map and a normal image reflection feature map corresponding to the normal image reflectance map; Calculating the reflectivity degradation loss function corresponding to the degradation perception processing according to the low-illumination image reflectivity map, the normal image reflectivity map, the low-illumination image reflection feature map, and the normal image reflection feature map, including: determining a reflectivity similarity loss function according to the low-illumination image reflectivity map and the normal image reflectivity map; determining a reflection feature invariance loss function according to the low-illumination image reflection feature map and the normal image reflection feature map; determining a reflection feature smoothness loss function according to the low-illumination image reflection feature map; determining the reflectivity degradation loss function according to the reflectivity similarity loss function, the reflection feature invariance loss function, and the reflection feature smoothness loss function; The parameters of the degradation perception processing are updated according to the calculated reflectivity degradation loss function until the calculation result of the reflectivity degradation loss function meets the first preset condition, and the degradation perception processing after the parameter update is determined as the final degradation perception processing, and the reflectivity degradation loss function corresponding to the final degradation perception processing is determined as the final reflectivity degradation loss function.

32. The low-light image enhancement device according to claim 31, wherein: Also includes: The brightness adaptation adjustment module is used to input the illumination map of the low-illumination image to be processed into the brightness adaptation module of the low-illumination image enhancement model to obtain a multi-channel illumination map of the low-illumination image to be processed.

33. The low-light image enhancement device according to claim 32, wherein: Also includes: The image enhancement integration module is used to input the illumination map of the multi-channel low-illumination image to be processed into the image enhancement module of the low-illumination image enhancement model to obtain the enhanced low-illumination image illumination map.

34. The low-light image enhancement device according to claim 33, wherein: Also includes: The image output module is used to calculate the dot product of the illumination map of the enhanced low-illumination image and the reflectivity map of the low-illumination image to be processed, so as to obtain the enhanced image corresponding to the low-illumination image to be processed after enhancement.

35. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction, and when the at least one instruction is executed by the processor, implements the training method of the low-light image enhancement model as described in any one of claims 1 to 22, and / or implements the low-light image enhancement method as described in any one of claims 23 to 26.

36. A computer-readable storage medium, characterized in that The computer-readable storage medium stores at least one instruction, which, when executed, implements the training method for the low-light image enhancement model as described in any one of claims 1-22, and / or implements the low-light image enhancement method as described in any one of claims 23-26.

Images