Low-light image enhancement method and device, electronic equipment, readable storage medium and chip

Abstract

The invention provides a low-light image enhancement method and device, electronic equipment, a readable storage medium and a chip, and the method comprises the steps: obtaining an image training set which comprises at least one training sample; determining a first training sample and a second training sample in the image training set; obtaining a first low-light image enhancement network; training the first training sample according to the first low-light image enhancement network, and determining at least one first training result; determining optimization parameters corresponding to the first low-light image enhancement network based on an ablation experiment; optimizing the first low-light image enhancement network according to the optimization parameter, and determining a second low-light image enhancement network; and training the second training sample according to the second low-light image enhancement network, and determining at least one second training result. Through the scheme of the invention, the dependence on data in low-light image enhancement is reduced, and the efficiency and accuracy of low-light image enhancement are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular, to a low-light image enhancement method, apparatus, electronic device, readable storage medium, and chip. Background Art

[0002] At present, the popularization of smart phones has led to a sharp increase in the number of images. Due to limitations in conditions such as noise, light, and hardware, the quality of captured images has declined, the content of low-light images is difficult to identify, and a large amount of detailed information is missing. Traditional image enhancement algorithms usually perform denoising and enhancement processing on low-light images, and optimize low-light image enhancement algorithms through deep learning. Existing deep learning-based low-light image enhancement networks are usually driven by paired data, such as the low-light image enhancement network RetinexNet, or unpaired data, such as the unsupervised generative adversarial network EnlightenGan. The scenes of paired low-light image datasets are often relatively single, and the colors are relatively monotonous; unpaired data is easier to obtain, but the constraints on the network output are relatively weak, and additional restrictions often need to be introduced to ensure that there is no information loss in the output results. The methods for enhancing low-light images are data-dependent, with high hardware requirements and image limitations. Summary of the Invention

[0003] In view of this, the present invention aims to solve the problems that the methods for enhancing low-light images are data-dependent, with high hardware requirements and image limitations.

[0004] Specifically, the present invention is implemented through the following technical solutions:

[0005] The first aspect of the present invention provides a low-light image enhancement method.

[0006] The second aspect of the present invention provides a low-light image enhancement apparatus.

[0007] The third aspect of the present invention provides an electronic device.

[0008] The fourth aspect of the present invention provides a readable storage medium.

[0009] The fifth aspect of the present invention provides a chip.

[0010] The low-light image enhancement method provided by the present invention includes: obtaining an image training set, the image training set including at least one training sample, and the training sample corresponding to an exposure sequence in an image intensifier; determining a first training sample and a second training sample in the image training set; obtaining a first low-light image enhancement network; training the first training sample according to the first low-light image enhancement network to determine at least one first training result; determining optimization parameters corresponding to the first low-light image enhancement network based on an ablation experiment; optimizing the first low-light image enhancement network according to the optimization parameters to determine a second low-light image enhancement network; and training the second training sample according to the second low-light image enhancement network to determine at least one second training result.

[0011] In some technical solutions, optionally, the first low-light image enhancement network is an unsupervised low-light image enhancement network based on an attention mechanism.

[0012] In some technical solutions, optionally, determining the first training sample and the second training sample in the image training set includes: obtaining exposure sequences in at least one image intensifier; randomly allocating the exposure sequences to determine the first training sample and the second training sample, and the number of the first training samples being greater than the number of the second training samples.

[0013] In some technical solutions, optionally, training the first training sample according to the first low-light image enhancement network to determine at least one first training result includes: determining a mapping curve corresponding to at least one first training sample; performing at least one low-light enhancement process on the first training sample according to the mapping curve to determine an illumination enhancement curve corresponding to a high-order curve; obtaining convolution layer setting parameters; setting the number of convolution layers of the first low-light image enhancement network according to the convolution layer setting parameters; performing a reference-free loss analysis on the illumination enhancement curve according to the first low-light image enhancement network after the convolution layer setting is completed to determine a total loss function; and determining the first training result according to the total loss function.

[0014] In some technical solutions, optionally, the reference-free loss analysis includes: spatial consistency loss, exposure control loss, color constancy loss, illumination smoothness loss, and structural similarity loss; the spatial consistency loss, exposure control loss, color constancy loss, illumination smoothness loss, and structural similarity loss respectively correspond to a spatial consistency loss weight, an exposure control loss weight, a color constancy loss weight, an illumination smoothness loss weight, and a structural similarity loss weight.

[0015] In some technical solutions, optionally, the optimization parameters corresponding to the first low-light image enhancement network are determined based on ablation experiments, including: obtaining a public dataset; comparing the public dataset with the first training result by means of ablation experiments to determine a reference evaluation result; determining loss function optimization parameters and convolutional layer optimization parameters according to the reference evaluation result; and determining the optimization parameters corresponding to the first low-light image enhancement network according to the loss function optimization parameters and the convolutional layer optimization parameters.

[0016] The second aspect of the present invention provides a low-light image enhancement device, including: an acquisition module for acquiring an image training set, where the image training set includes at least one training sample, and the training sample corresponds to an exposure sequence in an image intensifier; acquiring a first low-light image enhancement network; a sample determination module for determining a first training sample and a second training sample in the image training set; a training module for training the first training sample according to the first low-light image enhancement network to determine at least one first training result; an evaluation module for determining the optimization parameters corresponding to the first low-light image enhancement network based on ablation experiments; an optimization module for optimizing the first low-light image enhancement network according to the optimization parameters to determine a second low-light image enhancement network; and a result determination module for training the second training sample according to the second low-light image enhancement network to determine at least one second training result.

[0017] An embodiment of the third aspect of the present invention provides an electronic device, including a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, the steps in the first aspect are implemented.

[0018] An embodiment of the fourth aspect of the present invention provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by the processor, the steps in the first aspect are implemented.

[0019] An embodiment of the fifth aspect of the present invention provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the steps in the first aspect.

[0020] The technical solutions provided by the present invention at least bring the following beneficial effects:

[0021] The present invention provides a low-light image enhancement method, apparatus, electronic device, readable storage medium, and chip. By means of a low-light image enhancement method based on an attention mechanism, the performance of low-light images in image vision tasks is improved. The attention mechanism is used to improve a zero-reference deep learning image enhancement network, enhancing indicators such as the Mean Square Error (MSE), Root Mean Square Error (RMSE), Peak Signal to Noise Ratio (PSNR), and Structural Similarity Index (SSIM) of the image, improving the quality of the image, and reducing the dependence on data. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings herein are incorporated into and constitute a part of this specification, showing embodiments consistent with the present invention, and are used in conjunction with the specification to explain the principles of the present invention.

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or in the prior art, the following briefly introduces the accompanying drawings required for use in the description of the embodiments or related technologies. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0024] Figure 1 It is a schematic flowchart of the low-light image enhancement method provided by an embodiment of the present invention;

[0025] Figure 2 It is a partial schematic flowchart of the low-light image enhancement method provided by an embodiment of the present invention;

[0026] Figure 3 It is a partial schematic flowchart of the low-light image enhancement method provided by an embodiment of the present invention;

[0027] Figure 4 It is a partial schematic flowchart of the low-light image enhancement method provided by an embodiment of the present invention;

[0028] Figure 5 It is a schematic block diagram of the structure of the low-light image enhancement apparatus provided by an embodiment of the present invention;

[0029] Figure 6 It is a schematic block diagram of the structure of the electronic device provided by an embodiment of the present invention;

[0030] Figure 7 It is a schematic overall framework diagram of the image enhancement network provided by an embodiment of the present invention;

[0031] Figure 8Schematic diagram of the network structure provided by the embodiments of the present invention;

[0032] Figure 9 Schematic diagram of the structure of the attention module provided by the embodiments of the present invention;

[0033] Figure 10 Schematic diagram of the structure of channel attention in the attention module provided by the embodiments of the present invention;

[0034] Figure 11 Schematic diagram of the structure of spatial attention in the attention module provided by the embodiments of the present invention;

[0035] Figure 12 Schematic diagram of the optimization effect of the loss function in the ablation experiment provided by the embodiments of the present invention;

[0036] Figure 13 Schematic diagram of the optimization effect of the convolutional layer in the ablation experiment provided by the embodiments of the present invention;

[0037] Figure 14 Comparison chart of the visualization results of the low-light image enhancement method provided by the embodiments of the present invention and other methods.

[0038] Among them, Figure 5 and Figure 6 The corresponding relationship between the part names and labels is as follows:

[0039] 900: Low-light image enhancement device; 902: Acquisition module; 904: Sample determination module; 906: Training module; 908: Evaluation module; 910: Optimization module; 912: Result determination module; 1000: Electronic device; 1109: Memory; 1110: Processor. Specific implementation manners

[0040] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0041] See Figure 1 , the first aspect of the present invention provides a low-light image enhancement method, including the following steps:

[0042] Step S100: Obtain an image training set;

[0043] Step S102: Determine the first training sample and the second training sample in the image training set;

[0044] Step S104: Obtain the first low-light image enhancement network;

[0045] Step S106: Train the first training sample according to the first low-light image enhancement network to determine at least one first training result;

[0046] Step S108: Determine the optimization parameters corresponding to the first low-light image enhancement network based on ablation experiments;

[0047] Step S110: Optimize the first low-light image enhancement network according to the optimization parameters to determine the second low-light image enhancement network;

[0048] Step S112: Train the second training sample according to the second low-light image enhancement network to determine at least one second training result.

[0049] According to the low-light image enhancement method provided by the present invention, the image training set is subjected to low-light enhancement through the first low-light image enhancement network, and the first low-light image enhancement network is optimized according to ablation experiments and subjective and quantitative evaluations to continuously update the optimization parameters. Specifically, different from the existing deep learning algorithms based on paired data and unpaired data, the algorithm of this application only requires a certain number of degraded images for training, and the number of degraded images is set by the operator. Among them, the degraded images are obtained from the under-exposed single-image contrast enhancement (SICE) data set as the image training set. The image samples in the image training set are divided into a first training sample and a second training sample. The first training sample is used to train the network model, and the second training sample is used to verify the training result of the network model. Each image in the image training set corresponds to at least one exposure sequence, including an under-exposed single image and a low-light enhanced image corresponding to the single image. The first low-light image enhancement network used in this application is an image enhancement network based on Zero-Reference Deep Curve Estimation (Zero-DCE) and introducing an attention mechanism. The under-exposed low-light images in the first training sample are trained through the first low-light image enhancement network. The convolutional neural network is used to learn the mapping curve from the low-light image to the high-light image, and then the learned mapping curve is used to perform at least one iterative adjustment at the pixel level on the original image, so that the mapping curve of the original image is pixel-level enhanced within the dynamic range to obtain the corresponding first training result. After the first low-light image enhancement network finishes processing the first training sample, the ablation experiment is used to verify the rationality of the loss function design in the first low-light image enhancement network and the influence of the number of convolutional layers on the performance of the image enhancement network; and through subjective and quantitative evaluations, the enhancement results of the algorithm corresponding to the first low-light image enhancement network in this application and at least one classical algorithm are compared, and the enhanced images on the public data set are evaluated through no-reference and reference-based evaluation indicators to obtain the optimization parameters corresponding to the first low-light image. The optimization parameters include loss function weight optimization parameters, convolutional layer optimization parameters, and evaluation optimization parameters. After adjusting the corresponding parameters in the first low-light image enhancement network through at least one optimization parameter, the optimized low-light image enhancement network is determined as the second low-light image enhancement network, and the second training sample is input into the optimized low-light image enhancement network to determine the optimized low-light image enhancement result as the second training result. After obtaining the second training result, the optimization parameters can still be updated through ablation experiments or subjective and quantitative evaluations to determine the network model parameters with the best low-light image enhancement effect.

[0050] It is understandable that the performance of low-light images in image vision tasks is improved by a method of low-light image enhancement based on the attention mechanism. The main impacts of low-light conditions on images are changing the lighting angle, generating Gaussian noise and blurring due to changing the lighting intensity. In view of the above situations, this chapter proposes a zero-reference deep learning image enhancement network and uses the attention mechanism to improve it, further excavating image features and improving indicators such as the Mean Square Error (MSE), Root Mean Square Error (RMSE), Peak Signal to Noise Ratio (PSNR), and Structural Similarity Index (SSIM) of the image, thus improving the quality of the image. The proposed method does not rely on paired or unpaired data, and uses a reference-free loss function to guide the neural network structure with the attention mechanism to learn, so as to obtain the parameters for estimating the brightness curve, which reduces the workload from the perspective of training the model and has strong practicability.

[0051] Optionally, the low-light image enhancement method of the present invention can be used as a low-light image preprocessing step to provide a good data input for downstream image matching, image calibration, image tracking, etc.

[0052] In some embodiments, optionally, the first low-light image enhancement network is an unsupervised low-light image enhancement network based on the attention mechanism.

[0053] In this embodiment, the unsupervised low-light image enhancement network based on the attention mechanism is used as the first low-light image enhancement network, that is, using the zero-reference depth curve, the light enhancement is converted into an image-specific curve estimation problem, with the image as the input and the curve as the output, which is realized through a reference-free loss function, so as to obtain the low-light image enhanced image, and a lightweight network (DCE-NET) is trained to predict a pixel-level high-order curve, and the low-light image enhanced image is adjusted through this curve. In this application, the attention mechanism is introduced into the zero-reference depth curve model, and the skip-layer network structure in the original DCE-NET network structure is replaced with a network with an attention module added to form an unsupervised low-light image enhancement network based on the attention mechanism. The structure of the attention module is as Figure 9 shown, including channel attention and spatial attention. The input feature map outputs the feature map through the channel attention module and the spatial attention module to complete the enhancement. Among them, the structure of the channel attention module is as Figure 10 shown. After the input features, max-pooling and average-pooling are performed respectively, and the data is transmitted to the shared fully connected layer for convolution to obtain the channel attention weights; the structure of the spatial attention module is asFigure 11 As shown, the feature map processed by the channel attention module is passed through max pooling and average pooling convolutions again to determine the spatial attention weights. The replaced network structure is as Figure 8 shown, including a linear convolution layer (Conv+ReLU), an attention module (CBAMmoudle), and a hyperbolic convolution layer (Conv+Tanh). The linear convolution layer includes four convolution layers composed of 32 3×3 convolutional kernels; the attention module includes three convolution layers composed of 32 3×3 convolutional kernels; the hyperbolic convolution layer includes one convolution layer composed of 32 3×3 convolutional kernels.

[0054] It can be understood that the original zero-reference depth curve network model uses seven convolutional layers to form a simple CNN network. The first 6 convolutional layers are composed of Conv and Relu functions, and the last convolutional layer is composed of Conv and Tanh functions. The entire network uses a simple skip-layer network connection. The outputs of the 1st / 2nd / 3rd layers in the convolutional layer are concatenated with the 4th / 5th / 6th layers at the channel level. The number of channels in each convolution is 32, and the convolutional kernel size is 3×3. Finally, the values of the RGB three channels are iterated 8 times. The skip-layer structure in DCE-Net can effectively integrate multi-scale feature information, but this architecture is likely to introduce redundant feature information in the final result. By introducing an attention mechanism, redundant image information is screened, improving the processing efficiency and image enhancement quality during low-light image enhancement.

[0055] In some embodiments, optionally, as Figure 2 shown, determining the first training sample and the second training sample in the image training set includes:

[0056] Step S1022: Obtain the exposure sequence in at least one image intensifier;

[0057] Step S1024: Randomly allocate the exposure sequence to determine the first training sample and the second training sample.

[0058] In this embodiment, a large-scale multi-exposure image dataset is obtained. The multi-exposure dataset includes a multi-scene high-resolution low-light image sequence. The multi-exposure image dataset includes three single-image contrast enhancement (SICE) image enhancers based on Convolutional Neural Networks (CNNs). The first two SICE enhancers are an illumination enhancement network and a detail enhancement network respectively. The illumination enhancement network is used to enhance the image contrast, and the detail enhancement network is used to enhance the image details. The last SICE enhancer serves as an adjustment network to balance details and textures and avoid color distortion. The exposure sequence is determined based on the original image corresponding to the image enhancer and the image after illumination enhancement. At least one obtained exposure sequence is randomly assigned into two parts. The exposure sequences in the first part are used as the first training samples for the training of the first low-light image enhancement network, and the exposure sequences in the second part are used as the second training samples for the image samples verified after the optimization of the first low-light enhancement network. Among them, the image size is uniformly adjusted to 512×512×3 by the operator to accelerate the training speed and improve the algorithm operation efficiency.

[0059] In some embodiments, optionally, as Figure 3 shown, training the first low-light image enhancement network with the first training samples to determine at least one first training result includes:

[0060] Step S1062: Determine the mapping curve corresponding to at least one first training sample;

[0061] Step S1064: Perform at least one low-light enhancement process on the first training samples according to the mapping curve to determine the illumination enhancement curve corresponding to the high-order curve;

[0062] Step S1066: Obtain the convolutional layer setting parameters;

[0063] Step S1068: Set the number of convolutional layers of the first low-light image enhancement network according to the convolutional layer setting parameters;

[0064] Step S1070: Perform a reference-free loss analysis on the illumination enhancement curve according to the first low-light image enhancement network after the convolutional layer setting is completed to determine the total loss function;

[0065] Step S1072: Determine the first training result according to the total loss function.

[0066] In this embodiment, training the first training sample through an unsupervised low-light image enhancement network based on the attention mechanism to determine the training result specifically includes: First, the convolutional neural network in the network model is used to learn the mapping curve from the low-light image to the high-light image. The mapping curve parameter map learned by the network is related to the coordinates of the pixel points. The mapping curve parameter expression is as follows:

[0067] LE n (x) = LE n-1 (x) + A n (x)LE n-1 (x)(1 - LE n-1 )(x));

[0068] where n represents the number of iterations, and n is set to 8. LE n (x) is the enhanced version of the previous iteration curve, and A is the curve parameter map with the same size as the given image.

[0069] After determining the illumination enhancement curve corresponding to the high-order curve, obtain the convolutional layer number setting parameter of the first low-light image enhancement network. The setting parameter is x CBAM + n, where x CBAM + n means that there are x CBAM attention modules and n convolutional layers in the set network structure. After setting the number of attention modules and the number of convolutional layers in the network model, determine the values of the RGB three channels iteratively 8 times through the first low-light image enhancement network. In order to achieve reference-free learning in the first low-light image enhancement network, a set of reference-free losses is proposed to evaluate the quality of the enhanced image, guide the network to learn the mapping relationship between the low-light image and the curve parameter map, and determine the total loss function of the first low-light image enhancement network. The formula is as follows:

[0070] L totol = L spa + w exp L exp + w col L col + w tv L tv + w ssim L ssim ;

[0071] where L total is the total loss function, L spa is the spatial consistency loss, L exp is the exposure control loss, L col is the color constancy loss, L tv is the illumination smoothness loss, L ssim is the structural similarity loss; W exp is the exposure control loss weight, W col is the color constancy loss weight, W tvis the weight value of the illumination smoothness loss, W ssim is the weight value of the structural similarity loss.

[0072] The first low-light image enhancement network determines a first training result corresponding to the first training sample through the total loss function.

[0073] In some embodiments, optionally, the reference loss analysis includes: spatial consistency loss, exposure control loss, color constancy loss, illumination smoothness loss, and structural similarity loss;

[0074] The spatial consistency loss, exposure control loss, color constancy loss, illumination smoothness loss, and structural similarity loss respectively correspond to a spatial consistency loss weight value, an exposure control loss weight value, a color constancy loss weight value, an illumination smoothness loss weight value, and a structural similarity loss weight value.

[0075] In this embodiment, the reference-free loss function includes five losses: spatial consistency loss, exposure control loss, color constancy loss, illumination smoothness loss, and structural similarity loss.

[0076] Among them, the spatial consistency loss: The spatial consistency loss is mainly used to maintain the difference between adjacent regions between the input image and the enhanced image, and encourage the spatial consistency of the enhanced image. The formula is as follows:

[0077]

[0078] where K is the number of the local region, Ω(i) represents the set of adjacent regions (up, down, left, right) centered on i, γ and I respectively represent the average pixel values of the local regions in the enhanced image and the input image, and the local size is set to 4×4 according to experience.

[0079] Exposure control loss: To suppress underexposed / overexposed regions, an exposure control loss L exp is designed to control the exposure level. The exposure control loss evaluates whether the exposure of each region of the image is good by calculating the difference between the average intensity value of the local region and the good exposure intensity E. The exposure control loss is only calculated in the luminance space, and the good exposure intensity E is set to 0.6. However, experiments show that when the good exposure intensity E is set between [0.4, 0.7], it has a certain enhancement effect on low-light images. The exposure control loss formula is as follows:

[0080]

[0081] where M is the number of non-overlapping local regions of size 16×16, the average pixel value of the enhanced local region is denoted as Y, and E represents the good exposure level, which is set to 0.6 according to experience [61, 62].

[0082] Color constancy loss: Based on the gray world color constancy assumption, that is, the color of each sensor channel is gray on average across the entire image, a color constancy loss is established to correct potential color biases in the enhanced image and establish the relationship between enhanced images:

[0083]

[0084] where J P represents the average intensity of image channel p, and J Q represents the average intensity of image channel Q, and P, Q represent a pair of channels.

[0085] Illumination smoothness loss: To maintain the monotonic relationship between adjacent pixels, this paper adds an illumination smoothness loss to each curve parameter map a to avoid overexposure and underexposure, which can be expressed as:

[0086]

[0087] where N is the number of iterations, and x and y represent horizontal and vertical gradient operations respectively;

[0088] Structural similarity loss: The SSIM image evaluation index combines brightness, contrast, and structure to measure the error between a distorted image and the original image. The more similar the two images are, the closer the SSIM value is to 1. In this paper, during training, the structural similarity loss is used to correct potential structural biases in the enhanced image, which can be expressed as:

[0089] L ssim = 1 - SSIM(Y, I);

[0090] where Y represents the enhanced image, I represents the original image, and SSIM(Y, I) represents the similarity between the enhanced image and the original image.

[0091] The spatial consistency loss, exposure control loss, color constancy loss, illumination smoothness loss, and structural similarity loss in the reference - free loss function respectively correspond to spatial consistency loss weights, exposure control loss weights, color constancy loss weights, illumination smoothness loss weights, and structural similarity loss weights. For example, when the color constancy loss L col is discarded, serious color projection will occur. When the spatial consistency loss L spa is discarded, the contrast of the image is severely reduced. When the exposure control loss L exp is discarded, the low - light areas of the image cannot be restored. When the structural consistency loss L ssim is discarded, the details of the image will become blurred, and at the same time, the contrast and brightness will also decrease to a certain extent. When the illumination smoothness loss L tvWhen this happens, the correlation between regions will decrease, and obvious artifacts will appear. Therefore, the operator can adjust the weights of each loss function to achieve the corresponding optimal low-light image enhancement effect. For example, the weights of the exposure control loss, color constancy loss, illumination smoothness loss, and structural similarity loss are set to 30, 1600, 5, and 5 respectively, and the first training sample is trained with these initial loss weights in the first low-light image enhancement network.

[0092] It can be understood that by adjusting the weights of each loss function in the reference-free loss function, the image enhancement quality of the first low-light image enhancement network in this application can be improved.

[0093] In some embodiments, optionally, as Figure 4 shown, the optimization parameters corresponding to the first low-light image enhancement network are determined based on ablation experiments, including:

[0094] Step S1082: Obtain a public dataset;

[0095] Step S1084: Compare the public dataset with the first training result by means of ablation experiments to determine the reference evaluation result;

[0096] Step S1086: Determine the loss function optimization parameters and convolutional layer optimization parameters according to the reference evaluation result;

[0097] Step S1088: Determine the optimization parameters corresponding to the first low-light image enhancement network according to the loss function optimization parameters and convolutional layer optimization parameters.

[0098] In this embodiment, to verify the performance of the first low-light image enhancement network, it is experimentally compared with existing low-light image enhancement methods and restoration algorithms through ablation experiments and objective and quantitative evaluation methods. The experiments are respectively carried out on public datasets. First, through ablation experiments, the rationality of the loss function weights and the influence of the number of convolutional layers on the performance in the first low-light image enhancement network are analyzed to determine the reference evaluation results corresponding to the first low-light image enhancement network and to determine the optimization parameters. The optimization parameters are used to adjust the loss function weights and the number of convolutional layers in the first low-light image enhancement network. Then, the results are analyzed through no-reference and reference-based objective image quality evaluation metrics on the public dataset to determine the low-light image enhancement quality and conduct quantitative evaluation. Among them, by adjusting the number of attention modules and convolutional layers, the performance and efficiency of the image enhancement network can be maximized. For example, only two attention modules are needed to produce satisfactory results for the image, which proves the effectiveness of adding attention modules. When the number of attention modules is set to 3 and the number of convolutional layers is also set to 3, the most natural exposure and contrast effects can be produced. With the further increase of the number of attention modules, problems such as overexposure and image distortion may occur.

[0099] To verify the effectiveness of this algorithm, this paper compares the enhancement results of this algorithm with those of multiple classical algorithms and evaluates the enhanced images on the public dataset through no-reference and reference-based evaluation metrics; the low-light image dataset for no-reference evaluation is collected from existing low-light image enhancement papers, including MEF, LIME, NPE, VV, DICM; for the dataset used for reference-based quality evaluation, the dataset contains 500 low-light / normal-light image pairs. The size of all experimental images is adjusted to 1200×900×3.

[0100] It can be understood that by determining the optimization parameters through ablation experiments to adjust the parameters in the first low-light image enhancement network and re-evaluating the experimental results through objective and quantitative evaluation methods, the image quality and objective effect of low-light image enhancement of the unsupervised low-light image enhancement network based on the attention mechanism can be improved.

[0101] In a specific embodiment, the overall framework of an unsupervised low-light image enhancement network based on the attention mechanism is as Figure 7 shown, including a brightness enhancement curve graph (Curve Parameter Map), an unsupervised low-light image enhancement (Zero-Reference Deep Curve Estimation, Zero-DCE) Attention network based on the attention mechanism, and three loss functions (LE1, LE2, and LE3). The following will be introduced in three steps.

[0102] Step 1: The unsupervised low-light image enhancement network based on the attention mechanism (Zero-DCE Attention) is mainly divided into a feature extraction network and a brightness enhancement network. First, a convolutional neural network is used to learn the mapping curve from a low-light image to a high-light image, and then the learned mapping curve is used to perform multiple iterative adjustments on the original image at the pixel level to achieve the purpose of pixel-level image enhancement within the dynamic range. The enhancement curve parameter map learned by the network is related to the coordinates of the pixel points, and the designed enhancement curve parameter expression is as follows:

[0103] LE n (x) = LE n-1 (x) + A n (x)LE n-1 (x)(1 - LE n-1 (x));

[0104] Among them, n represents the number of iterations. Setting n to 8 can obtain a relatively optimal image enhancement effect; LE n (x) is the enhanced version of the previous iteration curve, and A is the curve parameter map with the same size as the given image.

[0105] The characteristics of the image enhancement curve are as follows: the pixel value of each enhanced image should be within the normalized range of [0, 1] to avoid information loss caused by overflow truncation; the curve should be monotonic to maintain the difference between adjacent pixels; in order to realize the backpropagation of gradients during the training process of the deep network model, the form of the curve should be as simple and differentiable as possible. The algorithm in this paper estimates the curve parameters for the R, G, and B color channels respectively.

[0106] Step 2: The input of Zero-DCE Attention is a low-light image, and the output is a per-pixel curve parameter map corresponding to a high-order curve. The original DCE-Net uses a simple CNN network composed of seven convolutional layers. The first 6 convolutional layers are composed of Conv and Relu functions, and the last convolutional layer is composed of Conv and Tanh functions. The entire network uses a simple skip-layer network connection. The outputs of the 1st / 2nd / 3rd layers in the convolutional layer are concatenated with the 4th / 5th / 6th layers at the channel level. The number of channels for each convolutional layer is 32, and the kernel size is 3×3. Finally, the values of the RGB three channels are iterated 8 times. The skip-layer structure in DCE-Net can effectively integrate multi-scale feature information, which is very important for forming the illumination enhancement curve. However, this architecture is likely to introduce redundant feature information in the final result.

[0107] To solve these problems, the proposed solution in this application introduces the attention mechanism, replaces the original skip-layer network structure with a network with an attention module added. Inspired by SE-net and CBAM-net, it is designed as Figure 8Replace the previous DCE-Net network structure with the model shown below. The network structure parameters are shown in Table 1:

[0108] Type Convolution Kernel Size Output Input - 3×256×256 Conv+Relu 3×3 32×256×256 Conv+Relu 3×3 32×256×256 CMAB module 1×256×256 CMAB module 1×256×256 CMAB module 1×256×256 Conv+Relu 3×3 32×256×256 Conv+Relu 3×3 32×256×256 Conv+Tanh 3×3 3×256×256

[0109] Table 1

[0110] The composition of the attention module is as shown in Figure 9 The attention module consists of channel attention and spatial attention, which are shown by Figure 10 and Figure 11 respectively.

[0111] Step 3: To achieve reference-free learning in Zero-DCE Attention, this paper proposes a set of differentiable reference-free losses to evaluate the quality of enhanced images and guide the network to learn the mapping relationship between low-light images and curve parameter maps. The reference-free loss function used to supervise the training of the DCE-Net Attention network in Zero-DCE Attention includes five losses: spatial consistency loss, exposure control loss, color constancy loss, illumination smoothness loss, and structural similarity loss.

[0112] Among them, the spatial consistency loss: The spatial consistency loss is mainly used to maintain the difference between adjacent regions of the input image and the enhanced image, and encourage the spatial consistency of the enhanced image. The formula is as follows:

[0113]

[0114] where K is the number of local regions, Ω(i) represents the set of adjacent regions (up, down, left, right) centered on i, and and represent the average pixel values of the local regions in the enhanced image and the input image respectively. The local size is set to 4×4 according to experience.

[0115] Exposure control loss: To suppress underexposed / overexposed regions, an exposure control loss Lexp is designed to control the exposure level. The exposure control loss evaluates whether the exposure of each region of the image is good by calculating the difference between the average intensity value of the local region and the good exposure intensity E. The exposure control loss is calculated only in the luminance space, and the good exposure intensity E is set to 0.6. However, experiments show that when the good exposure intensity E is set between [0.4, 0.7], it has a certain enhancement effect on low-light images.

[0116] The formula for the exposure control loss is as follows:

[0117]

[0118] Where M is the number of non-overlapping local regions of size 16×16, the average pixel value of the enhanced local region is denoted as Y, E represents a good exposure level, and is set to 0.6 according to experience [61,62].

[0119] Color constancy loss: Based on the gray world color constancy hypothesis, that is, the color of each sensor channel is gray on average across the entire image, a color constancy loss is established to correct potential color deviations in the enhanced image, and the relationship between the enhanced images is established as follows:

[0120]

[0121] Among them, J P represents the average intensity of image channel p, and J Q represents the average intensity of image channel Q, and P, Q represent a pair of channels.

[0122] Illumination smoothing loss: To maintain the monotonic relationship between adjacent pixels, this paper adds an illumination smoothness loss to each curve parameter map a to avoid overexposure and underexposure, which can be expressed as:

[0123]

[0124] Where N is the number of iterations, and x and y represent horizontal and vertical gradient operations respectively;

[0125] Structural similarity loss: The SSIM image evaluation index combines brightness, contrast, and structure to measure the error between a distorted image and the original image. The more similar the two images are, the closer the SSIM value is to 1. In this paper, during training, the structural similarity loss is used to correct potential structural deviations in the enhanced image, which can be expressed as:

[0126] L ssim = 1 - SSIM(Y, I);

[0127] Among them, Y represents the enhanced image, I represents the original image, and SSIM(Y, I) represents the similarity between the enhanced image and the original image.

[0128] The total loss function Ltotal of the low-light image enhancement network based on the attention mechanism is:

[0129] L totol = L spa + w exp L exp + w col L col + w tv L tv + w ssim L ssim ;

[0130] Among them, Wexp , W col , W tv and W ssim are the weights of the corresponding loss functions respectively.

[0131] The unsupervised low-light image enhancement network based on the attention mechanism is implemented based on the deep learning framework Pytorch. The CPU of the hardware device is AMD Ryzen 7 5800H CPU@3.20GHz, the memory is 16GB, the graphics card is Nvidia RTX 3070, and the operating system is Windows 11. The dataset and experimental parameters are introduced as follows:

[0132] Dataset setting: Different from the existing deep learning algorithms based on paired data and unpaired data, the algorithm in this paper only requires a certain number of degraded images for training. The image dataset used in the experimental training selects 360 multiple-exposure sequences from the first part of the SICE dataset. In this paper, 3022 images with different exposure levels in the Part1 subset are randomly divided into two parts (2422 for training and the remaining 600 for validation). The image size is adjusted to 512×512×3 to accelerate training.

[0133] Training environment and weight setting: The programming software PyCharm uses the Community 2021.3 version. The batch size is 8. In the initial training stage of the network, Adam is used as the model optimizer, the initial learning rate is 0.0001, and the weights W exp , W col , W tv and W ssim are set to 30, 1600, 5, and 5 respectively. The network is trained for 100 epochs in total.

[0134] Comparison algorithms and experimental settings: To verify the performance of Zero-DCE Attention, it is experimentally compared with a variety of existing low-quality image enhancement and restoration algorithms. The experiments are carried out on the public dataset and the asteroid surface dataset under complex lighting conditions in the previous chapter respectively. The experiments mainly include ablation experiments, subjective and quantitative evaluations. First, ablation experiments are carried out to verify the rationality of the design of the Zero-DCE Attention loss function and explore the influence of parameters on the performance. Then, the results are analyzed through the no-reference and reference-based objective image quality evaluation metrics running on the public dataset. The reference-based image quality evaluation selects the LOL dataset, and the quality is evaluated through the PSNR, SSIM, and MAE metrics. The no-reference image quality evaluation selects NIQE for quantitative evaluation. The ablation experiments are mainly used to verify the rationality of the algorithm design and implementation. The ablation experiments carried out in this paper include the influence of the loss function and the influence of parameter setting.

[0135] The influence of the loss function is as follows Figure 12 shown: It shows examples of the enhanced results of Zero-DCE Attention obtained by training the loss function under a set of different combined conditions. When the color constancy loss (w / o)L col is discarded, serious color projection will occur. When the spatial consistency loss L spa is discarded, the contrast of the image is severely reduced. When the exposure control loss L exp is discarded, the low-light areas of the image cannot be restored. When the structural consistency loss L ssim is discarded, the details of the image will become blurred, and at the same time, the contrast and brightness will also decrease to a certain extent. When the illumination smoothness loss L tv is discarded, it will cause a decrease in the correlation between regions and obvious artifacts;

[0136] The influence effects of the attention module and the convolutional layer settings are as follows Figure 13 shown: The setting of the parameters consists of the number of attention modules and the number of subsequent connected convolutional layers. By adjusting the number of attention modules and convolutional layers, the performance and efficiency of the image enhancement network can be maximized. x CBAM + n represents that the structure of the set network is annotated with x CBAM attention modules and n convolutional layers.

[0137] In Figure 13 , only two attention modules are needed, and the picture can produce satisfactory results, which proves the effectiveness of adding attention modules. When the number of attention modules is set to 3 and the number of convolutional layers is also set to 3, the most natural exposure and contrast effects can be produced.

[0138] As Figure 13 shown, with the further increase of the attention modules, problems such as overexposure and image distortion may occur. For details, see Table 2:

[0139] Method PSNR SSIM MAE 2CBAM+2 13.42 0.51 155.60 2CBAM+3 15.32 0.50 132.01 3CBAM+2 16.52 0.52 112.54 3CBAM+3 20.14 0.58 96.05 4CBAM+3 19.52 0.59 98.32

[0140] Table 2

[0141] To verify the effectiveness of this algorithm, this paper compares the enhanced results of this algorithm with those of various classical algorithms, and evaluates the enhanced images on the public dataset through non-reference and reference-based evaluation metrics: The low-light image dataset for non-reference evaluation is collected from existing low-light image enhancement papers, including MEF, LIME, NPE, VV, DICM;

[0142] The LOL dataset for reference-based quality evaluation. The LOL dataset contains 500 low-light / normal-light images, and the size of all experimental images is adjusted to 1200×900×3.

[0143] Among them, this application has both CNN-based methods (RetinexNet, Lighten-Net, LLNet, etc.) and GAN-based methods (EnlightenGAN).

[0144] Figure 14 As the test results of this article on the DICM dataset, for outdoor scenes, it is difficult for LightenNet, EnlightenGAN, and Zero-Dce++ to accurately enhance the backlit areas (such as faces), resulting in darker facial areas. LLNet will produce artifacts, and the overall details of the image are relatively rough, with a certain color distortion.

[0145] For RetinexNet, there are many overexposed situations in the image, while Zero-Dce and the method of this article perform better and can produce relatively clear and natural detail features;

[0146] For indoor scenes, LLNet and RetinexNet have relatively large noise information, and the image enhancement effect is not very good. There is a certain color deviation in the image enhanced by EnlightenGAN;

[0147] The method of this application and the methods of the Zero-DCE series can enhance the dark areas of the image while retaining the detail information of the image.

[0148] As a supplement to subjective evaluation, a no-reference quantitative evaluation index is used to quantitatively analyze the low-light image test dataset without a reference image;

[0149] The no-reference evaluation index NIQE is used to further evaluate the enhancement results of each algorithm.

[0150] The performance of the method proposed in this application is the best on the DICM dataset.

[0151] Details are shown in Table 3:

[0152] Method MEF LIME NPE VV DICM Average Input 4.27 4.38 4.32 3.53 4.26 4.10 LLNet 4.85 4.94 4.78 4.47 4.81 4.75 RetinexNet 4.15 4.42 4.49 2.06 4.20 3.55 EnlightenGAN 3.23 3.79 4.12 2.58 3.57 3.39 Zero-DCE 3.28 3.77 3.93 3.21 3.55 3.55 Zero-DCE++ 3.45 3.98 4.03 3.10 3.56 3.62 Ours 3.38 4.02 4.09 3.05 3.48 3.60

[0153] Table 3

[0154] For the reference-based image quality assessment, this application uses the Peak Signal-to-Noise Ratio (PSNR, dB), Structural Similarity (SSIM), and Mean Absolute Error (MAE) metrics to quantitatively compare the performance of different methods on the LOL dataset. Among them, the larger the SSIM metric value, the closer the enhanced image is to the original image in terms of brightness, contrast, and structure. When the SSIM value is 1, it indicates that the quality of the two images is the same. The larger the PSNR metric value, the closer the image quality is to the reference image. As can be seen from Table 3, the method proposed in this application has obtained the highest PSNR and the lowest MAE in all cases. Due to the introduction of the attention mechanism, compared with the Zero-DCE++ network, the PSNR index of the method in this application has increased by 21.3%, and the MAE index has decreased by 6%. Although the SSIM index is not the best, it is still very competitive. For details, see Table 4:

[0155] Method PSNR SSIM MAE SRIE 14.41 0.54 127.08 LIME 16.17 0.57 108.12 Li et al. 15.19 0.54 114.21 RetinexNet 15.99 0.53 104.81 Wang et al. 13.52 0.49 142.01 EnlightenGAN 16.21 0.59 102.78 Zero-DCE 16.57 0.59 98.78 Zero-DCE++ 16.42 0.58 102.87 Ours 20.14 0.58 96.05

[0156] Table 4

[0157] As Figure 5 shown, the second aspect of the present invention provides a low-light image enhancement device 900. The low-light image enhancement device 900 includes: an acquisition module 902 for acquiring an image training set, the image training set including at least one training sample, and the training sample corresponding to an exposure sequence in the image enhancer; acquiring a first low-light image enhancement network; a sample determination module 904 for determining a first training sample and a second training sample in the image training set; a training module 906 for training the first training sample according to the first low-light image enhancement network to determine at least one first training result; an evaluation module 908 for determining optimization parameters corresponding to the first low-light image enhancement network based on ablation experiments; an optimization module 910 for optimizing the first low-light image enhancement network according to the optimization parameters to determine a second low-light image enhancement network; and a result determination module 912 for training the second training sample according to the second low-light image enhancement network to determine at least one second training result.

[0158] The low-light image enhancement device 900 provided by the present invention performs low-light enhancement on an image training set through a first low-light image enhancement network, and optimizes the first low-light image enhancement network to determine a second low-light image enhancement network. Specifically, through an acquisition module 902, a certain number of degraded images are acquired from an underexposed single-image contrast enhancement (SICE) data set as the image training set; through a sample determination module 904, the image samples in the image training set are divided into a first training sample and a second training sample, the first training sample is used to train a network model, and the second training sample is used to verify the training result of the network model; a training module 906 obtains a mapping curve from a low-light image to a high-light image through the first low-light image enhancement network, and then uses the learned mapping curve to perform multiple iterative adjustments on the original image at the pixel level to achieve the purpose of pixel-level image enhancement within the dynamic range, thereby determining a first training result; an evaluation module 908 includes ablation experiments and relevant data for objective and quantitative evaluation, and through ablation experiments and objective and quantitative evaluation methods, it is experimentally compared with existing low-light image enhancement methods and restoration algorithms to determine the optimization parameters and evaluation results of the first low-light image enhancement network, and optimize the first low-light image to determine a second low-light image enhancement network; finally, a result determination module 912 determines the low-light image enhancement result of the second low-light image enhancement network.

[0159] It can be understood that the dynamic pixels of the input image are adjusted through a high-order curve to obtain an enhancement effect. It does not require any paired or unpaired data during the training process, but uses a non-reference loss function to adjust the parameters of the network, thereby reducing the dependence of the neural network on data during use. At the same time, due to its simple network structure, while improving performance, it also reduces the requirements of the network for hardware.

[0160] As Figure 6 shown, a third aspect of the present invention provides an electronic device 1000, including a processor 1110, a memory 1109, a program or instruction stored on the memory 1109 and executable on the processor 1110. When the program or instruction is executed by the processor 1110, it implements each process of the embodiment of the above low-light image enhancement method and can achieve the same technical effect. To avoid repetition, it will not be described in detail here.

[0161] Among them, the processor 1110 is configured to obtain an image training set, where the image training set includes at least one training sample, and the training sample corresponds to an exposure sequence in an image intensifier; determine a first training sample and a second training sample in the image training set; obtain a first low-light image enhancement network; train the first training sample according to the first low-light image enhancement network to determine at least one first training result; determine optimization parameters corresponding to the first low-light image enhancement network based on an ablation experiment; optimize the first low-light image enhancement network according to the optimization parameters to determine a second low-light image enhancement network; train the second training sample according to the second low-light image enhancement network to determine at least one second training result.

[0162] Optionally, the processor 1110 is further configured to obtain exposure sequences in at least one image intensifier; randomly allocate the exposure sequences to determine a first training sample and a second training sample, where the number of the first training samples is greater than the number of the second training samples.

[0163] Optionally, the processor 1110 is further configured to determine a mapping curve corresponding to at least one first training sample; perform at least one low-light enhancement process on the first training sample according to the mapping curve to determine an illumination enhancement curve corresponding to a high-order curve; obtain convolution layer setting parameters; set the number of convolution layers of the first low-light image enhancement network according to the convolution layer setting parameters; perform a reference-free loss analysis on the illumination enhancement curve according to the first low-light image enhancement network after the convolution layer setting is completed to determine a total loss function; determine the first training result according to the total loss function.

[0164] Optionally, the processor 1110 is further configured to obtain a public data set; compare the public data set with the first training result by means of an ablation experiment to determine a reference evaluation result; determine loss function optimization parameters and convolution layer optimization parameters according to the reference evaluation result; determine optimization parameters corresponding to the first low-light image enhancement network according to the loss function optimization parameters and the convolution layer optimization parameters.

[0165] A fourth aspect of the present invention provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, it implements each process of the embodiments of the above low-light image enhancement method and can achieve the same technical effect. To avoid repetition, it will not be elaborated here. In addition, the readable storage medium improves the data storage capacity corresponding to the low-light image enhancement method in the present application and the data processing speed of each step.

[0166] The method can be implemented in various different ways according to specific features and / or example applications. For example, these methods can be implemented by a combination of hardware, firmware, and / or software. For example, in a hardware implementation, a processor can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, electronic devices, other device units for performing the above functions, and / or combinations thereof.

[0167] A computer-readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. A computer-readable storage medium can be an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing devices, without limitation. A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer floppy disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile disks (DVDs), memory cards, floppy disks, encoding mechanical devices (such as punched cards or grooves with raised structures recording instructions), and any suitable combination of the foregoing devices. The computer-readable storage medium used herein should not be construed as a transmission signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagated through waveguides or other transmission media, or electrical signals transmitted through wires, etc.

[0168] Among them, the processor is the processor in the electronic device in the above embodiment. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs, etc.

[0169] The fifth aspect of the present invention provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement each process of the embodiment of the above low-light image enhancement method and can achieve the same technical effects. To avoid repetition, it will not be elaborated here. In addition, the chip improves the data processing speed of each step in the low-light image enhancement method of the present application.

[0170] Although this specification contains many specific implementation details, these should not be construed as limiting the scope of any invention or the scope of what is claimed, but rather as mainly for describing the features of specific embodiments of a particular invention. Certain features that are described in multiple embodiments in this specification may also be implemented in combination in a single embodiment. On the other hand, the various features described in a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. In addition, although features may act in certain combinations as described above and are even initially claimed as such, one or more features from a claimed combination may in some cases be removed from that combination, and the claimed combination may refer to a sub-combination or a variation of a sub-combination.

[0171] Similarly, although operations are depicted in the drawings in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or sequentially, or that all illustrated operations be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of the various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0172] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the acts recited in the claims may be performed in a different order and still achieve the desired result. In addition, the processes depicted in the drawings are not necessarily in the particular order or sequential order shown to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0173] It should be noted that in this document, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising a..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0174] The above are only specific embodiments of the present invention, enabling those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features claimed herein.

Claims

1. A low-light image enhancement method, characterized in that: include: Acquire an image training set, wherein the image training set includes at least one training sample, and the training sample corresponds to an exposure sequence in an image intensifier; Determine a first training sample and a second training sample in the image training set; Obtaining a first low-light image enhancement network; Training the first training sample according to the first low-light image enhancement network to determine at least one first training result; Determining optimization parameters corresponding to the first low-light image enhancement network based on ablation experiments; Optimizing the first low-light image enhancement network according to the optimization parameters to determine a second low-light image enhancement network; The second training sample is trained according to the second low-light image enhancement network to determine at least one second training result.

2. The low-light image enhancement method according to claim 1, characterized in that: The first low-light image enhancement network is an unsupervised low-light image enhancement network based on an attention mechanism.

3. The low-light image enhancement method according to claim 1, characterized in that: The determining of the first training sample and the second training sample in the image training set comprises: acquiring an exposure sequence in at least one of the image intensifiers; The exposure sequence is randomly allocated to determine a first training sample and a second training sample, wherein the number of the first training samples is greater than the number of the second training samples.

4. The low-light image enhancement method according to claim 1, characterized in that: The step of training the first training sample according to the first low-light image enhancement network to determine at least one first training result includes: determining a mapping curve corresponding to at least one of the first training samples; Performing at least one low-light enhancement process on the first training sample according to the mapping curve to determine a lighting enhancement curve corresponding to a high-order curve; Get the convolution layer setting parameters; Setting the number of convolutional layers of the first low-light image enhancement network according to the convolutional layer setting parameters; Performing a reference-free loss analysis on the illumination enhancement curve according to the first low-light image enhancement network after completing the convolutional layer setting to determine a total loss function; A first training result is determined according to the total loss function.

5. The low-light image enhancement method according to claim 4, characterized in that: The reference loss analysis includes: spatial consistency loss, exposure control loss, color constancy loss, illumination smoothness loss and structural similarity loss; The spatial consistency loss, the exposure control loss, the color constancy loss, the illumination smoothness loss and the structural similarity loss respectively correspond to a spatial consistency loss weight, an exposure control loss weight, a color constancy loss weight, an illumination smoothness loss weight and a structural similarity loss weight.

6. The low-light image enhancement method according to claim 1, characterized in that: The step of determining the optimization parameters corresponding to the first low-light image enhancement network based on the ablation experiment includes: Get public datasets; Comparing the public data set and the first training result by an ablation experiment method to determine a reference evaluation result; Determine the loss function optimization parameters and the convolution layer optimization parameters according to the reference evaluation results; Determine optimization parameters corresponding to the first low-light image enhancement network according to the loss function optimization parameters and the convolutional layer optimization parameters.

7. A low-light image enhancement device, characterized in that: include: An acquisition module, used for acquiring an image training set, wherein the image training set includes at least one training sample, and the training sample corresponds to an exposure sequence in an image intensifier; Obtaining a first low-light image enhancement network; A sample determination module, used to determine a first training sample and a second training sample in the image training set; a training module, configured to train the first training sample according to the first low-light image enhancement network to determine at least one first training result; An evaluation module, configured to determine optimization parameters corresponding to the first low-light image enhancement network based on an ablation experiment; an optimization module, configured to optimize the first low-light image enhancement network according to the optimization parameters to determine a second low-light image enhancement network; A result determination module is used to train the second training sample according to the second low-light image enhancement network to determine at least one second training result.

8. An electronic device, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method according to any one of claims 1 to 6.

9. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A chip, characterized in that: The chip includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the steps of the method according to any one of claims 1 to 6.

Images