Low-illumination image enhancement method based on zero reference depth curve estimation

Abstract

The invention belongs to the field of computer vision, and particularly relates to a low-illumination image enhancement method based on zero reference depth curve estimation. The method comprises the following steps: acquiring a low-illumination image data set, and inputting a low-illumination image into a curve estimation module to obtain a curve estimation result; the curve estimation result is used for processing the input image to obtain an enhanced image; calculating the total loss of the model according to the enhanced image and the input image, and adjusting model parameters according to the total loss of the model to obtain a trained low-illumination image enhancement model; performing low-illumination image enhancement by using the trained model; the network used by the method is simple in structure, the calculation amount and the cost during enhancement are smaller, and the image enhancement accuracy is improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of computer vision, and particularly relates to a low-light image enhancement method based on zero reference depth curve estimation. BACKGROUND

[0002] With the wide popularity of smart phones, security monitoring, medical imaging, and night photography, the demand for high-quality low-light images is increasing. However, images taken in low-light conditions are accompanied by problems such as increased noise, loss of details, blurred textures, and low brightness and contrast, which severely limit the visual quality of the images and the performance of subsequent processing (such as low-light target detection and low-light target classification). Therefore, by designing a low-light image enhancement model, the overall image contrast can be improved while the texture and detail information of the low-light image is restored, which can effectively improve the overall quality of the image.

[0003] Low-light image enhancement technology can specifically optimize the image performance in dark scenes: by improving the picture brightness, enhancing the detail recognition, and maintaining the color balance, the overall image quality is significantly improved. As a key data preprocessing step in advanced visual tasks, this technology can provide better input data for downstream tasks (such as target detection and image segmentation), thereby helping to achieve performance breakthroughs in algorithms. Therefore, low-light image enhancement technology plays an important role in both academic research and industrial applications, and has high research significance and commercial potential.

[0004] In recent years, the rapid development of deep learning technology has driven the paradigm innovation in the field of image enhancement. Deep learning-based solutions have gradually replaced traditional methods and become the mainstream. Compared with traditional methods that rely on hand-designed features, deep learning methods have better accuracy, stronger robustness, and higher processing efficiency, and have been a research hotspot in both academia and industry in recent years. The core logic is as follows: by constructing a model containing learnable parameters, training it using the original data set, and continuously optimizing the parameters to minimize the prediction loss, a model with generalization ability is finally obtained - this model can effectively enhance low-light images that did not participate in the training, and exhibits strong adaptive processing capabilities. However, as the enhancement effect improves, the network structure becomes more and more complex, which brings huge parameter quantity and high computational complexity. The high computational cost makes it difficult to deploy these models on devices with limited resources, thereby limiting the application of low-light image enhancement technology.

[0005] Therefore, it is necessary to research and develop image enhancement algorithms for low-light environments, which can extract more feature information while consuming less computational resources, to improve image quality and usability in applications. SUMMARY

[0006] To address the shortcomings of existing technologies, this invention proposes a low-light image enhancement method based on zero-reference depth curve estimation. The method includes: acquiring the low-light image to be enhanced, inputting it into a trained low-light image enhancement model, and obtaining the enhanced image.

[0007] The training process for low-light image enhancement models includes:

[0008] S1: Obtain the low-light image dataset for subsequent processing;

[0009] S2: Input the low-light image into the depth curve estimation module to obtain the curve estimation result;

[0010] S3: Apply the curve estimation result to the original low-light image to obtain the enhanced image;

[0011] S4: Calculate the total loss of the model and adjust the model parameters according to the loss to obtain a trained low-light image enhancement model.

[0012] Preferably, the depth curve estimation process includes: the low-light image is processed through seven convolutional layers and activation layers to obtain the final curve estimation result.

[0013] Furthermore, the seven convolutional layers consist of one regular convolutional layer, five asymmetric convolutional layers, and one regular output convolutional layer.

[0014] Furthermore, the asymmetric convolutional layer processing procedure includes: the input sequentially passes through a convolutional layer with a kernel of (3,1), a ReLU activation layer, a convolutional layer with a kernel of (1,3), and a ReLU activation layer to obtain the final asymmetric convolution result.

[0015] Furthermore, the inputs to the last three convolutional layers are the outputs of the previous layer and the outputs of the third, second, and first convolutional layers, respectively, which are then stacked after passing through the MECA attention mechanism module.

[0016] Preferably, the process of applying curve estimation to the original image satisfies the following formula:

[0017]

[0018] in This represents the low-light image after the nth enhancement, where n is a non-negative integer. For the low-light image, The curve estimation results are divided into three channels and eight layers of eigenvalues.

[0019] Preferably, the total loss function used for the total loss of the model is a weighted sum of color consistency loss, spatial consistency loss, exposure control loss, illumination smoothing loss, and perceptual loss;

[0020] The formula for calculating perceived loss is: in, This represents the low-light image after the nth enhancement, where n is a non-negative integer. For the low-light image, For the pre-trained VGG16 model used for feature extraction, Describing the L2 norm, , and Let represent the number of channels, height, and width of the i-th feature map extracted by the pre-trained network, and let Q represent the feature maps output by layers 4, 9, 16, and 23 of the pre-trained network.

[0021] The beneficial effects of the present invention are as follows: By using this low-light image enhancement network to enhance low-light images, the computational load during image enhancement can be reduced, and the image enhancement effect can be optimized. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the network structure of CDCE-Net in this invention.

[0023] Figure 2 This is a schematic diagram of the MECA attention mechanism of the present invention.

[0024] Figure 3 This is a schematic diagram illustrating the effect of curve estimation parameters on an image in this invention.

[0025] Figure 4 This is a flowchart of a neural network-based dark light enhancement method according to the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] This invention proposes a low-light image enhancement method based on a zero-reference depth curve, the method comprising:

[0028] Construct a depth curve estimation model based on zero-reference depth curve estimation, namely CDCE-Net, such as... Figure 1 As shown, it includes seven convolutional layers and three attention mechanism layers. The low-light image to be enhanced is obtained, input into a trained depth curve estimation model, and then applied to the original image to obtain the enhanced image.

[0029] The training process for low-light image enhancement models includes:

[0030] S1: Obtain the low-light image dataset for subsequent processing.

[0031] Collect a dataset of low-light images. Since this algorithm is unsupervised, it does not require normal-light images corresponding to the low-light images. The images can be obtained from public datasets and images of public datasets after transformation, as well as images obtained from electronic device sensors.

[0032] S2: Input the low-light image into the depth curve estimation module to obtain the curve estimation result.

[0033] The depth estimation module is primarily used to obtain curve estimation results from the input image to facilitate subsequent image enhancement. The data processing of the depth estimation module includes: a 3×3 input convolutional layer, followed by a ReLU activation layer, and then five asymmetric convolutional layers. The inputs of the last three convolutional layers are the outputs of the previous layer and the outputs of the third, second, and first convolutional layers, respectively, which are then stacked using the MECA attention mechanism. Finally, there is a 3×3 output convolutional layer, followed by ReLU activation and output.

[0034] Traditional convolution operations typically use symmetric kernels, where elements within the kernel are symmetrical in both the horizontal and vertical directions and have equal weights. However, in low-light image enhancement scenarios, since curve estimation results are pixel-level and independent of the surrounding pixels and the entire image, asymmetric features in the image become crucial for model performance. Therefore, this invention sets the middle convolutional layers as asymmetric convolutions. Asymmetric convolutions offer the advantages of easily combining with other structures without fine-tuning by introducing only a few hyperparameters, enhancing the feature representation capabilities of labeled convolutional kernels, effectively improving network accuracy without increasing inference time and computational overhead, and enhancing the model's robustness to rotationally distorted objects.

[0035] The data processing procedure for asymmetric convolution is as follows:

[0036] The input passes through a convolutional layer with a kernel of (3, 1), then is activated by a ReLU function, then passes through a convolutional layer with a kernel of (1, 3), then is activated by a ReLU function again, and finally outputs.

[0037] In traditional ECA attention mechanisms, to make the model parameters smaller and lighter, the authors ignored capturing information from all channels and used a (1×1) convolutional kernel for processing. However, in low-light image enhancement scenarios, the information between channels is interconnected for curve estimation. To better enhance features and address the aforementioned issues, this invention proposes the MECA attention mechanism, which can capture more information between channels while enhancing feature extraction, such as... Figure 2 As shown, it mainly includes the following key steps:

[0038] Global average pooling: Perform global average pooling on the input feature map to obtain global information for each channel.

[0039] One-dimensional convolution: The pooled features are processed by one-dimensional convolution. The size of the convolution kernel is adaptively determined, which can capture local channel information.

[0040] Activation function: The Sigmoid function is used to restrict the output of the one-dimensional convolution to the range (0-1) to obtain the channel weights.

[0041] Feature recalibration: The calculated channel weights are multiplied by the original feature map to achieve feature recalibration.

[0042] Channel information extraction: Perform a 2D convolution with a kernel of (1×1) on the input feature map to obtain channel information with 1 channel.

[0043] Channel information fusion: The extracted information is summed with the result of feature recalibration channel by channel to obtain the final result.

[0044] S3: Apply the curve estimation result to the original low-light image to obtain the enhanced image, as follows: Figure 3 As shown, the curve estimation result is divided into eight curve estimation channels based on the RGB three channels. The first layer curve estimation result is applied to the input image to obtain the first-order image, and the second layer curve estimation result is applied to the first-order image to obtain the second-order image. The specific formula is as follows:

[0045]

[0046] in This represents the low-light image after the nth enhancement, where n is a non-negative integer. For the low-light image, The curve estimation results are divided into three channels and eight layers of eigenvalues.

[0047] S4: Calculate the total loss of the model and adjust the model parameters according to the loss to obtain a trained low-light image enhancement model.

[0048] This invention constructs a simple and efficient loss function, which specifically includes color consistency loss, spatial consistency loss, exposure control loss, illumination smoothing loss, and perceptual loss.

[0049] The formula for spatial consistency loss is as follows:

[0050]

[0051] Where k is the number of local regions of size (3×3), and Ω(i) are the four adjacent regions (top, bottom, left, right) centered at region i. Y and I represent the average intensity values ​​of local regions in the enhanced version and the input image, respectively.

[0052] The formula for color consistency loss is as follows:

[0053]

[0054] in This represents the average intensity value of the p channel in the enhanced image. A pair of channels is represented as (p, q).

[0055] The formula for exposure control loss is as follows:

[0056]

[0057] Where M represents the number of non-overlapping local regions of size 16x16, and Y represents the average intensity value of the local regions in the enhanced image.

[0058] The formula for illumination smoothing loss is as follows:

[0059]

[0060] Where N is the number of iterations, and the horizontal and vertical gradient operations are respectively represented as follows: and .

[0061] The formula for calculating perceived loss is:

[0062]

[0063] in, This represents the low-light image after the nth enhancement, where n is a non-negative integer. For the low-light image, For the pre-trained VGG16 model used for feature extraction, Describing the L2 norm, , and Let represent the number of channels, height, and width of the i-th feature map extracted by the pre-trained network, and let Q represent the feature maps output by layers 4, 9, 16, and 23 of the pre-trained network.

[0064] In summary, the complete loss function is:

[0065]

[0066] The model parameters are adjusted based on the total model loss. Training stops when the loss function converges or reaches the maximum preset number of iterations, and the model parameters are saved, resulting in a trained low-light image enhancement model. The low-light image to be enhanced is then obtained and input into the trained low-light image enhancement model to produce the enhanced image.

[0067] Experimental verification

[0068] To illustrate the effectiveness of the present invention, this embodiment will compare multiple methods under the same experimental conditions.

[0069] 1. Experimental conditions

[0070] The hardware testing conditions for this experiment were: NVIDIA 5060 GPU, 8GB VRAM, CUDA 12.8. The low-light images used in the test were real-world paired low-light / normal-light datasets.

[0071] 2. Experimental Results

[0072] By comparing different low-light image enhancement schemes, the effectiveness of the enhancement method disclosed in this invention is verified from multiple angles and in all aspects.

[0073] Table 1. Performance comparison of low-light image enhancement algorithms on three different datasets.

[0074]

[0075] As can be seen from the results in Table 1, the method of this invention is based on the attention mechanism and achieves very good enhancement effect through unsupervised training. On the classic low-light image dataset LOL, the enhancement effect on low-light images is significantly better than other state-of-the-art methods.

[0076] It is worth noting that although the proposed method is unsupervised, it outperforms current supervised enhancement methods such as KinD and Retinex-Net. PSNR and SSIM are currently the most widely used and recognized image quality evaluation metrics, while NIQE is the most commonly used metric for judging the naturalness of generated images. Therefore, the results strongly demonstrate the effectiveness of the method disclosed in this invention, significantly outperforming other methods.

Claims

1. A low-light image enhancement method based on zero-reference depth curve estimation, characterized in that, include: Obtain the low-light image to be enhanced, input it into the trained low-light image enhancement model, and obtain the enhanced image; The training process for low-light image enhancement models includes: S1: Obtain the low-light image dataset for subsequent processing; S2: Input the low-light image into the depth curve estimation module to obtain the curve estimation result; S3: Apply the curve estimation result to the original low-light image to obtain the enhanced image; S4: Calculate the total loss of the model and adjust the model parameters according to the loss to obtain a trained low-light image enhancement model.

2. The low-light image enhancement method based on zero-reference curve estimation according to claim 1, characterized in that, The depth curve estimation process includes: the low-light image is processed through seven convolutional layers and activation layers to obtain the final curve estimation result.

3. The low-light image enhancement method based on zero-reference curve estimation according to claim 2, characterized in that, The seven convolutional layers consist of one regular convolutional layer, five asymmetric convolutional layers, and one regular output convolutional layer.

4. The low-light image enhancement method based on zero-reference curve estimation according to claim 3, characterized in that, The asymmetric convolutional layer processing procedure includes: the input sequentially passes through a convolutional layer with kernel (3,1), a ReLU activation layer, a convolutional layer with kernel (1,3), and a ReLU activation layer to obtain the final asymmetric convolution result.

5. The low-light image enhancement method based on zero-reference curve estimation according to claim 4, characterized in that, The inputs to the last three convolutional layers are the outputs of the previous layer and the outputs of the third, second, and first convolutional layers, respectively, which are then stacked after passing through the MECA attention mechanism module.

6. The low-light image enhancement method based on zero-reference curve estimation according to claim 5, characterized in that, The MECA attention mechanism includes key steps such as global average pooling, one-dimensional convolution, activation function, feature recalibration, channel information extraction, and channel information fusion.

7. The low-light image enhancement method based on zero-reference curve estimation according to claim 1, characterized in that, The process of applying curve estimation to the original image satisfies the following formula: L E n x = L E n - 1 x + S n L E n - 1 x 1 - L E n - 1 x in This represents the low-light image after the nth enhancement, where n is a non-negative integer. For the low-light image, The curve estimation results are divided into three channels and eight layers of eigenvalues.

8. The low-light image enhancement method based on zero-reference curve estimation according to claim 1, characterized in that, The total loss function used in the model is a weighted sum of color consistency loss, spatial consistency loss, exposure control loss, illumination smoothing loss, and perceptual loss.

9. A low-light image enhancement method based on zero-reference curve estimation according to claim 8, characterized in that, The formula for calculating the perceived loss is as follows: in, This represents the low-light image after the nth enhancement, where n is a non-negative integer. For the low-light image, For the pre-trained VGG16 model used for feature extraction, Describing the L2 norm, , and Let represent the number of channels, height, and width of the i-th feature map extracted by the pre-trained network, and let Q represent the feature maps output by layers 4, 9, 16, and 23 of the pre-trained network.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1-9.

11. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the electronic device, it implements the method as described in any one of claims 1-9.

Images