DCE-based Low Illumination Image Enhancement Method, System and Related Devices

Through the low-illumination image enhancement method based on the DCE model, the jump-connected DCE-Net module and mutual consistency loss function are used to solve the problem of information loss under color shift and complex lighting conditions in the traditional method, and a better image enhancement effect is achieved.

CN114663300BActive Publication Date: 2025-06-03SHENZHEN ANRUAN HUISHI TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210196296.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-01
Publication Date
2025-06-03
Estimated Expiration
2042-03-01

AI Technical Summary

Technical Problem

The traditional low-illumination image enhancement method based on histogram and Retinex theory is prone to color shifts, and errors are prone to estimating illuminated images under complex lighting conditions, resulting in the loss of dark details and edge information, and insufficient enhancement effect.

Method used

The low-illumination image enhancement method based on the DCE model is adopted. By obtaining the video data of the real scene, disassembly processing and filtering out the training data set and test data sets, the preset enhancement model is used for iterative training, including four sets of jump-connected DCE-Net modules, iterative parameter modules and curve iterative modules, and a mutual consistency loss function is added to protect the original image information and edge information.

Benefits of technology

It effectively prevents the loss of original image information during image enhancement, protects edge information in the image, and improves the enhancement effect of low-illumination images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114663300B_ABST
    Figure CN114663300B_ABST
Patent Text Reader

Abstract

The present invention is applicable to the field of image processing, and provides a low-light image enhancement method, system and related device based on DCE. The method includes: acquiring video data of a real scene; performing frame splitting processing on the video data to obtain frame-split pictures, screening out a test data set containing paired picture data from the frame-split pictures, and screening out a training data set containing pictures with different exposure degrees from the SICE data set; using the training data set to perform iterative training on a preset enhancement model, and saving the preset enhancement model that has completed iterative training as a low-light enhancement model; using the low-light enhancement model to perform low-light enhancement on the test data set. The present invention prevents the loss of original image information during the image enhancement process, further protects the edge information in the image, and improves the image enhancement effect.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and particularly relates to a low-light image enhancement method, system and related devices based on DCE. Background Art

[0002] In low-light environments and under the condition of unsatisfactory imaging devices, the collected images have problems such as low brightness, low contrast, and high noise, which not only affect the beauty of the images and the visual experience of humans, but also reduce the performance of high-level visual tasks using normal-light images.

[0003] With the continuous development of low-light image enhancement technology, researchers have proposed various methods to improve the subjective and objective quality of low-light images. Currently, relatively common low-light image enhancement algorithms include methods based on histogram equalization, low-light image enhancement algorithms based on Retinex theory, and low-light image enhancement algorithms based on deep learning. The former method based on histogram equalization is simple and efficient, but it is easy to have insufficient enhancement in dark areas and excessive enhancement in bright areas, resulting in color deviation problems; the low-light image enhancement algorithm based on Retinex theory regards the low-light image as the product of the illumination image and the reflectance image, but it is necessary to use prior knowledge or existing constraints to estimate the illumination map in the Retinex theory model. However, when the prior knowledge or existing constraints cannot describe the complex and changing illumination conditions, the estimation of the illumination image will have errors, thus affecting the accuracy of the subsequent reflectance image.

[0004] In recent research, the Zero-DCE (Zero-Reference Deep Curve Estimation) neural network model is often used for low-light image enhancement. It is a kind of basic DCE model. The training process of this model does not require any paired data, but uses a set of non-reference loss functions to supervise the network training to generate high-order curves, and performs pixel-level adjustment on the change range of the input low-light image. The problem is that the Zero-DCE model training often uses regular illumination images, making it easy to lose edge information such as dark details in the final image, resulting in insufficient enhancement effect. Summary of the Invention

[0005] Embodiments of the present invention provide a low-light image enhancement method, system and related devices, aiming to solve the problems of easy color deviation and errors under complex illumination conditions in traditional low-light image enhancement methods based on histograms and Retinex theory.

[0006] In a first aspect, embodiments of the present invention provide a low-light image enhancement method. The low-light image enhancement method is based on a DCE model and includes the following steps:

[0007] Obtain video data of the real scene;

[0008] Perform frame splitting on the video data to obtain split-frame pictures, screen out a test data set containing paired picture data from the split-frame pictures, and screen out pictures with different exposure degrees from the SICE data set as the training data set;

[0009] Use the training data set to iteratively train a preset enhancement model, and save the preset enhancement model that has completed iterative training as the low-light enhancement model;

[0010] Use the low-light enhancement model to perform low-light enhancement on the test data set.

[0011] Furthermore, the preset enhancement model includes four groups of DCE-Net modules with skip connections, an iterative parameter module, and a curve iteration module, where:

[0012] The DCE-Net module takes image data as input and outputs 8 groups of parameters to be saved in the iterative parameter module;

[0013] Each parameter in the iterative parameter module is used to determine a curve shape, and the pixels on the RGB channels in the image data adjust the original RGB pixel values according to the curve shape;

[0014] The curve iteration module takes the 8 groups of iterative parameters in the iterative parameter module as input and calculates the high-order luminance enhancement curve LE through iteration and pixelization.

[0015] Furthermore, the DCE-Net module includes eight convolutional layers connected in sequence. Among them, each convolutional layer in the first to fourth layers contains a convolution of 32*32 size, and each convolutional layer in the fifth to eighth layers contains two convolutions of 32*32 size. Moreover, the first convolutional layer is skip-connected to the eighth convolutional layer, the second convolutional layer is skip-connected to the seventh convolutional layer, the third convolutional layer is skip-connected to the sixth convolutional layer, and the fourth convolutional layer is skip-connected to the fifth convolutional layer.

[0016] Furthermore, the high-order luminance enhancement curve LE in the curve iteration module satisfies the following relational expression (1):

[0017] LE n (I (x) ; r) = LE n-1 + r * LE n-1 (1 - LE n-1 ) (1)

[0018] where, I (x) represents the input image data, LEn (I (x) ; r) represents the nth enhanced estimate, where r represents the iteration parameter and n represents the number of iterations. The value of r satisfies the following relational expression (2): (x) The nth enhanced estimate, r represents the iteration parameter, n represents the number of iterations, and the value of r satisfies the following relational expression (2):

[0019] r ∈ (-1, 1) (2).

[0020] Furthermore, the loss function used by the preset enhancement model during iterative training includes a basic loss Loss and a mutual consistency loss function L mc , where the basic loss Loss includes a spatial consistency loss, an exposure control loss, a color constancy loss, and a light smoothness loss.

[0021] Furthermore, the mutual consistency loss function L mc satisfies the following relational expression (3):

[0022] L mc = ∥M * exp(-c * M)∥ 1 (3)

[0023] where M represents the square of the gradient of the image data and its corresponding enhanced estimate on the plane coordinate axes, and c represents a preset penalty factor.

[0024] Furthermore, during the iterative training process of the preset enhancement model, the preset penalty factor c starts from a preset value and makes the mutual consistency loss function L mc gradually increase as M increases, and after the mutual consistency loss function L mc reaches the maximum value, it is gradually decreased to 0. The number of iterative training times of the preset enhancement model is at least 200 times.

[0025] In a second aspect, an embodiment of the present invention further provides a low-light image enhancement system, including:

[0026] A data acquisition module for acquiring video data of a real scene;

[0027] A data processing module for performing frame splitting on the video data to obtain split-frame pictures, screening out a test data set containing paired picture data from the split-frame pictures, and screening out pictures with different exposure degrees from the SICE data set as a training data set;

[0028] A model training module for iteratively training a preset enhancement model using the training data set and saving the preset enhancement model that has completed iterative training as a low-light enhancement model;

[0029] An image enhancement module, configured to perform low-light enhancement on the test data set by using the low-light enhancement model.

[0030] In a third aspect, an embodiment of the present invention further provides a computer device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps in the low-light image enhancement method according to any one of the above embodiments are implemented.

[0031] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the low-light image enhancement method according to any one of the above embodiments are implemented.

[0032] The beneficial effects achieved by the present invention are as follows: By using the optimized DCE-Net module as the basis of the neural network and adding a special mutual consistency loss to the original reference-free loss function, the loss of the original image information during the image enhancement process is prevented, and the edge information in the image is further protected, thereby improving the image enhancement effect. Description of the Drawings

[0033] Figure 1 is a flowchart of the steps of the low-light image enhancement method provided by an embodiment of the present invention;

[0034] Figure 2 is a schematic structural diagram of a preset enhancement model provided by an embodiment of the present invention;

[0035] Figure 3 is a schematic diagram of curves of different r values of the second-order brightness enhancement curve LE provided by an embodiment of the present invention;

[0036] Figure 4 is a schematic diagram of curves of different iteration times of the second-order brightness enhancement curve LE provided by an embodiment of the present invention;

[0037] Figure 5 is a schematic diagram of a loss function curve when the penalty factor c takes different values provided by an embodiment of the present invention;

[0038] Figure 6 is a schematic structural diagram of the low-light image enhancement system 200 provided by an embodiment of the present invention;

[0039] Figure 7 is a schematic structural diagram of the computer device provided by an embodiment of the present invention. Detailed Embodiments

[0040] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0041] Please refer to Figure 1 , Figure 1 which is a flowchart of the steps of the low-light image enhancement method provided by the embodiment of the present invention, specifically including the following steps:

[0042] S101. Obtain video data of a real scene.

[0043] In the embodiment of the present invention, the real scene may be a closed real space, such as an office area, a teaching area, etc. The method for obtaining the video data of the real scene may be to continuously shoot the real scene using a fixed-point camera. Moreover, in order to enable the performance of the video data to reflect the characteristics of low-light images and the corresponding bright scenes, the video data should preferably include data under different lighting conditions during day and night changes. For example, use a camera to perform fixed-point shooting at the entrance and exit of a teaching area, and within 24 consecutive hours, shoot and obtain 1000 segments of the video data, each segment having a duration of 3 seconds.

[0044] S102. Perform frame splitting on the video data to obtain split-frame pictures, screen out a test data set containing paired picture data from the split-frame pictures, and screen out pictures with different exposure degrees from the SICE data set as a training data set.

[0045] Specifically, the video data is a continuous frame image with a certain duration. In the embodiments of the present invention, each segment of the video data is split into the split-frame pictures according to a certain duration. For example, each segment of the video data with a duration of 3 seconds is split at a speed of extracting once every 0.5 seconds. After processing 1000 segments of the video data, 6000 split-frame pictures of the same scene with different illuminances are obtained. Then, through computer processing or manual screening, a test data set containing paired picture data is screened out from the split-frame pictures. The paired picture data means that for a split-frame picture with low illuminance, a split-frame picture of the same scene but with an illuminance significantly higher than that of the low-illuminance picture is used as its paired picture. Therefore, a pair of the paired picture data contains two split-frame pictures with different illuminances. In the embodiments of the present invention, the paired picture data contains at least 1000 pairs. The SICE data set (Single Image Contrast Enhancer) is a commonly used data set in low-illuminance image research. The pictures in the SISE data set do not contain paired data. In the embodiments of the present invention, pictures with different exposure degrees are randomly screened out from the SICE data set by computer or manual methods as the training data set for the preset enhancement model in the embodiments of the present invention.

[0046] S103. Iteratively train the preset enhancement model using the training data set, and save the preset enhancement model that has completed the iterative training as the low-illuminance enhancement model.

[0047] Specifically, please refer to Figure 2 , Figure 2 which is a schematic structural diagram of the preset enhancement model provided by the embodiments of the present invention. The preset enhancement model includes four groups of DCE-Net modules connected by skip connections, an iterative parameter module, and a curve iteration module, where:

[0048] The DCE-Net module takes image data as input and outputs 8 groups of parameters to be saved in the iterative parameter module;

[0049] Each parameter in the iterative parameter module is used to determine a curve shape, and the pixels on the RGB channels in the image data adjust the original RGB pixel values according to the curve shape;

[0050] The curve iteration module takes the 8 groups of iterative parameters in the iterative parameter module as input and calculates the high-order luminance enhancement curve LE through iteration and pixelization.

[0051] The DCE-Net module is constructed based on the Zero-DCE model, which is a neural network model for low-light image enhancement. The training process of this model does not require any paired data. Instead, it uses a set of non-reference loss functions to supervise the network training to generate high-order curves, and performs pixel-level adjustment on the change range of the input low-light image. The problem is that the Zero-DCE model still has insufficient image enhancement effect in scenes with uneven and repetitive illumination.

[0052] The DCE-Net module in the embodiment of the present invention includes eight convolutional layers connected in sequence. Among them, each of the first to fourth convolutional layers contains a set of convolutions of size 32*32, and each of the fifth to eighth convolutional layers contains two sets of convolutions of size 32*32. Moreover, the first convolutional layer is skip-connected to the eighth convolutional layer, the second convolutional layer is skip-connected to the seventh convolutional layer, the third convolutional layer is skip-connected to the sixth convolutional layer, and the fourth convolutional layer is skip-connected to the fifth convolutional layer. Compared with the original Zero-DCE model, the DCE-Net module in the embodiment of the present invention adds convolutional layers and makes a multi-level design for the last four convolutional layers with skip connections. This enables the neural network model in the embodiment of the present invention to improve the accuracy of image feature extraction while increasing the number of model parameters and enhancing the complexity of the model to cope with more complex scenarios.

[0053] The number of the iterative parameter modules corresponds to the output number of the DCE-Net module. In the embodiment of the present invention, the DCE-Net module outputs a total of 8 groups of parameters to the iterative parameter modules. Therefore, the iterative parameter modules contain 8 groups of the iterative parameters.

[0054] The high-order brightness enhancement curve LE in the curve iteration module satisfies the following relational expression (1):

[0055] LE n (I (x) ; r) = LE n-1 + r * LE n-1 (1 - LE n-1 ) (1)

[0056] Where, I (x) represents the input image data, and LE n (I (x) ; r) represents the nth enhancement estimate of I (x) , r represents the iterative parameter, n represents the number of iterations, and the value of r satisfies the following relational expression (2):

[0057] r ∈ (-1, 1) (2)

[0058] For the high-order brightness enhancement curve LE, at the first-order iteration, the first set of iteration parameters in the iteration parameter module is assigned to R. After the calculation of the first enhancement estimation is completed, the next set of iteration parameters in the iteration parameter module is continued to be used for calculation. For example, the second-order high-order brightness enhancement curve LE can be expressed as the following relational expression (4):

[0059] LE(I (x) ; r) = I (x) + r * I (x) (1 - I (x) ) (4)

[0060] Specifically, please refer to Figure 3 and Figure 4 , Figure 3 which are the schematic diagrams of the curves of different r values of the second-order brightness enhancement curve LE provided by the embodiments of the present invention, Figure 4 and Figure 4 which are the schematic diagrams of the curves of different iteration times of the second-order brightness enhancement curve LE provided by the embodiments of the present invention. In

[0061] the loss function used by the preset enhancement model during the iterative training process includes the basic loss Loss and the mutual consistency loss function L mc , where the basic loss Loss includes the spatial consistency loss, the exposure control loss, the color constancy loss, and the illumination smoothness loss. The basic loss Loss is applied to the Zero-DCE model, and the original Zero-DCE model does not include the mutual consistency loss function L mc .

[0062] Specifically, the mutual consistency loss function L mc satisfies the following relational expression (3):

[0063] L mc = ∥M * exp(-c * M)∥ 1 (3)

[0064] Wherein, M represents the square of the gradient of the image data and its corresponding enhanced estimate on the plane coordinate axis, and c represents a preset penalty factor. The preset penalty factor is used to control the shape of the loss function to correspond to different degrees of training adjustment forms. At the same time, the penalty factor in the embodiments of the present invention has an impact on the relationship between M and the mutual consistency loss function L mc Specifically, the smaller the penalty factor c is taken, the more significant the proportional relationship between M and L mc becomes; the larger the penalty factor c is taken, the stronger the nonlinearity, and L mc first rises and then falls as M increases. Please refer to Figure 5 , Figure 5 which is a schematic diagram of the loss function curve when the penalty factor c takes different values provided by the embodiments of the present invention. For example, when the penalty factor c is taken as 10, the total penalty for L mc first rises, and then decreases to 0 as M increases. At the same time, compared with the values of other penalty factors c, in this process, as the penalty factor c increases, M increases and makes the maximum value of L mc become smaller, and makes L mc start to decrease after reaching the maximum value, and finally makes the result value of L mc close to 0. On the premise of the above design, the mutual consistency loss function L mc in the embodiments of the present invention can encourage the retention of large-difference edge information in the image during the training process of the preset enhancement model, while making less small-difference edge information be strengthened, so as to retain more dark part details and improve the image enhancement effect.

[0065] In the embodiments of the present invention, the preset enhancement model is iteratively trained at least 200 times using the training data set, and the preset enhancement model that has completed the iterative training is saved and output as the low-light enhancement model.

[0066] S104. Use the low-light enhancement model to perform low-light enhancement on the test data set.

[0067] In this step, the low-light enhancement model is used to enhance the lower-brightness frame-split images in the test data set, and the images obtained after low-light enhancement are compared with the higher-brightness frame-split images in the paired data, so as to complete the low-light image enhancement process of the test data set.

[0068] Exemplarily, the low-light enhancement model in the embodiments of the present invention uses 10 pairs of data in the publicly available Kind test set and the test data set as the instance real data set for index calculation. The publicly available Kind test set includes 15 pairs of data from four data sets, namely LOL, LIME, NPE, and MEF. The indexes include MSE (mean square error), PSNR (Peak Signal-to-Noise Ratio), SSIM (Structure Similarity Index Measure), and AB (Average Brightness). The index calculation results of the low-light enhancement model in the publicly available Kind test set and the instance real data set are shown in Table 1 below.

[0069] Table 1 Index calculation results in the publicly available Kind test set and the instance real data set

[0070]

[0071] The above data shows that the indexes of the low-light enhancement model in the embodiments of the present invention have been improved well on the publicly available Kind test set. On the real data set, MSE and PSNR are almost the same, and there are significant improvements in SSIM and AB. Especially for the AB index, the index calculation results show that while the image is enhanced, the average brightness is significantly improved, and the enhancement effect is remarkable.

[0072] The beneficial effects achieved by the present invention are as follows. Since the DCE-Net module with an optimized structure is used as the basis of the neural network and a special mutual consistency loss is added to the original reference-free loss function, the loss of the original image information during the image enhancement process is prevented, and the edge information in the image is further protected, thereby improving the image enhancement effect.

[0073] The embodiments of the present invention also provide a low-light image enhancement system. Please refer to Figure 6 , Figure 6 which is a schematic structural diagram of the low-light image enhancement system 200 provided by the embodiments of the present invention. The low-light image enhancement system 200 includes:

[0074] A data acquisition module 201, configured to acquire video data of a real scene;

[0075] A data processing module 202, configured to perform frame splitting on the video data to obtain split-frame pictures, screen out a test data set including paired picture data from the split-frame pictures, and screen out pictures with different exposure degrees from the SICE data set as a training data set;

[0076] The model training module 203 is configured to iteratively train a preset enhancement model using the training data set, and save the preset enhancement model that has completed the iterative training as a low-light enhancement model;

[0077] The image enhancement module 204 is configured to perform low-light enhancement on the test data set using the low-light enhancement model.

[0078] The low-light image enhancement system 200 can implement the steps in the low-light image enhancement method in the above embodiments, and can achieve the same technical effects. Refer to the description in the above embodiments, and details are not described herein again.

[0079] An embodiment of the present invention further provides a computer device. Please refer to Figure 7 , Figure 7 which is a schematic structural diagram of the computer device provided by the embodiment of the present invention. The computer device 300 includes: a memory 302, a processor 301, and a computer program stored on the memory 302 and executable on the processor 301.

[0080] The processor 301 calls the computer program stored in the memory 302 to execute the steps in the park management method provided by the embodiment of the present invention. Please refer to Figure 1 , specifically including:

[0081] S101. Obtain video data of a real scene.

[0082] S102. Perform frame splitting on the video data to obtain split-frame pictures, and screen out a test data set containing paired picture data from the split-frame pictures, and screen out a training data set containing pictures with different exposure degrees from the SICE data set.

[0083] S103. Iteratively train a preset enhancement model using the training data set, and save the preset enhancement model that has completed the iterative training as a low-light enhancement model.

[0084] S104. Perform low-light enhancement on the test data set using the low-light enhancement model.

[0085] Furthermore, the preset enhancement model includes four groups of DCE-Net modules with skip connections, an iterative parameter module, and a curve iteration module, where:

[0086] The DCE-Net module takes image data as input and outputs 8 groups of parameters to be saved in the iterative parameter module;

[0087] Each parameter in the iterative parameter module is used to determine a curve shape, and the pixels on the RGB channels in the image data adjust the original RGB pixel values according to the curve shape;

[0088] The curve iteration module takes the 8 sets of iteration parameters in the iteration parameter module as input, and calculates the high-order brightness enhancement curve LE through iteration and pixelization.

[0089] Furthermore, the DCE-Net module contains eight convolutional layers connected in sequence. Among them, each convolutional layer in the first to fourth layers contains a convolution of size 32*32, and each convolutional layer in the fifth to eighth layers contains two convolutions of size 32*32. Moreover, the first convolutional layer is skip-connected to the eighth convolutional layer, the second convolutional layer is skip-connected to the seventh convolutional layer, the third convolutional layer is skip-connected to the sixth convolutional layer, and the fourth convolutional layer is skip-connected to the fifth convolutional layer.

[0090] Furthermore, the high-order brightness enhancement curve LE in the curve iteration module satisfies the following relational expression (1):

[0091] LE n (I (x) ; r) = LE n-1 + r * LE n-1 (1 - LE n-1 ) (1)

[0092] where, I (x) represents the input image data, LE n (I (x) ; r) represents the nth enhanced estimate of I (x) , r represents the iteration parameter, n represents the number of iterations, and the value of r satisfies the following relational expression (2):

[0093] r ∈ (-1, 1) (2).

[0094] Furthermore, the loss function used by the preset enhancement model during iterative training includes a basic loss Loss and a mutual consistency loss function L mc , where the basic loss Loss includes a spatial consistency loss, an exposure control loss, a color constancy loss, and a lighting smoothness loss.

[0095] Furthermore, the mutual consistency loss function L mc satisfies the following relational expression (3):

[0096] L mc = ∥M * exp(-c * M)∥ 1 (3)

[0097] where, M represents the square of the gradient of the image data and its corresponding enhanced estimate on the plane coordinate axis, and c represents a preset penalty factor.

[0098] Furthermore, during the iterative training process of the preset enhancement model, the preset penalty factor c starts from a preset value to make the mutual consistency loss function L mc gradually increase as M increases, and gradually decrease to 0 after the mutual consistency loss function L mc reaches the maximum value. The number of iterative training times of the preset enhancement model is at least 200 times.

[0099] The computer device 300 provided by the embodiment of the present invention can implement the steps in the low-light image enhancement method in the above embodiment and can achieve the same technical effect. Refer to the description in the above embodiment, and details are not described here again.

[0100] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements each process and step in the low-light image enhancement method provided by the embodiment of the present invention and can achieve the same technical effect. To avoid repetition, details are not described here again.

[0101] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0102] It should be noted that in this article, the term "comprising", "including" or any other variant thereof is intended to cover a non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of another identical element in the process, method, article or device including the element.

[0103] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described method of the embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions for causing a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0104] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. What is disclosed is only the preferred embodiments of the present invention. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many equivalent changes in form without departing from the purpose of the present invention and the scope protected by the claims, and all of them fall within the protection scope of the present invention.

Claims

1. A low-light image enhancement method, characterized in that, the low-light image enhancement method is based on the DCE model, and it includes the following steps: Obtain video data of a real scene; Perform frame splitting on the video data to obtain split-frame pictures, screen out a test data set containing paired picture data from the split-frame pictures, and screen out pictures with different exposure levels from the SICE data set as the training data set; Iteratively train a preset enhancement model using the training data set, and save the preset enhancement model that has completed the iterative training as a low-light enhancement model; Perform low-light enhancement on the test data set using the low-light enhancement model; Among them, the preset enhancement model includes four groups of DCE-Net modules with skip connections, an iterative parameter module, and a curve iteration module: The DCE-Net module takes image data as input and outputs 8 groups of parameters to be saved in the iterative parameter module; Each parameter in the iterative parameter module is used to determine a curve shape, and the pixels on the RGB channels in the image data adjust the original RGB pixel values according to the curve shape; Define the high-order brightness enhancement curve as LE. The curve iteration module takes the 8 groups of iterative parameters in the iterative parameter module as input and calculates the high-order brightness enhancement curve LE through iterative and pixelization methods; The DCE-Net module contains eight convolutional layers connected in sequence. Among them, each convolutional layer in the first to fourth layers contains a group of convolutions with a size of 32*32, and each convolutional layer in the fifth to eighth layers contains two groups of convolutions with a size of 32*32. Moreover, the first convolutional layer is skip-connected to the eighth convolutional layer, the second convolutional layer is skip-connected to the seventh convolutional layer, the third convolutional layer is skip-connected to the sixth convolutional layer, and the fourth convolutional layer is skip-connected to the fifth convolutional layer; The loss function used by the preset enhancement model during iterative training includes a basic loss Loss and a mutual consistency loss function L mc , where the basic loss Loss includes a spatial consistency loss, an exposure control loss, a color constancy loss, and a light smoothness loss; The mutual consistency loss function L mc satisfies the following relation (3): L mc = ∥M * exp(-c * M)∥ 1 (3) Among them, M represents the square of the gradient of the image data and its corresponding enhancement estimate on the plane coordinate axis, and c represents a preset penalty factor.

2. The low-light image enhancement method according to claim 1, characterized in that, the high-order brightness enhancement curve LE in the curve iteration module satisfies the following relational formula (1): LE n (I (x) ; r) = LE n-1 + r * LE n-1 (1 - LE n-1 ) (1) Among them, I (x) represents the input image data, LE n (I (x) ; r) represents the nth enhanced estimate of I (x) The value of r satisfies the following relational expression (2): r∈(-1,1)(2)。 3. The low-light image enhancement method according to claim 1, characterized in that, During the iterative training process of the preset enhancement model, the preset penalty factor c starts from a preset value to make the mutual consistency loss function L mc gradually increase as M increases, and when the mutual consistency loss function L mc reaches the maximum value, it is gradually decreased to 0. The number of iterative training times of the preset enhancement model is at least 200 times.

4. A low-light image enhancement system, characterized in that, includes: A data acquisition module for obtaining video data of a real scene; A data processing module for performing frame splitting on the video data to obtain split-frame pictures, screening out a test data set containing paired picture data from the split-frame pictures, and screening out pictures with different exposure levels from the SICE data set as the training data set; A model training module for iteratively training a preset enhancement model using the training data set, and saving the preset enhancement model that has completed the iterative training as a low-light enhancement model; An image enhancement module for performing low-light enhancement on the test data set using the low-light enhancement model; Among them, the preset enhancement model includes four groups of DCE-Net modules with skip connections, an iterative parameter module, and a curve iteration module: The DCE-Net module takes image data as input and outputs 8 groups of parameters to be saved in the iterative parameter module; Each parameter in the iterative parameter module is used to determine a curve shape, and the pixels on the RGB channels in the image data adjust the original RGB pixel values according to the curve shape; Define the high-order brightness enhancement curve as LE. The curve iteration module takes the 8 groups of iterative parameters in the iterative parameter module as input and calculates the high-order brightness enhancement curve LE through iteration and pixelization methods; The DCE-Net module includes eight convolutional layers connected in sequence. Among them, each convolutional layer in the first to fourth layers includes a group of convolutions with a size of 32*32, and each convolutional layer in the fifth to eighth layers includes two groups of convolutions with a size of 32*32. Moreover, the first convolutional layer is skip-connected to the eighth convolutional layer, the second convolutional layer is skip-connected to the seventh convolutional layer, the third convolutional layer is skip-connected to the sixth convolutional layer, and the fourth convolutional layer is skip-connected to the fifth convolutional layer; The loss function used by the preset enhancement model during iterative training includes a basic loss Loss and a mutual consistency loss function L mc , where the basic loss Loss includes a spatial consistency loss, an exposure control loss, a color constancy loss, and a light smoothness loss; The mutual consistency loss function L mc satisfies the following relational expression (3): L mc = ∥M * exp(-c * M)∥ 1 (3) Among them, M represents the square of the gradient of the image data and its corresponding enhancement estimate on the plane coordinate axis, and c represents a preset penalty factor.

5. A computer device, characterized in that it includes: a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the low-light image enhancement method described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements the steps in the low-light image enhancement method described in any one of claims 1 to 3.

Citation Information

Patent Citations

  • Low-illumination image enhancement method without normal illumination reference based on HSV color space and Retinex theory

    CN112465727A

  • Low-illumination image brightness enhancement and super-resolution method based on double-channel codec

    CN112614061A