Low-light image enhancement model, method, electronic device, and storage medium
By combining fully connected neural networks and the unfolding algorithm, the problem of unsatisfactory image effects in low-light image enhancement is solved, achieving better noise suppression and detail preservation, and improving the image enhancement effect.
Patent Information
- Application Number
- CN202210135560.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-02-14
- Publication Date
- 2025-12-12
- Estimated Expiration
- 2042-02-14
AI Technical Summary
Existing low-light image enhancement techniques are not ideal in terms of image enhancement results, especially in fields such as urban traffic and surveillance videos. Deep learning methods lack interpretability and ignore the prior assumptions of traditional iterative optimization algorithms, resulting in over-smoothing of images and residual noise.
A low-light image enhancement model is adopted, including an initialization module, an optimization module, an illumination adjustment module, and an image reconstruction module. The initialization decomposition is performed through a fully connected neural network, the unfolding algorithm is used for alternating iterative optimization, and the illumination adjustment is combined with the channel self-attention mechanism and gamma correction technology to ensure the image reconstruction effect.
It improves the robustness and flexibility of low-light image enhancement models, effectively suppresses noise and preserves detail information, thus enhancing image enhancement results.
Smart Images
Figure CN114638749B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of image processing, and particularly relates to a low-light image enhancement model and method, an electronic device and a storage medium. BACKGROUND
[0002] Visual perception is one of the most important ways for human beings to understand the world. With the rapid development of society, people's demand for information is also increasing. In real-world applications, unpredictable situations often cause image quality degradation. Among these degradation situations, low-light images are always one of the hot issues in computer society, especially in urban traffic, monitoring video, medical assistance and other fields. Low-light images are often caused by insufficient light intensity or too short exposure time, resulting in small overall pixel intensity and low contrast, making a large number of details invisible, and seriously affecting visual experience and algorithm processing accuracy.
[0003] In existing low-light image enhancement techniques, although most deep learning algorithms combine traditional Retinex theory and deep learning methods, the network is directly used to solve the intermediate variable in the solving process, ignoring the complex prior assumptions and iterative processes in traditional image processing. Although deep learning methods have the advantage of fast reasoning speed, the lack of interpretability limits the development of deep learning technology to some extent. In order to speed up the convergence of traditional iterative optimization algorithms, as early as 2010, famous scholars Lecun et al. designed an iterative solving framework integrating deep learning. The idea of this framework is that the nonlinear operator and linear matrix in each iteration of the traditional iterative algorithm can be replaced by the activation function and neural network in the deep learning neighborhood respectively, and through data-driven methods, the entire optimization process is more adaptive to the true distribution of the data, without relying on specific prior assumptions, and has stronger robustness, achieving better results with fewer iterations. After that, this kind of unfolding algorithm (replaced by unfolding algorithm in the following) is gradually applied to image super-resolution reconstruction, image denoising, pulse compression and other neighborhoods. Recently, Liu et al. applied the unfolding algorithm to low-light enhancement for illumination estimation and denoising. Although this work provides a new solution for unsupervised low-light enhancement, the separation of illumination and noise leads to the risk of over-smoothing and noise residue after enhancement, and in addition, due to the neglect of the reflection layer, the model is prone to overexposure in most cases, and the image enhancement effect is not ideal. SUMMARY
[0004] The purpose of the present application is to provide a low-light image enhancement model and method, an electronic device and a storage medium, which aims to solve the problem of unsatisfactory image enhancement effect in the prior art.
[0005] In one aspect, the present application provides a low-light image enhancement model, comprising an initialization module, an optimization module, a light adjustment module and an image reconstruction module connected in sequence, wherein,
[0006] The initialization module is configured to perform initialization decomposition on an input image to obtain an initialization light layer and an initialization reflection layer corresponding to the input image.
[0007] The optimization module is configured to perform a plurality of times of alternating iterative optimization on the initialization light layer and the initialization reflection layer by using an unfolding algorithm to obtain an optimized light layer and an optimized reflection layer.
[0008] The light adjustment module is configured to perform light adjustment on the optimized light layer to obtain a target light layer.
[0009] The image reconstruction module is configured to perform image reconstruction according to the target light layer and the optimized reflection layer to obtain a target light image.
[0010] Preferably, the initialization module is a fully connected neural network.
[0011] Preferably, the fully connected neural network is a fully convolutional neural network comprising four convolutional layers.
[0012] Preferably, the loss function used by the initialization module during training comprises a fidelity term and a prior term, the fidelity term is used to measure the closeness of an initialization image composed of an initialization light layer and an initialization reflection layer of a training sample to the training sample, and the prior term is used to measure the closeness of the initialization light layer of the training sample to the maximum values of R, G and B channels of the training sample.
[0013] Preferably, the loss function used by the initialization module during training is:
[0014]
[0015] wherein L init represents the loss of the initialization module, I represents the training sample, R0 represents the initialization reflection layer of the training sample, L0 represents the initialization light layer of the training sample, μ is a constant, and R, G and B represent red, green and blue channels, respectively.
[0016] Preferably, the optimization module comprises a variable calculation sub-network, a reflection layer repairing network and an illumination layer repairing network, the variable calculation sub-network is configured to calculate first and second intermediate variables after the current iteration optimization when the current iteration optimization is performed, the reflection layer repairing network is configured to obtain an optimized reflection layer after the current iteration optimization based on the first and second intermediate variables after the current iteration optimization, and the illumination layer repairing network is configured to obtain an optimized illumination layer after the current iteration optimization based on the second intermediate variable after the current iteration optimization.
[0017] Preferably, the variable calculation sub-network is configured to calculate the first and second intermediate variables after the current iteration optimization by using a least square method.
[0018] Preferably, the reflection layer repairing network is configured to perform convolution operation on the first and second intermediate variables after the current iteration optimization to obtain first and second intermediate feature maps, perform concatenation operation on the first and second intermediate feature maps to obtain a spliced feature map, perform channel attention calculation on the spliced feature map by using a channel self-attention mechanism to obtain a re-weighted feature map, obtain a noise distribution of the re-weighted feature map, and obtain the optimized reflection layer after the current iteration optimization based on the noise distribution and the first intermediate feature map.
[0019] Preferably, the illumination adjustment module comprises an adjustment factor expansion sub-module, a splicing sub-module and a brightness adjustment network connected in sequence, wherein,
[0020] The adjustment factor expansion sub-module is configured to expand a preset adjustment scale factor into a matrix with the same size as the optimized illumination layer.
[0021] The splicing sub-module is configured to splice the matrix and the optimized illumination layer to obtain a splicing result.
[0022] The brightness adjustment network is configured to perform brightness adjustment on the optimized illumination layer based on the splicing result to obtain the target illumination layer.
[0023] Preferably, the brightness adjustment network has the same network structure as the initialization module, and a size of a convolution kernel of a convolution layer of the brightness adjustment network is greater than a size of a convolution kernel of a convolution layer of the initialization module.
[0024] Preferably, the loss function employed by the brightness adjustment network during training comprises one or more combinations of a gradient level fidelity term, a color level fidelity term, and a structure level fidelity term, wherein the gradient level fidelity term is used to measure the horizontal or vertical gradient distance between the optimized illumination layer and the target illumination layer of the training sample, the color level fidelity term is used to measure the reconstruction loss of the target illumination image and the reference image of the training sample, and the structure level fidelity term is used to measure the distance between the target illumination image and the reference image of the training sample.
[0025] Preferably, the loss function employed by the brightness adjustment network during training is:
[0026]
[0027] wherein L adjust denotes the loss of the brightness adjustment network, denotes the gradient of the optimized illumination layer of the training sample in the horizontal or vertical direction, denotes the gradient of the target illumination layer of the training sample in the horizontal or vertical direction, I ref denotes the reference image, and R denotes the optimized reflectance layer of the training sample, denotes the target illumination layer of the training sample, and SSIM denotes the image quality loss function.
[0028] In another aspect, the present application provides a low-light image enhancement method based on the above low-light image enhancement model, the method comprising the following steps:
[0029] The input image is initialized and decomposed by the initialization module to obtain an initialized illumination layer and an initialized reflectance layer corresponding to the input image;
[0030] The initialized illumination layer and the initialized reflectance layer are iteratively optimized several times by the optimization module using an unfolding algorithm to obtain an optimized illumination layer and an optimized reflectance layer;
[0031] The optimized illumination layer is adjusted by the illumination adjustment module to obtain a target illumination layer;
[0032] The target illumination image is obtained by the image reconstruction module according to the target illumination layer and the optimized reflectance layer.
[0033] In another aspect, the present application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above method.
[0034] In another aspect, the present application also provides a computer readable storage medium, which stores a computer program, and the computer program, when executed by a processor, implements the steps of the above method.
[0035] The low-light image enhancement model comprises an initialization module, an optimization module, a light adjustment module and an image reconstruction module connected in sequence, wherein the initialization module is configured to initialize and decompose an input image to obtain an initialization light layer and an initialization reflection layer corresponding to the input image, the optimization module is configured to perform alternating iterative optimization on the initialization light layer and the initialization reflection layer for several times by using an unfolding algorithm to obtain an optimized light layer and an optimized reflection layer, the light adjustment module is configured to adjust the optimized light layer to obtain a target light layer, and the image reconstruction module is configured to perform image reconstruction according to the target light layer and the optimized reflection layer to obtain a target light image, so that the robustness of the low-light image enhancement model is improved while the flexibility and interpretability of the low-light image enhancement model are ensured. BRIEF DESCRIPTION OF DRAWINGS
[0036] Figure 1A is a structural schematic diagram of the low-light image enhancement model provided by the embodiment one of the present application;
[0037] Figure 1B is a working principle schematic diagram of the reflection layer repair network provided by the embodiment one of the present application;
[0038] Figure 1C is a working principle schematic diagram of the low-light image enhancement model provided by the embodiment one of the present application;
[0039] Figure 2A is a visualization result of the enhancement effect of the 10 different low-light image enhancement models provided by the embodiment two of the present application in the SICE data set;
[0040] Figure 2B is a visualization result of the enhancement effect of the 10 different low-light image enhancement models provided by the embodiment two of the present application in the LOL data set;
[0041] Figure 3 is an implementation flowchart of the low-light image enhancement method provided by the embodiment three of the present application; and
[0042] Figure 4 is a structural schematic diagram of the electronic device provided by the embodiment four of the present application. DETAILED DESCRIPTION
[0043] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.
[0044] The specific implementation of the present application is described in detail below in combination with specific embodiments:
[0045] Example One:
[0046] Figure 1A The structure of the low-light image enhancement model provided by the embodiment of the present application is shown, which includes initialization module 11, optimization module 12, light adjustment module 13 and image reconstruction module 14 connected in turn; wherein the initialization module is used for initializing decomposition of the input image to obtain the initialization light layer and the initialization reflection layer corresponding to the input image; the optimization module is used for alternating iterative optimization of the initialization light layer and the initialization reflection layer for several times by using the unfolding algorithm to obtain the optimized light layer and the optimized reflection layer; the light adjustment module is used for light adjustment of the optimized light layer to obtain the target light layer; the image reconstruction module is used for image reconstruction according to the target light layer and the optimized reflection layer to obtain the target light image. The above-mentioned input image is a low-light image to be enhanced.
[0047] For the above-mentioned initialization module, considering that the variable initialization plays an important role for the iterative optimization algorithm (such as ADMM), the above-mentioned initialization module can use the commonly used all-zero initialization decomposition and random initialization to initialize the decomposition of the input image. In order to enable the initialization decomposition to provide the correct direction for the subsequent optimization, therefore, the initialization light layer and the initialization reflection layer should contain important information of the input image, assuming that the reflection layer is an RGB three-channel image, the three channels share the same light layer, then the maximum value of the three channels of the input image can be directly assigned to the light layer as the initialization light layer, according to the Retinex theory, the color of an object is composed of the reflectivity of its surface and the light intensity falling on the surface, therefore, there is the following imaging expression:
[0048] I=R·L (1)
[0049] Wherein, I represents the input image, R represents the reflection layer of the input image, L represents the light layer of the input image, therefore, the initialization reflection layer can be obtained based on the above-mentioned Retinex theory.
[0050] But the above-mentioned initialization decomposition method will amplify the difference of the three-channel values in the reflection layer, and further destroy the statistical characteristics of R, G and B. Therefore, preferably, the initialization module uses a fully connected network to realize the initialization decomposition of the light layer and the reflection layer, so as to make the initialization light layer and the initialization reflection layer contain more information, while avoiding the destruction of the statistical characteristics of R, G and B. Further preferably, the fully connected neural network is a fully convolutional neural network containing 4 convolutional layers, so as to improve the running efficiency of the low-light image enhancement model.
[0051] Specifically, the input of the initialization module is a low-light image, and the output is an initialization illumination layer and an initialization reflection layer corresponding to the input image, which is abstractly represented as:
[0052]
[0053] wherein R0 represents the initialization reflection layer, L0 represents the initialization illumination layer, denotes the initialization module, I represents the input image, and θ D denotes the parameters of the initialization module.
[0054] In order to achieve the purpose of preserving the effective information of the input image in the initialization decomposition, preferably, the loss function used by the initialization module in the training process includes a fidelity term and a priori term, so that the initialization illumination layer and the initialization reflection layer after the initialization decomposition preserve the effective information of the input image. The fidelity term is used to measure the closeness between the initialization image composed of the initialization illumination layer and the initialization reflection layer of the training sample and the training sample, so as to ensure that the initialization decomposition satisfies the Retinex theory, and the priori term is used to measure the closeness between the initialization illumination layer of the training sample and the maximum values of R, G and B channels of the training sample, so as to make the initialization illumination layer learn more structural information. The training sample is the input image in the training process of the low-light image enhancement model.
[0055] Preferably, the fidelity term in the loss function used by the initialization module in the training process is calculated by using the L1 norm, and the priori term is calculated by using the L2 norm, so as to ensure the training effect of the initialization module, so that the illumination layer and the reflection layer after the initialization decomposition preserve the effective information of the input image as much as possible. The loss function used by the initialization module in the training process is as follows:
[0056]
[0057] wherein L init denotes the loss of the initialization module, I represents the training sample (the input image in the training process of the model), R0 represents the initialization reflection layer of the training sample, L0 represents the initialization illumination layer of the training sample, μ is a constant, and R, G and B represent red, green and blue channels respectively.
[0058] For the above optimization module, according to the Retinex theory, the decomposition problem of the above imaging expression (1) is a pathological problem, so a prior model needs to be established to constrain R and L when solving the problem, which is represented as:
[0059]
[0060] where Φ(R) and Ψ(L) represent the prior assumption of the reflectance layer R and the illumination layer L, respectively. To solve the problem, two new variables, i.e., a first intermediate variable P and a second intermediate variable Q, are introduced, and formula (4) can be rewritten as:
[0061]
[0062] At this point, formula (5) can be reasonably divided into a fidelity term and a prior term to be solved independently, and the following expression is obtained:
[0063]
[0064]
[0065]
[0066]
[0067] where P k and Q k represent the solution results of the fidelity term at the kth iteration, R k and L k represent the solution results of the prior term at the kth iteration. When k = 1, P k and Q k represent the initialized reflectance layer and the initialized reflectance layer illumination layer.
[0068] Considering that the initialized reflectance layer is often full of noise, which seriously affects the display of important details. The initialized illumination layer not only retains the structural information completely, but also retains a large degree of redundant texture details. Therefore, the purpose of the optimization module in the embodiment is to repair the initialized reflectance layer, which can retain effective details completely while removing noise. A satisfactory illumination layer should be complete in structure and smooth in texture details. In order to achieve the above purpose, based on the framework of deep neural network, the unfolding algorithm is used to alternately iterate and optimize the fidelity term and the prior term in the above formulas (6)-(9). Based on the comprehensive consideration of time and performance, preferably, the number of times of alternately iterating and optimizing the optimization module is 3, so as to determine the number of times of alternately iterating and optimizing according to the actual experimental effect.
[0069] Preferably, the optimization module comprises a variable calculation sub-network, a reflection layer repairing network and an illumination layer repairing network, when performing the current iteration optimization, the variable calculation sub-network is configured to calculate the first intermediate variable and the second intermediate variable after the current iteration optimization, the reflection layer repairing network is configured to obtain the optimized reflection layer after the current iteration optimization based on the first intermediate variable and the second intermediate variable after the current iteration optimization, and the illumination layer repairing network is configured to obtain the optimized illumination layer after the current iteration optimization based on the second intermediate variable after the current iteration optimization, so as to replace the manual priori solving based on the traditional Retinex optimization algorithm, and make the optimization module learn more robust priori information from data. The reflection layer repairing network and the illumination layer repairing network are both constructed based on a neural network. Preferably, the variable calculation sub-network is configured to calculate the first intermediate variable and the second intermediate variable after the current iteration optimization by using a least square method, so as to realize the calculation of the intermediate variable. In the specific implementation, when performing the kth iteration optimization, the first intermediate variable P k-1 k-1 The above formula (6) is solved to obtain the first intermediate variable P k after the current iteration optimization, then the first intermediate variable P k after the current iteration optimization is fixed, and the optimized illumination layer L k-1 after the previous iteration optimization is obtained, the above formula (7) is solved to obtain the second intermediate variable Q k The above formulas (6)-(7) can be understood as a classic least square problem, and therefore the following closed loop solution can be obtained by derivation:
[0070]
[0071]
[0072] Wherein, k≥1, and k is a positive integer, and λ is a constant.
[0073] After P k and Q k are obtained, the above priori terms R k and L k are learned.
[0074] According to the experimental findings, the noise on the reflection layer and the brightness distribution on the illumination layer have a high correlation, that is, the higher the brightness, the less the noise, and vice versa. Preferably, the reflection layer repair network is used to perform convolution operation on the calculated first intermediate variable and the second intermediate variable to obtain the first intermediate feature map and the second intermediate feature map, and perform fusion processing on the first intermediate feature map and the second intermediate feature map to obtain a fusion feature map. Then, based on the noise distribution of the fusion feature map and the first intermediate feature map, the optimized reflection layer after the current iteration is obtained, so as to repair the reflection layer by combining the information of the illumination layer, realize the denoising of the reflection layer, and improve the learning effect of the optimized reflection layer.
[0075] Further preferably, the reflection layer repair network is used to perform convolution operation on the first intermediate variable and the second intermediate variable after the current iteration optimization to obtain the first intermediate feature map and the second intermediate feature map, perform concatenation operation on the first feature map and the second intermediate feature map to obtain a spliced feature map, perform channel attention calculation on the spliced feature map by using the channel self-attention mechanism to obtain a re-weighted feature map, obtain the noise distribution of the re-weighted feature map, and obtain the optimized reflection layer after the current iteration optimization based on the noise distribution and the first intermediate feature map. In this way, the information can be better fused, and the calculation consumption is not too large, so as to ensure the efficiency of the low-light image enhancement model.
[0076] The optimized reflection layer Rk in the kth alternating iteration optimization in formula (8) k is abstractly represented as:
[0077]
[0078] wherein, θ R represents the parameters of the reflection layer repair network , P k represents the first intermediate variable in the kth alternating iteration optimization, and Q k represents the second intermediate variable in the kth alternating iteration optimization.
[0079] The optimized illumination layer Lk in the kth alternating iteration optimization in formula (9) k is abstractly represented as:
[0080]
[0081] wherein, θ L represents the parameters of the illumination layer repair network .
[0082] Figure 1B is a schematic diagram of the working principle of the reflection layer repair network in the embodiment. Figure 1BIn the specific implementation, the reflection layer repairing network includes a channel self-attention module and a noise extraction module, Conv represents convolution, C represents concatenation, Average pooling represents average pooling, FC represents full connection, represents element-wise multiplication, and represents denoising. The working principle of the reflection layer repairing network is as follows: first, the first intermediate variable P k and the second intermediate variable Q k are subjected to convolution to obtain the first intermediate feature map M1 and the second intermediate feature map M2. Then, the first intermediate feature map M1 and the second intermediate feature map M2 are subjected to concatenation to obtain the spliced feature map M3. The spliced feature map M3 is subjected to channel attention calculation by the channel self-attention module to obtain the reweighted feature map M4. The noise distribution M5 of the reweighted feature map M4 is obtained by the noise extraction module. The first intermediate feature map M1 is denoised based on the noise distribution M5 to obtain the optimized reflection layer M6 after the current iteration optimization.
[0083] For the above-mentioned illumination adjustment module, in the Retinex theory, the reflection layer is an inherent attribute of an object and does not change with the illumination condition. Based on the theory, low-illumination imaging is caused by a low intensity of the illumination layer. Therefore, after the optimized illumination layer and the optimized reflection layer are obtained, the optimized reflection layer can be fixed and the optimized illumination layer can be adjusted. The illumination adjustment module can use a gamma correction technique to adjust the illumination of the illumination layer. Although the gamma correction technique can achieve different degrees of brightness enhancement by adjusting the parameters, it is difficult to determine the adjustment scale factor by adjusting the scale. Preferably, the illumination adjustment module is configured to adjust the illumination of the optimized illumination layer according to a preset adjustment scale factor, and the adjustment scale factor is specified by a user to generate a target illumination layer according to a user-specified brightness adjustment scale. In the specific implementation, the input of the illumination adjustment module is the optimized illumination layer and the user-specified adjustment scale factor, and the output is the high-light illumination layer under the target adjustment scale, that is, the target illumination layer.
[0084] Preferably, the illumination adjustment module includes an adjustment factor expansion sub-module, a splicing sub-module, and a brightness adjustment network connected in sequence. The adjustment factor expansion sub-module is configured to expand the preset adjustment scale factor into a matrix with the same size as the optimized illumination layer. The splicing sub-module is configured to splice the matrix and the optimized illumination layer to obtain a splicing result. The brightness adjustment network is configured to adjust the brightness of the optimized illumination layer based on the splicing result to obtain the target illumination layer, so as to adjust the illumination intensity. In the specific implementation, the adjustment scale factor is first expanded into a matrix with the same size as the illumination layer, and then spliced with the optimized illumination layer as the input of the brightness adjustment network after brightness adjustment. Specifically, it can be represented as follows:
[0085]
[0086] wherein ω represents an adjustment scale factor, θ A denotes a parameter of the brightness adjustment network L represents an optimized illumination layer.
[0087] Preferably, the brightness adjustment network has the same network structure as the initialization module, i.e., the brightness adjustment network can be a fully convolutional neural network comprising four convolutional layers, and the size of the convolution kernel of the convolutional layer of the brightness adjustment network is larger than that of the convolutional layer of the initialization module, so as to maintain consistency and limit the smoothness of the illumination layer.
[0088] In order to better train the brightness adjustment network, considering that the high-brightness illumination layer output by the brightness adjustment network should be consistent in structure with the low-brightness illumination layer input, preferably, the loss function used in the training of the brightness adjustment network comprises a gradient layer fidelity term, so as to ensure the training effect of the brightness adjustment network. The gradient layer fidelity term is used to measure the horizontal or vertical gradient distance between the optimized illumination layer of the training sample and the target illumination layer.
[0089] In order to reconstruct the image under normal illumination based on the target illumination layer, preferably, the loss function used in the training of the brightness adjustment network comprises a color layer fidelity term, so as to further improve the training effect of the brightness adjustment network. The color layer fidelity term is used to measure the reconstruction loss of the target illumination image of the training sample and the reference image, so that the reconstructed image is an image under normal illumination, i.e., the reference image.
[0090] In order to make the reconstructed image consistent with the reference image in structure, brightness and contrast, preferably, the loss function used in the training of the brightness adjustment network comprises one or more combinations of the color layer fidelity term and the structure layer fidelity term, so as to further improve the training effect of the brightness adjustment network. The structure layer fidelity term is used to measure the distance between the target illumination image of the training sample and the reference image, so that the reconstructed image is consistent with the reference image in structure, brightness and contrast.
[0091] Preferably, the loss function used in the training of the brightness adjustment network comprises the gradient layer fidelity term, the color layer fidelity term and the structure layer fidelity term, so as to further improve the training effect of the brightness adjustment network through the constraints of the three aspects.
[0092] Further preferably, the gradient layer fidelity term is calculated by using the L1 norm, the color layer fidelity term is calculated by using the L2 norm, and the structure layer fidelity term is calculated by using the SSIM (image quality) loss, and the loss function used in the training of the brightness adjustment network is represented as:
[0093]
[0094] wherein, L adjust denotes the loss of the brightness adjustment network, denotes the gradient of the optimized illumination layer of the training sample in the horizontal or vertical direction, denotes the gradient of the target illumination layer of the training sample in the horizontal or vertical direction, I ref denotes the reference image, R denotes the optimized reflectance layer of the training sample, denotes the target illumination layer of the training sample, and SSIM denotes the image quality loss function.
[0095] After obtaining the target illumination layer, the image reconstruction module multiplies the target illumination layer and the optimized reflectance layer to perform image reconstruction to obtain a target illumination image.
[0096] Figure 1C A working principle schematic diagram of the low-illumination image enhancement model provided by the embodiment of the present application is shown in FIG. 1. Figure 1C In the embodiment, the input image I is decomposed by the initialization module to obtain the initialization illumination layer L0 and the initialization reflectance layer R0, and then the optimized illumination layer and the optimized reflectance layer are output after T times of alternating iterative optimization training by the optimization module, the optimized illumination layer is input to the illumination adjustment module, the adjustment scale factor ω is spliced with the optimized illumination layer, and the spliced feature map is input to the brightness adjustment network to perform brightness adjustment to obtain the target illumination layer After that, the target illumination layer and the optimized reflectance layer are used for image reconstruction, and the enhanced image, i.e., the target illumination image, is input.
[0097] In the embodiment of the present application, the low-illumination image enhancement model comprises an initialization module, an optimization module, an illumination adjustment module and an image reconstruction module connected in sequence, wherein the initialization module is used for initializing and decomposing the input image to obtain the initialization illumination layer and the initialization reflectance layer corresponding to the input image, the optimization module is used for performing several times of alternating iterative optimization on the initialization illumination layer and the initialization reflectance layer by using the unfolding algorithm to obtain the optimized illumination layer and the optimized reflectance layer, the illumination adjustment module is used for performing illumination adjustment on the optimized illumination layer to obtain the target illumination layer, and the image reconstruction module is used for performing image reconstruction according to the target illumination layer and the optimized reflectance layer to obtain the target illumination image, so as to improve the robustness of the low-illumination image enhancement model while ensuring the flexibility and interpretability of the low-illumination image enhancement model, and to retain the detail information while suppressing the noise of the low-illumination image enhancement model.
[0098] In the embodiments of the present application, each unit / module of the low-light image enhancement module can be implemented by a corresponding hardware or software unit, and each unit / module can be an independent software or hardware unit / module, or can be integrated into a software or hardware unit / module, which does not limit the present application.
[0099] Example Two:
[0100] The present embodiment further illustrates the low-light enhancement model described in Experimental Example One in combination with Experimental Examples.
[0101] This experimental example subjectively and objectively evaluates the unfolding-based low-light image enhancement model described in Experimental Example One on two public low-light image enhancement test sets. The two representative data sets are LOL and SICE, respectively. This experimental example uses common reference indicators for evaluating image quality, namely Mean Absolute Error (MAE), Structural Similarity (SSIM), Peak Signal to Noise Ratio (PSNR), and Learned Perceptual Image Patch Similarity (LPIPS). A good model should have high PSNR and SSIM indicator scores, but low MAE and LPIPS scores. This experimental example compares the low-light image enhancement model proposed in Experimental Example One with some existing benchmark models, namely LIME, NPE, SRIE, RRM, LR3M, Retinex-Net, KinD, Zero-DCE, and RUAS.
[0102] The results of the model performance comparison are given in Table One and Table Two, and it is clear that the low-light image enhancement model (URetinex-Net) proposed in Experimental Example One achieves good performance on the LOL and SICE data sets compared with other benchmark models.
[0103] Table One Experimental evaluation of the low-light image enhancement model described in Experimental Example One and benchmark models on the LOL database test set
[0104]
[0105]
[0106] Table Two Experimental evaluation of the low-light image enhancement model described in Experimental Example One and benchmark models on the SICE database test set
[0107]
[0108] Table I gives the experimental results of different low-light image enhancement models on the test set of LOL database. From it, it can be seen that the low-light image enhancement model (URetinex-Net) proposed in embodiment I obtains good performance. Compared with the traditional Retinex optimization model based on manual priori, the model of the present experimental example shows very superior effect on all indicators, which shows that the optimization module described in embodiment I can learn more robust priori rules from data. Compared with other deep learning based methods, the present experimental example is only slightly worse than KinD (0.0832 vs 0.0804) in the MAE index, and the difference is very small. But in other indicators (PSNR, SSIM and LPIPS), the low-light image enhancement model (URetinex-Net) proposed in embodiment I is significantly better than other models, which further shows the advantage of the enhancement mode based on iterative alternating optimization proposed in the present application.
[0109] In addition, in order to verify the generalization ability of the low-light image enhancement model proposed in embodiment I, the model trained on the LOL dataset is evaluated on the SICE dataset, and the comparison results are given in Table I. From Table I, it can be seen that the MAE, PSNR and SSIM index scores of the low-light image enhancement model (URetinex-Net) proposed in embodiment I are significantly better than those of other benchmark models. For the same dataset, the low-light image enhancement model proposed in embodiment I shows better noise suppression ability and ability to retain image structure information. This is enough to show that the low-light image enhancement model proposed in embodiment I has strong generalization ability, and can achieve good results even in scenes not appearing in the training set.
[0110] The experimental example in Figure 2A , 2B visually gives some test results of these low-light image enhancement models on the LOL and SICE datasets, including the low-light image enhancement model (Ours) proposed in embodiment I, and the ground truth is shown in the last row and the last column. Figure 2A , 2B The low-light image enhancement model provided in embodiment I is compared with LIME, NPE, SRIE, RRM, LR3M, Retinex-Net, KinD, Zero-DCE and RUAS model in Figure 2B . As shown in Figure 2B , the low-light enhancement model (Ours) proposed in embodiment I performs well in some challenging cases. For example, Figure 2BIt can be seen that the brightness in the source image (Input) is very low, and if only the contrast is improved, a large amount of noise will be introduced (such as LIME, NPE, SRIE, Retinex-Net and Zero-DCE), which seriously interferes with important texture details. While considering the noise algorithm (such as RRM, LR3M, KinD and RUAS), although the noise is significantly reduced, important details are lost due to excessive smoothing. In contrast, the low-light image enhancement model proposed in the application can not only remove noise sufficiently, but also retain important texture details. From Figure 2A It can be seen that the low-light image enhancement model (Ours) proposed in Embodiment One performs well in color fidelity, noise suppression or exposure.
[0111] Example Three:
[0112] Embodiment Three of the application is based on the low-light image enhancement model described in Embodiment One, Figure 3 The implementation process of the low-light image enhancement method provided by Embodiment Three of the application is shown, only the part related to the embodiment of the application is shown for the convenience of description, and the details are as follows:
[0113] In step S301, the input image is initialized and decomposed by the initialization module to obtain an initialized illumination layer and an initialized reflection layer corresponding to the input image.
[0114] In the embodiment of the application, the input image described above is a low-light image to be subjected to image enhancement. Considering that the variable initialization plays an important role for the iterative optimization algorithm (for example, ADMM), the commonly used all-zero initialization decomposition and random initialization can be used to initialize and decompose the input image. In order to enable the initialization decomposition to provide a correct direction for the subsequent optimization, therefore, the initialized illumination layer and the initialized reflection layer should contain important information of the input image. Assuming that the reflection layer is an RGB three-channel image, the three channels share the same illumination layer, then the maximum value of the three channels of the input image can be directly assigned to the illumination layer as the initialized illumination layer. According to the Retinex theory, the color of an object is composed of the reflectivity of its surface and the illumination intensity falling on the surface, therefore, there is the following imaging expression:
[0115] I=R·L (1)
[0116] Where I represents the input image, R represents the reflection layer of the input image, and L represents the illumination layer of the input image, therefore, the initialized reflection layer can be obtained based on the above Retinex theory.
[0117] But the above initialization decomposition method will amplify the difference of three-channel values in the reflection layer, and further destroy the statistical characteristics of R, G, B. Therefore, preferably, the initialization module adopts a fully connected network to realize the initialization decomposition of the illumination layer and the reflection layer, so as to make the initialized illumination layer and the initialized reflection layer contain more information, while avoiding the destruction of the statistical characteristics of R, G, B. Further preferably, the fully connected neural network is a fully convolutional neural network containing 4 convolutional layers, so as to improve the running efficiency of the low-light image enhancement model.
[0118] Specifically, the input of the initialization module is a low-light image, and the output is the initialized illumination layer and the initialized reflection layer corresponding to the input image, which is abstractly represented as:
[0119]
[0120] Wherein, R0 represents the initialized reflection layer, L0 represents the initialized illumination layer, represents the initialization module, I represents the input image, θ D represents the parameters of the initialization module.
[0121] In order to achieve the purpose of preserving the effective information of the input image in the initialization decomposition, preferably, the loss function adopted by the initialization module during training includes a fidelity term and a priori term, so that the initialized illumination layer and the initialized reflection layer after initialization decomposition preserve the effective information of the input image. Among them, the fidelity term is used to measure the closeness between the initialization image composed of the initialized illumination layer and the initialized reflection layer of the training sample and the training sample, so as to ensure that the initialization decomposition meets the Retinex theory, and the priori term is used to measure the closeness between the initialized illumination layer of the training sample and the maximum value of R, G, B three channels of the training sample, so as to make the initialized illumination layer learn more structural information. Among them, the training sample is the input image during the training of the low-light image enhancement model.
[0122] Preferably, the fidelity term in the loss function adopted by the initialization module during training is calculated by L1 norm, and the priori term is calculated by L2 norm, so as to ensure the training effect of the initialization module, so that the illumination layer and the reflection layer after initialization decomposition preserve the effective information of the input image as much as possible. The loss function formula adopted by the initialization module during training is as follows:
[0123]
[0124] Wherein, L init represents the loss of the initialization module, I represents the training sample (the input image during the training of the model), R0 represents the initialized reflection layer of the training sample, L0 represents the initialized illumination layer of the training sample, μ is a constant, and R, G, B represent red, green and blue channels respectively.
[0125] In step S302, the initialization illumination layer and the initialization reflection layer are iteratively optimized by the optimization module using an unfolding algorithm to obtain an optimized illumination layer and an optimized reflection layer.
[0126] In the embodiment of the present application, according to the Retinex theory, the decomposition problem of the above imaging expression (1) is a pathological problem, and therefore a prior model needs to be established to constrain R and L when solving the problem, which is expressed as:
[0127]
[0128] Wherein, Φ(R) and Ψ(L) represent the prior assumptions of the reflection layer R and the illumination layer L respectively. In order to solve the problem, two new variables, i.e. a first intermediate variable P and a second intermediate variable Q, are introduced, and formula (4) can be rewritten as:
[0129]
[0130] At this point, formula (3) can be reasonably divided into a fidelity term and a prior term to be solved independently, and the following expression is obtained:
[0131]
[0132]
[0133]
[0134]
[0135] Wherein, P k and Q k represent the fidelity term solving results at the kth iteration, R k and L k represent the prior term solving results at the kth iteration. When k=1, P k and Q k represent the initialization reflection layer and the initialization reflection layer illumination layer.
[0136] Considering that the initialization reflection layer is often full of noise, which seriously affects the display of important details. The initialization illumination layer not only retains the structure information completely, but also retains a large degree of redundant texture details. Therefore, the purpose of the optimization module in the embodiment is to repair the initialization reflection layer, which can retain effective details completely while removing noise. The satisfactory illumination layer should be complete in structure and smooth in texture details. In order to achieve the above purpose, the fidelity term and the prior term in the above formula (6)-(9) can be alternately iterated and optimized based on the framework of deep neural network by using unfolding algorithm. Based on the comprehensive consideration of time and performance, preferably, the number of times of alternately iterated and optimized by the optimization module is 3, so as to determine the number of times of alternately iterated and optimized according to the actual experimental effect.
[0137] Preferably, the optimization module comprises a variable calculation sub-network, a reflection layer repair network and an illumination layer repair network. When performing the current alternately iterated and optimized, the variable calculation sub-network is used to calculate the first intermediate variable and the second intermediate variable after the current iterated and optimized, the reflection layer repair network is used to obtain the optimized reflection layer after the current iterated and optimized based on the first intermediate variable and the second intermediate variable after the current iterated and optimized, and the illumination layer repair network is used to obtain the optimized illumination layer after the current iterated and optimized based on the second intermediate variable after the current iterated and optimized, so as to replace the manual prior solving based on the traditional Retinex optimization algorithm, so that the optimization module learns more robust prior information from data. The reflection layer repair network and the illumination layer repair network are both constructed based on neural network. Preferably, the variable calculation sub-network is used to calculate the first intermediate variable and the second intermediate variable after the current iterated and optimized by using the least square method, so as to realize the calculation of the intermediate variable. In the specific implementation, when performing the kth iterated and optimized, the first intermediate variable P k-1 k-1 and the second intermediate variable Q k after the current iterated and optimized can be solved by the above formula (6), then the first intermediate variable P k after the current iterated and optimized is fixed, and the optimized illumination layer L k-1 after the previous iterated and optimized is used to solve the above formula (7) to obtain the second intermediate variable Q k after the current iterated and optimized. The above formula (6)-(7) can be understood as a classic least square problem, so that the following closed loop solution can be obtained by derivation:
[0138]
[0139]
[0140] Wherein, k≥1, and k is a positive integer, and λ is a constant.
[0141] After obtaining P k and Qk After that, the prior term R k and L k are learned.
[0142] According to the experimental findings, there is a high correlation between the noise on the reflection layer and the brightness distribution on the illumination layer, that is, the higher the brightness, the less the noise, and vice versa. Preferably, the reflection layer repair network is used to perform convolution operations on the calculated feature maps of the first intermediate variable and the second intermediate variable to obtain the first intermediate feature map and the second intermediate feature map, and the first intermediate feature map and the second intermediate feature map are fused to obtain a fused feature map, and then based on the noise distribution of the fused feature map and the first intermediate feature map, the optimized reflection layer after the current iteration is obtained, thereby repairing the reflection layer by combining the information of the illumination layer, achieving denoising of the reflection layer and improving the learning effect of the optimized reflection layer.
[0143] Further preferably, the reflection layer repair network is used to perform convolution operations on the first intermediate variable and the second intermediate variable after the current iteration optimization to obtain the first intermediate feature map and the second intermediate feature map, and the first feature map and the second intermediate feature map are concatenated to obtain a spliced feature map, and the channel attention mechanism is used to perform channel attention calculation on the spliced feature map to obtain a re-weighted feature map, the noise distribution of the re-weighted feature map is obtained, and the optimized reflection layer after the current iteration optimization is obtained based on the noise distribution and the first intermediate feature map, so that the information can be better fused without causing too much calculation consumption, thereby ensuring the efficiency of the low-light image enhancement model.
[0144] The optimized reflection layer R k in the kth alternating iteration optimization is represented abstractly as:
[0145]
[0146] where θ R represents the parameters of the reflection layer repair network , P k represents the first intermediate variable of the kth alternating iteration optimization, and Q k represents the second intermediate variable of the kth alternating iteration optimization.
[0147] The optimized illumination layer L k in the kth alternating iteration optimization is represented abstractly as:
[0148]
[0149] where θ L represents the parameters of the illumination layer repair network .
[0150] In step S303, the optimized illumination layer is adjusted by the light adjustment module to obtain a target illumination layer.
[0151] In the embodiments of the present application, in the Retinex theory, the reflection layer is an inherent property of the object and does not change with the illumination condition. Based on this theory, the low-illumination image is caused by the low intensity of the illumination layer. Therefore, after obtaining the optimized illumination layer and the optimized reflection layer, the optimized reflection layer can be fixed, and the optimized illumination layer can be adjusted.
[0152] The light adjustment module can use the gamma correction technology to adjust the illumination layer. Although the gamma correction technology can achieve different degrees of brightness enhancement by adjusting the parameters, it is difficult to determine the adjustment scale factor by adjusting the scale. Preferably, the light adjustment module is configured to adjust the optimized illumination layer according to a preset adjustment scale factor specified by a user, so as to generate a target illumination layer according to the brightness adjustment scale specified by the user. In specific implementation, the input of the light adjustment module is the optimized illumination layer and the adjustment scale factor specified by the user, and the output is the high-light illumination layer under the target adjustment scale, i.e., the target illumination layer.
[0153] Preferably, the light adjustment module comprises an adjustment factor expansion submodule, a splicing submodule and a brightness adjustment network connected in sequence, wherein the adjustment factor expansion submodule is configured to expand the preset adjustment scale factor into a matrix with the same size as the optimized illumination layer, the splicing submodule is configured to splice the matrix with the optimized illumination layer to obtain a splicing result, and the brightness adjustment network is configured to adjust the brightness of the optimized illumination layer based on the splicing result to obtain the target illumination layer, so as to adjust the illumination intensity. In specific implementation, the adjustment scale factor is first expanded into a matrix with the same size as the illumination layer, and then spliced with the optimized illumination layer as the input of the brightness adjustment network, which can be specifically represented as:
[0154]
[0155] Wherein, ω represents the adjustment scale factor, θ A represents the parameters of the brightness adjustment network , and L represents the optimized illumination layer.
[0156] Preferably, the brightness adjustment network has the same network structure as the initialization module, i.e., the brightness adjustment network can be a full convolutional neural network comprising four convolutional layers, and the size of the convolution kernel of the convolutional layer of the brightness adjustment network is greater than that of the convolutional layer of the initialization module, so as to maintain consistency and limit the smoothness of the illumination layer.
[0157] To better train the brightness adjustment network, considering that the high brightness illumination layer output by the brightness adjustment network should be consistent with the low brightness illumination layer input in structure, preferably, the loss function used in the training of the brightness adjustment network includes a gradient layer fidelity term to ensure the training effect of the brightness adjustment network. The gradient layer fidelity term is used to measure the horizontal or vertical gradient distance between the optimized illumination layer of the training sample and the target illumination layer.
[0158] To reconstruct the image under normal illumination based on the target illumination layer, preferably, the loss function used in the training of the brightness adjustment network includes a color layer fidelity term to further improve the training effect of the brightness adjustment network. The color layer fidelity term is used to measure the reconstruction loss of the target illumination image of the training sample and the reference image, so that the reconstructed image is the image under normal illumination, i.e. the reference image.
[0159] To make the reconstructed image consistent with the reference image in structure, brightness and contrast, preferably, the loss function used in the training of the brightness adjustment network includes one or more combinations of the color layer fidelity term and the structure layer fidelity term to further improve the training effect of the brightness adjustment network. The structure layer fidelity term is used to measure the distance between the target illumination image of the training sample and the reference image, so that the reconstructed image is consistent with the reference image in structure, brightness and contrast.
[0160] Preferably, the loss function used in the training of the brightness adjustment network includes the gradient layer fidelity term, the color layer fidelity term and the structure layer fidelity term to further improve the training effect of the brightness adjustment network through three aspects of constraints.
[0161] Further preferably, the gradient layer fidelity term is calculated by L1 norm, the color layer fidelity term is calculated by L2 norm, and the structure layer fidelity term is calculated by SSIM (image quality) loss. The loss function used in the training of the brightness adjustment network is represented as:
[0162]
[0163] Wherein, L adjust represents the loss of the brightness adjustment network, represents the gradient of the optimized illumination layer of the training sample in the horizontal or vertical direction, represents the gradient of the target illumination layer of the training sample in the horizontal or vertical direction, I ref represents the reference image, and R represents the optimized reflectance layer of the training sample, represents the target illumination layer of the training sample, and SSIM represents the image quality loss function.
[0164] In step S304, the image is reconstructed according to the target illumination layer and the optimized reflectance layer by the image reconstruction module to obtain the target illumination image.
[0165] In this embodiment of the invention, the image reconstruction module multiplies the target illumination layer and the optimized reflection layer to perform image reconstruction and obtain the target illumination image.
[0166] In this embodiment of the invention, the input image is initialized and decomposed to obtain an initial illumination layer and an initial reflection layer corresponding to the input image. The initial illumination layer and the initial reflection layer are then optimized through several alternating iterations using an unfolding algorithm to obtain an optimized illumination layer and an optimized reflection layer. The optimized illumination layer is then adjusted to obtain a target illumination layer. Based on the target illumination layer and the optimized reflection layer, image reconstruction is performed to obtain the target illumination image. This approach ensures the flexibility and interpretability of the low-light image enhancement model while improving its robustness and allowing the low-light image enhancement model to retain detail information while suppressing noise.
[0167] Example Four:
[0168] Figure 4 The structure of the electronic device provided in Embodiment 4 of the present invention is shown. For ease of explanation, only the parts related to the embodiments of the present invention are shown.
[0169] The electronic device 4 of this embodiment includes a processor 40, a memory 41, and a computer program 42 stored in the memory 41 and executable on the processor 40. When the processor 40 executes the computer program 42, it implements the steps described in the above method embodiments, for example... Figure 3 The steps S301 to S304 are shown. Alternatively, when the processor 40 executes the computer program 42, it implements the functions of each module in the above-described low-light image enhancement model embodiment, for example... Figure 1A The functions of modules 11 to 14 are shown.
[0170] Example Five:
[0171] In this embodiment of the invention, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps in the method embodiments described above. For example... Figure 3 The steps S301 to S304 are shown. Alternatively, when the computer program is executed by a processor, it implements the functions of each module in the above-described low-light image enhancement model embodiment, for example... Figure 1A The functions of modules 11 to 14 are shown.
[0172] The computer-readable storage medium in embodiments of the present invention may include any entity or device capable of carrying computer program code, a recording medium, such as ROM / RAM, disk, optical disk, flash memory, etc.
[0173] The above description is only the preferred embodiment of the present application, and is not intended to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A low-light image enhancement device, characterized by, The low-light image enhancement device comprises an initialization module, an optimization module, a light adjustment module and an image reconstruction module connected in sequence, wherein The initialization module is configured to perform initialization decomposition on an input image to obtain an initialization light layer and an initialization reflection layer corresponding to the input image; The optimization module is configured to perform several times of alternating iterative optimization on the initialization light layer and the initialization reflection layer by using an unfolding algorithm to obtain an optimized light layer and an optimized reflection layer; The light adjustment module is configured to perform light adjustment on the optimized light layer to obtain a target light layer; The image reconstruction module is configured to perform image reconstruction according to the target light layer and the optimized reflection layer to obtain a target light image; The optimization module comprises a variable calculation subnetwork, a reflection layer repair network and a light layer repair network, wherein when performing the current alternating iterative optimization, the variable calculation subnetwork is configured to calculate first and second intermediate variables after the current iterative optimization, the reflection layer repair network is configured to perform convolution operation on the first and second intermediate variables after the current iterative optimization to obtain first and second intermediate feature maps, perform concatenation operation on the first and second intermediate feature maps to obtain a spliced feature map, perform channel attention calculation on the spliced feature map by using a channel self-attention mechanism to obtain a re-weighted feature map, obtain a noise distribution of the re-weighted feature map, and obtain the optimized reflection layer after the current iterative optimization based on the noise distribution and the first intermediate feature map, and the light layer repair network is configured to obtain the optimized light layer after the current iterative optimization based on the second intermediate variable after the current iterative optimization.
2. The low-light image enhancement device of claim 1, wherein, The initialization module is a fully connected neural network, and the fully connected neural network is a fully convolutional neural network comprising four convolutional layers.
3. The low-light image enhancement device of claim 1, wherein, The loss function used by the initialization module during training comprises a fidelity term and a prior term, wherein the fidelity term is used to measure the closeness of an initialization image composed of the initialization light layer and the initialization reflection layer of a training sample to the training sample, and the prior term is used to measure the closeness of the initialization light layer of the training sample to the maximum values of R, G and B channels of the training sample; The loss function used by the initialization module during training is as follows: wherein, denotes the loss of the initialization module, denotes the training sample, denotes the initialization reflectance layer of the training sample, denotes the initialization illumination layer of the training sample, is a constant, denote the red, green, and blue channels, respectively.
4. The low-light image enhancement device of claim 1, wherein, The variable calculation subnetwork is configured to calculate the first and second intermediate variables after the current iterative optimization by using the least square method.
5. The low-light image enhancement device of claim 1, wherein, The light adjustment module comprises an adjustment factor expansion submodule, a splicing submodule and a brightness adjustment network connected in sequence, wherein The adjustment factor expansion submodule is configured to expand a preset adjustment scale factor into a matrix with the same size as the optimized light layer; The splicing submodule is configured to splice the matrix and the optimized light layer to obtain a splicing result; and The brightness adjustment network is configured to perform brightness adjustment on the splicing result to obtain the target light layer. The brightness adjustment network is configured to perform brightness adjustment on the optimized illumination layer based on the splicing result to obtain the target illumination layer, wherein the brightness adjustment network has the same network structure as the initialization module, and a size of a convolution kernel of a convolution layer of the brightness adjustment network is greater than that of the initialization module.
6. The low-light image enhancement device of claim 5, wherein, The loss function used by the brightness adjustment network during training includes one or more combinations of a gradient level fidelity term, a color level fidelity term, and a structure level fidelity term, wherein the gradient level fidelity term is used to measure a horizontal or vertical gradient distance between the optimized illumination layer and the target illumination layer of a training sample, the color level fidelity term is used to measure a reconstruction loss of the target illumination image and a reference image of the training sample, and the structure level fidelity term is used to measure a distance between the target illumination image and the reference image of the training sample. The loss function used by the brightness adjustment network during training is: wherein, denotes a loss of the brightness adjustment network, denotes a gradient in a horizontal or vertical direction of an optimized illumination layer of the training sample, denotes a gradient in a horizontal or vertical direction of a target illumination layer of the training sample, denotes the reference image, denotes an optimized reflectance layer of the training sample, denotes a target illumination layer of the training sample, denotes an image quality loss function.
7. A low-light image enhancement method based on the low-light image enhancement device of any one of claims 1-6, characterized in that, The method comprises the following steps: The initialization module is configured to perform initialization decomposition on an input image to obtain an initialization illumination layer and an initialization reflectance layer corresponding to the input image; The optimization module is configured to perform a plurality of times of alternating iterative optimization on the initialization illumination layer and the initialization reflectance layer by using an unfolding algorithm to obtain an optimized illumination layer and an optimized reflectance layer; The illumination adjustment module is configured to perform illumination adjustment on the optimized illumination layer to obtain a target illumination layer; The image reconstruction module is configured to perform image reconstruction based on the target illumination layer and the optimized reflectance layer to obtain a target illumination image.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the function of the low-illumination image enhancement device according to any one of claims 1 to 6.
9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. The computer program is executed by the processor to realize the function of the low-illumination image enhancement device according to any one of claims 1 to 6.