Low light enhancement method and device based on deep expansion network

Abstract

The low-light enhancement method and device based on the deep expansion network have excellent performance in the aspects of objective and subjective quality evaluation, and can better realize low-illuminance image enhancement. The method comprises the following steps: (1) inputting a low-illuminance image; (2) performing initialization operation on the illuminance and the reflectivity, and storing five diagonals of the five-point positive definite matrix as a new matrix according to the matrix property of the five-point positive definite matrix by iteratively solving two sub-problem optimization of the illuminance and the reflectivity, and optimizing the product operation of the five-point positive definite matrix by adopting Hadamard product operation on the new matrix; (3) enhancing the illumination component through an adaptive method of a local region to obtain an enhanced illumination component with relatively good exposure; (4) de-noising the enhanced illumination component and the reflection component to obtain de-noised reflectivity; (5) performing element product on the enhanced illumination component and the de-noised reflectivity to obtain an enhanced result; and (6) outputting the enhanced image.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a low-light enhancement method based on a deep unfolding network, and a low-light enhancement device based on a deep unfolding network. Background Art

[0002] Images captured under low-light conditions usually have problems such as low brightness, low contrast, and color distortion, which poses challenges to the visual effects of the image and subsequent visual processing tasks such as object detection and face detection. Low-light images obscure a large amount of image information due to the low light, resulting in less available information and a large amount of noise. Although traditional histogram equalization and gamma correction methods both improve the brightness of the image to a certain extent by changing the grayscale value of pixels, there is still a phenomenon of detail loss and excessive noise.

[0003] Retinex theory is based on the human eye's perception of color and the principle of color constancy, and represents an image as the product of the illumination component and the reflectance component S = L·R. Early methods based on Retinex theory enhanced the image by estimating the reflectance by calculating the illumination. Although these methods try to retain the detailed information in the image, estimating the reflectance alone may cause the image to be overexposed and produce significant noise interference. After that, many model-based methods introduced prior knowledge to optimize the decomposition of illumination and reflectance. JieP enhances low-light images by combining internal and external priors of the image, and STAR better decomposes illumination and reflectance while improving noise processing. Although these methods perform well in interpretability and specific image enhancement, achieving sufficient adaptability in different scenarios remains a major problem. In addition, the above methods require iterative optimization of each image, which significantly increases the time cost, thereby affecting the efficiency of image processing.

[0004] Many deep learning low-light enhancement methods have been proposed to improve the above problems. The KinD method establishes a subnetwork for layer decomposition, illumination adjustment, and reflection restoration to effectively achieve image enhancement and denoising. The Zero-DCE method estimates the high-order curve of the image through the input of the low-light image, and adjusts the image pixel by pixel to obtain an enhanced image. The SCI method improves exposure stability and greatly reduces the computational burden by introducing a self-calibration module. The above methods still retain good details when dealing with complex image enhancement tasks, and obtain good enhancement performance.

[0005] Recently, URetinex proposed a low-light image enhancement method based on a deep unfolding network of Retinex. This method combines Retinex theory with neural networks, improves network performance through iterative optimization, and ensures the texture details of the image while enhancing the illumination, thus achieving effective enhancement of low-light images. However, this method only uses a simple prior and ignores local illumination information when adjusting the illumination, resulting in problems such as abnormal color and contrast restoration and overexposure of some areas. Summary of the invention

[0006] In order to overcome the defects of the prior art, the technical problem to be solved by the present invention is to provide a low-light enhancement method based on a deep unfolding network, which has superior performance in both objective and subjective quality evaluation and can better achieve low-light image enhancement.

[0007] The technical solution of the present invention is: the low-light enhancement method based on the deep unfolding network comprises the following steps:

[0008] (1) Input low-light image;

[0009] (2) Initialize the illumination and reflectivity, denoted as L (0) With R (0) Then, L is optimized by iteratively solving the two sub-problems of illumination and reflectivity. (k) With R (k) , where L (k) With R (k) Represent the illumination and reflectance of the kth stage respectively. According to the matrix properties, the five diagonals of the five-point positive definite matrix are stored as a new matrix, and the new matrix is ​​subjected to Hadamard product operation to optimize the five-point positive definite matrix product operation.

[0010] (3) The illumination component L is adjusted by an adaptive method in the local area. u Enhanced to obtain enhanced lighting component with better exposure

[0011] (4) and the reflected component R u Perform denoising to obtain the denoised reflectivity

[0012] (5) and Get the enhanced result by element-wise multiplication;

[0013] (6) Output the enhanced image.

[0014] A low-light enhancement device based on a deep unfolding network is also provided, the device comprising:

[0015] An input module, which inputs a low-light illumination image;

[0016] Initialization module, which performs initialization operations on illuminance and reflectance, recorded as L(0) and R(0), and then optimizes L(k) and R(k) by iteratively solving the two sub-problems of illuminance and reflectance. k ), where L(k) and R( k ) represent the illumination and reflectance of the kth stage respectively, store the five diagonals of the five-point positive definite matrix as a new matrix according to its matrix properties, and use the Hadamard product operation on the new matrix to optimize the five-point positive definite matrix product operation;

[0017] The enhancement module uses an adaptive method in the local area to adjust the illumination component L u Enhanced to obtain enhanced lighting component with better exposure

[0018] The denoising module and the reflected component R u Perform denoising to obtain the denoised reflectivity

[0019] The computing module will and Get the enhanced result by element-wise multiplication;

[0020] Output module, which outputs the enhanced image.

[0021] The present invention combines image shape, texture and external illumination prior with neural network to build a deep expandable network framework (UREST-Net), which includes three modules, namely, a deep expansion module for decomposing illumination and reflectance, an illumination adjustment module for adaptive enhancement of illumination area, and a denoising module for denoising. The deep expansion module decomposes the optimization of reflectance and illumination into two sub-problems, introduces shape prior and illumination prior to the illumination optimization problem to establish an illumination optimization model, introduces texture prior to the reflectance optimization problem to establish a reflectance optimization model, and better decomposes illumination and reflectance in the iterative optimization process of the two sub-problems; the illumination adjustment module correctly restores the contrast of the image while ensuring the normal exposure of the enhanced image by introducing a regional adaptive method; the denoising module removes noise as much as possible by combining illumination and reflectance features while ensuring the image texture details. Experimental results show that compared with the most advanced methods, this method has superior performance in both objective and subjective quality evaluation, and can better achieve low-light illumination image enhancement. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1A model framework diagram of a low-light enhancement method based on a deep unfolding network according to the present invention is shown.

[0023] Figure 2 A flowchart of a low-light enhancement method based on a deep unfolding network according to the present invention is shown.

[0024] Figure 3 A flow chart of the depth expansion module to realize illumination and reflectivity is shown.

[0025] Figure 4 A flow chart of the illumination adjustment module for achieving illumination enhancement is shown.

[0026] Figure 5 A flowchart of the denoising module for implementing reflectivity denoising is shown. DETAILED DESCRIPTION

[0027] like Figure 2 As shown, this low-light enhancement method based on a deep unfolding network includes the following steps:

[0028] (1) Input low-light image;

[0029] (2) Initialize the illumination and reflectivity, denoted as L (0) With R (0) Then, L is optimized by iteratively solving the two sub-problems of illumination and reflectivity. (k) With R (k) , where L (k) With R (k) Represent the illumination and reflectance of the kth stage respectively. According to the matrix properties, the five diagonals of the five-point positive definite matrix are stored as a new matrix, and the new matrix is ​​subjected to Hadamard product operation to optimize the five-point positive definite matrix product operation.

[0030] (3) The illumination component L is adjusted by an adaptive method in the local area. u Enhanced to obtain enhanced lighting component with better exposure

[0031] (4) and the reflected component R u Perform denoising to obtain the denoised reflectivity

[0032] (5) and Get the enhanced result by element-wise multiplication;

[0033] (6) Output the enhanced image.

[0034] The present invention combines image shape, texture and external illumination prior with neural network to build a deep expandable network framework (UREST-Net), which includes three modules, namely, a deep expansion module for decomposing illumination and reflectance, an illumination adjustment module for adaptive enhancement of illumination area, and a denoising module for denoising. The deep expansion module decomposes the optimization of reflectance and illumination into two sub-problems, introduces shape prior and illumination prior to the illumination optimization problem to establish an illumination optimization model, introduces texture prior to the reflectance optimization problem to establish a reflectance optimization model, and better decomposes illumination and reflectance in the iterative optimization process of the two sub-problems; the illumination adjustment module correctly restores the contrast of the image while ensuring the normal exposure of the enhanced image by introducing a regional adaptive method; the denoising module removes noise as much as possible by combining illumination and reflectance features while ensuring the image texture details. Experimental results show that compared with the most advanced methods, this method has superior performance in both objective and subjective quality evaluation, and can better achieve low-light illumination image enhancement.

[0035] Preferably, in step (2), the decomposition of illuminance and reflectivity is expressed as two sub-problems: formula (1) and formula (2):

[0036]

[0037] Where L (k) represents the illumination component of the kth stage, R (k) represents the reflection component of the kth stage, u x 、u y Respectively represent the weight parameters of L in the horizontal component and the vertical component, v x 、v y Respectively represent the weight parameters of R in the horizontal component and the vertical component, B is the bright channel of the image, is the gradient operator, α, β and λ are variable parameters; the two sub-problems are solved by gradient descent method:

[0038]

[0039] Where vec(·) is a vectorized operation, and Unfld(·) is an inverse vectorized operation, where are the weight matrices of L and R respectively, where and Is The resulting Toeplitz matrix, They are The matrix formed, It is obtained through network learning, and different update optimizations are performed at different iteration stages. The solution process is expressed as:

[0040]

[0041] in It is a lightweight U-Net network, which consists of five encoding blocks and four decoding blocks. In the encoding stage, a pooling layer with a step size of 2 is added between two adjacent encoding blocks to downsample the feature map. In the decoding stage, a bilinear interpolation layer with a scale factor of 2 is used to upsample the feature map. Each layer of the network uses a 3×3 Conv filter. At the end of the decoding stage, two weight generators are added, and their convolution kernel sizes are 3×1 and 1×3 respectively to learn the weight parameters in the horizontal direction. The weight parameter in the vertical direction

[0042] Preferably, in step (2), the five diagonals of the weight matrix in the form of a five-point positive definite matrix are stored as new matrices P and Q:

[0043]

[0044] where diag k (·) represents the diagonal line with matrix offset k, where A is a set representing the offset of the five diagonals; then the Hadamard product is introduced to optimize matrix multiplication:

[0045] X (k) =(P (k) vec(L (k) ))e (9)

[0046] Y (k) =(Q (k) vec(R (k) ))e (10)

[0047] Where · represents the Hadamard product, and e is a column vector of all ones. Through the above optimization, the solutions of equations (3) and (4) are:

[0048] L (k+1) =L (k) -2δ{R (k)T (R (k) ·L (k) -S)+αUnfold(X (k) )+λ(L (k) -B)} (11)

[0049] R (k+1) =R (k) -2δ{L (k+1)T (L (k+1) ·R (k) -S)+βUnfold(Y (k))) (12).

[0050] Preferably, the loss function of step (2) is:

[0051]

[0052] Among them, Θ is the network parameter, For the deep expansion module, I z is the product of the decomposed illuminance and reflectivity under the network parameter Θ, This is an image with normal lighting.

[0053] Preferably, in step (3),

[0054] The decomposed illuminance L and the illuminance H of the highlight image are divided into There are n×n regions, and the enhancement weight for each region is:

[0055]

[0056] where r t represents the enhancement weight of the tth region, represents the (i, j)th element of the tth region of the highlight image, represents the (i, j)th element of the tth region of the low-light image; finally, the enhancement weights of the n regions are concatenated to obtain the image brightness enhancement matrix r. L and r are first concatenated through a fully connected operation, and then sent to three Conv+LeakyRelu networks and one Conv+Relu network for feature learning, so that L is enhanced in a region-adaptive manner to obtain the enhanced illumination. During the training process, the loss function is:

[0057]

[0058] in For a highlight image, is the enhanced illumination, SSIM(·) is the structural similarity index,

[0059] is the network parameter, and Z represents the total number of training samples.

[0060] Preferably, in step (3), in the test phase, the characteristic relationship between the low-light image and the high-light image is learned to map an approximate high-light image HF, and then substituted into equation (14) to obtain the enhancement weight of each region,

[0061]

[0062] Preferably, in step (4), the denoising of the reflectivity is achieved by jointly denoising the reflection component and the illumination component, and the loss function is:

[0063]

[0064] in is the reflectivity after denoising, SSIM(·) is the structural similarity index, φ(·) is the feature extractor, is the network parameter, and Z represents the total number of training samples.

[0065] A person skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes the steps of the above-mentioned embodiment method, and the storage medium can be: ROM / RAM, disk, optical disk, memory card, etc. Therefore, corresponding to the method of the present invention, the present invention also includes a low-light enhancement device based on a deep unfolding network, which is usually represented in the form of functional modules corresponding to the steps of the method. The device includes:

[0066] An input module, which inputs a low-light illumination image;

[0067] Initialization module, which performs initialization operations on illumination and reflectivity, denoted as L (0) With R (0) Then, L is optimized by iteratively solving the two sub-problems of illumination and reflectivity. (k) With R (k) , where L (k) With R (k) Represent the illumination and reflectance of the kth stage respectively. According to the matrix properties, the five diagonals of the five-point positive definite matrix are stored as a new matrix, and the new matrix is ​​subjected to Hadamard product operation to optimize the five-point positive definite matrix product operation.

[0068] The enhancement module uses an adaptive method in the local area to adjust the illumination component L u Enhanced to obtain enhanced lighting component with better exposure

[0069] The denoising module and the reflected component R u Perform denoising to obtain the denoised reflectivity

[0070] The computing module will and Get the enhanced result by element-wise multiplication;

[0071] Output module, which outputs the enhanced image.

[0072] Preferably, in the initialization module, the decomposition of illuminance and reflectivity is expressed as two sub-problems: Formula (1) and Formula (2):

[0073]

[0074] Where L (k) represents the illumination component of the kth stage, R (k) represents the reflection component of the kth stage, u x 、u y Respectively represent the weight parameters of L in the horizontal component and the vertical component, v x 、v y Respectively represent the weight parameters of R in the horizontal component and the vertical component, B is the bright channel of the image, is the gradient operator, α, β and λ are variable parameters; the two sub-problems are solved by gradient descent method:

[0075]

[0076] Where vec(·) is a vectorized operation, and Unfld(·) is an inverse vectorized operation, where are the weight matrices of L and R respectively, where and Is The resulting Toeplitz matrix, They are The matrix formed, It is obtained through network learning, and different update optimizations are performed at different iteration stages. The solution process is expressed as:

[0077]

[0078] in It is a lightweight U-Net network, which consists of five encoding blocks and four decoding blocks. In the encoding stage, a pooling layer with a step size of 2 is added between two adjacent encoding blocks to downsample the feature map. In the decoding stage, a bilinear interpolation layer with a scale factor of 2 is used to upsample the feature map. Each layer of the network uses a 3×3 Conv filter. At the end of the decoding stage, two weight generators are added, and their convolution kernel sizes are 3×1 and 1×3 respectively to learn the weight parameters in the horizontal direction. The weight parameter in the vertical direction

[0079] Preferably, in the initialization module, the five diagonals of the weight matrix in the form of a five-point positive definite matrix are stored as new matrices P and Q:

[0080]

[0081] where diag k (·) represents the diagonal line with matrix offset k, where A is a set representing the offset of the five diagonals; then the Hadamard product is introduced to optimize matrix multiplication:

[0082] X (k) =(P (k) vec(L (k) ))e (9)

[0083] Y (k) =(Q (k) vec(R (k) ))e (10)

[0084] Where · represents the Hadamard product, and e is a column vector of all ones. Through the above optimization, the solutions of equations (3) and (4) are:

[0085] L (k+1) =L (k) -2δ{R (k)T (R (k) ·L (k) -S)+αUnfold(X (k) )+λ(L( k) -B)} (11)

[0086] R (k+1) =R (k) -2δ{L (k+1)T (L (k+1) ·R (k) -S)+βUnfold(Y (k) )} (12).

[0087] In order to verify the effectiveness of the proposed low-light enhancement method of joint prior and deep unfolding network, only the training dataset of LOL dataset is used for training, and the model is tested on datasets of various scenes to verify the adaptability and effectiveness of our proposed method, including LOLv1 dataset, LOLv2-Synthetic dataset and LOLv2-Real dataset. The model is compared with several state-of-the-art methods, including STAR, LIME, JIEP, RetinexNet, KinD, Zero-DCE, RUAS, KinD++, Zero-DCE++, SCI, R2RNet, URetinex and PairLIE. Three widely used image quality evaluation indicators are used to compare the enhancement performance, including mean absolute error (MAE), peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). The higher the PSNR, SSIM or the lower the MAE of the enhanced image, the better the quality.

[0088] Table 1 shows the average enhancement performance of 13 competing methods on the LOLv1 dataset, LOLv2-Synthetic dataset, and LOLv2-Real dataset. It can be seen that the results obtained by the model of the present invention have the highest PSNR and almost the highest MAE and SSIM results respectively.

[0089] Table 1

[0090]

[0091] More specifically, the network framework of the present invention is trained on the LOL dataset. The LOL dataset includes 500 pairs of low-light / normal-light images collected at different exposure times, of which 485 pairs are used for training and the remaining 15 pairs are used for testing. In addition to the 15 pairs of test sets (LOLv1) provided by the LOL dataset, the test set also includes the LOLv2-Synthetic dataset and the LOLv2-Real dataset, wherein the LOLv2-Synthetic dataset and the LOLv2-Real dataset each have 100 test images.

[0092] The proposed method is compared with traditional methods STAR, LIME, JieP and deep learning methods RetinexNet, KinD, Zero-DCE, RUAS, KinD++, Zero-DCE++, SCI, R2RNet, URetinex and PairLIE on the LOLv1 dataset, LOLv2-Synthetic dataset and LOLv2-Real dataset. The deep learning-based comparison methods are tested using their pre-trained models.

[0093] In the experiment, the image size of the dataset of the present invention is adjusted to 256×256×3. Pytorch is used to implement the framework of the present invention on NVIDIA 4090 GPU. The batch size is 4. α=0.001, β=0.0001, λ=0.25 are set respectively, the total number of training samples Z=485, the local area size n=20, and the number of iterative optimization T=4.

[0094] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the protection scope of the technical solution of the present invention.

Claims

1. A low-light enhancement method based on a deep unfolding network, characterized by: The method comprises the following steps: (1) Input low-light image; (2) Initialize the illumination and reflectivity, denoted as L (0) With R (0) Then, L is optimized by iteratively solving the two sub-problems of illumination and reflectivity. (k) With R (k) , where L (k) With R (k) Represent the illumination and reflectance of the kth stage respectively. According to the matrix properties, the five diagonals of the five-point positive definite matrix are stored as a new matrix, and the new matrix is ​​subjected to Hadamard product operation to optimize the five-point positive definite matrix product operation. (3) The decomposed illumination component L is processed by an adaptive method in the local area. u Enhanced to obtain enhanced lighting component with better exposure (4) and the decomposed reflection component R u Perform denoising to obtain the denoised reflectivity (5) and Get the enhanced result by element-wise multiplication; (6) Output the enhanced image.

2. The low-light enhancement method based on deep unfolding network according to claim 1, characterized in that: In step (2), the decomposition of illuminance and reflectivity is expressed as two sub-problems: equation (1) and equation (2): Where L (k) represents the illumination component of the kth stage, R (k) represents the reflection component of the kth stage, u x 、u y Respectively represent the weight parameters of L in the horizontal component and the vertical component, v x 、v y Respectively represent the weight parameters of R in the horizontal component and the vertical component, B is the bright channel of the image, is the gradient operator, α, β and λ are variable parameters; the two sub-problems are solved by gradient descent method: Where vec(·) is a vectorized operation, and Unfld(·) is an inverse vectorized operation, where are the weight matrices of L and R respectively, where and Is The resulting Toeplitz matrix, They are The matrix formed, It is obtained through network learning, and different update optimizations are performed at different iteration stages. The solution process is expressed as: in It is a lightweight U-Net network, which consists of five encoding blocks and four decoding blocks. In the encoding stage, a pooling layer with a step size of 2 is added between two adjacent encoding blocks to downsample the feature map. In the decoding stage, a bilinear interpolation layer with a scale factor of 2 is used to upsample the feature map. Each layer of the network uses a 3×3 Conv filter. At the end of the decoding stage, two weight generators are added, and their convolution kernel sizes are 3×1 and 1×3 respectively to learn the weight parameters in the horizontal direction. The weight parameter in the vertical direction 3. The low-light enhancement method based on deep unfolding network according to claim 2, characterized in that: In step (2), the five diagonals of the weight matrix in the form of a five-point positive definite matrix are stored as new matrices P and Q: where diag k (·) represents the diagonal line with matrix offset k, where A is a set representing the offset of the five diagonals; then the Hadamard product is introduced to optimize matrix multiplication: X (k) =(P (k) ·old(L (k) )e (9) Y (k) =(Q (k) ·vec(R (k) ))e (10) Where · represents the Hadamard product, and e is a column vector of all ones. Through the above optimization, the solutions of equations (3) and (4) are: L (k+1) =L (k) -2δ{R (k)T (R (k) ·L (k) -S)+αUnfold(X (k) )+λ(L (k) -B)} (11) R (k+1) =R (k) -2δ{L (k+1)T (L (k+1) ·R (k) -S)+βUnfold(Y (k) )} (12)。 4. The low-light enhancement method based on deep unfolding network according to claim 3, characterized in that: The loss function of step (2) is: Among them, Θ is the network parameter, For the deep expansion module, I z is the product of the decomposed illuminance and reflectivity under the network parameter Θ, This is an image with normal lighting.

5. The low-light enhancement method based on deep unfolding network according to claim 4, characterized in that: In the step (3), The decomposed illuminance L and the illuminance H of the highlight image are divided into There are n×n regions, and the enhancement weight for each region is: where r t represents the enhancement weight of the tth region, represents the (i,j)th element of the tth region of the highlight image, represents the (i, j)th element of the tth region of the low-light image; finally, the enhancement weights of the n regions are concatenated to obtain the image brightness enhancement matrix r. L and r are first concatenated through a fully connected operation, and then sent to three Conv+LeakyRelu networks and one Conv+Relu network for feature learning, so that L is enhanced in a region-adaptive manner to obtain the enhanced illumination. During the training process, the loss function is: in For a highlight image, is the enhanced illumination, SSIM(·) is the structural similarity index, is the network parameter, and Z represents the total number of training samples.

6. The low-light enhancement method based on deep unfolding network according to claim 4, characterized in that: In the step (3), during the test phase, the feature relationship between the low-light image and the high-light image is learned to map an approximate high-light image HF, and then substituted into equation (14) to obtain the enhancement weight of each region:

7. The low-light enhancement method based on deep unfolding network according to claim 5 or 6, characterized in that: In the step (4), the denoising of the reflectivity is achieved by jointly denoising the reflection component and the illumination component, and the loss function is: in is the reflectivity after denoising, SSIM(·) is the structural similarity index, φ(·) is the feature extractor, is the network parameter, and Z represents the total number of training samples.

8. A low-light enhancement device based on a deep unfolding network, characterized in that: It includes: An input module, which inputs a low-light illumination image; Initialization module, which performs initialization operations on illumination and reflectivity, denoted as L (0) With R (0) , Then, L is optimized by iteratively solving the two sub-problems of illumination and reflectivity. (k) With R (k) , where L (k) With R (k) Represent the illumination and reflectance of the kth stage respectively. According to the matrix properties, the five diagonals of the five-point positive definite matrix are stored as a new matrix, and the new matrix is ​​subjected to Hadamard product operation to optimize the five-point positive definite matrix product operation. The enhancement module uses an adaptive method in the local area to adjust the illumination component L u Enhanced to obtain enhanced lighting component with better exposure The denoising module and the reflected component R u Perform denoising to obtain the denoised reflectivity The computing module will and Get the enhanced result by element-wise multiplication; Output module, which outputs the enhanced image.

9. The low-light enhancement device based on deep unfolding network according to claim 8, characterized in that: In the initialization module, the decomposition of illumination and reflectivity is expressed as two sub-problems: equation (1) and equation (2): Where L (k) represents the illumination component of the kth stage, R (k) represents the reflection component of the kth stage, u x 、u y Respectively represent the weight parameters of L in the horizontal component and the vertical component, v x 、v y Respectively represent the weight parameters of R in the horizontal component and the vertical component, B is the bright channel of the image, is the gradient operator, α, β and λ are variable parameters; the two sub-problems are solved by gradient descent method: Where vec(·) is a vectorized operation, and Unfld(·) is an inverse vectorized operation, where are the weight matrices of L and R respectively, where and Is The resulting Toeplitz matrix, They are The matrix formed, It is obtained through network learning, and different update optimizations are performed at different iteration stages. The solution process is expressed as: in It is a lightweight U-Net network, which consists of five encoding blocks and four decoding blocks. In the encoding stage, a pooling layer with a step size of 2 is added between two adjacent encoding blocks to downsample the feature map. In the decoding stage, a bilinear interpolation layer with a scale factor of 2 is used to upsample the feature map. Each layer of the network uses a 3×3 Conv filter. At the end of the decoding stage, two weight generators are added, and their convolution kernel sizes are 3×1 and 1×3 respectively to learn the weight parameters in the horizontal direction. The weight parameter in the vertical direction 10. The low-light enhancement device based on deep unfolding network according to claim 9, characterized in that: In the initialization module, the five diagonals of the weight matrix in the form of a five-point positive definite matrix are stored as new matrices P and Q: where diag k (·) represents the diagonal line with matrix offset k, where A is a set representing the offset of the five diagonals; then the Hadamard product is introduced to optimize matrix multiplication: X (k) =(P (k) ·old(L (k) ))e (9) Y (k) =(Q (k) ·vec(R (k) ))e (10) Where · represents the Hadamard product, and e is a column vector of all ones. Through the above optimization, the solutions of equations (3) and (4) are: L (k+1) =L (k) -2δ{R (k)T (R (k) ·L (k) -S)+αUnfold(X (k) )+λ(L (k) -B)} (11) R (k+1) =R (k) -2δ{L (k+1)T (L (k+1) ·R (k) -S)+βUnfold(Y (k) )} (12)。

Images