Low-light image enhancement quality assessment method based on neural network unfolding filter decomposition

Abstract

The present invention discloses a low-light image enhancement quality evaluation method based on neural network expanded filtering decomposition, comprising the following steps: inputting a low-light enhanced image and a reference image, obtaining illumination detail information and structural information of the low-light enhanced image and the reference image through filtering decomposition; using a neural network to perform feature extraction on the illumination detail information, structural information and overall information of the low-light enhanced image and the reference image, and obtaining depth features of the illumination detail features, structural features and overall features of the low-light enhanced image and the reference image respectively; comparing the similarity of the illumination detail depth features, structural depth features and overall features of the low-light enhanced image and the reference image, and obtaining a final quality score through training and learning.

Description

Technical Field

[0001] The present invention relates to the field of image quality assessment and image enhancement technology, in particular to the field of low-light image quality assessment technology, and specifically to a low-light image enhancement quality assessment method based on neural network expansion filter decomposition. Background Art

[0002] With the rapid development of the internet, image resources are increasing, and low-quality, highly distorted images are also on the rise. Low-light images are one example of low-quality, highly distorted images. Low-light images often result in unclear visual effects and blurred image details, which not only affects people's subjective visual experience but also limits the processing performance of machine vision systems. Therefore, low-light image enhancement is an important research direction in the field of image processing and a crucial preprocessing process. It aims to use algorithms to enhance under-illuminated and underexposed images to a range acceptable to the human eye and algorithms, thereby extracting valuable information from the images and improving the performance of computer vision systems. Low-light image enhancement technology has multiple application scenarios and plays an indispensable role in computer vision, machine learning, security detection, medical imaging, space exploration, deep-sea exploration, and other fields.

[0003] Low-light images have diverse content, complex low-light scenes, and additional interference is generated during the image acquisition process. Therefore, existing low-light image enhancement algorithms may have problems such as over / underexposure, inaccurate color restoration, and loss of details. A good evaluation standard is needed to help and guide low-light image enhancement and improve the enhancement quality and effect.

[0004] At the same time, although there are many methods for image quality assessment, such as LPIPS (Simonyan K, Zisserman A. Verydeep convolutional networks for large-scale image recognition [J]. arXiv preprint arXiv: 1409.1556, 2014.), which uses the hidden layer output of a pre-trained neural network as perceptual features, calculates the distance between features, and performs linear combination to obtain the final score, it can detect defects that are difficult to detect with traditional methods; DISTS (Ding K, Ma K, Wang S, et al. Image Quality Assessment: Unifying Structure and Texture Similarity [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.), which also calculates feature distances by extracting perceptual features and draws on the SSIM method to calculate the final score, there is a lack of corresponding methods for the problem of low-light image enhancement quality evaluation, making it difficult to effectively evaluate the enhancement quality of low-light enhanced images. Summary of the Invention

[0005] The purpose of this paper is to propose a low-light image enhancement quality assessment method based on neural network unrolling filter decomposition. This method uses a neural network unrolling (hereinafter referred to as "unrolling") method to perform filter decomposition and evaluate the image quality of the input image enhanced by the low-light enhancement algorithm, obtaining a better evaluation result that is more consistent with human visual perception.

[0006] The present invention is achieved through at least one of the following technical solutions.

[0007] The low-light image enhancement quality evaluation method based on neural network unfolding filter decomposition includes the following steps:

[0008] The low-light enhanced image and the reference image are input, and the illumination detail information and structural information of the low-light enhanced image and the reference image are obtained by filtering and decomposing them using the neural network unfolding (hereinafter referred to as Unrolling) filtering decomposition method;

[0009] Extracting features of the illumination detail information and structural information of the low-light enhanced image and the reference image obtained by the neural network filtering decomposition to obtain illumination detail depth features and structural depth features of the illumination detail information and structural information of the low-light enhanced image and the reference image, respectively;

[0010] Extracting features from the low-light enhanced image and the reference image using a neural network to obtain overall features of the low-light enhanced image and the reference image respectively;

[0011] comparing similarities of illumination detail depth features, structural depth features, and overall features of the low-light enhanced image and the reference image;

[0012] The final quality score is obtained through training and learning based on the similarity of the illumination detail depth features, structural depth features and overall features of the low-light enhanced image and the reference image.

[0013] Furthermore, the process of filtering decomposition based on neural network is as follows:

[0014] Input the image to be decomposed, and perform image color space conversion on the image to be decomposed, from RGB color space to CIELab color space;

[0015] The L channel of the CIELab color space image is used as input to the Unrolling filter decomposition module to obtain the corresponding structural information. The expression is as follows:

[0016] u struct =UnrollingFilter(I L )

[0017] Among them, UnrollingFilter() represents the Unrolling filter decomposition module, I L Represents the input L channel image, u struct Represents the structural information obtained after decomposition;

[0018] The structural information obtained by decomposition is subtracted from the L channel of the CIELab color space image to obtain the lighting detail information, which is expressed as follows:

[0019] u illumination =I L -u struct

[0020] Among them, I L Represents the input L channel image, u illiumination Represents the lighting detail information obtained after decomposition.

[0021] Furthermore, the neural network expansion filter decomposition module includes an image filter decomposition submodule and an intermediate variable updating submodule during the optimization process; the neural network expansion filter decomposition method requires several alternating iterative optimizations, and the decomposition results and intermediate variables obtained from this iterative calculation are used as inputs for the next iterative calculation, and the next result is calculated, and the above iterative process is repeated until the final decomposition result is obtained.

[0022] Furthermore, a CNN neural network is used to extract the features of the illumination details of the low-light enhanced image. The process is as follows:

[0023] The illumination detail information of the low-light enhanced image is used as input to the VGGNet CNN neural network to calculate the depth features corresponding to the illumination detail information of the low-light enhanced image and the reference image;

[0024] In the VGGNet CNN neural network, the outputs of the n hidden convolutional layers are taken as the depth features corresponding to the illumination detail information of the low-light enhanced image;

[0025] The VGGNet CNN neural network is used to obtain n depth features of illumination detail information of low-light enhanced images.

[0026] Furthermore, a CNN neural network is used to extract features from the structural information of the low-light enhanced image. The process is as follows:

[0027] The structural information of the low-light enhanced image is used as input to the VGGNet CNN neural network to calculate the depth features corresponding to the structural information of the low-light enhanced image;

[0028] In the VGGNet CNN neural network, the outputs of the n hidden convolutional layers are taken as the deep features corresponding to the structural information of the low-light enhanced image;

[0029] The VGGNet CNN neural network is used to obtain n structural information and depth features of the low-light enhanced image;

[0030] The CNN neural network is used to extract the features of the illumination details of the reference image. The process is as follows:

[0031] The illumination details of the reference image are used as input to the VGGNet CNN neural network to calculate the depth features corresponding to the illumination details of the reference image.

[0032] In the VGGNet CNN neural network, the outputs of the n hidden convolutional layers are taken as the depth features corresponding to the illumination detail information of the reference image;

[0033] The VGGNet CNN neural network is used to obtain n depth features of the illumination detail information of the reference image.

[0034] Furthermore, a CNN neural network is used to extract features from the structural information of the reference image. The process is as follows:

[0035] The structural details of the reference image are input into the VGGNet CNN neural network to calculate the deep features corresponding to the structural information of the reference image;

[0036] In the VGGNet CNN neural network, the outputs of the n hidden convolutional layers are taken as the deep features corresponding to the structural information of the reference image;

[0037] The VGGNet CNN neural network is used to obtain n structural information and deep features of the reference image.

[0038] Furthermore, a CNN neural network is used to extract overall features of the low-light enhanced image. The process is as follows:

[0039] The entire low-light enhanced image is used as input to the VGGNet CNN neural network to calculate the depth features corresponding to the low-light enhanced image and the reference image;

[0040] In the VGGNet CNN neural network, the outputs of the n hidden convolutional layers are taken as the overall features of the low-light enhanced image;

[0041] The VGGNet CNN neural network is used to obtain n overall features of the low-light enhanced image.

[0042] Furthermore, the CNN neural network is used to extract the overall features of the reference image. The process is as follows:

[0043] The entire reference image is used as input to the VGGNet CNN neural network to calculate the deep features corresponding to the low-light enhanced image and the reference image;

[0044] In the VGGNet CNN neural network, the outputs of the n hidden convolutional layers are taken as the overall features of the low-light enhanced image and the reference image respectively;

[0045] The VGGNet CNN neural network is used to obtain n overall features of the low-light enhanced image and the reference image respectively.

[0046] Furthermore, the calculation formula for the similarity between the illumination detail depth feature, the structural depth feature, and the overall feature of the low-light enhanced image and the reference image is as follows:

[0047]

[0048]

[0049]

[0050] Among them, i represents the i-th feature, and Represent the similarity of the i-th illumination detail depth feature, the i-th structural depth feature and the i-th overall feature, and They represent the i-th illumination detail depth feature, i-th structural depth feature and i-th overall feature of the low-light enhanced image respectively. and They represent the i-th illumination detail depth feature, i-th structural depth feature and i-th overall feature of the reference image respectively.

[0051] Furthermore, based on the similarity of illumination detail depth features, structural depth features, and overall features, the final quality score is obtained through training and learning as follows:

[0052] S=w illumination D illumination +w strust D struct +w I D I

[0053] Among them, w illumination 、w strust and w I They are the score weights of illumination detail depth feature similarity, structural feature similarity and overall feature similarity obtained through training, D illumination 、D struct and D I They are the overall similarity of illumination detail depth features, the overall similarity of structural depth features, and the overall similarity of overall features, namely:

[0054]

[0055] The present invention has the following advantages and effects compared to the prior art:

[0056] (1) The present invention utilizes a neural network to carry out filtering decomposition to decompose the illumination detail information and structural information of the image. Compared with the traditional decomposition method, it combines the neural network end-to-end training method, which has faster processing speed and better effect.

[0057] (2) Lighting detail information can be used as an evaluation indicator for image blur and interference, and can better determine whether blur and interference still exist after enhancement. By decomposing lighting detail information and structural information, the quality of low-light enhanced images can be better evaluated, and the low-light image enhancement algorithm can be better improved.

[0058] (3) In addition to being able to play a role in low-light enhanced image quality evaluation, the present invention is also versatile and applicable to general image quality evaluation problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 This is a flow chart of the low-light image enhancement quality evaluation method based on neural network expansion filter decomposition of the present invention;

[0060] Figure 2 It is an algorithm flow chart of the neural network expansion filter decomposition module of the present invention;

[0061] Figure 3 It is a neural network structure diagram of the Q function in the neural network expansion filter decomposition module of the present invention;

[0062] Figure 4 It is an output effect diagram of the neural network expansion filtering decomposition module of the present invention. DETAILED DESCRIPTION

[0063] To help those skilled in the art better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It is apparent that the embodiments described are only a portion of the present invention, not all of the embodiments. All other embodiments derived by those skilled in the art based on the embodiments of the present invention without inventive effort are also within the scope of protection of the present invention.

[0064] Example 1

[0065] like Figure 1 The low-light image enhancement quality evaluation method based on neural network unfolding filter decomposition shown in FIG, the specific steps are as follows:

[0066] 1 Color space conversion

[0067] First input the image to be evaluated X I and reference image Y I , and convert from RGB color space to CIELab color space.

[0068] 2 Filter decomposition

[0069] The image to be evaluated X I and reference image Y I The L channel containing illumination information in the CIELab color space is input to the neural network expansion filter decomposition module.

[0070] like Figure 2 As shown, this embodiment includes several Unrolling filter decomposition modules, each of which includes an image filter decomposition submodule and an optimization process intermediate variable update submodule; the image filter decomposition submodule is Figure 2 The filter decomposition module U Model in the optimization process is used to filter and decompose the input image. The intermediate variable update submodule in the optimization process is Figure 2The intermediate variable Z update module z Model, the constraint parameter μ update module μModel and the constraint parameter α update module αModel in the filter decomposition are updated to update the intermediate variables used for filtering decomposition.

[0071] The filter decomposition module U Model performs filter decomposition on the input image based on the filter decomposition output and intermediate variables output by the previous Unrolling filter decomposition module. The expression is:

[0072]

[0073] Where I is the input image decomposed by Unrolling filter, Z k 、μ k and α k They represent the intermediate variable Z and constraint coefficients μ and α in the kth iteration, that is, the intermediate variable Z and constraint coefficients μ and α output by the kth Unrolling filter decomposition module. k+1 represents the output image u in the k+1th iteration, that is, the output image u output by the k+1th Unrolling filter decomposition module.

[0074] The intermediate variable Z update module z Model adopts the structure of CNN neural network, inputs the output image u of the filter decomposition module U Model and the constraint coefficient α output by the previous Unrolling filter decomposition module, and updates the intermediate variable Z. The expression is:

[0075] Z k+1 =Q(α k +u k+1 )

[0076] where α k represents the constraint coefficient α in the kth iteration, that is, the constraint coefficient α output by the kth Unrolling filter decomposition module. k+1 represents the output image u in the k+1th iteration, that is, the output image u output by the k+1th Unrolling filter decomposition module, Z k+1 It represents the intermediate variable Z in the k+1th iteration, that is, the intermediate variable Z output by the k+1th Unrolling filter decomposition module.

[0077] The constraint parameter μ update module μModel doubles the constraint coefficient μ output by the previous Unrolling filter decomposition module. The expression is:

[0078] μ k+1 =2*μ k

[0079] where μ kRepresents the constraint coefficient μ in the kth iteration, that is, the constraint coefficient μ output by the kth Unrolling filter decomposition module. k+1 Represents the constraint coefficient μ in the k+1th iteration, that is, the constraint coefficient μ output by the k+1th Unrolling filter decomposition module.

[0080] The constraint parameter α update module αModel inputs the output image u, intermediate variable Z and constraint coefficient μ updated by the current Unrolling filter decomposition module, as well as the constraint coefficient α output by the previous Unrolling filter decomposition module, updates the constraint coefficient α and outputs it. The expression is:

[0081] α k+1 =α k +μ k+1 (u k+1 -Z k+1 )

[0082] where α k Represents the constraint coefficient α in the kth iteration, that is, the constraint coefficient α output by the kth Unrolling filter decomposition module. k+1 、u k+1 and μ k+1 They represent the intermediate variable Z, output image u and constraint coefficient μ in the j+1th iteration, that is, the intermediate variable Z, output image u and constraint coefficient μ output by the k+1th Unrolling filter decomposition module.

[0083] The specific derivation process of the above expression is as follows:

[0084] Unrolling filter decomposition uses an iterative solution method, and the constraints are as follows:

[0085]

[0086] Where I is the input image decomposed by the Unrolling filter, u is the output image, w(Z) represents the constraint term related to the output image u, and λ is the constraint term coefficient.

[0087] The complex constraints are integrated into a constraint term related to the output image u. Solve the constraint and write the Lagrangian augmentation formula of the above constraint:

[0088]

[0089] Where μ and α represent the constraint coefficients of the Lagrangian augmentation formula, and Z represents the intermediate variable.

[0090] Using ADMM method to solve, we can get

[0091]

[0092] Taking the derivative of the above formula and setting it equal to 0, we get

[0093]

[0094] Perform iterative solution. In each iteration, update the output image u, intermediate variable Z, and constraint coefficients μ and α respectively. The expression can be obtained:

[0095]

[0096] where Z k 、μ k and α k They represent the intermediate variable Z and constraint coefficients μ and α in the kth iteration respectively. k+1 、Z k+1 , μ k+1 and α k+1 They represent the output image u, the intermediate variable Z, and the constraint coefficients μ and α in the k+1th iteration respectively, and Q() is a shrink function.

[0097] Use the Unrolling method to expand the above iterative update process, as shown in the attached Figure 2 As shown, combined with neural network training technology, the iterative optimization process is converted into an end-to-end training process, which encompasses the prior knowledge of traditional optimization iterative algorithms and the advantages of neural network technology.

[0098] The above iterative update process expressions are respectively Figure 2 The image filter decomposition submodule U Model in the neural network expansion filter decomposition module, Figure 2 The expressions of the intermediate variables updating submodules z Model, μModel and αModel in the optimization process of the neural network unfolding filter decomposition module.

[0099] In order to obtain better processing results, the function Q() uses a neural network to calculate the output result. The neural network structure adopts a fully convolutional network, as shown in the attached figure. Figure 3 In this embodiment, the number of Unrolling iterations is 3.

[0100] The L channels of the image to be evaluated and the reference image are input into the Unrolling filter module to obtain the filtered structural information.

[0101] u struct =UnrollingFilter(I L )

[0102] Wherein, UnrollingFilter() represents the Unrolling filter decomposition module, I L Represents the input L channel image, u struct Represents the structural information obtained after decomposition;

[0103] Subtract the decomposed structural information u from the input L channel image struct , get the lighting detail information, the expression is as follows:

[0104] u illumination =I L -u struct

[0105] Among them, I L Represents the input L channel image, u illiumination Represents the lighting detail information obtained after decomposition.

[0106] 3 Feature Extraction

[0107] The input image I and the structural information u obtained by filtering decomposition are respectively struct and lighting detail information u illumination Feature extraction is performed to extract illumination detail depth perception features, structural depth perception features, and overall depth perception features, which can better evaluate the quality of low-light image enhancement. Feature extraction uses a pre-trained VGG16 network, taking the output of its five hidden layers as the perception features.

[0108] The lighting detail information u of the low-light enhanced image and the reference image is illumination and structural information u struct As input, it is input into the VGGNet CNN neural network (Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition [J]. arXiv, 2014: 1409.1556.), which calculates the illumination detail information depth features and structural information depth features of the low-light enhanced image and the reference image;

[0109] In the VGGNet CNN neural network, the outputs of the five hidden convolutional layers are taken as the illumination detail information depth features and structural information depth features of the low-light enhanced image and the reference image respectively;

[0110] The low-light enhanced image and the reference image are input into the VGGNet CNN neural network to calculate the deep overall features corresponding to the low-light enhanced image and the reference image;

[0111] In the VGGNet CNN neural network, the outputs of the five hidden convolutional layers are taken as the overall features of the low-light enhanced image and the reference image respectively;

[0112] The VGGNet CNN neural network is used to obtain five depth features of illumination detail information, five depth features of structural information, and five overall features of the low-light enhanced image and the reference image. Figure 4 As shown in a, b, and c, they are the input image decomposed by Unrolling filter, the structural information of the output of Unrolling filter decomposition, and the illumination detail information obtained by Unrolling filter decomposition, respectively.

[0113] After obtaining the illumination detail depth perception features, structural depth perception features, and overall depth perception features, the similarity between the corresponding features of the image to be evaluated and the reference image is compared.

[0114] The calculation formula for the similarity of illumination detail depth features, structural depth features and overall features is as follows:

[0115]

[0116]

[0117]

[0118] Where i represents the i-th feature, i = 1, 2, 3, 4, 5, and Represent the similarity of the i-th illumination detail depth feature, the i-th structural depth feature and the i-th overall feature, and They represent the i-th illumination depth feature, i-th structural depth feature and i-th overall feature of the low-light enhanced image respectively, and They represent the i-th illumination detail depth feature, i-th structural depth feature and i-th overall feature of the reference image respectively.

[0119] 4 evaluation scores

[0120] The quality evaluation score is obtained by calculating the linear superposition of the similarity between the corresponding perceptual features of the reference image and the image to be evaluated. The final quality score formula is as follows:

[0121] S=w illumination D illumination +w struct D struct +w I D I

[0122] Among them, willumination 、w struct and w I They are the score weights of illumination detail depth feature similarity, structural feature similarity and overall feature similarity obtained through training, D illumination 、D struct and D I They are the overall similarity of illumination detail depth features, the overall similarity of structural depth features, and the overall similarity of overall features, namely:

[0123]

[0124] This embodiment further illustrates the low-light image enhancement quality evaluation method based on neural network expansion filter decomposition of Example 1 with the help of experiments:

[0125] This experiment evaluates the quality evaluation effect of the low-light image enhancement quality evaluation method based on neural network unfolded filter decomposition of Example 1 on low-light enhanced images in a low-light dataset, and tests the versatility of the low-light image enhancement quality evaluation method based on neural network unfolded filter decomposition of Example 1 on the public dataset LIVE. This experiment uses the Spearman rank-order correlation coefficient (SROCC) and Kendall rank-order correlation coefficient (KROCC), which are common indicators for evaluating image quality evaluation methods. And compare with commonly used image quality evaluation algorithms. Table 1 and Table 2 are the experimental results of low-light image enhancement quality evaluation and the experimental results of the LIVE dataset, respectively.

[0126] Table 1 Experimental results of low-light image enhancement quality evaluation

[0127]

[0128]

[0129] As shown in Table 1, in the low-light enhancement dataset, both srocc and krocc are higher than other IQA methods. This shows that our UnrollingIQA model is effective in the image quality assessment problem of low-light image enhancement. It is more suitable for the quality assessment problem of low-light image enhancement than other methods and has better quality assessment results.

[0130] Table 2 Experimental results of LIVE dataset

[0131] Method SROCC KROCC SSIM 0.948 0.797 VSI 0.952 0.806 MAD 0.967 0.842 FSIMc 0.965 0.836 GMSD 0.960 0.827 DeepIQA 0.947 0.791 LPIPSvgg 0.932 0.765 DISTS 0.954 0.811 Unrolling 0.97910 0.87979

[0132] As shown in Table 2, the results are quite good on the LIVE dataset, with both SROCC and KROCC reaching SOTA, proving that our method is quite versatile and can be applied to general image quality assessment problems, in addition to low-light image enhancement quality assessment.

[0133] In summary, according to the experimental results of Example 2, Example 1 of the present invention uses a neural network to carry out filtering decomposition to decompose the illumination detail information and structural information of the image. Compared with the traditional decomposition method, it combines the neural network end-to-end training method, with faster processing speed and better effect. By decomposing the illumination detail information and structural information, the quality of low-light enhanced images can be better evaluated, and the low-light image enhancement algorithm can be better assisted in improving. In addition to being able to play a role in the problem of low-light enhanced image quality evaluation, the present invention also has a certain degree of versatility and is also applicable to general image quality evaluation problems.

[0134] Example 2

[0135] This embodiment provides a low-light image enhancement quality assessment system based on neural network unfolding filter decomposition, including:

[0136] The color space conversion module converts the input image from RGB space to CIELAB space, which is more convenient for feature extraction;

[0137] The layer decomposition module decomposes the input low-light enhanced image and reference image using a neural network unfolding filter decomposition method to obtain the corresponding structural information and lighting detail information;

[0138] The feature extraction module inputs the structural information and illumination detail information obtained by the layer decomposition module and the overall information contained in the input image, performs feature extraction on the input structural information, illumination detail information and the overall information contained in the input image, and uses the VGGNet neural network to output the hidden layer of the neural network as the corresponding structural features, illumination detail features and overall features;

[0139] The evaluation module inputs the structural features, lighting detail features and overall features extracted by the feature extraction module, calculates the feature similarity between the input image to be evaluated and the corresponding features of the reference image, and obtains an evaluation score.

[0140] The low-light image enhancement quality evaluation system based on neural network expanded filter decomposition of this embodiment can execute the low-light image enhancement quality evaluation method based on neural network expanded filter decomposition provided in Example 1 of the method of the present invention, and has the corresponding functions and beneficial effects of the method.

[0141] Example 3

[0142] This embodiment also provides a low-light image enhancement quality assessment system based on neural network unfolding filter decomposition, including:

[0143] at least one processor;

[0144] at least one memory for storing at least one program;

[0145] When the at least one program is executed by the at least one processor, the at least one processor implements Figure 1 The method shown.

[0146] The low-light image enhancement quality evaluation system based on neural network expanded filter decomposition of this embodiment can execute the low-light image enhancement quality evaluation method based on neural network expanded filter decomposition provided in embodiment 1 of the method of the present invention, and can execute any combination of implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.

[0147] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the present invention to the specific embodiments described. Obviously, many modifications and variations are possible based on the content of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A low-light image enhancement quality assessment method based on neural network unfolding filter decomposition, characterized by: The following steps are involved: Inputting a low-light enhanced image and a reference image, and filtering and decomposing the low-light enhanced image and the reference image using a neural network unfolding filtering and decomposition method to obtain illumination detail information and structural information of the low-light enhanced image and the reference image; the neural network unfolding is referred to as Unrolling; Extracting features of the illumination detail information and structural information of the low-light enhanced image and the reference image obtained by the neural network filtering decomposition to obtain illumination detail depth features and structural depth features of the illumination detail information and structural information of the low-light enhanced image and the reference image, respectively; The process of expanding the filter decomposition based on the neural network is as follows: Input the image to be decomposed, and perform image color space conversion on the image to be decomposed, from RGB color space to CIELab color space; The L channel of the CIELab color space image is used as input to the Unrolling filter decomposition module to obtain the corresponding structural information. The expression is as follows: u struct =UnrollingFilter(I L ) Among them, UnrollingFilter() represents the Unrolling filter decomposition module, I L Represents the input L channel image, u Struct Represents the structural information obtained after decomposition; The structural information obtained by decomposition is subtracted from the L channel of the CIELab color space image to obtain the lighting detail information, which is expressed as follows: in ill mind i night i on =I L -in struct Among them, I L Represents the input L channel image, u illumination Represents the lighting detail information obtained after decomposition; A neural network is used to extract features from the low-light enhanced image and the reference image to obtain overall features of the low-light enhanced image and the reference image, respectively. A CNN neural network is used to extract features from illumination detail information of the low-light enhanced image, and the process is as follows: The illumination detail information of the low-light enhanced image is used as input to the VGGNet CNN neural network to calculate the depth features corresponding to the illumination detail information of the low-light enhanced image and the reference image; In the VGGNet CNN neural network, the outputs of the n hidden convolutional layers are taken as the depth features corresponding to the illumination detail information of the low-light enhanced image; The VGGNet CNN neural network is used to obtain n depth features of illumination detail information of low-light enhanced images; comparing similarities of illumination detail depth features, structural depth features, and overall features of the low-light enhanced image and the reference image; The final quality score is obtained through training and learning based on the similarity of the illumination detail depth features, structural depth features and overall features of the low-light enhanced image and the reference image.

2. The low-light image enhancement quality assessment method based on neural network unfolding filter decomposition according to claim 1, characterized in that: The neural network expansion filter decomposition module includes an image filter decomposition submodule and an intermediate variable update submodule during the optimization process; the neural network expansion filter decomposition method requires several alternating iterative optimizations, and the decomposition results and intermediate variables obtained from this iterative calculation are used as inputs for the next iterative calculation, and the next result is calculated, and the above iterative process is repeated until the final decomposition result is obtained.

3. The low-light image enhancement quality assessment method based on neural network expansion filter decomposition according to claim 1, characterized in that: The CNN neural network is used to extract the structural information of the low-light enhanced image. The process is as follows: The structural information of the low-light enhanced image is used as input to the VGGNet CNN neural network to calculate the depth features corresponding to the structural information of the low-light enhanced image; In the VGGNet CNN neural network, the outputs of the n hidden convolutional layers are taken as the deep features corresponding to the structural information of the low-light enhanced image; The VGGNet CNN neural network is used to obtain n structural information and depth features of the low-light enhanced image; The CNN neural network is used to extract the features of the illumination details of the reference image. The process is as follows: The illumination details of the reference image are used as input to the VGGNet CNN neural network to calculate the depth features corresponding to the illumination details of the reference image. In the VGGNet CNN neural network, the outputs of the n hidden convolutional layers are taken as the depth features corresponding to the illumination detail information of the reference image; The VGGNet CNN neural network is used to obtain n depth features of the illumination detail information of the reference image.

4. The low-light image enhancement quality assessment method based on neural network unfolding filter decomposition according to claim 1, characterized in that: The CNN neural network is used to extract features from the structural information of the reference image. The process is as follows: The structural details of the reference image are input into the VGGNet CNN neural network to calculate the deep features corresponding to the structural information of the reference image; In the VGGNet CNN neural network, the outputs of the n hidden convolutional layers are taken as the deep features corresponding to the structural information of the reference image; The VGGNet CNN neural network is used to obtain n structural information and deep features of the reference image.

5. The low-light image enhancement quality assessment method based on neural network unfolding filter decomposition according to claim 1, characterized in that: The CNN neural network is used to extract the overall features of the low-light enhanced image. The process is as follows: The entire low-light enhanced image is used as input to the VGGNet CNN neural network to calculate the depth features corresponding to the low-light enhanced image and the reference image; In the VGGNet CNN neural network, the outputs of the n hidden convolutional layers are taken as the overall features of the low-light enhanced image; The VGGNet CNN neural network is used to obtain n overall features of the low-light enhanced image.

6. The low-light image enhancement quality assessment method based on neural network unfolding filter decomposition according to claim 1, characterized in that: The CNN neural network is used to extract the overall features of the reference image. The process is as follows: The entire reference image is used as input to the VGGNet CNN neural network to calculate the deep features corresponding to the low-light enhanced image and the reference image; In the VGGNet CNN neural network, the outputs of the n hidden convolutional layers are taken as the overall features of the low-light enhanced image and the reference image respectively; The VGGNet CNN neural network is used to obtain n overall features of the low-light enhanced image and the reference image respectively.

7. The low-light image enhancement quality assessment method based on neural network unfolding filter decomposition according to any one of claims 1 to 6, characterized in that: The calculation formula for the similarity between the illumination detail depth feature, structural depth feature and overall feature of the low-light enhanced image and the reference image is as follows: Among them, i represents the i-th feature, and Represent the similarity of the i-th illumination detail depth feature, the i-th structural depth feature and the i-th overall feature, and They represent the i-th illumination detail depth feature, i-th structural depth feature and i-th overall feature of the low-light enhanced image respectively. and They represent the i-th illumination detail depth feature, i-th structural depth feature and i-th overall feature of the reference image respectively.

8. The low-light image enhancement quality assessment method based on neural network unfolding filter decomposition according to claim 7, characterized in that: Based on the similarity of illumination detail depth features, structural depth features, and overall features, the final quality score is obtained through training and learning as follows: S=w illumination D illumination +w struct D struct +w I D I Among them, w illumination 、w struct and w I They are the score weights of illumination detail depth feature similarity, structural feature similarity and overall feature similarity obtained through training, D illumination 、D struct and D I They are the overall similarity of illumination detail depth features, the overall similarity of structural depth features, and the overall similarity of overall features, namely:

Images