Low-light image enhancement method based on reversible neural network

Abstract

The invention relates to the technical field of low-light images, in particular to a low-light image enhancement method based on a reversible neural network. Comprising the steps of 1, building a reversible neural network; step 2, working process and training of the reversible neural network; and step 3, applying: inputting an image shot under a low-light condition into the trained reversible neural network for enhancement processing. Compared with the prior art, the method has the advantages that the enhancement effect and interpretability of the model are improved, the decomposition accuracy and the generalization ability of the model are improved, a memory mechanism is introduced, the brightness information of the whole data set is learned, the recovery process of the image in the brightness aspect is guided, and the enhancement effect of the model is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of low-light images, and in particular to a low-light image enhancement method based on a reversible neural network. Background Art

[0002] Images captured under low-light conditions often suffer from visual degradation problems such as low brightness, low contrast, and severe noise pollution. These visual degradation problems can cause the loss of image information, leading to malfunction or failure of computer vision systems. To address this challenge, low-light image enhancement (LLIE) has become a research direction that has attracted much attention in the field of computer vision. Low-light image enhancement aims to improve the visibility of low-light images and effectively suppress noise and artifacts caused by low light in the shooting environment. Through low-light image enhancement technology, various computer vision applications can benefit from improved low-light images and improve system performance and robustness.

[0003] In the early stages of the development of computer vision, low-light image enhancement mainly relied on traditional image processing techniques. These traditional LLIE methods can be divided into two categories, one based on image priors and the other based on physical models. Methods based on image priors rely on prior knowledge of image processing and computer vision. For example, histogram equalization is a common method based on image priors, which adjusts the global intensity distribution in a linear or nonlinear manner to improve the brightness and contrast of the image. However, these methods usually require manual parameter adjustment, are sensitive to noise, and have poor generalization ability when dealing with complex scenes. On the other hand, methods based on physical models guide image enhancement through imaging theory. For example, Retinex theory decomposes an image into an illumination component and a reflection component, and then processes the reflection component to achieve enhancement. This type of method has strong generalization ability and can adapt to different scenes. However, these methods are less effective in scenes with complex lighting conditions.

[0004] In recent years, deep learning technology has been widely used in LLIE tasks and has achieved excellent performance. The LLIE method based on deep learning improves the performance of low-light image enhancement by improving the structural design of the model. Compared with traditional methods, the deep LLIE method can obtain better results with the help of the powerful capabilities of Convolutional Neural Network (CNN) and Transformer neural network. Summary of the invention

[0005] Purpose of the present invention:

[0006] Most learning-based LLIE methods learn the mapping relationship between low-light images and normal-light images by using some paired data. However, in real scenarios, a low-light image can correspond to multiple normal-light images of different brightness. Models trained based on paired data often fuse multiple possible results as output, leading to problems such as uneven brightness enhancement, residual noise removal, and image artifacts. Some methods propose to learn the bidirectional mapping relationship between low-light images and normal-light images, and use the auxiliary information of the reverse process to guide the learning of the forward process. However, these methods simply use reversible networks without specific design for the task.

[0007] In addition, in order to further improve the quality of image enhancement, many methods introduce Retinex theory into the model. Since Retinex theory simulates the color perception of human vision, deep learning methods based on Retinex often achieve better results and better generalization capabilities. However, the illumination component and reflection component obtained by Retinex decomposition lack true values, and the accuracy and rationality of the decomposition results cannot be guaranteed. These works solve this problem by introducing additional priors or regularizers, but these manually designed regularization functions are difficult to apply to all scenarios. Joint optimization with too many constraints can also lead to lack of adaptability and efficiency.

[0008] The present invention mainly proposes optimization for the following problems of low-light image enhancement:

[0009] (1) Aiming at the specific task of low-light image enhancement, the present invention integrates Retinex theory into the reversible neural network, making the whole process more interpretable.

[0010] (2) Furthermore, Retinex decomposition can improve low-light image enhancement results, but the illumination component and the reflection component lack true values, and the accuracy and rationality of the decomposition results cannot be guaranteed. In order to accurately decompose and improve generalization ability, the present invention uses cross-color ratio imaging theory to guide Retinex decomposition.

[0011] (3) Furthermore, the present invention introduces a memory mechanism to improve the brightness restoration effect, learns the brightness information of the entire data set, and uses this information to guide low-light image enhancement.

[0012] Technical solution:

[0013] A low-light image enhancement design method based on a reversible neural network, characterized by comprising:

[0014] Step 1: Build a reversible neural network;

[0015] Step 2: Working process and training of reversible neural network;

[0016] Step 3 Application: Input the images taken under low light conditions into the reversible neural network trained above for enhancement processing.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] (1) Interpretability of the enhancement process: The present invention redesigns the reversible network structure so that the reversible network enhancement process can be explained using the Retinex theory, thereby improving the enhancement effect and interpretability of the model.

[0019] (2) Generalizability of image decomposition: The present invention introduces a cross color ratio physical model to guide the image decomposition process based on the Retinex theory, thereby increasing the accuracy of decomposition and the generalization ability of the model.

[0020] (3) Global guidance of brightness information: The present invention introduces a memory mechanism to learn the brightness information of the entire data set, guide the image restoration process in terms of brightness, and improve the enhancement effect of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is the overall framework flow chart of the present invention;

[0022] Figure 2 for Figure 1 Detailed description of the feature enhancement module in ;

[0023] Figure 3 Visual comparison results on the LOL-v1 dataset;

[0024] Figure 4 Visual comparison results on the LOLv2-real dataset;

[0025] Figure 5 Visual comparison results on the SID dataset;

[0026] Figure 6 Visual comparison results on the SMID dataset;

[0027] Figure 7 Visual comparison results on the SDSD-in dataset;

[0028] Figure 8 Figure 2 is a visual comparison of the results on the SDSD-out dataset. DETAILED DESCRIPTION

[0029] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0030] The present invention proposes a low-light image enhancement method based on a reversible neural network, which integrates the Retinex theory with the reversible neural network to better learn the one-to-one mapping between low-light images and normal-light images. Specifically, the present invention redesigns the forward process and the reverse process of the reversible neural network so that the forward process can be expressed as image enhancement based on the Retinex theory, and the reverse process can be expressed as image degradation based on the Retinex theory. Furthermore, the present invention introduces the cross color ratio (Cross Color Ratio, CCR) physical model to guide the decomposition results of Retinex, improve the accuracy of the decomposition and the generalization ability of the model. In addition, the present invention introduces a memory mechanism to learn the brightness information of the data set, further improving the brightness restoration effect of low-light images.

[0031] like Figure 1 , Figure 2 As shown in the figure, a low-light image enhancement design method based on a reversible neural network is described in detail as follows:

[0032] Step 1: Build a reversible neural network

[0033] The reversible neural network consists of a feature encoding module φ(·) and And the stacked n reversible modules θ(·), and the feature decoding module ψ(·), the feature encoding module φ(·) and After parallel connection, it is sequentially connected with the stacked reversible module θ(·) and feature decoding module ψ(·).

[0034] The feature encoding module φ(·) and They are all composed of two convolutional layers, where the size of the convolution kernel is 3x3 and the step size is 1.

[0035] The reversible module θ(·) is composed of four feature enhancement modules G1(·), H1(·), G2(·), and H2(·), according to Figure 1 They are connected in the manner shown and have the same network structure.

[0036] The feature decoding module ψ(·) is also composed of two convolutional layers, where the size of the convolution kernel is 3x3 and the step size is 1.

[0037] More specifically,

[0038] like Figure 2As shown in (a), the feature enhancement modules G1(·), H1(·), G2(·), and H2(·) are composed of a dual-branch attention module (DBAM) with residual and a feed-forward neural network (FFN) with residual, wherein the feed-forward neural network (FFN) is implemented by a 3x3 convolutional layer.

[0039] like Figure 2 As shown in (b), the dual-branch attention module (DBAM) consists of a maximum pooling layer (Max Pooling), a Transformer branch, a convolutional neural network (CNN) branch (referred to as: CNN branch), a channel concatenation layer (Concatenate), a 3x3 convolution layer and an interpolation layer (Interpolate). The Transformer branch and the CNN branch are connected in parallel and are sequentially connected to other layers.

[0040] in:

[0041] The CNN branch is composed of a dual attention unit (DAU) with residuals. The network structure of DAU is as follows: Figure 2 (b) As shown in the dotted box, it also contains two branches. The first branch consists of two 1x1 convolutional layers with a scaled exponential linear unit (SELU) activation function, and the second branch adds two 1x1 convolutional layers with a SELU activation function after the global average pooling (GAP) layer. The results of the two branches are added and normalized by the Sigmoid function, and the normalized result is used as the weight to weight the second input of DAU.

[0042] The Transformer branch consists of a cross-attention module and a dual attention unit (DAU) in sequence. The cross-attention module consists of a 1x1 convolutional layer and a Softmax function, and additionally introduces a global variable M and an interpolation layer (Interpolate).

[0043] As mentioned above, the dual attention unit (DAU) of the Transformer branch and the dual attention unit (DAU) of the CNN branch have the same network structure, but the input data are different.

[0044] Step 2: Working process and training of reversible neural network

[0045] Includes the design steps of the forward process

[0046] Step 1.1 Calculate the cross-color ratio map.

[0047] Low light image I low As the input of the forward process of the reversible neural network, the cross color ratio is first calculated by formula f ccr (·) Calculate the cross color ratio map

[0048]

[0049] Step 1.2 Feature extraction.

[0050] The low light image I low and cross color ratio image are input into the feature encoder φ(·) and In the above example, we extract their respective features. and

[0051]

[0052] Step 1.3 Feature transformation.

[0053] feature and Input into the n stacked reversible modules θ(·) for feature conversion to obtain the converted features and Corresponding to the enhanced image features and the features of the enhanced image cross color ratio map. For the i-th reversible module θ i (·), input features and Based on the feature enhancement modules G1(·), H1(·), G2(·), H2(·), feature conversion operations are performed. The key innovation is that the feature conversion process is combined with the Retinex theory. In addition, the cross color ratio map is used to guide the decomposition results of Retienx. Based on the Retienx theory, the input feature and It can be interpreted as:

[0054]

[0055] in represents the reflection component, Represents the illumination component, and represents the noise component. Further, the present invention converts the output of the feature enhancement module G2(·) into Considered as noise component The output of feature enhancement module H2(·) Considered as light component Then the reversible module θ i (·) can be achieved by Remove noise components and add lighting component To convert it into features

[0056]

[0057] Similarly, by taking the output of the feature enhancement module G1(·) Considered as noise component The output of feature enhancement module H1(·) Considered as light component The reversible module θ i (·) Further by removing the noise component and remove the illumination component The characteristics Convert to Features

[0058]

[0059]

[0060] The feature enhancement modules G1(·), H1(·), G2(·), and H2(·) are used to enhance the input features, and have the following characteristics: Figure 2 (a) shows the structure. Taking G1(·) as an example, the input feature x is processed as follows:

[0061] G1(x)=f FFN (f DBAM (x)+x)+(f DBAM (x)+x)

[0062] where f DBAM (·) denotes the dual-branch attention module (DBAM), f FFN (·) represents the feedforward neural network (FNN).

[0063] like Figure 2 As shown in (b), the dual-branch attention module (DBAM) transforms the input feature x into feature x ′ The input feature x first passes through the maximum pooling layer f MP (·) downsamples to reduce the computational complexity of the model. Then it is split into features x along the channel dimension cnn and feature x trans, used as input to the CNN branch and the Transformer branch:

[0064] {x cnn ,x trans} = f MP (x).

[0065] The CNN branch is mainly composed of dual attention units (DAU) with residuals, which are used to cnn To enhance:

[0066] x′ cnn =f DAU (x cnn ,x cnn )+x cnn .

[0067] like Figure 2 (b) As shown in the dotted box, the dual attention unit (DAU) accepts two features as input. The first feature x cnn After the dual-branch structure is enhanced, the second feature x cnn To perform weighting:

[0068]

[0069] The Transformer branch is composed of a criss-cross attention module and a dual attention unit (DAU) in sequence. For the criss-cross attention module, first start from x trans Calculate the key and value in the CNN branch and convert the result x′ into cnn It is used as a query to calculate the cross attention. In addition, it is observed that the existing model has poor effect in restoring image brightness information. Only by modulating the global brightness information of the true value image can the reconstruction effect and index be effectively improved. Therefore, the cross attention module introduces a memory mechanism and uses the global variable M∈R 1×C×3×3 To learn the brightness information of the data set and modify the size of M using interpolation, then Q Modulate for better reconstruction:

[0070] {x K ,x V} = f conv (x trans ),x Q = x′ cnn ×f interp (M)

[0071]

[0072] Enhanced feature x′ transThe DAU has the same network structure as the DAU in the CNN branch, but the input features are different to achieve x′ cnn Weighting of:

[0073]

[0074] in:

[0075]

[0076] The result x″ of the CNN branch cnn and the result x′ of the Transformer branch trans Concatenate along the channel dimension, then input to the 3x3 convolution for processing, and restore to the input size through interpolation:

[0077]

[0078] Step 1.4 Image reconstruction.

[0079] Enhance image features and the features of the cross color ratio map of the enhanced image Input to the feature decoding module ψ(·) to obtain the enhanced image I′ norm :

[0080]

[0081] and design steps including the reverse process

[0082] Step 2.1 Calculate the cross-color ratio map.

[0083] Normal image I norm As the input of the reversible neural network reverse process, firstly, the cross color ratio is calculated by formula f ccr (·) Calculate the cross color ratio map

[0084]

[0085] Step 2.2 Feature extraction.

[0086] The normal image I norm and cross color ratio image are input into the feature encoder φ(·) and In the above example, we extract their respective features. and

[0087]

[0088] Step 2.3: Feature inverse transformation.

[0089] feature and Input into the n stacked reversible modules θ(·) for feature inverse transformation to obtain the inverse transformed feature and Corresponding to the degraded image features and the features of the degraded image cross-color ratio map. For the i-th reversible module θ i (·), input features and Based on the feature enhancement modules G1(·), H1(·), G2(·), H2(·), the feature inverse transformation operation is performed. Based on the Retienx theory, the input feature and It can be interpreted as:

[0090]

[0091] in represents the reflection component, Represents the illumination component, and Represents the noise component. As in the forward process, the present invention converts the output of the feature enhancement module G1(·) into Considered as noise component The output of feature enhancement module H1(·) Considered as light component Then the reversible module θ i (·) can be achieved by and add lighting component and add noise components To convert it into features

[0092]

[0093] Similarly, by taking the output of the feature enhancement module G2(·) Considered as noise component The output of feature enhancement module H2(·) Considered as light component The reversible module θ i (·) Further by removing the illumination component and add noise components The characteristics Convert to Features

[0094]

[0095]

[0096] The feature enhancement modules G1(·), H1(·), G2(·), and H2(·) are used to enhance the input features, and have the following characteristics: Figure 2 (a) shows the structure.

[0097] Step 2.4 Image reconstruction.

[0098] Characteristics of Degraded Images The characteristics of the cross color ratio map of the degraded image Input to the feature decoding module ψ(·) to obtain the degraded image I′ low :

[0099]

[0100] And the design steps including loss function

[0101] In the training phase, the L1 distance is used as the loss function of the method of the present invention. For the forward process (forward) and the backward process (backward), there are:

[0102]

[0103] Since using only L1 loss may result in smooth images, perceptual loss is further used for constraints to improve the perceptual quality of the image:

[0104]

[0105] Among them VGG i (·) represents the i-th layer of VGG-19 pre-trained on ImageNet, i∈{1,3,5,9,13}. In addition, ω i is the weight corresponding to the i-th layer. In the present invention, ω i Set to 1. In addition, the forward process needs to denoise the low-light image, so the edge-aware smoothing loss is introduced to improve the denoising effect:

[0106]

[0107] in and Represents the gradient in the horizontal and vertical directions. Therefore, the total loss of the present invention is the weighted sum of the above losses:

[0108]

[0109] where λ i Represents the weight, take λ1=λ4=0.05, λ2=λ3=1.

[0110] Step 3 Application

[0111] Based on the above design, images taken under low-light conditions are input into the trained reversible neural network for enhancement processing.

[0112] The present invention achieves good results on multiple public data sets. For a detailed description, please refer to the examples.

[0113] Key points of the technical solution design of the present invention

[0114] (1) Reversible neural network integrating Retinex theory.

[0115] In order to make the reversible neural network more suitable for low-light image enhancement tasks and improve the interpretability of the network, the present invention redesigns the structure of the reversible network so that the feature transformation process of the reversible neural network can be explained by the Retinex theory. Specifically, the forward process can be explained as enhancing the illumination component while removing the noise component, and the reverse process can be explained as degrading the illumination component while adding the noise component. Based on this design, the process of image enhancement by the reversible neural network can be well explained, and the reversible neural network can handle low-light image enhancement tasks more specifically.

[0116] (2) Retinex decomposition guided by cross-color ratio graph.

[0117] In order to solve the problem of lack of effective constraints in Retinex decomposition and improve the generalization ability of the model, the present invention adopts the imaging theory of cross color ratio map to guide the decomposition process. Specifically, the Retinex theory decomposes the image into illumination component and reflection component, but lacks the corresponding decomposition result true value for constraint. In imaging theory, the cross color ratio map represents the gradient of illumination invariance, and its value is related to the reflectivity. Therefore, the cross color ratio map can be used to guide the reflection component, thereby effectively constraining and guiding the result of image decomposition.

[0118] (3) Dataset brightness learning based on memory mechanism.

[0119] In order to encode the overall brightness information of the data set and improve the model's ability to restore image brightness information, the present invention adopts a feature enhancement module based on a memory mechanism. Specifically, the reversible neural network implements the feature conversion process through the feature enhancement module G / H, and each feature enhancement module is implemented by a complex neural network. The present invention adopts a model design combining CNN and Transformer to extract global and local information. On this basis, the present invention further introduces a memory mechanism to learn the brightness information M of the entire data set and modulate the Q component of the Transformer.

[0120] Advantages of the present invention

[0121] (1) Most methods only learn the mapping relationship from low-light images to normal-light images, but this mapping relationship is ill-posed and the model tends to learn the mean of multiple mappings, which affects the model effect. Some methods propose to learn the bidirectional mapping relationship between low-light images and normal-light images, and use the auxiliary information of the reverse process to guide the learning of the forward process. These methods simply use reversible networks and lack improvements for specific tasks. The present invention redesigns the structure of the reversible neural network for the low-light image enhancement task, so that the enhancement process of the reversible network can be explained by the Retinex theory, thereby improving the performance of the model.

[0122] (2) Since the Retinex theory simulates the color perception of human vision, many methods have introduced it into the low-light image enhancement task and achieved good results. However, the illumination component and reflection component obtained by Retinex decomposition lack true value constraints, and the accuracy and rationality of the decomposition results cannot be guaranteed. Some methods solve this problem by introducing additional priors or regularizers, but lack generalization ability and are difficult to apply to more scenes. The present invention introduces the cross color ratio theory and uses this illumination invariant to guide the reflection component of Retinex decomposition, achieving better decomposition results and generalization ability.

[0123] (3) Existing methods are not very effective in brightness restoration. The model is updated based on limited data each time and cannot effectively learn the brightness information of the entire dataset. This method introduces a memory mechanism that can receive information from all images in the dataset during training. Therefore, it can be used to learn the brightness information of the dataset and guide brightness restoration in low-light image enhancement.

[0124] Example

[0125] Dataset

[0126] The experiments were conducted on commonly used low-light image enhancement benchmark datasets, including LOL-v1, LOL-v2, SID, SMID, SDSD, and LSRW. In order to prove the generalization ability of the model, the present invention was also tested on 5 datasets without true values, including DICM, LIME, MEF, NPE, and VV.

[0127] Method application

[0128] A method process for low-light image enhancement based on a reversible neural network, comprising:

[0129] Forward process description: First, based on the CCR theory ccr (·), using low-light image Ilow Compute the cross color ratio map Next, the low-light image I low and cross color ratio are input into the feature encoding module φ(·) and Extracting features from low-light images and the corresponding cross color ratio map features Then, the features and Input into n reversible modules θ(·) for transformation to obtain enhanced features and Finally, the features and Input to the feature decoding module ψ(·) to obtain the enhanced image I′ norm .

[0130] Description of the reverse process: First, based on the CCR theory ccr (·), using the normal image I norm Compute the cross color ratio map Next, the normal image I norm and cross color ratio are input into the feature encoding module φ(·) and Extract the features of normal images and the corresponding cross color ratio map features Then, the features and Input into n reversible modules θ(·) for inverse transformation to obtain the degraded features and Finally, the features and Input to the feature decoding module ψ(·) to obtain the degraded image I′ low .

[0131] The unfinished parts are the same as above and Figure 1-Figure 2 The network structure and training algorithm (loss function) in the embodiment are not described in detail. The present invention is based on the Pytorch framework and is trained on an NVIDIA 4090 GPU. The present invention uses the Adam optimizer for training, where β1 is set to 0.9 and β2 is set to 0.999. The initial learning rate is set to 3×e-4 and decays to 1×e-6 using a cosine annealing scheme. In addition, the batch size and patch size are set to 5 and 256×256, respectively, and the number of iterations is set to 2.5×10 5 The training data is randomly augmented, including horizontal flipping, vertical flipping, and 90° rotation.

[0132] Comparison of implementation effects

[0133] For comparison purposes, we selected several state-of-the-art methods that have been improved for low-light image enhancement tasks, including KinD, LLFlow, SNR-Net, LLFormer, PyDiff, UHDFour, and DiffLL.

[0134] First, we compare the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) metrics on multiple paired datasets. There are two test methods for existing methods, one is direct testing and the other is testing after adjustment using the average of the ground-truth images. For fair comparison, we implemented both test methods on all methods and compared the corresponding full-reference metrics respectively. The metrics without ground-truth adjustment are denoted as PSNR and SSIM, and the metrics with ground-truth adjustment are denoted as PSNR-G and SSIM-G. For methods with pre-trained weights, the weights are used directly for testing. For methods without pre-trained weights, the original parameters are kept unchanged and trained for 300,000 iterations. Tables 1-9 show the performance of our method and the comparison methods on 6 datasets and the average results. Our method achieves the best or sub-optimal results on multiple datasets, while achieving the best average results. Then we compare the generalization ability of the model on five unpaired datasets. We uniformly use the weights trained on the LOLv1 dataset for testing. Table 10 shows the Natural Image Quality Evaluator (NIQE) indicators of the method of the present invention and the comparative method. The method of the present invention achieves optimal or suboptimal results on multiple datasets, and has suboptimal average results, indicating that the method of the present invention has good generalization ability.

[0135] Table 1 Comparison of experimental results of the present invention with other models on the LOL-v1 dataset

[0136]

[0137] Table 2 Comparison of experimental results of the present invention with other models on the LOL-v2-real dataset

[0138]

[0139]

[0140] Table 3 Comparison of experimental results of the present invention with other models on the SID dataset

[0141]

[0142] Table 4 Comparison of experimental results of the present invention with other models on the SMID dataset

[0143]

[0144] Table 5 Comparison of experimental results of the present invention with other models on the SDSD-in dataset

[0145]

[0146] Table 6 Comparison of experimental results of the present invention with other models on the SDSD-out dataset

[0147]

[0148] Table 7 Comparison of experimental results of the present invention with other models on the LSRW-Huawei dataset

[0149]

[0150]

[0151] Table 8 Comparison of experimental results of the present invention with other models on the LSRW-Nikon dataset

[0152]

[0153] Table 9 Comparison of average experimental results of the present invention and other models on multiple data sets

[0154]

[0155] Table 10 Quantitative results of NIQE on DICM, LIME, MEF, NPE and VV datasets

[0156]

[0157]

[0158] Figure 3 to Figure 8 The visual comparison results of the present invention and the previous methods on multiple data sets are as follows:

[0159] Figure 3 The visualization results on the LOLv1 dataset are shown. KinD performs image enhancement based on Retinex theory and reduces chromatic aberration, but the brightness restoration is limited. Although the SNR-Net, LLFormer, and UHD-Four methods further restore the brightness and contrast of the image, there are obvious noise and artifacts.

[0160] Both the LLFlow method and the PyDiff method are based on generative models to achieve low-light image enhancement. Although the visual effect is good, the restored image is too smooth and a lot of detail information is lost. Since the DiffLL method finally uses the inverse wavelet transform to obtain the image, there are obvious checkerboard artifacts. In contrast, the present invention not only removes noise artifacts, but also retains detailed information, and has the best visual effect. We also visualized the test results on the LOL-v2-real, SID, SMID, SDSD-in, and SDSD-out datasets, as shown in Figure 2. Figure 4-Figure 8 shown.

Claims

1. A low-light image enhancement design method based on a reversible neural network, characterized in that: include: Step 1: Build a reversible neural network; Step 2: Working process and training of reversible neural network; Step 3 Application: Input the images taken under low light conditions into the reversible neural network trained above for enhancement processing.

2. A low-light image enhancement design method based on a reversible neural network as claimed in claim 1, characterized in that: Step 1: Build a reversible neural network: The reversible neural network consists of a feature encoding module φ(·) and And the stacked n reversible modules θ(·), and the feature decoding module ψ(·), the feature encoding module φ(·) and After parallel connection, it is sequentially connected with the stacked reversible module θ(·) and feature decoding module ψ(·), where: The feature encoding module φ(·) and They are all composed of two convolutional layers (ConvolutionalLayer); the reversible module θ(·) is composed of four feature enhancement modules G1(·), H1(·), G2(·), H2(·), which have the same network structure; the feature decoding module ψ(·) is also composed of two convolutional layers.

3. A low-light image enhancement design method based on a reversible neural network as claimed in claim 2, characterized in that: The feature enhancement modules G1(·), H1(·), G2(·), and H2(·) are composed of a dual-branch attention module DBAM with residual and a feed-forward neural network FFN with residual; The dual-branch attention module DBAM consists of a maximum pooling layer, a Transformer branch, a CNN branch, a channel concatenation layer, a 3x3 convolution layer, and an interpolation layer. The Transformer branch and the CNN branch are connected in parallel and sequentially connected to other layers. in: The CNN branch is composed of a dual attention unit DAU with residual. The dual attention unit DAU contains two branches. The first branch consists of two 1x1 convolutional layers with a scaled exponential linear unit SELU activation function. The second branch adds two 1x1 convolutional layers with a SELU activation function after the global average pooling layer GAP. The results of the two branches are added and normalized by the Sigmoid function. The normalized result is used as the weight to weight the input of the dual attention unit DAU. The Transformer branch consists of a cross-attention module and a dual attention unit DAU in sequence. The cross-attention module consists of a 1x1 convolutional layer and a Softmax function, and additionally introduces a global variable M and an interpolation layer Interpolate.

4. The low-light image enhancement design method based on a reversible neural network as claimed in claim 1, characterized in that: Step 2: Working process and training of reversible neural network. Includes design steps for the forward process; and design steps including the reverse process; And the design steps including loss function.

5. A low-light image enhancement design method based on a reversible neural network as claimed in claim 4, characterized in that: The design steps of the forward process include: Step 1.1 Calculate the cross color ratio map Low light image I low As the input of the forward process of the reversible neural network, the cross color ratio is first calculated by formula f ccr (·) Calculate the cross color ratio map Step 1.2 Feature extraction The low light image I low and cross color ratio image are input into the feature encoder φ(·) and In the above example, we extract their respective features. and Step 1.3 Feature transformation feature and Input into the n stacked reversible modules θ(·) for feature conversion to obtain the converted features and Corresponding to the enhanced image features and the features of the enhanced image cross color ratio map. For the i-th reversible module θ i (·), input features and Based on the feature enhancement modules G1(·), H1(·), G2(·), H2(·), feature conversion operation is performed; based on Retienx theory, the input feature and Interpretation: in represents the reflection component, Represents the illumination component, and represents the noise component; further, the output of the feature enhancement module G2(·) Considered as noise component The output of feature enhancement module H2(·) Considered as light component Then the reversible module θ i (·) By Remove noise components and add lighting component To convert it into features Similarly, by taking the output of the feature enhancement module G1(·) Considered as noise component The output of feature enhancement module H1(·) Considered as light component The reversible module θ i (·) Further by removing the noise component and remove the illumination component The characteristics Convert to Features The feature enhancement modules G1(·), H1(·), G2(·), H2(·) are used to enhance the input features. Taking G1(·) as an example, the processing process of the input feature x is as follows: G1(x)=f FFN (f DBAM (x)+x)+(f DBAM (x)+x) where f DBAM (·) represents the dual-branch attention module DBAM, f FFN (·) represents the feedforward neural network FNN; The dual-branch attention module DBAM transforms the input feature x into feature x′; The input feature x first passes through the maximum pooling layer f MP (·) downsamples to reduce the computational complexity of the model; then it is split into features x along the channel dimension cnn and feature x trans , used as input to the CNN branch and the Transformer branch: {x cnn ,x trans }=f MP (x). The CNN branch is mainly composed of a dual attention unit DAU with residual, which is used to cnn To enhance: x′ cnn =f DAU (x cnn ,x cnn )+x cnn . The dual attention unit DAU accepts two features as input; The first feature x cnn After the dual-branch structure is enhanced, the second feature x cnn To perform weighting: The Transformer branch consists of a cross-attention module and a dual-attention unit DAU sequence; For the cross attention module, first start from x trans Calculate the key Key and value Value, and the result x′ of the CNN branch cnn Used as a query to calculate cross attention; Introduce a memory mechanism in the cross-attention module, using a global variable M∈R 1×C×3×3 To learn the brightness information of the dataset and modify the size of M using interpolation, then Q Modulate for better reconstruction: {x K ,x V }=f conv (x trans ),x Q =x′ cnn ×f interp (M) Enhanced feature x′ trans The dual attention unit DAU is further input for processing; the dual attention unit DAU has the same network structure as the dual attention unit DAU in the CNN branch, but the input features are different to achieve x′ cnn Weighting of: in: The result x″ of the CNN branch cnn and the result x′ of the Transformer branch trans Concatenate along the channel dimension, then input to the 3x3 convolution for processing, and restore to the input size through interpolation: Step 1.4 Image reconstruction Enhance image features and the features of the cross color ratio map of the enhanced image Input to the feature decoding module ψ(·) to obtain the enhanced image I′ norm :

6. A low-light image enhancement design method based on a reversible neural network as claimed in claim 4, characterized in that: The design steps of the reverse process include: Step 2.1 Calculate the cross color ratio map Normal image I norm As the input of the reversible neural network reverse process, firstly, the cross color ratio is calculated by formula f ccr (·) Calculate the cross color ratio map Step 2.2 Feature extraction The normal image I norm and cross color ratio image are input into the feature encoder φ(·) and In the above example, we extract their respective features. and Step 2.3 Feature inverse transformation feature and Input into the n stacked reversible modules θ(·) for feature inverse transformation to obtain the inverse transformed feature and Corresponding to the features of the degraded image and the features of the cross-color ratio map of the degraded image; for the i-th reversible module θ i (·), input features and Based on the feature enhancement modules G1(·), H1(·), G2(·), H2(·), feature inverse transformation operation is performed; based on Retienx theory, the input feature and Interpretation: in represents the reflection component, Represents the illumination component, and represents the noise component; Same as the forward process, the output of the feature enhancement module G1(·) Considered as noise component The output of feature enhancement module H1(·) Considered as light component Then the reversible module θ i (·) By and add lighting component and add noise components To convert it into features Similarly, by taking the output of the feature enhancement module G2(·) Considered as noise component The output of feature enhancement module H2(·) Considered as light component The reversible module θ i (·) Further by removing the illumination component and add noise components The characteristics Convert to Features Step 2.4 Image reconstruction Characteristics of Degraded Images The characteristics of the cross color ratio map of the degraded image Input to the feature decoding module ψ(·) to obtain the degraded image I′ low :

7. A low-light image enhancement design method based on a reversible neural network as claimed in claim 4, characterized in that: The design steps of the loss function are: In the training phase, L1 distance is used as the loss function. For the forward process (forward) and the backward process (backward), there are: The perceptual loss is further constrained to improve the perceptual quality of the image: Among them VGG i (·) represents the i-th layer of VGG-19 pre-trained on ImageNet, i∈{1, 3, 5, 9, 13}; in addition, ω i is the weight corresponding to the i-th layer; In the forward process, the low-light image is denoised and edge-aware smoothing loss is introduced to improve the denoising effect: in and Represents horizontal and vertical gradients; In summary, the total loss is the weighted sum of the above losses: where λ i Represents the weight, take λ1=λ4=0.05, λ2=λ3=1.

Images