Dark light enhancement method based on multi-task twin neural network

Abstract

The invention discloses a dark light enhancement method based on a multi-task twin neural network, and relates to the technical field of image enhancement, and the training process comprises the steps: collecting a dark light image and a normal light image, and carrying out the preprocessing; inputting the dark light image and the normal light image into an RL twinborn decomposition network to obtain a decomposed dark light reflection image, a normal light reflection image, a dark light brightness image and a normal light brightness image; splicing the feature maps extracted by the twin neural network, and sending the spliced feature maps to a foreground segmentation network branch to obtain a foreground prediction mask map; sending the dark light brightness image into a brightness adjustment network to obtain a brightness image after dark light enhancement, performing dot product on the brightness image after dark light enhancement and the dark light reflection image to obtain an enhanced image, and comprehensively training the network; the use process comprises the following steps: deleting auxiliary training branches, forming an end-to-end model, and directly inputting a dark light image into an image enhancement model to obtain an enhanced image; according to the dark light enhancement method, the dark light image enhancement effect is optimized, the reasoning time is shortened, and the deployment complexity is reduced.

Description

Technical Field

[0001] The present invention relates to the field of image enhancement technology, and in particular to a dark light enhancement method based on a multi-task twin neural network. Background Art

[0002] Dark-light images refer to images taken in low-light or extremely dark environments. Due to insufficient lighting, such images usually have problems such as low brightness, poor contrast, high noise, and color distortion. These defects not only affect the visual quality of the image, but also bring great challenges to subsequent computer vision tasks (such as target detection, classification, and segmentation). In the fields of security monitoring, autonomous driving, medical imaging, smartphone photography, astronomical observation, etc., improving the quality of dark-light images is an urgent problem to be solved. The goal of dark-light image enhancement methods is to increase brightness, reduce noise, and restore color while retaining the original details of the image through algorithmic means, so as to better meet the needs of visual perception and downstream tasks.

[0003] Dark-light image enhancement methods have gone through a development process from traditional methods to modern methods driven by deep learning. Traditional methods mainly use techniques based on pixel value adjustment to enhance images. For example, histogram equalization improves contrast by adjusting the distribution of image pixels, gamma correction improves image brightness through nonlinear transformation, and the Retinex method simulates the human eye's visual mechanism and decomposes the image into two parts: reflected light and ambient light for enhancement. These methods are simple and efficient, but often lead to loss of image details, color distortion or over-enhancement during the enhancement process, and cannot adapt well to complex low-light scenes.

[0004] In recent years, deep learning has provided a new solution for low-light image enhancement. End-to-end models based on convolutional neural networks (CNNs) and generative adversarial networks (GANs) can learn nonlinear mappings from a large number of low-light and high-light image pairs, significantly improving the enhancement effect. For example, LLNet is an early CNN-based low-light enhancement model that can achieve denoising and brightness enhancement at the same time. Although the model has a simple structure and is easy to implement, the receptive field of CNN is limited, making it difficult to capture global dependencies in images, and simple networks may have difficulty optimizing denoising and detail enhancement tasks at the same time. Models such as EnlightenGAN and Zero-DCE++ use generative adversarial learning or brightness curve estimation in an unsupervised or weakly supervised framework to achieve more natural image enhancement effects, but the generator and discriminator of GAN need to be finely balanced, and the training process is prone to mode collapse or gradient vanishing problems. In addition, because unsupervised learning is highly dependent on the design of the loss function, it is difficult to train a high-performance model.

[0005] Transformer can capture global correlations in images through the attention mechanism, solving the problem of limited receptive field in CNN. At the same time, in low-light images, Transformer's modeling of dependencies between long-distance pixels helps retain more details. However, Transformer also has problems such as high computing resource requirements, strong dependence on data, and complex architecture. Summary of the invention

[0006] Based on the technical problems existing in the background technology, the present invention proposes a dark light enhancement method based on a multi-task twin neural network, which optimizes the dark light image enhancement effect, while reducing the inference time and reducing the deployment complexity.

[0007] The present invention proposes a dark light enhancement method based on a multi-task twin neural network, which inputs a dark light image into a trained image enhancement model to obtain an enhanced image;

[0008] The image enhancement model includes an RL twin decomposition network, a foreground segmentation network branch, and a brightness adjustment network, wherein the RL twin decomposition network includes two RL decomposition network branches, and when the trained image enhancement model is used, the foreground segmentation network branch and the normal light RL decomposition network branch are deleted;

[0009] The training process of the image enhancement model is as follows:

[0010] Step 1: Collect dark light images and normal light images and preprocess them to construct an image training data set;

[0011] Step 2: Input the dark light image and the normal light image into the RL twin decomposition network to obtain the decomposed dark light reflection image, normal light reflection image, dark light brightness image and normal light brightness image;

[0012] Step 3: splice the four feature images output by the RL twin neural network and send them to the foreground segmentation network branch to output the foreground prediction mask image;

[0013] Step 4: Send the dark light brightness image to the brightness adjustment network to output the dark light enhanced brightness image, and do a dot product between the dark light enhanced brightness image and the dark light reflection image to obtain the enhanced image;

[0014] Step 5: Construct a total loss function to adjust the parameters to be trained in the image enhancement model.

[0015] Furthermore, two RL decomposition networks are used for decomposing dark light images and normal light images respectively, and decompose the dark light images into dark light reflection images and dark light brightness images, and decompose the normal light images into normal light reflection images and normal light brightness images;

[0016] The normal light image decomposition branch and the dark light image decomposition branch are used as the twin network R, and the dark light brightness image decomposition branch and the normal light brightness image decomposition branch are used as the twin network L. All weights of the two branches in the twin network R are shared, and all weights of the two branches in the twin network L are shared.

[0017] Further, in step three, the foreground segmentation network branch includes a two-dimensional convolutional layer 1, a normalization layer 1, an activation layer 1, a position encoding layer 1 and a transformer encoding layer 1;

[0018] The spliced ​​image passes through two-dimensional convolution layer 1, normalization layer 1, activation layer 1, position encoding layer 1, and transformer encoding layer 1 in sequence to obtain the foreground prediction mask map.

[0019] Further, in step 4, the brightness adjustment network includes a two-dimensional convolution layer 2, a normalization layer 2, an activation layer 2, a position encoding layer 2, and a transformer encoding layer 2;

[0020] The dark-light brightness image passes through two-dimensional convolution layer 2, normalization layer 2, activation layer 2, position encoding layer 2, and transformer encoding layer 2 in sequence to output a dark-light enhanced brightness image.

[0021] Furthermore, in step five, the total loss function Loss is constructed as follows:

[0022] Loss=α·L1+β·L2+γ·L3+λ·L4;

[0023] Among them, α, β, γ, and λ are hyperparameters, L1 is the mean square error loss function, L2 is the auxiliary supervision loss function, L3 is the auxiliary branch loss function, and L4 is the optimized supervision loss function.

[0024] Further, L1 is as follows:

[0025]

[0026] Among them, R 暗 is the dark light reflection image output by the RL twin decomposition network, R 常 Normal light reflection image output by the RL twin decomposition network.

[0027] Furthermore, L2 is as follows:

[0028]

[0029] Among them, I 暗 is the input dark light image, I 常 is the input normal light image, R 暗is the dark light reflection image output by the RL twin decomposition network, R 常 is the normal light reflection image output by the RL twin decomposition network, L 暗 is the dark-light brightness image output by the RL twin decomposition network, L 常 It is the normal light brightness image output by the RL twin decomposition network, and ⊙ represents the Hadamard product.

[0030] Furthermore, L3 is as follows:

[0031]

[0032] Among them, mask 预测 The foreground prediction mask map output by the foreground segmentation network branch, mask 真实 is the real foreground mask.

[0033] Furthermore, L4 is as follows:

[0034]

[0035] Among them, x represents the normal light image of the original input, y represents the enhanced dark light image output by the image enhancement model, and μ x represents the mean of the normal light image, μ y represents the mean value of the enhanced dark light image, σ x represents the variance of the normal light image, σ y represents the variance of the enhanced dark image, σ xy represents the covariance of the normal light image and the enhanced dark light image, c1=(k1L) 2 ,c2=(k2L) 2 , where k1, k2, and L are constants. In this embodiment, k1=0.01, k2=0.03. L represents the pixel value range. In this embodiment, the number of image pixel bits is 8, so L=255.

[0036] A computer-readable storage medium stores a plurality of classification programs, wherein the plurality of classification programs are used to be called by a processor and execute the dark light enhancement method as described above.

[0037] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various media that can store program codes.

[0038] The advantage of a dark light enhancement method based on a multi-task twin neural network provided by the present invention is that: the dark light enhancement method based on a multi-task twin neural network provided in the structure of the present invention combines the traditional Retinex method with deep learning, and uses a deep learning network to decompose the dark light image and adjust the brightness image, thereby enhancing the dark light image. The innovation of this embodiment is that Transformer is used instead of CNN in the decomposition network and the brightness adjustment network, and the twin Vit (Vision Transformer) is used to decompose the image and adjust the brightness, which utilizes the characteristics of the twin network and the stronger feature extraction ability of the Transformer, and enhances the image decomposition and brightness adjustment capabilities through the network. At the same time, an auxiliary foreground segmentation supervision branch is also designed to help the model better focus on the main foreground target during the training phase, thereby optimizing the dark light image enhancement effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a structural schematic diagram of the present invention;

[0040] Figure 2 This is the training flowchart of the image enhancement model;

[0041] Figure 3 This is the execution flow chart after the image enhancement model training is completed;

[0042] Figure 4 Detailed flowchart for image enhancement model training

[0043] Figure 5 Schematic diagram of the structure of the transformer encoder block. DETAILED DESCRIPTION

[0044] Below, the technical solution of the present invention is described in detail through specific embodiments. Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific implementation disclosed below.

[0045] The Retinex method is based on the fact that the human visual system perceives color constancy under different lighting conditions, that is, the color of an object is not affected by lighting non-uniformity and is consistent. Retinex proposes that the image I seen by the human eye is composed of a reflection image R and a brightness image: I = R⊙L, where R represents the reflection image. Ideally, the decomposed R in dark light and normal light scenes should be the same, and the separated L represents the brightness of the image. If the brightness image L decomposed in dark light can be adjusted to the brightness in normal light, and finally restored to I 增 =R⊙L增 , the new image I 增 It is the normal light image after the dark light image is enhanced. The key to this process is how to better extract R and L, and properly adjust L to finally generate a higher quality enhanced image. The traditional method relies on manual design and extraction, which is difficult to achieve high quality. Therefore, this embodiment combines the deep learning method and uses the powerful feature extraction and adjustment adaptability of the deep learning network to optimize the above process.

[0046] like Figures 1 to 5 As shown, the present invention proposes a dark light enhancement method based on a multi-task twin neural network, which inputs the dark light image into a trained image enhancement model to obtain an enhanced image.

[0047] The image enhancement model includes an RL twin decomposition network, a foreground segmentation network branch, and a brightness adjustment network. The RL twin decomposition network includes two RL decomposition network branches. The training process of the image enhancement model is as follows:

[0048] Step 1: Collect dark light images and normal light images and preprocess them to construct an image training data set;

[0049] Data processing stage: First, retrieve and download the open source dataset in the field of dark light enhancement. At the same time, create dark light and normal light scenes in the same scene, and collect dark light-normal light data pairs. Finally, preprocess all data, including labeling, cleaning data with differences or large noise, and preprocessing into a unified format.

[0050] Step 2: Input the dark light image and the normal light image into the RL twin decomposition network to obtain the decomposed dark light reflection image, normal light reflection image, dark light brightness image and normal light brightness image;

[0051] In this example, the Vit model with a patch size of 16 is used, and the input image size is (H, W, 3). Then the size of the intermediate feature map should be This 768 dimension is obtained by 16x16x3. Figure 4 For the convenience of explanation, the input image is taken as 224x224x3 as an example.

[0052] Two RL decomposition networks are used for decomposing dark light images and normal light images respectively, and decompose the dark light image into dark light reflection image and dark light brightness image, and decompose the normal light image into normal light reflection image and normal light brightness image; the normal light image decomposition branch and the dark light image decomposition branch are used as the twin network R, and the dark light brightness image decomposition branch and the normal light brightness image decomposition branch are used as the twin network L. All weights of the two branches in the twin network R are shared, and all weights of the two branches in the twin network L are shared.

[0053] First, the dark-normal light image pair is described through the twin network R. For the dark-light image decomposition branch: the image is subjected to a 2D convolution with a kernel size of 16x16, a step size of 16, and an output channel of 768, as well as a normalization layer (BN) and an activation function (ReLU), and a 14x14x768 output is obtained. Then the output is flattened to 196x768 and the position code is superimposed. Finally, after passing through three transformer encoder layers (transformer encoder blocks), a 196x768 feature map is output and adjusted to a 224x224x3 dark-light reflection image.

[0054] For the normal light image decomposition branch, you can directly refer to the dark light image decomposition branch, and finally get a 224x224x3 normal light reflection image.

[0055] When the dark light brightness image and the normal light brightness image pass through the twin network L, the dark light brightness image decomposition branch: the image passes through a two-dimensional convolution with a convolution kernel size of 16x16, a step size of 16, and an output channel of 768, as well as a normalization layer (BN) and an activation function (ReLU), and a 14x14x768 output is obtained. Then the output is flattened to 196x768 and superimposed with a position code, and finally adjusted to 224x224x3 dimensions after passing through three transformer encoding layers (transformer encoder blocks), and then a 1x1, step size of 1, output channel number 1 two-dimensional convolution, BN and ReLU are added to obtain a dark light brightness image with an output of 224x224x1.

[0056] For the normal light image decomposition branch, you can directly refer to the dark light brightness image decomposition branch, and finally get a 224x224x1 normal light brightness image.

[0057] Step 3: Dark light reflection image R 暗 , Normal light reflection image R 常 , dark light brightness image L 暗 And the normal light brightness image L 常 After splicing, it is sent to the foreground segmentation network branch to output the foreground prediction mask map;

[0058] For the foreground segmentation network branch serving as an auxiliary supervision branch, the foreground segmentation network branch includes a two-dimensional convolution layer 1, a normalization layer 1, an activation layer 1, a position encoding layer 1, and a transformer encoding layer 1; the spliced ​​image is sequentially passed through a two-dimensional convolution layer 1, a normalization layer 1, an activation layer 1, a position encoding layer 1, and a transformer encoding layer 1 to obtain a foreground prediction mask map.

[0059] The dark light reflected image R 暗 , Normal light reflection image R 常 , dark light brightness image L 暗 And the normal light brightness image L 常 The image is concatenated and resized to 224x224x12 dimensions, and then a 1x1, stride 1, 1 output channel 2D convolution as well as BN and ReLU are used to resize the dimensions to 224x224x1. After that, a 16x16 convolution kernel, stride 16, 256 output channels 2D convolution + BN + ReLU is performed, and then position encoding is added and the image is resized to 224x224x1 dimensions after passing through 3 transformer encoding layers 1 (transformerencoder block). Finally, the foreground prediction mask image (i.e. foreground prediction mask) is output after Sigmod probabilization.

[0060] Step 4: Transform the dark light brightness image L 暗 The image is sent to the brightness adjustment network to output a dark-light enhanced brightness image, and the dark-light enhanced brightness image is dot-producted with the dark-light reflection image to obtain an enhanced image.

[0061] The brightness adjustment network includes a two-dimensional convolution layer 2, a normalization layer 2, an activation layer 2, a position encoding layer 2, and a transformer encoding layer 2; the dark-light brightness image passes through a two-dimensional convolution layer 2, a normalization layer 2, an activation layer 2, a position encoding layer 2, and a transformer encoding layer 2 in sequence, and then outputs a dark-light enhanced brightness image.

[0062] The L obtained above 暗 After a two-dimensional convolution with a convolution kernel size of 16x16, a step size of 16, and an output channel of 256, as well as BN and ReLU and adding position encoding, the dark image is adjusted to 224x224x1 after passing through a transformer encoder block to obtain the enhanced brightness image L 增 Finally, the enhanced brightness image is expanded to 224x224x3 and the dark light reflection image R segmented in the previous step 暗 Doing the dot product, we get the enhanced image.

[0063] The transformer encoder block in this embodiment is the encoder structure in the standard Vit (Vision transformer). For details, see Figure 5The input feature map is normalized, multi-headed attention mechanism, regularized, and encoded by the multi-layer perception layer. For the 3-layer transformer encoder block, the output of the previous layer of transformer encoder block is used as the input of the current layer of transformer encoder block, and the image output by the last layer of transformer encoder block is used as the output feature map of the 3-layer transformer encoder block.

[0064] Step 5: Construct a total loss function to adjust the parameters to be trained in the image enhancement model.

[0065] In this embodiment, the total loss function is obtained based on four loss combinations:

[0066] For the RL twin decomposition network, R 暗 and R 常 , use mean square error MSE to supervise the optimization and minimize R 暗 , R 常 The difference between them, from which we can get the mean square error loss function L1 used:

[0067]

[0068] Among them, R 暗 is the dark light reflection image output by the RL twin decomposition network, R 常 Normal light reflection image output by the RL twin decomposition network.

[0069] At the same time, in order to eliminate the influence of degradation or noise and strengthen the auxiliary supervision of the ideal decomposition of image I into R and L, the mean square error supervision optimization is also used, and the auxiliary supervision loss function L2 is introduced:

[0070]

[0071] Among them, I 暗 is the input dark light image, I 常 is the input normal light image, L 暗 is the dark-light brightness image output by the RL twin decomposition network, L 常 It is the normal light brightness image output by the RL twin decomposition network, and ⊙ represents the Hadamard product.

[0072] The foreground segmentation network branch as an auxiliary supervision branch is to make the image enhancement model focus on the foreground and edge details, thereby improving the effect of generating the enhanced image. The Dice coefficient is used to supervise the prediction and true mask, and the auxiliary branch loss function L3 is introduced:

[0073]

[0074] Among them, mask 预测 The foreground prediction mask map output by the foreground segmentation network branch, mask 真实 is the real foreground mask.

[0075] In this embodiment, the decomposed L is not used. 常 and the enhanced brightness image L 增 Instead of doing supervised optimization, we use the final generated I 增 and the input normal light image I 常 To supervise the optimization. The reasons are: first, we hope that it is an end-to-end image enhancement model in the use stage, which can directly output the enhanced image without additional operations; second, we hope that the image enhancement model can also combine the brightness adjustment network and the RL twin decomposition network to optimize the dot product results of R and L as a whole, rather than only optimizing the brightness enhancement effect through the brightness network and the twin network L. The similarity of two images can be measured using structural similarity (SSIM). The more similar the SSIM value is, the closer it is to 1. Therefore, the optimization supervision loss function L4 is introduced:

[0076]

[0077] Among them, x represents the normal light image of the original input, y represents the enhanced dark light image output by the image enhancement model, and μ x represents the mean of the normal light image, μ y represents the mean value of the enhanced dark light image, σ x represents the variance of the normal light image, σ y represents the variance of the enhanced dark image, σ xy represents the covariance of the normal light image and the enhanced dark light image, c1=(k1L) 2 ,c2=(k2L) 2 , where k1, k2, and L are constants. In this embodiment, k1=0.01, k2=0.03. L represents the pixel value range. In this embodiment, the number of image pixel bits is 8, so L=255.

[0078] During training, the above four loss functions are combined to obtain the final loss function Loss, where α, β, γ, and λ are hyperparameters used to control the weights of each loss supervision model. In this embodiment, their values ​​are 0.1, 0.05, 0.01, and 1 respectively.

[0079] Loss=α·L1+γ·L2+γ·L3+λ·L4.

[0080] Through steps one to five, the processed data pairs are sent to the constructed image enhancement model, which is composed of an RL twin decomposition network, an illumination adjustment network, and an auxiliary foreground segmentation network branch. Using the designed multi-task loss function and data pair supervised learning, the training is stopped after a certain number of steps or the loss error is less than a certain value to obtain the image enhancement model. (This process can be iterated repeatedly according to the test results of the image enhancement model, so that the generalization ability of the image enhancement model is stronger).

[0081] Therefore, this embodiment combines the traditional Retinex method with deep learning, and uses a deep learning network to decompose dark-light images and adjust brightness images, thereby enhancing dark-light images. The innovation of this embodiment is that Transformer is used instead of CNN in the decomposition network and the brightness adjustment network, and Twin Vit (Vision Transformer) is used to decompose images and adjust brightness, which utilizes the characteristics of the Twin network and the stronger feature extraction capability of Transformer, and enhances the ability of images to be decomposed and adjusted in brightness through the network. At the same time, this embodiment also designs an auxiliary foreground segmentation supervision branch, which is used in the training phase to help the model better focus on the main foreground targets and optimize the dark-light image enhancement effect.

[0082] During the execution of the image enhancement model after training, there will only be one dark light image to be enhanced and no foreground segmentation is needed to assist supervised learning. Therefore, the RL decomposition network and foreground segmentation network branches corresponding to the normal light branch in the image enhancement model are removed, and the dark light image is decomposed by the RL decomposition network, and then the enhanced image is directly generated by combining the brightness adjustment network to form an end-to-end image enhancement model, which reduces the inference time and deployment complexity.

[0083] Therefore, the specific process of using the image enhancement model is as follows:

[0084] The dark-light image to be enhanced is input into the RL twin decomposition network. Since the RL decomposition network corresponding to the normal-light branch is removed, the dark-light image to be enhanced outputs a dark-light reflection image and a dark-light brightness image through the RL decomposition network corresponding to the dark-light branch.

[0085] The dark-light brightness image is input into the brightness adjustment network to obtain a dark-light enhanced brightness image;

[0086] The dot product of the dark light enhanced brightness image and the dark light reflection image is performed to obtain the enhanced image.

[0087] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A dark light enhancement method based on a multi-task twin neural network, characterized in that: Input the dark light image into the trained image enhancement model to obtain the enhanced image; The image enhancement model includes an RL twin decomposition network, a foreground segmentation network branch, and a brightness adjustment network, wherein the RL twin decomposition network includes two RL decomposition network branches, and when the trained image enhancement model is used, the foreground segmentation network branch and the normal light RL decomposition network branch are deleted; The training process of the image enhancement model is as follows: Step 1: Collect dark light images and normal light images and preprocess them to construct an image training data set; Step 2: Input the dark light image and the normal light image into the RL twin decomposition network to obtain the decomposed dark light reflection image, normal light reflection image, dark light brightness image and normal light brightness image; Step 3: Splice the four feature images output by the RL twin neural network and send them to the foreground segmentation network branch to output the foreground prediction mask image; Step 4: Send the dark light brightness image to the brightness adjustment network to output the dark light enhanced brightness image, and do a dot product between the enhanced brightness image and the dark light reflection image to obtain the enhanced image; Step 5: Construct a total loss function to adjust the parameters to be trained in the image enhancement model.

2. The dark light enhancement method based on multi-task twin neural network according to claim 1 is characterized in that: Two RL decomposition networks are used for decomposing dark light images and normal light images respectively, and decompose dark light images into dark light reflection images and dark light brightness images, and decompose normal light images into normal light reflection images and normal light brightness images; The normal light image decomposition branch and the dark light image decomposition branch are used as the twin network R, and the dark light brightness image decomposition branch and the normal light brightness image decomposition branch are used as the twin network L. All weights of the two branches in the twin network R are shared, and all weights of the two branches in the twin network L are shared.

3. The dark light enhancement method based on multi-task twin neural network according to claim 1 is characterized in that: In step 3, the foreground segmentation network branch includes a two-dimensional convolutional layer 1, a normalization layer 1, an activation layer 1, a position encoding layer 1, and a transformer encoding layer 1; The spliced ​​image passes through two-dimensional convolution layer 1, normalization layer 1, activation layer 1, position encoding layer 1, and transformer encoding layer 1 in sequence to obtain the foreground prediction mask map.

4. The dark light enhancement method based on multi-task twin neural network according to claim 1, characterized in that: In step 4, the brightness adjustment network includes a two-dimensional convolution layer 2, a normalization layer 2, an activation layer 2, a position encoding layer 2, and a transformer encoding layer 2; The dark-light brightness image passes through two-dimensional convolution layer 2, normalization layer 2, activation layer 2, position encoding layer 2, and transformer encoding layer 2 in sequence to output a dark-light enhanced brightness image.

5. The dark light enhancement method based on multi-task twin neural network according to claim 1, characterized in that: In step 5, the total loss function Loss is constructed as follows: Loss=α·L1+β·L2+γ·L3+λ·L4 Among them, α, β, γ, and λ are hyperparameters, L1 is the mean square error loss function, L2 is the auxiliary supervision loss function, L3 is the auxiliary branch loss function, and L4 is the optimized supervision loss function.

6. The dark light enhancement method based on multi-task twin neural network according to claim 5 is characterized in that: L1 is as follows: Among them, R 暗 is the dark light reflection image output by the RL twin decomposition network, R 常 Normal light reflection image output by the RL twin decomposition network.

7. The dark light enhancement method based on multi-task twin neural network according to claim 5 is characterized in that: L2 is as follows: Among them, I 暗 is the input dark light image, I 常 is the input normal light image, R 暗 is the dark light reflection image output by the RL twin decomposition network, R 常 is the normal light reflection image output by the RL twin decomposition network, L 暗 is the dark-light brightness image output by the RL twin decomposition network, L 常 It is the normal light brightness image output by the RL twin decomposition network, and ⊙ represents the Hadamard product.

8. The dark light enhancement method based on multi-task twin neural network according to claim 5, characterized in that: L3 is as follows: Among them, mask 预测 The foreground prediction mask map output by the foreground segmentation network branch, mask 真实 is the real foreground mask.

9. The dark light enhancement method based on multi-task twin neural network according to claim 5, characterized in that: L4 is as follows: Among them, x represents the normal light image of the original input, y represents the enhanced dark light image output by the image enhancement model, and μ x represents the mean of the normal light image, μ y represents the mean value of the enhanced dark light image, σ x represents the variance of the normal light image, σ y represents the variance of the enhanced dark image, σ xy represents the covariance of the normal light image and the enhanced dark light image, c1=(k1L) 2 ,c2=(k2L) 2 , where k1, k2, and L are constants. In this embodiment, k1=0.01, k2=0.

03. L represents the pixel value range. In this embodiment, the number of image pixel bits is 8, so L=255.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of classification programs, and the plurality of classification programs are used to be called by a processor and execute the dark light enhancement method as described in any one of claims 1 to 9.

Images