A low-light image enhancement method based on multimodal guided recurrent generative network

Abstract

The present invention proposes a low-light image enhancement method based on a multimodal guided recurrent generative network. A real normal-light image is sent to a low-light image generation network to obtain a synthetic low-light image. The real low-light image is sent to a low-light image enhancement network after passing through a preprocessing network to obtain a synthetic normal-light image. The image / text encoder in the multimodal teacher model is then used to encode the generated result to obtain a feature vector. The feature vector is used to evaluate the consistency of the low-light image enhancement / generation network before and after image enhancement, the image enhancement effect, and other aspects, to guide the low-light image enhancement / generation network to obtain better results. While ensuring low network complexity and computational consumption, this method uses an unsupervised training method to get rid of the dependence on paired training data. The addition of multimodal information enables the low-light image enhancement network to adapt to a variety of different real low-light environments, achieving better image enhancement effects.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and digital image processing, and specifically provides a low-illumination image enhancement method based on a multimodal guided cyclic generation network. Background Art

[0002] The goal of low-light image enhancement is to improve image quality, visibility, visual quality, and information content by improving image contrast, brightness, clarity, and color. This improves visual comfort and helps people navigate complex dark environments. Low-light image enhancement research has broad applications and significant significance, with applications and demands in multiple fields, such as smartphones, security surveillance, and autonomous driving.

[0003] Low-light image enhancement technology faces numerous difficulties and challenges in practical applications. Nighttime imagery is a diverse field, with varying levels of image degradation. Further research is needed to develop low-light image enhancement methods that effectively enhance low-light images in a variety of scenarios. Most existing solutions (i.e., supervised learning methods) rely on paired images. However, capturing both low-light images and paired normal-light images simultaneously in the real world is difficult. Synthesizing low-light images from clear images often yields suboptimal data. While these data may resemble low-light images, they rarely accurately reflect low-light characteristics, such as noise and over- and under-exposure areas. This results in models trained using synthesized paired images being unsuitable for real-world scenarios. Furthermore, the brightness of normal-light images varies significantly, hindering the performance of supervised learning methods.

[0004] This method, by introducing multimodal information, can help the network learn different prior knowledge, thereby obtaining results that are more suitable for human visual perception. Therefore, this low-light image enhancement method, through unsupervised training and the introduction of multimodal information, can help the network achieve better image enhancement effects and can run stably at real-time frame rates. Summary of the Invention

[0005] To address the current problems of low-light image enhancement, such as reliance on paired images, poor enhancement effect, and difficulty in adapting to diverse environments, the present invention proposes a low-light image enhancement method based on a multimodal guided recurrent generative network.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A low-light image enhancement method based on a multimodal guided recurrent generative network comprises the following steps:

[0008] Step 1: Sample the real low-light images in the LSRW (Low-Light Real-world) dataset and the real normal-light images in the BSDS300 dataset to obtain an image training set. Scale the images in the training set, unify the image size, and normalize them. Then, perform data augmentation operations such as image rotation and flipping on the training set.

[0009] Step 2: The real low-light image in the training set obtained in step 1 is fed into the preprocessing network; the real low-light image after the preprocessing network is then fed into the low-light image enhancement network to obtain a synthetic normal-light image; the real normal-light image is then fed into the low-light image generation network to obtain a synthetic low-light image;

[0010] Step 3: Design text prompt word pairs, i.e., text descriptions for low-light and normal-light images. Based on the image enhancement / generation results and the text prompt word pairs, use the multimodal teacher model's image encoder and text encoder to encode the image enhancement / generation results and the text prompt word pairs, respectively, and obtain an enhancement effect evaluation score based on the feature vectors.

[0011] Step 4: Use the multimodal teacher model image encoder to encode the enhanced and generated images respectively, and use the encoded feature vectors to obtain the image enhancement consistency evaluation score;

[0012] Step 5: Send the synthesized low-light image and the synthesized normal-light image to the image discriminator network to determine the image type.

[0013] Step 6: Optimize and iterate the multimodal guided cycle generation network by enhancing the consistency evaluation score, the enhancement effect evaluation score, and the discriminator judgment result.

[0014] Existing technologies generally use supervised learning methods that rely on paired image pairs for training. This paper proposes a low-light image enhancement method based on a multimodal guided recurrent generative network. This method breaks away from the reliance on paired image pairs and introduces multimodal information, improving the low-light image enhancement effect.

[0015] Furthermore, in step 2, the preprocessing network should include 5 stages: the first feature extraction layer, the first downsampling layer, the first upsampling layer, the residual layer and the encoding layer. The first feature extraction layer contains 5 convolution kernels of size The first convolution layer, normalization layer and activation layer. The first downsampling layer contains 2 convolution kernels of size The first upsampling layer contains two convolution kernels of size The residual layer consists of 3 residual blocks, each of which consists of 3 convolution kernels with a size of The network consists of a convolutional layer, a normalization layer, and an activation layer. The normalization layer is BN, and the activation layer is RELU6. The encoding layer contains a fully connected layer with an input dimension of 612, an output dimension of 3, and 3 layers of 256-dimensional hidden layers, as well as a position encoder. The network input is a size of The 3-channel image is output as The 3-channel preprocessed image of size .

[0016] Furthermore, in step 2, the low-light image enhancement network and the low-light image generation network have the same structure, and the network includes five parts: the network consists of a second feature extraction layer, a second downsampling layer, a second upsampling layer, a residual layer, and a post-processing layer; the second feature extraction layer consists of a reflection filling layer, a convolution kernel size of The third convolution layer, normalization layer and activation layer of the second downsampling layer contain 3 convolution kernels of size The fourth convolution layer and activation layer of . The second upsampling layer contains 3 convolution kernels of size The residual layer consists of 3 residual blocks, each of which consists of 3 convolution kernels with a size of The network consists of a convolutional layer, a normalization layer, and an activation layer. The normalization layer is BN, and the activation layer is ReLU; the post-processing layer consists of a reflection padding layer, a fifth convolutional layer, and an activation layer. The activation layer is a Tanh activation function. The network input is a size of The 3-channel image is output as The 3-channel enhanced image of size .

[0017] Furthermore, in step 3-4, the image encoder consists of a pre-processing layer, a visual converter, and a normalization layer; the pre-processing layer contains a convolution kernel with a size of The sixth convolutional layer and position encoder of the visual converter are composed of 3 residual attention layers, which include 1 multi-head attention layer and 1 fully connected layer; the fully connected layer consists of a linear layer and an activation layer; the activation layer is ReLU. The normalization layer is BN. The input is The 3-channel image is output as a 512-dimensional image encoding.

[0018] The text encoder consists of a backbone layer and a bottleneck layer; the backbone layer consists of 3 backbone blocks and 1 convolution kernel size Each backbone block consists of a convolution kernel of size The convolution layer, 1 activation layer, and 1 normalization layer are composed of BN and ReLU respectively; the bottleneck layer is composed of 4 bottleneck blocks, and the bottleneck block is composed of 1 convolution kernel of size The input is a pair of text prompts, and the output is dimensional text prompt word pair encoding.

[0019] Furthermore, in step 5, the image discriminator network is composed of a normal illumination image discriminator and a normal illumination image Gaussian low-pass discriminator, and the network is composed of three discriminant layers, each of which is composed of one The seventh convolutional layer with the same convolution kernel size and the activation function is LeakyReLU. The image discriminator network outputs a judgment on whether the image type is a normal lighting image, with the output ranging from 0 to 1. The larger the number, the more likely the image discriminator network believes that the image is a normal lighting image.

[0020] The beneficial effects of the present invention are:

[0021] This paper proposes a low-light image enhancement method based on a multimodal guided recurrent generative network. By using a multimodal guided recurrent generative network for low-light image enhancement, network training eliminates the reliance on paired data. By incorporating a massive unpaired dataset and multimodal information, the network's robustness against low-light images captured in diverse environments is improved. The multimodal guided recurrent generative network offers enhanced enhancement effects and is suitable for deployment on platforms with limited computing power and memory, such as embedded systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a flow chart of a low-light image enhancement method based on a multimodal guided cyclic generative network according to the present invention;

[0023] Figure 2 This is a diagram of the image discriminator network structure of the present invention;

[0024] Figure 3 This is a flow chart for calculating the image enhancement consistency evaluation score and the image enhancement effect evaluation score of the present invention;

[0025] Figure 4 Schematic diagram of an original image input in an embodiment of the present invention;

[0026] Figure 5 Schematic diagram of the image enhancement result output in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The present invention provides a low-light image enhancement method based on a multimodal guided recurrent generative network. Figure 1 As shown in Figure 1, the multimodal guided cycle generation network includes a multimodal teacher model, a low-light image enhancement network, a low-light image generation network, and an image discriminator network, including the following steps:

[0028] Step 1: The present invention samples real low-light images from the LSRW (Low-Light Real-world) dataset and real normal-light images from the BSDS300 dataset, unifies the image size and normalizes them, and then performs data augmentation on the normalized images to obtain the image training set. Step 1 mainly includes the following two parts:

[0029] Step 1.1: Sample 1000 images from the real low-light images in the LSRW dataset and the real normal-light images in the BSDS300 dataset, and scale the image sizes to The image is normalized so that the pixel values ​​are limited to between 0 and 1.

[0030] Step 1.2: Randomly flip the image processed in step 1.1 horizontally, vertically, rotate 90 degrees, and rotate it at random angles between 0 and 180, with the random probability set at 20%, to obtain the image training set.

[0031] Step 2: The real low-light image in the training set obtained in step 1 is fed into the preprocessing network; the real low-light image after the preprocessing network is then fed into the low-light image enhancement network to obtain a synthetic normal-light image; the real normal-light image is then fed into the low-light image generation network to obtain a synthetic low-light image. Step 2 mainly includes the following three parts:

[0032] Step 2.1: Send the low-light images in the training set obtained in step 1 to the preprocessing network to obtain low-light images with balanced degradation levels.

[0033] The preprocessing network consists of five stages: the first feature extraction layer, the first downsampling layer, the first upsampling layer, the residual layer, and the encoding layer. The first feature extraction layer contains five convolution kernels of size The first convolution layer, normalization layer and activation layer. The downsampling layer contains 2 convolution kernels of size The upsampling layer consists of two convolutional kernels with a size of The residual layer consists of 3 residual blocks, each of which consists of 3 convolution kernels with a size of The network consists of a convolutional layer, a normalization layer, and an activation layer. The normalization layer is BN, and the activation layer is RELU6. The encoding layer contains a fully connected layer with an input dimension of 612, an output dimension of 3, and 3 layers of 256-dimensional hidden layers, as well as a position encoder. The network input is a size of The 3-channel image is output as The 3-channel preprocessed image of size .

[0034] Step 2.2: Use the low-light image enhancement network to perform low-light image enhancement on the preprocessed real low-light image obtained in step 2.1 to obtain a synthetic normal-light image.

[0035] Step 2.3: Use the low-light image generation network to generate low-light images from the real normal-light images in the training set to obtain synthetic low-light images.

[0036] The low-light image enhancement network has the same structure as the low-light image generation network. The network consists of five parts: the network consists of a second feature extraction layer, a second downsampling layer, a second upsampling layer, a residual layer, and a post-processing layer; the second feature extraction layer consists of a reflection filling layer, a convolution kernel size of The third convolution layer, normalization layer and activation layer of the downsampling layer contain 3 convolution kernels of size The fourth convolution layer and activation layer of . The upsampling layer contains 3 convolution kernels with a size of The residual layer consists of 3 residual blocks, each of which consists of 3 convolution kernels with a size of The convolution layer, normalization layer, and activation layer are composed of . The normalization layer is BN, and the activation layer is ReLU; the post-processing layer is composed of a reflection filling layer, a fifth convolution layer, and an activation layer, and the activation layer is the Tanh activation function. Finally, we get 3-channel synthetic normal lighting image of size and 3-channel synthetic low-light image of different sizes.

[0037] Step 3: Design text prompt word pairs, i.e. text language descriptions of low-light images and normal-light images. Figure 3 As shown, using the synthetic normal illumination image, synthetic low illumination image and text prompt word pair obtained in step 2, the multimodal teacher model image encoder and text encoder are used to obtain the image encoding of the synthetic normal illumination image and synthetic low illumination image and the text prompt word pair encoding, respectively, and the image enhancement effect evaluation score is obtained based on the feature vector. Step 3 mainly includes the following four parts:

[0038] Step 3.1. Design text prompt pairs: “Images with low clarity, contrast and brightness, dull colors and difficult to distinguish details” and “Images with high clarity, contrast and brightness, vibrant colors and detailed” as the text language descriptions of low-light images and normal-light images.

[0039] Step 3.2: Use the multimodal teacher model image encoder to obtain the image encoding of the synthetic normal illumination image and the synthetic low illumination image respectively.

[0040] The image encoder consists of a pre-processing layer, a visual converter, and a normalization layer; the pre-processing layer contains a convolution kernel with a size of The sixth convolutional layer and position encoder of the visual converter are composed of 3 residual attention layers and 1 fully connected layer. The fully connected layer consists of a linear layer and an activation layer. The activation layer is ReLU. The normalization layer is BN. The input is The 3-channel image is output as a 512-dimensional image encoding.

[0041] Step 3.3: Use the multimodal teacher model text encoder to obtain the text prompt word pair encoding of the text prompt word pair.

[0042] The text encoder consists of a backbone layer and a bottleneck layer; the backbone layer consists of 3 backbone blocks and 1 convolution kernel size Each backbone block consists of a convolution kernel of size The convolution layer, 1 activation layer, and 1 normalization layer are composed of BN and ReLU respectively; the bottleneck layer is composed of 4 bottleneck blocks, and the bottleneck block is composed of 1 convolution kernel of size The input is a pair of text prompts, and the output is dimensional text prompt word pair encoding.

[0043] Step 3.4: Encode the image code and text prompt word pair obtained in steps 3.1 and 3.2, and calculate the enhancement effect evaluation score. The steps are as follows:

[0044] Step 3.4.1. Calculate the tensor product of the image encoding of the synthesized normal illumination image and the encoding of the text prompt word pair.

[0045] Step 3.4.2: Calculate the tensor product of the image encoding of the synthesized low-light image and the text prompt word pair encoding.

[0046] Step 3.4.3: Subtract and normalize the tensor products obtained in steps 3.4.1 and 3.4.2 to obtain the enhancement effect evaluation score.

[0047] Step 4: Figure 3 As shown in Figure 2, the image encoding of the real normal illumination image and the real low illumination image is obtained by using the multimodal teacher model image encoder. The image encoding of the synthetic normal illumination image and the synthetic low illumination image obtained in step 3 is combined to calculate the image enhancement consistency evaluation score. Step 4 mainly includes the following three parts:

[0048] Step 4.1: Send the synthesized normal illumination image and the synthesized low illumination image into the multimodal teacher model image encoder to obtain the encoding results respectively.

[0049] The image encoder consists of a pre-processing layer, a visual converter, and a normalization layer; the pre-processing layer contains a convolution kernel with a size of The sixth convolutional layer and position encoder of the visual converter are composed of 3 residual attention layers and 1 fully connected layer. The fully connected layer consists of a linear layer and an activation layer. The activation layer is ReLU. The normalization layer is BN. The input is The 3-channel image is output as a 512-dimensional image encoding.

[0050] Step 4.2: Encode the images obtained in steps 3.2 and 4.1 and calculate the enhanced consistency evaluation score. The steps are as follows:

[0051] Step 4.2.1. Calculate the tensor product of the image encoding of the synthesized normal illumination image and the actual low illumination image encoding as the low illumination image enhancement consistency evaluation score.

[0052] Step 4.2.2: Calculate the tensor product of the image encoding of the synthetic low-light image and the encoding of the real normal-light image as the normal-light image enhancement consistency evaluation score.

[0053] Step 4.2.3: After summing and normalizing the tensor products obtained in steps 4.2.1 and 4.2.2, the enhanced consistency evaluation score is obtained.

[0054] Step 5: Send the synthesized low-light image and the synthesized normal-light image to the image discriminator network to judge the image type. The discriminator network structure is as follows: Figure 2 As shown in the figure, it includes a normal illumination image discriminator and a normal illumination image Gaussian low-pass discriminator. The normal illumination image Gaussian low-pass discriminator adds a Gaussian low-pass filter before the discriminator network to enhance the image edge details and improve the discrimination ability. Except for the filter, the other parts of the network of the two discriminators are consistent. Step 5 mainly includes:

[0055] Step 5.1: Send the synthesized low-light image to the normal-light image discriminator and the normal-light image Gaussian low-pass discriminator respectively to determine whether the image is a normal-light image.

[0056] Step 5.2: Send the synthesized normal illumination image to the normal illumination image discriminator and the normal illumination image Gaussian low-pass discriminator respectively to determine whether the image is a normal illumination image.

[0057] The discriminator network consists of three discriminant layers, each of which consists of one The image is composed of the seventh convolution layer of the convolution kernel size, the activation function and the normalization layer; the normalization layer is BN, and the activation layer is LeakyReLU; the image discriminator network outputs the judgment on whether the image type belongs to a normal lighting image.

[0058] In addition, the loss calculation method of the cycle generation network of the present invention is as follows:

[0059] The loss function of the recurrent generation network consists of the following parts:

[0060] Cycle consistency loss :

[0061] ;

[0062] in, and Represent the low-light image enhancement network and the low-light image generation network, respectively. x represents a true low-light image, and y represents a true normal-light image. n represents the number of true low-light images in the training set, and i represents the sequence number of the current true low-light image. m represents the number of true normal-light images in the training set, and j represents the sequence number of the current true normal-light image.

[0063] Discriminative loss :

[0064]

[0065]

[0066] in, and Represent low-light image enhancement network and low-light image generation network respectively, and Represents the normal illumination image discriminator and the normal illumination image Gaussian low-pass discriminator, the intermediate parameters express and All in accordance with The calculation is performed using the formula represented by . n represents the number of true low-light images in the training set, i represents the sequence number of the current true low-light image, m represents the number of true normal-light images in the training set, and j represents the sequence number of the current true normal-light image.

[0067] Multimodal text loss :

[0068]

[0069]

[0070]

[0071]

[0072] in, and They represent low-light image enhancement network and low-light image generation network respectively. represents the synthetic low-light image, represents a synthesized normal-light image. n represents the number of real low-light images in the training set, and i represents the index of the current real low-light image. m represents the number of real normal-light images in the training set, and j represents the index of the current real normal-light image. s represents a preset text prompt word pair, P represents the evaluation score output by the multimodal teacher model, and E represents the image encoder of the multimodal teacher model.

[0073] Total loss:

[0074]

[0075] The total loss is the weighted sum of cycle consistency loss, discrimination loss and multimodal text loss, which is used to measure the enhanced performance of the multimodal guided cycle generation network. , and are hyperparameters that control the proportion of cycle consistency loss, discrimination loss, and multimodal text loss in the total loss.

[0076] Example

[0077] The experimental environment of the present invention is configured as follows: Pytorch is used as the neural network framework, an RTX3090 32GB GPU is used for training, and the model inference rate is tested. The specific steps of the present invention are summarized as follows:

[0078] (1) 1000 images of real low-light images in the LSRW dataset and real normal-light images in the BSDS300 dataset were sampled and scaled, the image sizes were unified and normalized, and then data enhancement operations were performed on the images to obtain the image training set.

[0079] (2) The low-light images in the image training set are sent to the preprocessing network, and then the preprocessed real low-light images are input into the low-light image enhancement network to obtain synthetic normal-light images, and the real normal-light images are sent to the low-light image generation network to obtain synthetic low-light images.

[0080] (3) Design text prompt pairs. Based on the image enhancement results and the text prompt word pairs, use the multimodal teacher model image encoder and text encoder to encode the image enhancement results and the text prompt word pairs respectively, and obtain the enhancement effect evaluation score based on the feature vector.

[0081] (4) Use the multimodal teacher model image encoder to encode the images before and after enhancement respectively, and use the encoded feature vectors to obtain the enhancement consistency evaluation scores of normal illumination images and low illumination images.

[0082] (5) The synthesized low-light image and the synthesized normal-light image are respectively sent to the image discriminator network to judge the image type.

[0083] (6) Train the entire network, input the images in the training set into the neural network structure, obtain the enhanced consistency evaluation score, the enhanced effect evaluation score and the discriminator discrimination result, use them as network loss to adjust the network parameters through the gradient back transfer method in the Pytorch framework, and then use the stochastic gradient descent method in the Pytorch framework to adjust the parameter learning rate, and finally obtain the trained model parameters and save them.

[0084] (7) Use the trained model parameters to perform inference calculations on the test set images in the LOL-V2 dataset to obtain the corresponding enhancement results of the test set images, and calculate the accuracy of the model with the true value of the corresponding normal lighting images in the test set.

[0085] (8) Use Pytorch to deploy the model on the RTX3090 graphics card and test the model speed.

[0086] The low-light image enhancement results are as follows Figure 4 , Figure 5 As shown, low-light image enhancement is performed on the low-light image actually taken. Figure 4 and Figure 5 Compared to the original image, the enhanced image shows a significant improvement in overall brightness. Previously dim and unclear areas are now brighter and clearer. This method effectively restores image details lost in low-light conditions, such as texture and edges, providing richer visual information. The enhanced image is visually more comfortable and natural, suitable for the human eye, and reduces visual fatigue caused by low light conditions.

[0087] Compared with the existing technology, the benefits of the present invention are as follows: it addresses the problem that the current low-light image enhancement technology has a large amount of computation and is difficult to meet real-time requirements. By designing a lightweight cyclic generative network, the network complexity is reduced and the time required for image enhancement is reduced; by using a multimodal teacher model to guide the cyclic generative network for low-light image enhancement, the network training is freed from the dependence on paired data, and the network's robustness for low-light images taken in different environments is improved through massive unpaired data sets. Design a network optimization mechanism guided by a multimodal teacher model. This method designs a low-light image enhancement method based on a multimodal guided cyclic generative network, and uses a multimodal teacher model to guide the cyclic generative network for optimization to achieve unsupervised low-light image enhancement. While reducing network complexity and computational consumption, the method of the present invention ensures considerable accuracy and real-time performance, and is suitable for deployment and application on embedded platforms with low computing / memory resources. The comparison results between the method of the present invention and the currently commonly used methods for low-light image enhancement are shown in Table 1. Compared with other methods, the method of the present invention significantly outperforms other methods in terms of accuracy (PSNR) and maintains a leading advantage in terms of speed. It can also achieve real-time enhancement of 2K images on the RTX3090.

[0088] Table 1

[0089]

[0090] The embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and changes can be made based on the contents of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents. The contents not described in detail in the present specification belong to the common knowledge of professional and technical personnel in this field.

Claims

1. A low-light image enhancement method based on a multimodal guided recurrent generative network, characterized in that: The implementation steps are as follows: Step 1: Sample the LSRW dataset and BSDS300 dataset respectively to obtain training sets containing real low-light images and real normal-light images. Scale the images in the training sets, unify the image size and normalize them, and then perform image rotation and flipping data augmentation operations. Step 2: Send the real low-light images in the training set processed in step 1 to the preprocessing network; Then the real low-light image that has passed the preprocessing network is input into the low-light image enhancement network to obtain a synthetic normal-light image; The real normal illumination image processed in step 1 in the training set is sent to the low illumination image generation network to obtain a synthetic low illumination image; Step 3: Design text prompt word pairs. Using the synthetic normal-light image, synthetic low-light image, and text prompt word pairs obtained in Step 2, obtain image encodings of the synthetic normal-light image and synthetic low-light image and text prompt word pair encodings through the multimodal teacher model image encoder and text encoder, respectively. Calculate the image enhancement effect evaluation score based on the feature vector. Step 4: Use the multimodal teacher model image encoder to obtain the image encoding of the real normal illumination image and the real low illumination image. Combined with the image encoding of the synthetic normal illumination image and the synthetic low illumination image obtained in step 3, the image enhancement consistency evaluation score is calculated based on the feature vector. Step 5: Send the synthesized low-light image and the synthesized normal-light image to the image discriminator network to discriminate the image type; Step 6: Optimize and iterate the multimodal guided cycle generation network based on the image enhancement consistency evaluation score, image enhancement effect evaluation score, and the discrimination results of the image discriminator network.

2. The low-light image enhancement method based on a multimodal guided recurrent generative network according to claim 1, characterized in that: In step 1, the image preprocessing steps are as follows: Step 1.1: Sample the training sets containing real low-light images and real normal-light images from the LSRW dataset and the BSDS300 dataset respectively. Scale the images in the training sets to a uniform scale and then perform normalization. Step 1.2: For the image training set obtained in step 1, randomly perform data augmentation operations on the images, such as horizontal flipping, vertical flipping, 90-degree rotation, and random angle rotation.

3. The low-light image enhancement method based on a multimodal guided recurrent generative network according to claim 1, characterized in that: The step 2 includes: Step 2.1: Send the real low-light image in the training set processed in step 1 to the preprocessing network to obtain the preprocessed low-light image. The preprocessing network includes a first feature extraction layer, a first downsampling layer, a first upsampling layer, a residual layer, and an encoding layer. The first feature extraction layer includes a reflection filling layer, a first convolution layer, a normalization layer, and an activation layer; the first downsampling layer includes a second convolution layer, a normalization layer, and an activation layer; the first upsampling layer includes a deconvolution layer, a normalization layer, and an activation layer; the normalization layer is a batch normalization layer, and the activation layer is a ReLU; the encoding layer includes a position encoder and a fully connected layer; Step 2.2: Input the preprocessed real low-light image obtained by the preprocessing network in step 2.1 into the low-light image enhancement network to obtain a synthetic normal-light image, and input the real normal-light image processed in step 1 in the training set into the low-light image generation network to obtain a synthetic low-light image; the low-light image enhancement network and the low-light image generation network have the same structure, both including a second feature extraction layer, a second downsampling layer, a second upsampling layer, a residual layer and a post-processing layer; the second feature extraction layer includes a reflection filling layer, a third convolution layer, a normalization layer and an activation layer; the second downsampling layer includes a fourth convolution layer, a normalization layer and an activation layer; the second upsampling layer includes a transposed convolution layer, a normalization layer and an activation layer; the post-processing layer includes a reflection filling layer, a fifth convolution layer and a Tanh activation layer.

4. The low-light image enhancement method based on a multimodal guided recurrent generative network according to claim 1, characterized in that: The specific implementation steps of step 3 are as follows: Step 3.1: First, design a text prompt word pair, then feed the text prompt word pair into the multimodal teacher model text encoder to obtain the text prompt word pair encoding result; the multimodal teacher model text encoder includes a backbone layer and a bottleneck layer; the backbone layer includes a backbone block and a first average pooling layer; the bottleneck layer includes a second average pooling layer, an activation layer, and a normalization layer; Step 3.2: Send the synthesized normal illumination image and the synthesized low illumination image to the multimodal teacher model image encoder to obtain encoding results respectively; The multimodal teacher model image encoder includes a pre-processing layer, a visual converter, and a normalization layer; the pre-processing layer includes the sixth convolutional layer and a position encoder, the visual converter includes a residual attention layer and a fully connected layer, and the fully connected layer includes a linear layer and an activation layer; Step 3.3, calculating a low-light image enhancement result score based on the text prompt word pair encoding result and the synthesized normal-light image encoding result, and calculating a low-light image generation result score based on the text prompt word pair encoding result and the synthesized low-light image encoding result; Step 3.4: Subtract the low-light image enhancement result score from the low-light image generation result score to obtain the image enhancement effect evaluation score.

5. The low-light image enhancement method based on a multimodal guided recurrent generative network according to claim 1, characterized in that: The step 4 comprises: Step 4.1: Send the synthesized normal illumination image and the real low illumination image to the multimodal teacher model image encoder to obtain the encoding result, and use the encoded first eigenvector to obtain the low illumination image enhancement consistency evaluation score; Step 4.2: Send the synthesized low-light image and the real normal-light image to the multimodal teacher model image encoder to obtain the encoding result, and use the encoded second eigenvector to obtain the normal-light image enhancement consistency evaluation score; Step 4.3: The low-light image enhancement consistency evaluation score is summed with the normal-light image enhancement consistency evaluation score to obtain the image enhancement consistency evaluation score.

6. The low-light image enhancement method based on a multimodal guided recurrent generative network according to claim 1, characterized in that: The step 5 comprises: The synthetic low-light image and the synthetic normal-light image are respectively sent to the image discriminator network to discriminate the image type and whether the image belongs to a normal-light image; the image discriminator network includes two parts: the normal-light image discriminator and the normal-light image Gaussian low-pass discriminator; the normal-light image Gaussian low-pass discriminator adds a Gaussian low-pass filter before the discriminator network; the discriminator network consists of the seventh convolutional layer, an activation function and a normalization layer; the normalization layer is BN, and the activation function is LeakyReLU.

Images