Visible light image to infrared image conversion method based on enhanced generative adversarial network
By enhancing the U-Net architecture, Transformer and Mamba dual encoders, and differential product learning modules of the generative adversarial network, the accuracy and computational complexity problems of visible light image to infrared image conversion are solved, and efficient infrared image generation is achieved.
Patent Information
- Application Number
- CN202411805337.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-10
- Publication Date
- 2025-10-03
- Estimated Expiration
- 2044-12-10
AI Technical Summary
Existing technologies make it difficult to efficiently convert visible light images into high-quality infrared images, especially when dealing with differences in atmospheric radiation and target radiation, and complex objects. The generated results are inaccurate and the computational complexity is high.
A method based on enhanced generative adversarial networks is adopted, using the generator and discriminator of the U-Net architecture, combined with the dual encoder of Transformer and Mamba, introducing the differential product learning module, and designing spatial and channel attention modules to optimize the loss function to improve feature extraction and generation quality.
The quality of infrared image generation is improved, the number of parameters is reduced, the computational complexity is lowered, and more efficient visible light image to infrared image conversion is achieved.
Smart Images
Figure CN119624757B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to a method for converting visible light images into infrared images based on an enhanced generative adversarial network. Background Art
[0002] Infrared images are widely used in military applications, security monitoring, and autonomous driving because they can capture thermal radiation information, provide clear global outlines of targets, and remain robust in low light, smoke, and camouflage interference. However, collecting infrared images requires specialized and expensive equipment, and acquiring images of specific scenes is time-consuming and laborious. This has led to a scarcity of infrared image datasets, limiting the research and application of computer vision technology in the infrared field. In contrast, visible light images are readily available and abundant, prompting researchers to explore methods for converting visible light images to infrared images.
[0003] Early methods for obtaining infrared images mainly relied on simulation modeling and manual feature processing. Although these methods took into account various physical and low-level features of different objects, thereby providing a certain degree of physical credibility, the wide variety of objects in nature and the complex and unpredictable surface temperature patterns made manual analysis, segmentation and modeling of each image very time-consuming and labor-intensive, and prone to inaccuracies and inconsistencies.
[0004] With recent advances in deep learning, researchers have attempted to use deep models to convert visible light images into infrared images. However, due to the influence of atmospheric radiation and the intrinsic radiation of the target, the temperature of different regions in infrared images varies significantly. Therefore, accurately matching and predicting the temperature of corresponding regions when converting from visible light images to infrared images becomes extremely challenging. Furthermore, the complexity of visible light images, which often contain numerous objects, further complicates the conversion process.
[0005] Convolution-based deep models are limited by their local receptive field, which greatly hinders their ability to capture long-range information, often leading to insufficient feature extraction and suboptimal generation results. Although Transformer-based models address the above issues and excel in global modeling, the quadratic complexity of the self-attention mechanism imposes a high computational burden, especially in tasks requiring dense predictions (such as infrared image conversion).
[0006] Mamba, based on the classic state-space model (SSM), aims to model long-range dependencies while maintaining computational complexity that scales linearly with the input size. Mamba has been widely studied in various fields, including language understanding and general vision tasks. Despite its many advantages, its selective state-space mechanism selectively propagates or forgets information based on the current input, which can lead to the loss of some detailed information in image processing. Summary of the Invention
[0007] Technical problems to be solved
[0008] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a visible light image to infrared image conversion method based on enhanced generative adversarial network, which solves the technical problem of how to achieve high-quality visible light image to infrared image conversion.
[0009] Technical Solution
[0010] In order to achieve the above objectives, the main technical solutions adopted by the present invention include:
[0011] In a first aspect, the present invention provides a method for converting visible light images to infrared images based on an enhanced generative adversarial network, comprising:
[0012] S1, construct a dataset of visible light images and corresponding real infrared images;
[0013] S2, build the discriminator and generator of the enhanced generative adversarial network;
[0014] S3, construct the loss function of the discriminator and generator;
[0015] S4, inputting the visible light image into the generator to obtain the generated infrared image Fake-IR;
[0016] S5, the Fake-IR is spliced with the corresponding visible light image and then input into the discriminator, and the discriminator outputs fake;
[0017] S6, the visible light image and the corresponding real infrared image are spliced and input into the discriminator, and the discriminator outputs true;
[0018] S7, derive the loss functions of the discriminator and generator, and use the backpropagation algorithm to update the parameters of the generator and discriminator;
[0019] S8, after repeating S4 to S7 for a preset number of rounds, the training of the generator is completed, and the visible light image is input into the trained generator to generate the corresponding infrared image.
[0020] Optionally, build a discriminator and generator for an enhanced generative adversarial network, including:
[0021] The generator and discriminator adopt U-Net structure;
[0022] A dual encoder based on Transformer and Mamba is used as the encoder of the discriminator;
[0023] A differential product learning module is introduced in the dual encoder.
[0024] Optionally, construct the generator's loss function, including:
[0025] Use the mean squared error method to construct the discriminator output as the loss of the true label;
[0026] Use the L1 method to construct the loss between Fake-IR and the corresponding real infrared image;
[0027] A structural similarity method is used to construct the loss between Fake-IR and the corresponding real infrared image.
[0028] Optionally, the generator loss function is:
[0029]
[0030] is the mean square error loss, where represents the discrimination result at pixel (i, j), and Represents the discrimination result of the entire image; is the structural similarity loss, is L1 loss; represents the computational expectation; X represents the visible light image, G(X) represents the output of the generator, Y represents the real infrared image, B is the batch size, represents the Transformer-based encoder, stands for Mamba-based encoder, D dec Represents the decoder module.
[0031] Optionally, construct the discriminator loss function, including:
[0032] Use the mean square error method to construct f1 as the loss of false labels and r1 as the loss of true labels;
[0033] Use the mean square error method to construct fd1 as the loss of false labels and rd1 as the loss of true labels;
[0034] Among them, the Fake-IR and the corresponding visible light image are spliced and input into the discriminator, and the output of the discriminator is f1 and the output of each layer of the dual encoder in the discriminator is fd1.
[0035] Optionally, the discriminator loss function is:
[0036]
[0037]
[0038] in, Indicates the authenticity judgment of the entire image, Represents the authenticity judgment of pixel (i, j), Measures the authenticity of the feature value at position (i, j) in the DPM output layer of the dual encoder; L represents the number of network layers, FD represents the output of the FD branch in DPM, and FP represents the output of the FP branch in DPM.
[0039] In a second aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements a method for converting visible light images to infrared images based on an enhanced generative adversarial network as described in any one of the first aspects above.
[0040] In a third aspect, the present invention provides a storage device comprising a storage medium and a processor, wherein the storage medium stores a computer program, and when the program is executed by the processor, the method for converting visible light images into infrared images based on an enhanced generative adversarial network as described in any one of the first aspects above is implemented.
[0041] Beneficial effects
[0042] The beneficial effects of the present invention are as follows: a method for converting visible light images to infrared images based on an enhanced generative adversarial network is proposed. First, from the perspective of improving the quality of infrared image generation while reducing the number of parameters required by the method, it is proposed to adopt a U-Net architecture in the generator and discriminator, and replace the traditional convolution module in the U-Net with a Mamba model to solve the problem of its local receptive field. Secondly, a spatial and channel attention module is constructed and integrated into the U-Net jump connection to provide more diverse features. Thirdly, a dual encoder based on Transformer and Mamba is proposed and used to replace the encoder in the discriminator to improve the performance of the discriminator. Then, a differential product learning module is designed based on the dual encoder to enhance the ability of the dual encoder to extract difference and consistency features. Next, the loss function required for method training is given. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 A schematic diagram of the overall network structure framework of the infrared image conversion method provided by an embodiment of the present invention;
[0044] Figure 2 Schematic diagram of the infrared image conversion method generator framework and spatial and channel attention modules provided by an embodiment of the present invention;
[0045] Figure 3 The discriminator framework and difference-product learning module of the infrared image conversion method provided by the embodiment of the present invention;
[0046] Figure 4 Schematic diagram of generation results on various data sets provided in embodiments of the present invention. DETAILED DESCRIPTION
[0047] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation methods in conjunction with the accompanying drawings.
[0048] In the visible-to-infrared image conversion task, the temperatures of different objects and regions in the infrared image can vary significantly due to the influence of atmospheric radiation and the target's own radiation characteristics. Therefore, accurately matching and predicting the temperatures of corresponding regions when converting from visible-to-infrared images becomes extremely challenging. Furthermore, visible-to-infrared images often contain a large number of complex objects, further increasing the difficulty of visible-to-infrared image conversion.
[0049] The visible light image to infrared image conversion method based on enhanced generative adversarial network proposed in an embodiment of the present invention can sequentially learn a superior feature extraction paradigm from the method training stage to obtain a network with better generation effect, and then use the trained network to infer the visible light image to obtain the corresponding infrared image.
[0050] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. Instead, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.
[0051] First, refer to Figure 1 、 Figure 2 and Figure 3 This embodiment provides a method for converting visible light images to infrared images based on an enhanced generative adversarial network. The method, based on a generative adversarial network, consists of three main components: a generator, a discriminator, and a loss function. The generator and discriminator employ a U-Net structure, and Mamba replaces the traditional convolutional modules in the U-Net structure. The U-Net structure improves overall method performance, while Mamba addresses the local receptive field problem of traditional convolutional modules and further improves image generation quality.
[0052] To maintain the simplicity and efficiency of the overall method, this method uses a slightly modified generator as the discriminator. The main modifications include: 1. Replacing the generator's encoder with a dual encoder based on Transformer and Mamba to enhance feature extraction; 2. Introducing a differential product learning module in the dual encoder to strengthen the extraction of differential and consistent features; 3. Adjusting the input of the discriminator to allow simultaneous input of a visible light image and its corresponding infrared image (or a visible light image and a generated infrared image).
[0053] The loss function of the method includes the generator loss function and the discriminator loss function. In addition to the loss of the basic generative adversarial network, the generator loss function introduces L1 loss and loss based on structural similarity SSIM, and the discriminator introduces L1 loss and multi-layer feature loss based on dual encoders.
[0054] This embodiment provides a method for converting visible light images to infrared images based on an enhanced generative adversarial network, including:
[0055] S1, construct a dataset of visible light images and corresponding real infrared images;
[0056] S2, build the discriminator and generator of the enhanced generative adversarial network;
[0057] S3, construct the loss function of the discriminator and generator;
[0058] S4, inputting the visible light image into the generator to obtain the generated infrared image Fake-IR;
[0059] S5, the Fake-IR is spliced with the corresponding visible light image and then input into the discriminator, and the discriminator outputs fake;
[0060] S6, the visible light image and the corresponding real infrared image are spliced and input into the discriminator, and the discriminator outputs true;
[0061] S7, derive the loss functions of the discriminator and generator, and use the backpropagation algorithm to update the parameters of the generator and discriminator;
[0062] S8, after repeating S4 to S7 for a preset number of rounds, the training of the generator is completed, and the visible light image is input into the trained generator to generate the corresponding infrared image.
[0063] Optionally, build a discriminator and generator for an enhanced generative adversarial network, including:
[0064] The generator and discriminator adopt U-Net structure;
[0065] A dual encoder based on Transformer and Mamba is used as the encoder of the discriminator;
[0066] A differential product learning module is introduced in the dual encoder.
[0067] Optionally, construct the generator's loss function, including:
[0068] Use the mean squared error method to construct the discriminator output as the loss of the true label;
[0069] Use the L1 method to construct the loss between Fake-IR and the corresponding real infrared image;
[0070] A structural similarity method is used to construct the loss between Fake-IR and the corresponding real infrared image.
[0071] Optionally, the generator loss function is:
[0072]
[0073] is the mean square error loss, where represents the discrimination result at pixel (i, j), and Represents the discrimination result of the entire image; is the structural similarity loss, is L1 loss; represents the computational expectation; X represents the visible light image, G(X) represents the output of the generator, Y represents the real infrared image, B is the batch size, represents the Transformer-based encoder, stands for Mamba-based encoder, D dec Represents the decoder module.
[0074] Optionally, construct the discriminator loss function, including:
[0075] Use the mean square error method to construct f1 as the loss of false labels and r1 as the loss of true labels;
[0076] Use the mean square error method to construct fd1 as the loss of false labels and rd1 as the loss of true labels;
[0077] Among them, the Fake-IR and the corresponding visible light image are spliced and input into the discriminator, and the output of the discriminator is f1 and the output of each layer of the dual encoder in the discriminator is fd1.
[0078] Optionally, the discriminator loss function is:
[0079]
[0080]
[0081] in, Indicates the authenticity judgment of the entire image. Represents the authenticity judgment of pixel (i, j), Measures the authenticity of the feature value at position (i, j) in the DPM output layer of the dual encoder; L represents the number of network layers, FD represents the output of the FD branch in DPM, and FP represents the output of the FP branch in DPM.
[0082] This method uses a generative adversarial network as the main framework, which consists of two parts: a generator and a discriminator. Therefore, the loss function of the method is also divided into two parts: the generator loss and the discriminator loss, and the two jointly train the entire network.
[0083] For each generated infrared image, the overall structure of the corresponding real infrared image should be preserved. Structural similarity loss is introduced to ensure that the generated image remains structurally similar to the input image.
[0084] It represents the absolute difference between the corresponding pixel values of the generated infrared image and the real infrared image.
[0085] The discriminator also uses three different loss functions. The output of the discriminator dual encoder is used as the loss Part of the focus is on the authenticity of the entire image; the output of the discriminator is used as the loss Focus on the authenticity of each pixel; the difference and consistency network output features in the differential product learning module are used as multi-layer loss It mainly considers the authenticity of each pixel at different feature scales.
[0086] The spatial and channel attention modules are designed as follows Figure 2 As shown. Considering the heat distribution between different targets and different parts in infrared images, as well as the influence of different channels on the generation effect when generating infrared images from visible light images, we design a simple but effective feedforward convolutional attention module - spatial and channel attention module. Specifically, given a feature map f, the module first infers the attention map f in parallel along the spatial and channel dimensions. c and f s , to enhance the diversity of feature extraction. These attention maps are then connected, convolved, and processed using activation functions to generate combined spatial and channel feature maps f csc Then f csc With f c Multiply to generate a new feature map f′ c Then, f′ c With f s Multiply to get f′ sFinally, by combining the residual mechanism, f′ s Added to the original input feature map f, thereby retaining the original low-level features. The mathematical representation of this module is as follows:
[0087] f s =SAB(f)
[0088] f c =CAB(f)
[0089]
[0090] f′ c =f csc ☉f c
[0091] f′ s =f′ c ☉f s
[0092] f scam =f s +f
[0093] Among them, SAB(·) represents the spatial attention mechanism, CAB(·) represents the channel attention mechanism, Conv2D(·) represents the two-dimensional convolution operation, and f scam Represents the output of the spatial and channel attention modules. The main purpose of designing the spatial and channel attention modules is to enhance the diversity of the output features of each layer of the encoder, provide more detailed features to the decoder through the skip connection of the U-Net, and thus improve the performance of the generator and discriminator.
[0094] Dual encoder design based on Transformer and Mamba Figure 3 shown.
[0095] While the Transformer model excels at modeling global features, its computational complexity grows quadratically due to its self-attention mechanism, requiring significant computational resources, particularly for tasks requiring dense predictions, such as infrared image generation. Mamba, based on a classic state-space model, aims to model long-range dependencies while maintaining computational complexity that scales linearly with the input size, thereby optimizing GPU memory usage. Despite its numerous advantages, Mamba's selective state-space mechanism selectively propagates or forgets information based on the current input, potentially resulting in the loss of some detailed information.
[0096] Therefore, in order to combine the advantages of the two models, this method designs a dual encoder based on Transformer and Mamba, which not only retains the global modeling capability of Transformer, but also benefits from the ability of Mamba model to capture long-range information with linear computational complexity.
[0097] The difference product learning module is designed as follows Figure 3 The main purpose of designing this module is to give full play to the performance of the dual encoder to extract the differences and consistency of features between the Transformer and Mamba encoders.
[0098] Specifically, the differential product learning module takes as input the feature outputs of the corresponding layers of the Transformer and Mamba encoders. First, the features of each layer of the dual encoders are passed through spatial and channel attention modules to extract spatial and channel features. Then, feature diversity and consistency are extracted using feature diversity and consistency networks, respectively. Finally, the output features of the diversity and consistency networks are concatenated to enhance feature diversity.
[0099] The differential product learning module has two main functions in the method: first, it enhances the diversity of feature extraction at each layer of the dual encoder; second, the difference and consistency network output features are integrated into the discriminator loss function as multi-layer loss features. This integration provides pixel-level feedback on method training at different scales, thereby optimizing the method and improving its performance.
[0100] This paper adopts the U-Net architecture in the generator and discriminator of a generative adversarial network and replaces the traditional convolutional neural network module in the U-Net with the Mamba model, aiming to establish long-range dependencies between features while maintaining computational complexity that scales linearly with the input size. Furthermore, the paper constructs a dual encoder based on Transformer and Mamba and integrates it into the discriminator to improve the method's performance. Spatial and channel attention modules are designed and integrated into the U-Net skip connection to more accurately capture spatial and channel features, thereby preserving the diversity of input features. Furthermore, this module integrates a residual mechanism to combine low-level features to improve generator performance. A differential product learning module is designed and introduced into the discriminator's dual encoder to extract feature differences and consistency between the two encoders, thereby enhancing the discriminator's performance. Furthermore, the difference and consistency features output by each layer of the dual encoder are incorporated into the discriminator's loss function, providing the discriminator with pixel-level training feedback at different scales.
[0101] The following is a further description of a method for converting visible light images to infrared images based on an enhanced generative adversarial network in accordance with the present application, which is implemented by the following steps:
[0102] Step 1: Data preparation and evaluation indicators.
[0103] The method uses four datasets: VEDAI, KAIST, LLVIP, and DroneVehicle. Each dataset contains visible light images and paired infrared images, including near-infrared, long-wave infrared, and medium-wave infrared. For each dataset, 1200 image pairs are selected as the training set for training the method, 200 image pairs are selected as the validation set for verification, and 200 image pairs are selected as the test set for inference and prediction. The generated infrared images are evaluated using five classic image evaluation metrics: the structural similarity metric (SSIM), the mean structural similarity metric (MSSIM), the peak signal-to-noise ratio (PSNR), the L1 loss, and the Euclidean metric (LPIPS) between image features.
[0104] The second step is parameter setting.
[0105] The parameters involved are:
[0106] (1) The image size in both training and inference stages is set to 512x512;
[0107] (2) The method was implemented using the PyTorch library and trained on a Linux platform (Ubuntu 22.04.4LTS) using an NVIDIA GeForce RTX 4090 24G GPU;
[0108] (3) The optimizer uses Adam’s mini-batch SGD. The learning rate of the generator is set to 2x10-4, the learning rate of the discriminator is set to 2x10-6, and the momentum parameters are set to β1 = 0.5 and β2 = 0.999. The learning rate is dynamically adjusted after every 100 dataset iterations.
[0109] (4) The batch size is 1 and the number of training rounds is 200.
[0110] The third step is model training.
[0111] The system consists of two parts: a generator and a discriminator, so two models need to be trained. The discriminator identifies fake images generated by the generator, while the generator generates high-quality images to deceive the discriminator. Eventually, the two models reach a Nash equilibrium, completing training. During each round of training, the generator first converts visible light images into infrared images. The discriminator is then trained. The generated infrared image is concatenated with the corresponding visible light image and input into the discriminator to identify the output as fake. The real infrared image is concatenated with the corresponding visible light image and input into the discriminator to identify the output as real. Finally, the generator is trained. The generated infrared image is concatenated with the corresponding visible light image and input into the discriminator to identify the output as real. The generated images are tested and their performance indicators are calculated. After the model is trained, the weight parameters of the generator and discriminator are saved.
[0112] The fourth step is the reasoning stage.
[0113] The performance of the method is tested using the test dataset and real visible light images from the four data. In the inference prediction stage, the visible light image is input into the trained generator to obtain the generated infrared image.
[0114] The results on each dataset are currently the best. Table 1 shows the results of this solution on various datasets.
[0115] Table 1
[0116]
[0117] Figure 4 Schematic diagram of the generation results of the above scheme on various data sets.
[0118] In a second aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements a method for converting visible light images to infrared images based on an enhanced generative adversarial network as described in any one of the first aspects above.
[0119] In a third aspect, an embodiment of the present invention provides a storage device comprising a storage medium and a processor, wherein the storage medium stores a computer program, and when the program is executed by the processor, the method for converting a visible light image to an infrared image based on an enhanced generative adversarial network as described in any one of the first aspects above is implemented.
[0120] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0121] Obviously, those skilled in the art may make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if such modifications and variations fall within the scope of the claims and their equivalents, the present invention shall also include such modifications and variations.
[0122] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may alter, modify, replace and modify the above embodiments within the scope of the present invention.
Claims
1. A method for converting visible light images to infrared images based on enhanced generative adversarial networks, characterized in that: include: S1, construct a dataset of visible light images and corresponding real infrared images; S2, build the discriminator and generator of the enhanced generative adversarial network; S3, construct the loss function of the discriminator and generator; S4, inputting the visible light image into the generator to obtain the generated infrared image Fake-IR; S5, the Fake-IR is spliced with the corresponding visible light image and then input into the discriminator, and the discriminator outputs fake; S6, the visible light image and the corresponding real infrared image are spliced and input into the discriminator, and the discriminator outputs true; S7, derive the loss functions of the discriminator and generator, and use the backpropagation algorithm to update the parameters of the generator and discriminator; S8, repeating S4 to S7 for a preset number of rounds to complete the training of the generator, inputting the visible light image into the trained generator to generate the corresponding infrared image; Build the discriminator and generator of the enhanced generative adversarial network, including: The generator and discriminator adopt the U-Net structure, and the Mamba model replaces the traditional convolution module in the U-Net. Construct spatial and channel attention modules and integrate them into U-Net skip connections; A dual encoder based on Transformer and Mamba is used as the encoder of the discriminator; A differential product learning module is introduced in the dual encoder.
2. The method for converting visible light images to infrared images based on enhanced generative adversarial networks according to claim 1, characterized in that: Construct the generator's loss function, including: Use the mean squared error method to construct the discriminator output as the loss of the true label; Use the L1 method to construct the loss between Fake-IR and the corresponding real infrared image; A structural similarity method is used to construct the loss between Fake-IR and the corresponding real infrared image.
3. The method for converting visible light images to infrared images based on enhanced generative adversarial networks according to claim 2, characterized in that: The generator loss function is: is the mean square error loss, where represents the discrimination result at pixel (i, j), and Represents the discrimination result of the entire image; is the structural similarity loss, is L1 loss; represents the computational expectation; X represents the visible light image, G(X) represents the output of the generator, Y represents the real infrared image, B is the batch size, represents the Transformer-based encoder, stands for Mamba-based encoder, D dec Represents the decoder module.
4. The method for converting visible light images to infrared images based on enhanced generative adversarial networks according to claim 3, characterized in that: Construct the loss function of the discriminator, including: Use the mean square error method to construct f1 as the loss of false labels and r1 as the loss of true labels; Use the mean square error method to construct fd1 as the loss of false labels and rd1 as the loss of true labels; Among them, the Fake-IR and the corresponding visible light image are spliced and input into the discriminator, and the output of the discriminator is f1 and the output of each layer of the dual encoder in the discriminator is fd1.
5. The method for converting visible light images to infrared images based on enhanced generative adversarial networks according to claim 4, characterized in that: The discriminator loss function is: in, Indicates the authenticity judgment of the entire image, Represents the authenticity judgment of pixel (i, j), Measures the authenticity of the feature value at position (i, j) in the DPM output layer of the dual encoder; L represents the number of network layers, FD represents the output of the FD branch in DPM, and FP represents the output of the FP branch in DPM.
6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for converting visible light images to infrared images based on an enhanced generative adversarial network as described in any one of claims 1 to 5 is implemented.
7. A storage device comprising a storage medium and a processor, wherein the storage medium stores a computer program, wherein: When the processor executes the computer program, the method for converting visible light images to infrared images based on an enhanced generative adversarial network as described in any one of claims 1 to 5 is implemented.