Method for converting visible light image into infrared image based on enhanced generative adversarial network

Abstract

The invention relates to a visible light image-to-infrared image conversion method based on an enhanced generative adversarial network. The method comprises the following steps: constructing a data set of a visible light image and a corresponding real infrared image; constructing a discriminator and a generator of the enhanced generative adversarial network; constructing loss functions of the discriminator and the generator; inputting the visible light image into a generator to obtain a generated infrared image Fake-IR; splicing the Fake-IR and the corresponding visible light image and then inputting the spliced image into a discriminator, wherein the output of the discriminator is false; the visible light image and the corresponding real infrared image are spliced and then input into a discriminator, and the output of the discriminator is real; carrying out derivation calculation on loss functions of the discriminator and the generator, and updating parameters of the generator and the discriminator by using a back propagation algorithm; and repeating the above steps for a preset number of rounds to complete the training of the generator, inputting the visible light image into the trained generator, and generating a corresponding infrared image. The method has the beneficial effect that high-quality conversion from a visible light image to an infrared image is realized.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method for converting a visible light image into an infrared image based on an enhanced generative adversarial network. Background Art

[0002] Infrared images are widely used in military applications, security monitoring, and autonomous driving because they can capture thermal radiation information, provide clear global outlines of targets, and remain robust in low light, smoke, and camouflage interference. However, collecting infrared images requires professional and expensive equipment, and acquiring images of specific scenes is time-consuming and laborious. This has led to a scarcity of infrared image datasets, limiting the research and application of computer vision technology in the infrared field. In contrast, visible light images are easy to obtain and rich and diverse, prompting researchers to explore methods for converting visible light images to infrared images.

[0003] Early methods for obtaining infrared images mainly relied on simulation modeling and manual feature processing. Although these methods take into account various physical and low-level features of different objects, thereby providing a certain degree of physical credibility, the wide variety of objects in nature and the complex and unpredictable surface temperature patterns make manual analysis, segmentation and modeling of each image very time-consuming and laborious, and prone to inaccuracies and inconsistencies.

[0004] In recent years, with the advancement of deep learning, researchers have tried to use deep models to convert visible light images into infrared images. However, due to the influence of atmospheric radiation and the intrinsic radiation of the target, the temperature of infrared images in different regions varies significantly. Therefore, when converting from visible light images to infrared images, it becomes extremely challenging to accurately match and predict the temperature of the corresponding area. In addition, visible light images usually contain many objects, and their complexity further increases the difficulty of converting infrared images.

[0005] Convolution-based deep models are limited by their local receptive field, which greatly hinders their ability to capture long-range information, often leading to insufficient feature extraction and unsatisfactory generation results. Although Transformer-based models solve the above problems and excel in global modeling, the quadratic complexity of the self-attention mechanism brings a high computational burden, especially in tasks that require dense predictions (such as infrared image conversion).

[0006] Mamba is based on the classic state-space model (SSM) and aims to establish long-range dependencies while maintaining a computational complexity that grows linearly with the input size. Mamba has been widely studied in multiple fields, including language understanding and general vision tasks. Despite its many advantages, its selective state-space mechanism selectively propagates or forgets information based on the current input, which may lead to the loss of some detailed information in image processing. Summary of the invention

[0007] Technical issues to be solved

[0008] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a visible light image to infrared image conversion method based on enhanced generative adversarial network, which solves the technical problem of how to achieve high-quality conversion of visible light images to infrared images.

[0009] Technical Solution

[0010] In order to achieve the above object, the main technical solutions adopted by the present invention include:

[0011] In a first aspect, the present invention provides a method for converting a visible light image to an infrared image based on an enhanced generative adversarial network, comprising:

[0012] S1, construct a dataset of visible light images and corresponding real infrared images;

[0013] S2, build the discriminator and generator of the enhanced generative adversarial network;

[0014] S3, construct the loss function of the discriminator and generator;

[0015] S4, inputting the visible light image into the generator to obtain a generated infrared image Fake-IR;

[0016] S5, concatenate the Fake-IR and the corresponding visible light image and input them into the discriminator, and the discriminator outputs false;

[0017] S6, splicing the visible light image and the corresponding real infrared image and inputting them into the discriminator, and the discriminator outputs true;

[0018] S7, derive the loss functions of the discriminator and the generator, and use the back propagation algorithm to update the parameters of the generator and the discriminator;

[0019] S8, after repeating S4 to S7 for a preset number of rounds, the training of the generator is completed, and the visible light image is input into the trained generator to generate a corresponding infrared image.

[0020] Optionally, build a discriminator and generator for an enhanced generative adversarial network, including:

[0021] The generator and discriminator adopt U-Net structure;

[0022] A dual encoder based on Transformer and Mamba is used as the encoder of the discriminator;

[0023] A differential product learning module is introduced in the dual encoder.

[0024] Optionally, construct the generator's loss function, including:

[0025] Use the mean squared error method to construct the discriminator output as the loss of the true label;

[0026] Use the L1 method to construct the loss between Fake-IR and the corresponding real infrared image;

[0027] The loss between Fake-IR and the corresponding real infrared image is constructed using a structural similarity method.

[0028] Optionally, the generator loss function is:

[0029]

[0030] is the mean square error loss, where represents the discrimination result at pixel (i, j), and Represents the discrimination result of the entire image; is the structural similarity loss, is L1 loss; represents the computational expectation; X represents the visible light image, G(X) represents the output of the generator, Y represents the real infrared image, B is the batch size, represents the Transformer-based encoder, stands for Mamba-based encoder, D dec Represents a decoder module.

[0031] Optionally, construct the loss function of the discriminator, including:

[0032] Use the mean square error method to construct f1 as the loss of false labels and r1 as the loss of true labels;

[0033] Use the mean square error method to construct fd1 as the loss of the false label and rd1 as the loss of the true label;

[0034] Among them, the Fake-IR is spliced ​​with the corresponding visible light image and then input into the discriminator, so that the output of the discriminator is f1 and the output of each layer of the dual encoder in the discriminator is fd1.

[0035] Optionally, the discriminator loss function is:

[0036]

[0037]

[0038] in, Represents the authenticity judgment of the entire image. Represents the authenticity judgment of pixel (i, j), Measures the authenticity of the feature value at position (i, j) in the DPM output layer in the dual encoder; L represents the number of network layers, FD represents the output of the FD branch in DPM, and FP represents the output of the FP branch in DPM.

[0039] In a second aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements a method for converting a visible light image to an infrared image based on an enhanced generative adversarial network as described in any one of the first aspects above.

[0040] In a third aspect, the present invention provides a storage device comprising a storage medium and a processor, wherein the storage medium stores a computer program, and when the program is executed by the processor, the method for converting a visible light image to an infrared image based on an enhanced generative adversarial network as described in any one of the first aspects above is implemented.

[0041] Beneficial Effects

[0042] The beneficial effects of the present invention are as follows: a visible light image to infrared image conversion method based on an enhanced generative adversarial network of the present invention, firstly, from the perspective of improving the quality of infrared image generation while reducing the amount of parameters required by the method, proposes to adopt a U-Net architecture in the generator and the discriminator, and replace the traditional convolution module in the U-Net with the Mamba model to solve the problem of its local receptive field. Secondly, a spatial and channel attention module is constructed and integrated into the U-Net jump connection to provide more diverse features. Thirdly, a dual encoder based on Transformer and Mamba is proposed and used to replace the encoder in the discriminator to improve the performance of the discriminator. Then, a differential product learning module is designed based on the dual encoder to enhance the ability of the dual encoder to extract difference and consistency features. Next, the loss function required for method training is given. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of the overall network structure framework of the infrared image conversion method provided by an embodiment of the present invention;

[0044] Figure 2 A schematic diagram of a generator framework and a spatial and channel attention module for an infrared image conversion method provided in an embodiment of the present invention;

[0045] Figure 3 The discriminator framework and the difference product learning module of the infrared image conversion method provided by the embodiment of the present invention;

[0046] Figure 4 A schematic diagram of generation results on various data sets provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation modes in conjunction with the accompanying drawings.

[0048] In the visible light image to infrared image conversion task, due to the influence of atmospheric radiation and the radiation characteristics of the target itself, the temperature of different targets and different areas in the infrared image may be significantly different. Therefore, when converting from visible light images to infrared images, it becomes extremely challenging to accurately match and predict the temperature of corresponding areas. In addition, visible light images usually contain a large number of complex objects, which further increases the difficulty of converting visible light images to infrared images.

[0049] The visible light image to infrared image conversion method based on enhanced generative adversarial network proposed in an embodiment of the present invention can sequentially learn superior feature extraction paradigms from the method training stage to obtain a network with better generation effect, and then use the trained network to infer the visible light image to obtain the corresponding infrared image.

[0050] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0051] First, refer to Figure 1 , Figure 2 and Figure 3 This embodiment provides a method for converting visible light images to infrared images based on an enhanced generative adversarial network. The main body of the method is based on a generative adversarial network, which mainly includes three parts, namely a generator, a discriminator, and a loss function. The generator and the discriminator use a U-Net structure, and Mamba is used to replace the traditional convolution module in the U-Net structure. The use of the U-Net structure can improve the performance of the overall method. The use of Mamba is intended to solve the problem of the local receptive field of the traditional convolution module, while further improving the image generation quality.

[0052] In order to keep the overall method simple and efficient, this method uses a slightly modified generator as the discriminator. The main modifications include: 1. Replacing the encoder of the generator with a dual encoder based on Transformer and Mamba to enhance feature extraction capabilities; 2. Introducing a differential product learning module in the dual encoder to enhance the extraction of differential and consistent features of the dual encoder; 3. Adjusting the input of the discriminator to allow the simultaneous input of visible light images and their corresponding infrared images (or visible light images and generated infrared images).

[0053] The loss function of the method includes the generator loss function and the discriminator loss function. In addition to the loss of the basic generative adversarial network, the generator loss function introduces L1 loss and loss based on structural similarity SSIM, and the discriminator introduces L1 loss and multi-layer feature loss based on dual encoders.

[0054] This embodiment provides a visible light image to infrared image conversion method based on enhanced generative adversarial network, including:

[0055] S1, construct a dataset of visible light images and corresponding real infrared images;

[0056] S2, build the discriminator and generator of the enhanced generative adversarial network;

[0057] S3, construct the loss function of the discriminator and generator;

[0058] S4, inputting the visible light image into the generator to obtain a generated infrared image Fake-IR;

[0059] S5, concatenate the Fake-IR and the corresponding visible light image and input them into the discriminator, and the discriminator outputs false;

[0060] S6, splicing the visible light image and the corresponding real infrared image and inputting them into the discriminator, and the discriminator outputs true;

[0061] S7, derive the loss functions of the discriminator and the generator, and use the back propagation algorithm to update the parameters of the generator and the discriminator;

[0062] S8, after repeating S4 to S7 for a preset number of rounds, the training of the generator is completed, and the visible light image is input into the trained generator to generate a corresponding infrared image.

[0063] Optionally, build a discriminator and generator for an enhanced generative adversarial network, including:

[0064] The generator and discriminator adopt U-Net structure;

[0065] A dual encoder based on Transformer and Mamba is used as the encoder of the discriminator;

[0066] A differential product learning module is introduced in the dual encoder.

[0067] Optionally, construct the generator's loss function, including:

[0068] Use the mean squared error method to construct the discriminator output as the loss of the true label;

[0069] Use the L1 method to construct the loss between Fake-IR and the corresponding real infrared image;

[0070] The loss between Fake-IR and the corresponding real infrared image is constructed using a structural similarity method.

[0071] Optionally, the generator loss function is:

[0072]

[0073] is the mean square error loss, where represents the discrimination result at pixel (i, j), and Represents the discrimination result of the entire image; is the structural similarity loss, is L1 loss; represents the computational expectation; X represents the visible light image, G(X) represents the output of the generator, Y represents the real infrared image, B is the batch size, represents the Transformer-based encoder, stands for Mamba-based encoder, D dec Represents a decoder module.

[0074] Optionally, construct the loss function of the discriminator, including:

[0075] Use the mean square error method to construct f1 as the loss of false labels and r1 as the loss of true labels;

[0076] Use the mean square error method to construct fd1 as the loss of the false label and rd1 as the loss of the true label;

[0077] Among them, the Fake-IR is spliced ​​with the corresponding visible light image and then input into the discriminator, so that the output of the discriminator is f1 and the output of each layer of the dual encoder in the discriminator is fd1.

[0078] Optionally, the discriminator loss function is:

[0079]

[0080]

[0081] in, Represents the authenticity judgment of the entire image. Represents the authenticity judgment of pixel (i, j), Measures the authenticity of the feature value at position (i, j) in the DPM output layer in the dual encoder; L represents the number of network layers, FD represents the output of the FD branch in DPM, and FP represents the output of the FP branch in DPM.

[0082] This method uses a generative adversarial network as the main framework, which consists of two parts: a generator and a discriminator. Therefore, the loss function of the method is also divided into two parts: a generator loss and a discriminator loss, and the two jointly train the entire network.

[0083] For each generated infrared image, the overall structure with the corresponding real infrared image should be preserved. Structural similarity loss is introduced to ensure that the generated image remains structurally similar to the input image.

[0084] It represents the absolute difference between the corresponding pixel values ​​of the generated infrared image and the real infrared image.

[0085] The discriminator also uses three different loss functions. The output of the discriminator dual encoder is used as the loss The part mainly focuses on the authenticity of the entire image; the output of the discriminator is used as the loss The focus is on the authenticity of each pixel; the difference and consistency network output features in the difference product learning module are used as multi-layer losses It mainly considers the authenticity of each pixel at different feature scales.

[0086] The spatial and channel attention modules are designed as follows Figure 2 As shown in the figure. Considering the heat distribution between different targets and different parts in infrared images, as well as the influence of different channels on the generation effect when generating infrared images from visible light images, we design a simple but effective feed-forward convolutional attention module - the spatial and channel attention module. Specifically, given a feature map f, the module first infers the attention map f in parallel along the spatial and channel dimensions. c and f s , to enhance the diversity of feature extraction. These attention maps are then concatenated, convolved, and processed using activation functions to generate combined spatial and channel feature maps f csc Then f csc With f c Multiply to generate a new feature map f′ c Then, f′ c With f s Multiply to get f′ sFinally, by combining the residual mechanism, f′ s Added to the original input feature map f, thereby retaining the original low-level features. The mathematical representation of this module is as follows:

[0087] f s =SAB(f)

[0088] f c =CAB(f)

[0089]

[0090] f′ c =f csc ☉f c

[0091] f′ s =f′ c ☉f s

[0092] f scam =f s +f

[0093] Among them, SAB(·) represents the spatial attention mechanism, CAB(·) represents the channel attention mechanism, Conv2D(·) represents the two-dimensional convolution operation, and f scam Represents the output of the spatial and channel attention modules. The main purpose of designing the spatial and channel attention modules is to enhance the diversity of the output features of each layer of the encoder, and to provide more detailed features to the decoder through the jump connection of U-Net, thereby improving the performance of the generator and discriminator.

[0094] The dual encoder design based on Transformer and Mamba is as follows Figure 3 shown.

[0095] Although the Transformer model performs well in global feature modeling, its computational complexity grows quadratically due to the self-attention mechanism, requiring a lot of computing resources, especially for tasks that require intensive predictions (such as infrared image generation). Mamba is based on the classic state-space model and aims to establish long-range dependencies while maintaining a computational complexity that scales linearly with the input size, thereby optimizing GPU memory usage. Despite its many advantages, the selective state-space mechanism adopted by Mamba selectively propagates or forgets information based on the current input, which may lead to the loss of some detailed information.

[0096] Therefore, in order to combine the advantages of the two models, this method designs a dual encoder based on Transformer and Mamba, which not only retains the global modeling ability of Transformer, but also benefits from the ability of Mamba model to capture long-range information with linear computational complexity.

[0097] The difference product learning module is designed as follows Figure 3 The main purpose of designing this module is to give full play to the performance of the dual encoder to extract the differences and consistency of features between the Transformer and Mamba encoders.

[0098] Specifically, the differential product learning module takes the feature outputs of the corresponding layers of the Transformer and Mamba encoders as input. First, the features of each layer of the dual encoder are extracted through the spatial and channel attention modules for spatial and channel features. Then, the feature difference and consistency networks are used to extract the diversity and consistency features respectively. Finally, the difference and consistency network output features are concatenated to enhance the diversity of features.

[0099] The difference product learning module has two main functions in the method: first, it enhances the diversity of feature extraction at each layer of the dual encoder; second, the difference and consistency network output features are integrated into the discriminator's loss function as multi-layer loss features. This integration provides pixel-level feedback for method training at different scales, thereby optimizing the method and improving its performance.

[0100] The present invention adopts the U-Net architecture in the generator and discriminator of the generative adversarial network, and replaces the traditional convolutional neural network module in the U-Net with the Mamba model, aiming to establish long-range dependencies of features while maintaining a computational complexity that scales linearly with the input size. In addition, the present invention constructs a dual encoder based on Transformer and Mamba and integrates it into the discriminator to improve the performance of the method. The spatial and channel attention modules are designed and integrated into the U-Net jump connection to more accurately capture the spatial and channel features to retain the diversity of the input features. In addition, the module integrates a residual mechanism to combine low-level features to improve the performance of the generator. The differential product learning module is designed and introduced into the dual encoder of the discriminator to extract the feature differences and consistency between the dual encoders, thereby enhancing the performance of the discriminator. In addition, the difference and consistency features of the outputs of each layer of the dual encoder are introduced into the loss function of the discriminator, providing the discriminator with pixel-level training feedback of different scales.

[0101] The following is a specific example to further illustrate a method for converting a visible light image to an infrared image based on an enhanced generative adversarial network. The method is implemented by the following steps:

[0102] Step 1: Data preparation and evaluation indicators.

[0103] The method uses four data sets: VEDAI, KAIST, LLVIP, and DroneVehicle. Each data set contains visible light images and paired infrared images. The infrared types include near infrared, long-wave infrared, and medium-wave infrared. For each data set, 1200 pairs of images are selected as training sets to train the method, 200 pairs of images are selected as validation sets to verify the method, and 200 pairs of images are selected as test sets to infer and predict the method. Five classic image evaluation indicators, including the structural similarity index SSIM, the average structural similarity index MSSIM, the peak signal-to-noise ratio PSNR, the L1 loss, and the Euclidean metric LPIPS between image features, are used to evaluate the generated infrared images.

[0104] The second step is parameter setting.

[0105] The parameters involved are:

[0106] (1) The image size in both training and inference stages is set to 512x512;

[0107] (2) The method was implemented using the PyTorch library and trained on a Linux platform (Ubuntu 22.04.4LTS) using an NVIDIA GeForce RTX 4090 24G GPU;

[0108] (3) The optimizer uses Adam's mini-batch SGD. The learning rate of the generator is set to 2x10-4, the learning rate of the discriminator is set to 2x10-6, the momentum parameters are set to β1=0.5 and β2=0.999, and the learning rate is dynamically adjusted after every 100 data set iterations;

[0109] (4) The batch size is 1 and the number of training rounds is 200.

[0110] The third step is model training.

[0111] It contains two parts: the generator and the discriminator, so two models need to be trained. The discriminator makes a false judgment on the image generated by the generator, and the generator generates high-quality images to deceive the discriminator. Finally, the two models reach Nash equilibrium and the training is completed. In each round of training, the generator first converts the visible light image into an infrared image. Then the discriminator is trained, and the generated infrared image is spliced ​​with the corresponding visible light image and input into the discriminator to make a false judgment on the output result. The real infrared image is spliced ​​with the corresponding visible light image and input into the discriminator to make a true judgment on the output result. Finally, the generator is trained, and the generated infrared image is spliced ​​with the corresponding visible light image and input into the discriminator to make a true judgment on the output result. Test the generated image and calculate its index performance. After training the model, save the weight parameters of the generator and discriminator.

[0112] The fourth step is the reasoning stage.

[0113] The performance of the method is tested using the test data set and real visible light images from the four data. In the inference prediction stage, the visible light image is input into the trained generator to obtain the generated infrared image.

[0114] The results on each data set are currently the best. Table 1 shows the results of this solution on each data set.

[0115] Table 1

[0116]

[0117] Figure 4 Schematic diagram of the generation results of the above scheme on various data sets.

[0118] In a second aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements a method for converting a visible light image to an infrared image based on an enhanced generative adversarial network as described in any one of the first aspects above.

[0119] In a third aspect, an embodiment of the present invention provides a storage device, including a storage medium and a processor, wherein the storage medium stores a computer program, and when the program is executed by the processor, the method for converting a visible light image to an infrared image based on an enhanced generative adversarial network as described in any one of the first aspects above is implemented.

[0120] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0121] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention should also include these modifications and variations.

[0122] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may alter, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A visible light image to infrared image conversion method based on enhanced generative adversarial network, characterized in that: include: S1, construct a dataset of visible light images and corresponding real infrared images; S2, build the discriminator and generator of the enhanced generative adversarial network; S3, construct the loss function of the discriminator and generator; S4, inputting the visible light image into the generator to obtain a generated infrared image Fake-IR; S5, concatenate the Fake-IR and the corresponding visible light image and input them into the discriminator, and the discriminator outputs false; S6, stitching the visible light image and the corresponding real infrared image and inputting them into the discriminator, and the discriminator outputs true; S7, derive the loss functions of the discriminator and the generator, and use the back propagation algorithm to update the parameters of the generator and the discriminator; S8, after repeating S4 to S7 for a preset number of rounds, the training of the generator is completed, and the visible light image is input into the trained generator to generate a corresponding infrared image.

2. According to claim 1, a method for converting visible light images to infrared images based on enhanced generative adversarial networks is characterized in that: Construct the discriminator and generator of the enhanced generative adversarial network, including: The generator and discriminator adopt U-Net structure; A dual encoder based on Transformer and Mamba is used as the encoder of the discriminator; A differential product learning module is introduced in the dual encoder.

3. The method for converting visible light images to infrared images based on enhanced generative adversarial networks according to claim 2, characterized in that: Construct the loss function of the generator, including: Use the mean squared error method to construct the discriminator output as the loss of the true label; Use the L1 method to construct the loss between Fake-IR and the corresponding real infrared image; The loss between Fake-IR and the corresponding real infrared image is constructed using a structural similarity method.

4. The method for converting visible light images to infrared images based on enhanced generative adversarial networks according to claim 3, characterized in that: The generator loss function is: is the mean square error loss, where represents the discrimination result at pixel (i, j), and Represents the discrimination result of the entire image; is the structural similarity loss, is L1 loss; represents the computational expectation; X represents the visible light image, G(X) represents the output of the generator, Y represents the real infrared image, B is the batch size, represents the Transformer-based encoder, stands for Mamba-based encoder, D dec Represents a decoder module.

5. The method for converting visible light images to infrared images based on enhanced generative adversarial networks according to claim 4, characterized in that: Construct the loss function of the discriminator, including: Use the mean square error method to construct f1 as the loss of false labels and r1 as the loss of true labels; Use the mean square error method to construct fd1 as the loss of the false label and rd1 as the loss of the true label; Among them, the Fake-IR is spliced ​​with the corresponding visible light image and then input into the discriminator, so that the output of the discriminator is f1 and the output of each layer of the dual encoder in the discriminator is fd1.

6. The method for converting visible light images to infrared images based on enhanced generative adversarial networks according to claim 5, characterized in that: The discriminator loss function is: in, Represents the authenticity judgment of the entire image. Represents the authenticity judgment of pixel (i, j), Measures the authenticity of the feature value at position (i, j) in the DPM output layer in the dual encoder; L represents the number of network layers, FD represents the output of the FD branch in DPM, and FP represents the output of the FP branch in DPM.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for converting a visible light image to an infrared image based on an enhanced generative adversarial network as described in any one of claims 1 to 6 is implemented.

8. A storage device comprising a storage medium and a processor, wherein the storage medium stores a computer program, characterized in that: When the processor executes the computer program, the method for converting a visible light image to an infrared image based on an enhanced generative adversarial network as described in any one of claims 1 to 6 is implemented.

Images