A method, system and device for generating long-wave infrared images

Abstract

The present invention belongs to the field of image generation technology, and in particular, is a method for generating long-wave infrared images, and the specific steps are: S1, constructing a network model: constructing a generative adversarial network including a generator and a discriminator; S2, preparing a data set: pre-training the generative adversarial network according to a first data set of aligned infrared images. The present invention trains the network model by using paired visible light images and long-wave infrared images, thereby realizing the generation of long-wave infrared images, and the specific steps are: constructing a network model, preparing a data set, selecting a minimized loss function, training the network model, fine-tuning the model, and saving the model. The present invention adopts a generative adversarial network structure, utilizes the game characteristics between the generator and the discriminator, strengthens the extraction of deep information of the image, solves the problem of missing long-wave infrared image data sets, enhances the naturalness and authenticity of the generated long-wave infrared images, and makes the generated long-wave infrared images more in line with human eye perception.

Description

Technical Field

[0001] The present invention relates to the technical field of image generation, and in particular to a method, system and device for generating long-wave infrared images. Background Art

[0002] Long-wave infrared image generation is a technique that generates long-wave infrared images from visible light images by learning the modal relationship between visible light and long-wave infrared images. The acquisition of infrared images requires specific equipment and environmental conditions, and is affected by factors such as weather and lighting. This results in a limited number of infrared image datasets that are actually available. Long-wave infrared image generation can solve the problem of missing long-wave infrared sample datasets. Infrared images can show the thermal distribution of objects, so they have advantages in dark nights or low-light environments. Visible light images are more suitable for bright environments and scenes where color information is important. Infrared waves can penetrate some media, such as fog and smoke, while visible light waves have poor penetration. Infrared images and visible light images summarize different properties of the scene. Infrared and visible images of the same scene look similar but not the same. The existence of these objective problems exacerbates the difficulty of achieving infrared image generation. Although a lot of research has been done on long-wave infrared image generation, the quality of the generated long-wave infrared images is average due to the unsatisfactory results of existing algorithms.

[0003] The Chinese patent publication number is "CN111814875B", and the name is "Ship sample expansion method in infrared images based on style generation adversarial network". This method first constructs a visible light-infrared image dataset, then constructs a generator network and a discriminator network and sets parameters, then constructs a style generation adversarial network, and finally uses the trained generator network to output infrared image samples to complete infrared image sample expansion. The infrared image obtained by this method has poor quality, does not take into account temperature information, does not meet the visual effect of the human eye, and the implementation process is complex and inefficient. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the deficiencies in the prior art, the present invention provides a method, system and device for generating long-wave infrared images, which solve the problem of poor quality of generated long-wave infrared images mentioned in the above background technology.

[0006] (II) Technical solution

[0007] In order to achieve the above-mentioned purpose, the present invention specifically adopts the following technical solutions:

[0008] A method for generating a long-wave infrared image, comprising the following specific steps:

[0009] S1. Build a network model: Build a generative adversarial network including a generator and a discriminator;

[0010] S2. Prepare a data set: pre-train the generative adversarial network based on the first data set of aligned infrared images;

[0011] S3. Select the minimization loss function: By minimizing the set loss function, the generated infrared image is closer to the real infrared image. At the same time, each parameter is updated through reverse gradient propagation, allowing the model to continue to iterate;

[0012] S4, training network model: input the data set prepared in S2 into the network model built in S1 for training;

[0013] S5, fine-tuning the model: retraining and fine-tuning the network model using the second dataset of aligned infrared images to obtain the final model;

[0014] S6. Save the model: solidify the parameters of the final model and save the model;

[0015] Furthermore, in S1, the generator includes a cross-feature application module, and the discriminator uses a multi-scale discriminator to better distinguish visible light and infrared images by utilizing multi-scale information;

[0016] Furthermore, in S2, the first dataset of aligned infrared images is the KAIST dataset. We scale the original dataset to a size of 256×256, and perform image stitching on the visible light image and the long-wave infrared image. The size of the stitched image is 256×512 for image training.

[0017] Furthermore, in S3, the generator uses adversarial loss, L1 loss, temperature loss and perceptual loss to better bundle the generated image;

[0018] Furthermore, in S4, the data set processed in S2 is used for training, and the network is converged by the set loss function to obtain a generation model;

[0019] Furthermore, in S5, the second infrared image data set is aligned to a FLIR data set, and the model is fine-tuned after the data set is preprocessed.

[0020] A long-wave infrared image generation system, the system comprising:

[0021] Image acquisition module: used to acquire data for training; the training data includes input image information and output image information;

[0022] Image processing module: used to preprocess each image in the training data; the preprocessing includes geometric correction, data standardization and image enhancement;

[0023] Feature extraction module: used to perform multi-scale feature extraction on the preprocessed image, obtain a feature map set for each image, and extract higher image texture details;

[0024] Model training module: used for supervised model training of preprocessed images, and finally obtains the long-wave infrared image generation model;

[0025] Image generation module: used for inputting the visible light image into the long-wave infrared image generation model to perform a long-wave infrared image generation operation;

[0026] Storage medium: used to store long-wave infrared image generation system.

[0027] A long-wave infrared image generating device, comprising:

[0028] Sample acquisition unit: used to acquire a training sample set, wherein the training samples include visible light images and long-wave infrared images;

[0029] Generative adversarial network acquisition unit: acquires a pre-established generative adversarial network, uses the pre-established generative network as a training reference network, learns the relationship between visible light images and long-wave infrared images, and uses the pre-established discriminant network to determine whether the input image is a generated long-wave infrared image or a real long-wave infrared image;

[0030] Training unit: using deep learning methods, the pre-established generative adversarial network is trained based on the training sample set, the first data set of aligned infrared images is used for pre-training, and then the second data set of aligned infrared images is used for fine-tuning, and the trained generative network is determined as a long-wave infrared image generation model.

[0031] (III) Beneficial effects

[0032] Compared with the prior art, the present invention provides a method, system and device for generating long-wave infrared images, which have the following beneficial effects:

[0033] 1. The present invention uses the method of generative adversarial network and the game between the generator and the discriminator to perform deep feature extraction on the visible light image, generate the long-wave infrared image through the visible light image, improve the authenticity of the generated image, and make the generated long-wave infrared image more consistent with the visual characteristics of the human eye.

[0034] 2. The present invention connects the low-level features and high-level features of the feature map by applying cross-feature application module 1, cross-feature application module 2, cross-feature application module 3, cross-feature application module 4 and cross-feature application module 5, thereby reducing the loss of feature information in the feature map, enhancing the network feature extraction capability, and generating higher quality long-wave infrared images.

[0035] 3. The present invention proposes a composite loss function composed of generation loss, adversarial loss, L1 loss, temperature loss and perception loss to optimize the quality of generated long-wave infrared images from two aspects: high-frequency information and visual perception. The temperature loss designed by the present invention can learn the thermal information of the long-wave infrared image, making the generated long-wave infrared image more realistic.

[0036] 4. The present invention uses a multi-scale discriminator network to discriminate between generated long-wave infrared images and real long-wave infrared images. The discriminator structure has a receptive field size of 70×70, which is equivalent to discriminating an image block of 70×70 in the image, thereby ensuring the local features of the image and making the generated long-wave infrared image of better quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A schematic diagram of a method for generating a long-wave infrared image according to the present invention;

[0038] Figure 2 A schematic diagram of the network structure of a method for generating a long-wave infrared image according to the present invention;

[0039] Figure 3 A schematic diagram of the generator structure of the generative adversarial network of the present invention;

[0040] Figure 4 A schematic diagram of the discriminator structure of the generative adversarial network of the present invention;

[0041] Figure 5 It is a structural diagram of a feature cross-application module in the generator structure of the present invention;

[0042] Figure 6 Schematic diagram of the convolutional layer structure using the L-type activation function in the present invention;

[0043] Figure 7 Schematic diagram of the convolutional layer structure using the P-type activation function in the present invention;

[0044] Figure 8 Schematic diagram of the convolutional layer structure using the T-type activation function in the present invention;

[0045] Fig. 9 Schematic diagram of the convolutional layer structure using the S-type activation function in the present invention;

[0046] Fig.10 It is a schematic diagram of the main modules of the long-wave infrared image generation system of the present invention;

[0047] Fig.11 It is a schematic diagram of the main modules of the long-wave infrared image generating device of the present invention;

[0048] Fig.12The figure is a schematic diagram of the internal structure of an electronic device for realizing the long-wave infrared image generation method of the present invention. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] Example 1

[0051] like Figure 1 As shown, a method for generating a long-wave infrared image provided by an embodiment of the present invention includes the following steps:

[0052] S1. Constructing the network model: The whole generative adversarial network includes a generator structure and a discriminator structure, wherein the generator consists of seven convolution blocks, seven transposed convolution blocks, and five feature cross-application modules, and each convolution block consists of a convolution layer, a normalization layer, and a loss function. Convolution blocks 1, 2, 3, and 4 perform primary feature extraction on the input image, and convolution blocks 5, 6, and 7 extract and reorganize high-level features in the image, remove redundant information, combine and retain useful information, and down-sampling operations extract deep semantic information from the image. Transposed convolution blocks 1, 2, 3, 4, 5, 6, and 7 upsample feature maps to restore the original image size.

[0053] S2. Prepare the data set: The first data set of aligned infrared images uses the KAIST data set, including 10,000 visible light images and aligned long-wave infrared images. The second data set of aligned infrared images is the FLIR data set, including 4,000 visible light images and aligned long-wave infrared images. After preprocessing the data set, input it into the training network;

[0054] S3. Select the minimization loss function: By minimizing the set loss function, the generated infrared image is closer to the real infrared image. At the same time, each parameter is updated through reverse gradient propagation, allowing the model to continue to iterate;

[0055] S4. Training network model: The entire generative adversarial network is first trained with the first dataset of aligned infrared images. During the training process, the first dataset of aligned infrared images is used, the KAIST image size is adjusted to 256×256, and supervised training is performed on the images in the dataset.

[0056] During the training process, choose to minimize the loss function value and the optimal evaluation index. Specifically, by minimizing the loss function between the network output image and the real image, until the number of training times reaches the set threshold or the value of the loss function reaches the set range, the model parameters are considered to have been pre-trained and the model parameters are saved. At the same time, the optimal evaluation index is selected to measure the accuracy of the algorithm and evaluate the performance of the system; during the training process, the loss function uses a composite loss function. The generator uses adversarial loss, L1 loss, perceptual loss and temperature loss, and the discriminator uses adversarial loss; the choice of loss function affects the quality of the model, can truly reflect the difference between the predicted value and the true value, and can correctly feedback the quality of the model; the peak signal-to-noise ratio and structural similarity are suitable evaluation indicators, which can effectively evaluate the quality of the long-wave infrared image generation results and the degree of image distortion, and measure the role of the generation network;

[0057] S5. Fine-tune the model: Use the second dataset of aligned infrared images to train and fine-tune the model to obtain stable and usable model parameters, and further improve the model's ability to generate long-wave infrared images. Ultimately, the model is more effective in generating long-wave infrared images; use the FLIR dataset in the process of fine-tuning the model parameters;

[0058] S6. Save the model: solidify the final model parameters. Directly input the visible light image into the generation network to obtain the final generated long-wave infrared image.

[0059] like Fig.10 As shown, the present invention also provides a long-wave infrared image generation system, the system comprising:

[0060] An image acquisition module, used to acquire data for training; the training data includes input image information and output image information;

[0061] An image processing module, used for preprocessing each image in the training data; the preprocessing includes geometric correction, data standardization and image enhancement;

[0062] The feature extraction module is used to perform multi-scale feature extraction on the preprocessed image to obtain a set of feature maps for each image, and to extract higher image texture details;

[0063] The model training module is used to perform supervised model training on the preprocessed images and finally obtain the long-wave infrared image generation model;

[0064] An image generation module, used for inputting a visible light image into the long-wave infrared image generation model to perform a long-wave infrared image generation operation;

[0065] A storage medium for storing a long-wave infrared image generation system;

[0066] like Fig.11 As shown, the present invention also provides a long-wave infrared image generation device, which includes: a sample acquisition unit, a generative adversarial network acquisition unit, and a training unit.

[0067] like Fig.12 As shown, the present invention also provides an electronic device for generating long-wave infrared images, the device includes a central processing unit (CPU), which can perform various appropriate actions and processes according to the program in the read-only memory (ROM) or the program loaded from the storage part to the random access memory (RAM). In the RAM, various programs and data required for system operation are also stored. The CPU, ROM and RAM are connected to each other through a bus. The input part, the output part, the storage part, and the communication part are also connected to the bus.

[0068] Example 2

[0069] A method for generating a long-wave infrared image proposed by the present invention is shown in Example 1 of S1 as follows: Figure 2 As shown, the generator and discriminator network model structures are as follows Figure 3 and Figure 4 shown.

[0070] The generator in S1 consists of seven convolution blocks, seven transposed convolution blocks, and five feature cross-application modules, including convolution block 1, convolution block 2, convolution block 3, convolution block 4, convolution block 5, convolution block 6, and convolution block 7. The convolution kernel size is 4×4, the step size is 2, and the L-type activation function is used. The specific composition of the convolution layer is as follows: Figure 6 As shown. Transposed convolution block 1, transposed convolution block 2, transposed convolution block 3, transposed convolution block 4, transposed convolution block 5, transposed convolution block 6, the convolution kernel size is 4×4, the step size is 2, and the P-type activation function is used. The specific composition of the convolution layer is as follows Figure 7 As shown in the figure, the convolution kernel size of the transposed convolution block 7 is 4×4, the step size is 2, and the T-type activation function is used. The specific composition of the convolution layer is as follows Figure 8 shown.

[0071] The cross-feature application module in S1 is as follows Figure 5 As shown in the figure, the feature map is divided into four groups along the channel dimension, and the low-level features and high-level features of each group are connected together to obtain four groups of fused features. For these four groups of fused features, 1×1, 3×3, 5×5 and 7×7 convolution kernels are used for feature extraction respectively, and then the valuable information is retained through the channel attention module. After that, the extracted features are connected in the channel dimension, and the feature information is integrated through a 1×1 convolution kernel. This can reduce the amount of parameters and better extract feature information, which is conducive to generating long-wave infrared images with better texture details.

[0072] Each discriminator of the multi-scale discriminator in S1 is composed of five convolution blocks. The convolution kernel size of convolution block 1, convolution block 2, convolution block 3, and convolution block 4 is 4×4, the step size is 2, and the L-type activation function is used. The convolution kernel size of convolution block 5 is 4×4, the step size is 1, and the S-type activation function is used. The specific composition of the convolution layer is as follows: Fig. 9 As shown in the figure. Input a 256×256 infrared image, output a 30×30 feature response, and the receptive field size is 70×70, which is equivalent to discriminating a 70×70 image block in the image, thereby ensuring the local features of the image. The final output of the discriminator is obtained by averaging all responses, and the final output result is between 0 and 1. The closer to 1, the more it resembles a real infrared image.

[0073] In general, the process of generating long-wave infrared image samples is to input a visible light image, extract the low-frequency and high-frequency information in the visible light image through 7 downsampling operations, restore the feature map size through 7 upsampling operations, and reconstruct the infrared image. Finally, the generated infrared image and visible light image are sent to the discriminator at the same time to determine whether the generated image is true or false.

[0074] In order to ensure the robustness of the network, retain more structural information, and fully extract image features, the present invention uses four activation functions, namely P-type function, T-type function, L-type function, and S-type function. The activation functions in the discriminator are L-type function and S-type function, and the last layer of the generator is T-type function. The definitions of P-type function, T-type function, L-type function, and S-type function are as follows:

[0075] P-type function:

[0076]

[0077] T-type function:

[0078]

[0079] L-type function:

[0080]

[0081] Sigmoid function:

[0082]

[0083] The KAIST dataset in S2 contains various daily traffic scenes in schools, streets and villages, which are generally used for pedestrian detection. The color camera is PointGrey Flea3, the color image size is 640×480, the infrared camera is FLIR-A35, the infrared image size is 320×256, and after camera calibration, the image pair of 640×512 size is finally obtained. In the experiment, images taken during the day in the KAIST dataset were selected and scaled to 256×256 size to adapt to the network input. The adversarial training of the KAIST dataset can initialize a set of parameters to speed up the training process of the subsequent network. The dataset preprocessing operation in S2 uses the opencv function to scale the image to 256×256 size, and splice the visible light image and the long-wave infrared image. The size of the spliced ​​image is 256×512. The dataset involved has undergone image cleaning work, and finally achieves better long-wave infrared image generation results.

[0084] The FILR dataset in S5 contains more than 14,000 pairs of thermal imaging images and visible light images, covering different scenes, objects and distances. The infrared image size is 640×512, the color camera uses an IMX250 camera and a 52.8-degree HFV Edmund optical lens, and the visible light image size is 1800×1600. We use 4000 image pairs for training in FLIR and 500 pairs for testing.

[0085] The output of the network in S3 and the label calculate the loss function, and a better generation effect is achieved by minimizing the loss function. The loss function uses a composite loss function, the generator uses generation loss, L1 loss, temperature loss and perceptual loss, and the discriminator uses adversarial loss.

[0086] Generative adversarial loss is a loss function used to train generative adversarial networks. Specifically, generative adversarial loss consists of two parts: generator loss and discriminator loss. Generator loss is the difference between the data output by the generator and the real data, while discriminator loss is the ability of the discriminator to correctly classify the real data and the data generated by the generator. By minimizing these two losses, GAN can learn how to generate more realistic data. Generative adversarial loss is defined as:

[0087] L(G,D)=E v,i [logD(v,i)]+E v [log(1-D(v,G(v)))]

[0088] For the visible light image source domain V and the infrared image target domain I, the generator G converts the visible light image v into the infrared image G(v), and the discriminator D not only judges G(v) but also the real infrared image i. The goal of the generator is to maximize D(G(v)), and the goal of the discriminator is to minimize D(G(v)), and the two are in competition with each other.

[0089] The L1 norm loss function, also known as the minimum absolute deviation, is used to help the generator produce images with more structure and details. It is used to calculate the difference between the generator's output and the target image, encouraging the generated image to be more similar to the target image at the pixel level.

[0090] The L1 norm loss function is defined as:

[0091] L L1 (G) = E v,i [||iG(v)||1]

[0092] The Pix2pix loss function is:

[0093] L pix2pix (G,D)=E v,i [logD(v,i)]+E v [log(1-D(v,G(v)))]+λL L1 (G)

[0094] There is a problem of missing temperature information when converting visible light images into long-wave infrared images. To solve this problem, we use temperature loss, which is defined as:

[0095] L s (G) = E v,i [||iG(S(v))||1]

[0096] Where S = R × 0.299 + G × 0.587 + B × 0.114.

[0097] Using adversarial loss can easily produce distorted textures. To alleviate this problem, a perceptual loss based on the ReLU activation of the VGG-16 network is used. The perceptual loss is conducive to keeping the texture in the high-level space and is defined as:

[0098]

[0099] where Φ(·) k represents the feature representation of the kth maximum pooling layer in the VGG-16 network. C, W, and H represent the number of channels, height, and width of the image, respectively. k H k W krepresents the size of these feature representations, k represents the k-th layer, i and j represent the indices of pixels, and we use perceptual loss to recover high-level content.

[0100] Therefore, the final loss function is:

[0101] L(G,D)=L pix2pix +λ1L s +λ2L preceptual

[0102] Among them, λ, λ1, and λ2 are hyperparameters.

[0103] The appropriate evaluation indicators in S4 are peak signal-to-noise ratio and structural similarity. Peak signal-to-noise ratio is based on the error between corresponding pixels, that is, error-sensitive image quality evaluation; structural similarity measures image similarity from three aspects: brightness, contrast, and structure. It is an indicator used to measure the similarity between two images. Peak signal-to-noise ratio and structural similarity are defined as follows:

[0104]

[0105]

[0106] where μ x and μ y Represent the mean and variance of images x and y respectively, and Represents the standard deviation of images x and y, σ xy Represents the covariance of image x and y, C1 and C2 are constants. Set the number of training times to 200, set the learning rate of the first 100 training processes to 0.0002, and gradually decrease the learning rate from 0.0002 to 0 in the next 100 training processes; the number of images input to the network each time is affected by the performance of the computer graphics processor. Selecting the number of images input to the network according to the performance of the image processor can make the network training more stable and the training results better, and can ensure the fast fitting of the network. The Adam optimizer is selected as the network parameter optimizer. It is known for its simple implementation, efficient calculation, low memory requirements, and parameter updates that are not affected by gradient scaling. When the discriminator's ability to identify fake images and the generator's ability to deceive the discriminator are balanced, the network can be considered to have basically completed training.

[0107] In S5, the model is trained and fine-tuned using the aligned infrared image second dataset, and the FLIR dataset is used in the process of fine-tuning the model parameters. The FLIR dataset consists of long-wave infrared images and visible light images. For the FLIR dataset, 4000 images are used for fine-tuning and 500 images are used for testing.

[0108] After the network training is completed in S6, all parameters in the network need to be saved, and then the visible light data set used for our test is input into the network to obtain a high-quality generated long-wave infrared image. The network has no requirements for the input image size, and any size is acceptable.

[0109] like Fig.10 As shown, the present invention also provides a long-wave infrared image generation system, which mainly includes an image acquisition module, an image processing module, a feature extraction module, a model training module, an image generation module and a storage medium;

[0110] An image acquisition module, used to acquire data for training; the training data includes input image information and output image information;

[0111] An image processing module, used for preprocessing each image in the training data; the preprocessing includes geometric correction, data standardization and image enhancement;

[0112] The feature extraction module is used to perform multi-scale feature extraction on the preprocessed image to obtain a set of feature maps for each image, and to extract higher image texture details;

[0113] The model training module is used to perform supervised model training on the preprocessed images and finally obtain the long-wave infrared image generation model;

[0114] An image generation module, used for inputting a visible light image into the long-wave infrared image generation model to perform a long-wave infrared image generation operation;

[0115] A storage medium for storing a long-wave infrared image generation system;

[0116] Furthermore, the image processing module is also used for data enhancement of the training data set, and commonly used methods include: image cropping, image flipping and image translation;

[0117] Furthermore, the model training module is also used to resize each image from an arbitrary size of the dataset to a fixed size of 256×256; both the generator and the discriminator are trained with a strategy of 200 epochs and a batch size of 8; initially, the learning rate is set to 0.0002 in the first 100 epochs, and the learning rate is linearly decreased to 0 in the next 100 epochs; we use the Adam optimizer with β1=0.5, β2=0.999; the discriminator and the generator are trained alternately until the model converges; further, the model training module is also used to resize each image from an arbitrary size of the dataset to a fixed size.

[0118] Furthermore, the image generation module is also used to obtain a visible light image, input the visible light image information into a long-wave infrared image generation model, and generate a long-wave infrared image using a forward propagation algorithm.

[0119] like Fig.11 As shown, the present invention also provides a long-wave infrared image generation device, which mainly includes a sample acquisition unit, a generative adversarial network acquisition unit and a training unit:

[0120] A sample acquisition unit, used to acquire a training sample set, wherein the training samples include visible light images and long-wave infrared images;

[0121] A generative adversarial network acquisition unit acquires a pre-established generative adversarial network, uses the pre-established generative network as a training reference network, learns the relationship between the visible light image and the long-wave infrared image, and uses the pre-established discriminant network to determine whether the input image is a generated long-wave infrared image or a real long-wave infrared image;

[0122] The training unit uses a deep learning method to train a pre-established generative adversarial network based on a training sample set, uses a first data set of aligned infrared images for pre-training, and then uses a second data set of aligned infrared images for fine-tuning, and determines the trained generative network as a long-wave infrared image generation model.

[0123] like Fig.12 As shown, the present invention also provides a schematic diagram of the internal structure of an electronic device for a long-wave infrared image generation method. The present invention also provides an electronic device for long-wave infrared image generation, the device comprising: a central processing unit (CPU), a read-only memory (ROM), a random access memory (RAM), an input part, an output part, a storage part and a communication part.

[0124] Further, the memory may store a program, and when the program stored in the memory is executed by the processor, the processor and the communication interface are used to perform each step of the training method for the long-wave infrared image generation network of the embodiment of the present invention;

[0125] The processor further adopts a CPU, which is used to execute relevant programs to realize the functions that the units in the long-wave infrared generation training system of the present invention need to perform, or to perform the long-wave infrared generation training method of the present invention;

[0126] Further, the communication interface uses, but is not limited to, a transceiver system such as a transceiver to achieve communication between the system and other devices or communication networks; for example, the image to be processed or the initial feature map of the image to be processed can be obtained through the communication interface;

[0127] Further, the bus may include a pathway for transferring information between various components of the system (e.g., CPU, memory portion, communication interface);

[0128] Python language is further used to write computer program codes for performing the operations of the present application. The program codes can be executed completely on the user's computer, partially on the computer, or on a remote computer or server.

[0129] The present invention constructs a method, system and device for generating long-wave infrared images, which are used to convert visible light images into long-wave infrared images. The generation method does not have manual generation rules. Under the same conditions, the relevant indicators of the image are obtained by calculation with the existing method, which further verifies the feasibility and superiority of the method. The comparison of the relevant indicators of the prior art and the method proposed in the present invention is shown in Table 1:

[0130] Table 1 Comparison of relevant indicators of the prior art and the method proposed in the present invention

[0131]

[0132] It can be seen from the table that the method proposed in the present invention has higher peak signal-to-noise ratio and structural similarity than the existing method. These indicators also further illustrate that the method proposed in the present invention has higher quality of generating infrared images.

[0133] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for generating a long-wave infrared image, characterized in that: The specific steps are: S1. Build a network model: Build a generative adversarial network including a generator and a discriminator; The generator in S1 includes a cross-feature application module, a downsampling module, an attention module, and an upsampling module; The cross-feature application module includes: cross-feature application module one, cross-feature application module two, cross-feature application module three, cross-feature application module four and cross-feature application module five; The cross-feature application module divides the feature map into four groups along the channel dimension, connects the low-level features and high-level features of each group together, and obtains four groups of fused features. For these four groups of fused features, 1×1, 3×3, 5×5 and 7×7 convolution kernels are used for feature extraction, and then the valuable information is retained by the channel attention module. After that, the extracted features are connected in the channel dimension, and the feature information is integrated by a 1×1 convolution kernel, which can reduce the number of parameters and better extract feature information, which is conducive to generating long-wave infrared images with better texture details. The downsampling module is used to continuously reduce the size of the feature map and extract deep semantic information of the image; The attention module is used to retain valuable information in the channel dimension; The upsampling module is used to gradually restore the size of the feature map; The discriminator in S1 is a multi-scale discriminator, which can more accurately judge the authenticity of the image; S2. Prepare a data set: pre-train the generative adversarial network based on the first data set of aligned infrared images; The first data set of aligned infrared images in S2 is the KAIST data set; S3. Select the minimization loss function: By minimizing the set loss function, the generated infrared image is closer to the real infrared image. At the same time, each parameter is updated through reverse gradient propagation, allowing the model to continue to iterate; The loss function in S3 uses a composite loss function. The loss functions used by the generator include generation loss, L1 loss, temperature loss and perception loss; the discriminator uses adversarial loss; The temperature loss is defined as: ； in, represents a visible light image, represents the long-wave infrared image, , represents the L1 norm; S4, training the network model: inputting the first data set of aligned infrared images into the constructed network model for training; S5, fine-tuning the model: retraining and fine-tuning the network model using the second dataset of aligned infrared images to obtain the final model; The second infrared image data set aligned in S5 is a FLIR data set; S6. Save the model: solidify the parameters of the final model and save the model.

2. A long-wave infrared image generation system based on a generative adversarial network, characterized in that: The system is used to execute a method for generating a long-wave infrared image according to claim 1, comprising: Image acquisition module: used to acquire training data for training; the training data includes input image information and output image information; Image processing module: used to preprocess each image in the training data; the preprocessing includes geometric correction, data standardization and image enhancement; Feature extraction module: used to perform multi-scale feature extraction on the preprocessed image, obtain a feature map set for each image, and extract higher image texture details; Model training module: used for supervised model training of preprocessed images, and finally obtains the long-wave infrared image generation model; Image generation module: used for inputting the visible light image into the long-wave infrared image generation model to perform a long-wave infrared image generation operation; Storage medium: used to store long-wave infrared image generation system.

3. A long-wave infrared image generating device, characterized in that: The device is used to execute a method for generating a long-wave infrared image according to claim 1, comprising: Sample acquisition unit: used to acquire a training sample set, wherein the training samples include visible light images and long-wave infrared images; Generative adversarial network acquisition unit: acquires a pre-established generative adversarial network, uses the pre-established generative network as a training reference network, learns the relationship between visible light images and long-wave infrared images, and uses the pre-established discriminant network to determine whether the input image is a generated long-wave infrared image or a real long-wave infrared image; Training unit: using deep learning methods, the pre-established generative adversarial network is trained based on the training sample set, the first data set of aligned infrared images is used for pre-training, and then the second data set of aligned infrared images is used for fine-tuning, and the trained generative network is determined as a long-wave infrared image generation model.

Images