An infrared image generation method and system based on a physical prior constraint generative adversarial network
By introducing generative adversarial networks with prior physical knowledge, the problem of insufficient physical realism and radiation characteristic representation in infrared image generation methods is solved, and infrared image generation with high physical consistency is achieved.
Patent Information
- Application Number
- CN202610878175.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-17
- Publication Date
- 2026-07-24
Smart Images
Figure CN122453975A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of infrared image generation, and specifically to an infrared image generation method and system based on physical prior constraint generative adversarial networks. Background Technology
[0002] Infrared detectors acquire infrared images through a series of imaging processes, including infrared radiation reception, electrical signal conversion, and amplification. They offer advantages such as all-weather operation, independence from lighting conditions, and long detection range. Due to their unique imaging mechanism, infrared images provide crucial data support for research in target detection and image segmentation. However, obtaining high-resolution infrared images through real-world shooting is costly, requiring significant time and resources, as it relies on specialized infrared detection equipment. Therefore, utilizing computer computing resources to obtain high-quality infrared images has significant application value for expanding infrared image data and improving the performance of infrared detection equipment.
[0003] Infrared image generation methods are generally divided into two categories: imaging simulation based on physical models and image generation based on deep learning. Imaging simulation based on physical models has strong theoretical basis and physical consistency, but it involves large computational loads and complex parameters, failing to meet the diverse and real-time requirements of infrared image generation. With the development of deep learning in the field of visible light image generation, deep learning-based infrared image generation methods have gained considerable attention in recent years. Image generation methods represented by Generative Adversarial Networks (GANs), Transformers, and diffusion models have driven the rapid development of the infrared image generation field.
[0004] Deep learning-based infrared image generation methods, to some extent, circumvent the limitations of traditional methods, such as computational complexity and limited flexibility. They also overcome the difficulties and high costs associated with acquiring real infrared data, and improve infrared image generation capabilities in areas such as texture detail restoration and structural consistency. However, existing image training data cannot comprehensively represent the characteristics of infrared radiation, and deep learning methods suffer from a lack of interpretability, thus compromising the physical realism of the generated results. Therefore, there is an urgent need to propose an infrared image generation method that enhances the ability of current deep learning-based infrared image generation methods to represent infrared radiation characteristics and achieve highly physically realistic infrared image generation. Summary of the Invention
[0005] This invention addresses the shortcomings of existing infrared image generation methods, such as insufficient physical realism and inadequate representation of infrared radiation characteristics. It proposes an infrared image generation method based on a physical prior constraint generative adversarial network (GAN). By introducing physical prior knowledge into the GAN, the performance of the infrared image generation model is improved at the physical characteristic representation level. A physical prior knowledge learning network is constructed based on the U-Net network to predict the infrared radiation parameter map in infrared images. Through the pre-trained physical prior knowledge learning network, the GAN, composed of an attention-based U-Net generator and a multi-scale PatchGAN discriminator, is constrained, thereby enhancing the physical characteristic representation capability of the infrared image generation model and resulting in an infrared image generation model with high physical consistency.
[0006] To achieve the above objectives, the present invention provides the following technical solution: Option 1: This invention designs a physical prior knowledge learning network. Based on the infrared radiation intensity calculation model received by the detector, it trains a U-Net network to learn infrared radiation-related features from real infrared images and predicts the corresponding infrared radiation parameter map.
[0007] Therefore, the present invention provides a method for constructing a physical prior knowledge learning network, the method comprising the following steps: Step 1: Construct an infrared radiation intensity calculation model, which links infrared radiation intensity with the physical parameters of the target, environment, and atmosphere; Step 2: Preprocess the real infrared image; Step 3: Build the U-Net network. Based on the preprocessed real infrared image and the infrared radiation intensity calculation model, complete the U-Net network training through parameter estimation, image reconstruction and error loss constraints. Step 4: Save the optimal model parameters after training to obtain the physical prior knowledge learning network.
[0008] Furthermore, in step 1 of this invention, the infrared radiation intensity calculation model is constructed in conjunction with the infrared imaging mechanism, and the physical parameters in the model include the target body temperature, target emissivity, atmospheric transmittance, environmental reflected radiation, and atmospheric path radiation.
[0009] Furthermore, step 3 of the present invention specifically includes: Construct a U-Net network based on an encoder and decoder structure; During training, preprocessed real infrared images are input into the U-Net network. The U-Net network estimates the parameters of the infrared radiation intensity calculation model pixel by pixel and outputs the predicted images of each parameter through its own output channel. The predicted images of each parameter are calculated pixel by pixel according to the infrared radiation intensity calculation model to obtain the reconstructed infrared image. The mean square error loss is calculated between the original infrared image and the reconstructed infrared image to constrain the training of U-Net.
[0010] Furthermore, the encoder of the present invention is composed of multi-level feature extraction modules, each of which sequentially includes a convolution operation unit, a normalization unit, a nonlinear activation unit, and a downsampling unit; The decoder includes an upsampling module, a convolution module, and skip connections.
[0011] Option 2: This invention also combines the trained physical prior knowledge learning network with GAN to build a generative adversarial network based on physical prior constraints, thereby realizing infrared image generation based on physical prior constraints.
[0012] Therefore, this invention provides an infrared image generation method based on a physical prior constraint generative adversarial network, comprising the following steps: Step 1: Build an attention U-Net generator with physical prior constraints. Input the visible light image into the physical prior knowledge learning network and extract the potential infrared radiation features as physical prior features. Then, concatenate the physical prior features with the visible light image in the channel dimension and input them into the attention U-Net backbone network. The backbone network outputs the infrared image and predicts multiple infrared-related physical parameter maps. Step 2: Build a multi-scale PatchGAN discriminator. Stitch the visible light image with the real infrared image and the generated infrared image to obtain image pairs. Input the two sets of image pairs into the PatchGAN discriminator at each scale to obtain the real and fake discrimination results of the local area of the image. Step 3: Training the Infrared Image Generative Adversarial Network Based on Physical Prior Constraints. The infrared image generative adversarial network based on physical prior constraints, consisting of an attention-based U-Net generator with physical prior constraints and a multi-scale PatchGAN discriminator, is trained. The optimizer and learning rate parameters are configured, and the generator and discriminator are jointly constrained by physical consistency loss, pixel-level loss, and feature matching loss to complete the model training and achieve infrared image generation.
[0013] Furthermore, this invention extracts potential infrared radiation features in visible light images that are related to the corresponding infrared images as physical prior features through a parameter-frozen physical prior knowledge learning network.
[0014] Furthermore, each PatchGAN discriminator in this invention consists of multiple convolutional layers, normalization layers, and activation functions. Each PatchGAN discriminator outputs a discrimination map through a local receptive field, and each element in the discrimination map corresponds to the true or false score of a local region in the input image.
[0015] Based on the same inventive concept, the infrared image generation method based on physical prior constraints generative adversarial networks described in this invention can be entirely implemented using computer software. Therefore, correspondingly, this invention also provides an infrared image generation system based on physical prior constraints generative adversarial networks.
[0016] Furthermore, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs either the method for constructing a physical prior knowledge learning network described above or the method for generating infrared images based on a physical prior constraint generative adversarial network described above.
[0017] Furthermore, the present invention also provides a computer device, which includes a memory and a processor. The memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes either the method for constructing a physical prior knowledge learning network described above or the method for generating infrared images based on a physical prior constraint generative adversarial network described above.
[0018] The beneficial effects of this invention are as follows: (1) This invention proposes a physical prior knowledge learning network to learn prior representations related to infrared radiation characteristics from infrared images and to predict infrared radiation-related parameters through a deep learning network. Based on the infrared imaging mechanism, this network introduces complete learnable parameters into the constructed infrared radiation intensity calculation model, which can provide prior information with physical constraints for the subsequent infrared image generation process, thereby enhancing the ability of the generation model to express infrared radiation characteristics.
[0019] (2) Based on the proposed physical prior knowledge learning network, this invention designs a generative adversarial network based on physical prior constraints. By introducing physical prior features into the image generation process, the generator's ability to learn the overall distribution of infrared radiation and its ability to represent the characteristics of infrared radiation are enhanced. This method breaks through the limitation of traditional generators that only rely on shallow texture mapping and improves the physical consistency of infrared image generation methods. Attached Figure Description
[0020] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0021] Figure 1 This is a block diagram of the physical prior knowledge learning network structure of the present invention; Figure 2 This is a block diagram of the generative adversarial network structure based on physical prior constraints of the present invention. Figure 3 This is a flowchart of the model training process of the present invention. Detailed Implementation
[0022] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. The following examples will help those skilled in the art to further understand the present invention, but do not limit the present invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention.
[0023] Example 1: This example provides a physics prior knowledge learning network, the structure of which is as follows. Figure 1 As shown, this learning network is based on the infrared radiation intensity calculation model received by the detector. It learns infrared radiation-related features from real infrared images by training the U-Net network and predicts the corresponding infrared radiation parameter map.
[0024] The method for constructing a physics prior knowledge learning network is as follows: Step 1: Construct an infrared radiation intensity calculation model, which links infrared radiation intensity with the physical parameters of the target, environment, and atmosphere; Step 2: Preprocess the real infrared image; Step 3: Build the U-Net network. Based on the preprocessed real infrared image and the infrared radiation intensity calculation model, complete the U-Net network training through parameter estimation, image reconstruction and error loss constraints. Step 4: Save the optimal model parameters after training to obtain the physical prior knowledge learning network.
[0025] Furthermore, in a specific embodiment, the above construction method will be described in detail; Construct an infrared radiation intensity calculation model.
[0026] The infrared radiation received by an infrared detector is influenced by a variety of factors, including the target's own temperature, the emissivity of its surface material, ambient reflected radiation, atmospheric transmittance, and atmospheric path radiation. For a practical infrared imaging system, the intensity of the infrared radiation received by the detector can be expressed as: (1) in, I This refers to the infrared radiation intensity value received by the infrared detector. Atmospheric transmittance, For the target emission rate, T The target body temperature, The target body radiation is calculated using Planck's law. For environmental reflected radiation, This refers to atmospheric path radiation.
[0027] Construction and training of U-Net network.
[0028] First, the input real infrared images are preprocessed, including uniform size cropping and normalization, to improve the consistency of sample distribution.
[0029] Subsequently, a U-Net network based on an encoder and decoder structure is constructed. The encoder part consists of a multi-level feature extraction module, each level including convolution operation, normalization processing, non-linear activation function and downsampling module. The decoder part mainly consists of an upsampling module, a convolution module and skip connections.
[0030] Finally, during U-Net training, the processed real infrared image is input, and the parameters in the infrared radiation intensity calculation model (Equation 1) are estimated pixel by pixel. The predicted images of each parameter are then output through the output channel of U-Net. The reconstructed infrared image is obtained by calculating the parameters pixel by pixel according to Equation (1), and the mean squared error loss is calculated with the original infrared image to constrain the training of U-Net, as shown in Equation (2). After training, the optimal model parameters are saved for subsequent network construction and physical consistency loss calculation.
[0031] (2) in, This indicates the reconstruction of the infrared image. Represents a true infrared image. N This indicates the total number of pixels.
[0032] In summary, this embodiment proposes a physical prior knowledge learning network to learn prior representations related to infrared radiation characteristics from infrared images and to predict infrared radiation-related parameters through a deep learning network. Based on the infrared imaging mechanism, this network introduces complete learnable parameters into the constructed infrared radiation intensity calculation model, providing physically constrained prior information for the subsequent infrared image generation process, thereby enhancing the generative model's ability to express infrared radiation characteristics.
[0033] Example 2: This example combines the physics prior knowledge learning network trained in Example 1 with a GAN to build a generative adversarial network based on physics prior constraints (e.g., Figure 2 As shown in the figure, infrared image generation based on physical prior constraints is thus achieved.
[0034] The infrared image generation method based on physical prior constraints and generative adversarial networks includes the following steps: Step 1: Build an attention U-Net generator with physical prior constraints. Input the visible light image into the physical prior knowledge learning network and extract the potential infrared radiation features as physical prior features. Then, concatenate the physical prior features with the visible light image in the channel dimension and input them into the attention U-Net backbone network. The backbone network outputs the infrared image and predicts multiple infrared-related physical parameter maps. Step 2: Build a multi-scale PatchGAN discriminator. Stitch the visible light image with the real infrared image and the generated infrared image to obtain image pairs. Input the two sets of image pairs into the PatchGAN discriminator at each scale to obtain the real and fake discrimination results of the local area of the image. Step 3: Training the Infrared Image Generative Adversarial Network Based on Physical Prior Constraints. The infrared image generative adversarial network based on physical prior constraints, consisting of an attention-based U-Net generator with physical prior constraints and a multi-scale PatchGAN discriminator, is trained. The optimizer and learning rate parameters are configured, and the generator and discriminator are jointly constrained by physical consistency loss, pixel-level loss, and feature matching loss to complete the model training and achieve infrared image generation.
[0035] Furthermore, in a specific embodiment, the above-described infrared image generation method will be described in detail; Step 1: Building a Physically Prior-Constrained Attention U-Net Generator. The physically prior-constrained attention U-Net generator combines a pre-trained physical prior knowledge learning network with an attention U-Net network. First, the visible light image is input into the physical prior knowledge learning module. This module extracts latent infrared radiation features related to the corresponding infrared image from the visible light image as physical prior features through a parameter-frozen physical prior knowledge learning network. Then, the model concatenates the physical prior features with the visible light image along the channel dimension and inputs them together into the backbone network. The generator backbone network adopts an attention U-Net structure, including a downsampling encoder, a bottleneck self-attention module, attention-gated skip connections, and an upsampling decoder. The generator ultimately outputs the generated infrared image and simultaneously predicts physical parameter maps of emissivity, temperature, transmittance, atmospheric radiation, and environmental reflected radiation through a parameter output header.
[0036] Step 2: Construction of a Multi-Scale PatchGAN Discriminator. The discriminator consists of multiple PatchGAN discriminators at different scales, used to determine the local realism of the input image at different resolutions. Each PatchGAN discriminator outputs a discrimination map through its local receptive field, where each element in the discrimination map corresponds to a real / fake score for a local region in the input image. Specifically, the real infrared image and the visible light image are stitched together to form a real image pair, and the generated infrared image and the visible light image are stitched together to form a generated image pair. The two sets of image pairs are downsampled stepwise and used as input to each PatchGAN discriminator. Each PatchGAN discriminator consists of multiple layers of convolution, normalization, and activation functions, capturing local details, structural distribution, and global content, respectively. The discriminator ultimately obtains a real / fake discrimination map of local image patches, used to determine the realism of the generated infrared image.
[0037] Step 3: Training the Infrared Image Generative Adversarial Network Based on Physical Prior Constraints. The infrared image generative adversarial network based on physical prior constraints, consisting of an attention-based U-Net generator with physical prior constraints and a multi-scale PatchGAN discriminator, is trained. The optimizer and initial learning rate are set, and relevant model parameters are configured. Physical consistency loss, pixel-level loss, and feature matching loss are used to jointly constrain the generator and discriminator. The physical consistency loss is calculated from the mean square error between the reconstructed infrared image and the real infrared image during the training of the physical prior knowledge learning network, and the calculation formula is shown in Equation (2). The pixel-level loss uses L1 distance to calculate the pixel-level loss between the generated infrared image and the real infrared image, as shown in Equation (3). The feature matching loss calculates the L1 distance between the features of the intermediate layers of the discriminator at different scales for the real image pair and the generated image pair, as shown in Equation (4).
[0038] (3) (4) in, This indicates the generation of an infrared image. Represents a true infrared image. Represents a visible light image. S Indicates the number of discriminator scales. Indicates the first s In the scale discriminator, the first... l Features of layer output, This indicates the number of elements in the corresponding feature map.
[0039] The loss function for model training consists of three types of losses: (5) in, These are the weights of the loss terms.
[0040] In summary, this invention designs a generative adversarial network based on physical prior knowledge learning network, which enhances the generator's ability to learn the overall distribution of infrared radiation and its ability to represent the characteristics of infrared radiation by introducing physical prior features into the image generation process. This method overcomes the limitation of traditional generators that rely solely on shallow texture mapping, and improves the physical consistency of infrared image generation methods.
[0041] Example 3: Combining Examples 1 and 2 above, this example provides a specific implementation of an infrared image generation method based on physical prior constraints and generative adversarial networks, as follows: Figure 3 As shown, the implementation process is as follows: Construct the image dataset required for model training, validation, and testing. Paired visible light and infrared images are organized into the image dataset needed for the model, with 70% for training, 20% for validation, and 10% for testing. Visible light and infrared images are divided into two parts. To improve the stability of network training and the consistency of data distribution, uniform data preprocessing is performed on the visible light and infrared images before inputting them into the network. First, all images are adjusted to the same spatial size to meet the network input size requirements. Second, the pixel values of the images are normalized to ensure they are distributed within a uniform numerical range, thereby reducing the impact of differences in grayscale ranges between different images on model training. Furthermore, to enhance the model's adaptability to different spatial locations and local structural changes, random cropping and flipping data augmentation operations are performed on the images.
[0042] The training process of the physical prior knowledge learning network in this example mainly includes four parts: input image preprocessing, infrared radiation parameter prediction based on U-Net, infrared image reconstruction, and loss constraints. First, the input infrared image is sized and normalized before being fed into the U-Net network. The U-Net network consists of a four-level encoder, a four-level decoder, and a bottleneck module. Each encoder level includes two sets of convolutional layers and non-linear activation functions, and progressively reduces the spatial resolution of the feature maps through a downsampling module. Each decoder level includes an upsampling module and two sets of convolutional layers and non-linear activation functions. Encoders and decoders at corresponding scales are connected via skip connections, fusing the shallow features extracted during the encoding stage with the upsampled deep features. The bottleneck module, located between the encoder and decoder, consists of two sets of convolutional layers and non-linear activation functions. It is used to further extract high-level abstract features and serves as the core connection for the transition from encoded features to the decoding and reconstruction process.
[0043] During forward propagation, the U-Net network predicts radiation parameter maps related to the infrared imaging process based on a constructed infrared radiation intensity calculation model. These parameters include emissivity, temperature, transmittance, atmospheric path radiation, and environmental reflected radiation. Subsequently, the predicted radiation parameter maps are re-introduced into the infrared radiation intensity calculation model to calculate the reconstructed infrared image. The reconstructed infrared image is compared with the real infrared image at the pixel level, and the mean squared error is calculated as a physical consistency loss function for constraint. The loss function calculation error is then propagated to the U-Net network through backpropagation to update the network parameters, ensuring that the predicted radiation parameter maps better conform to the imaging characteristics of the real infrared image, thereby improving the physical consistency of the prediction results and the accuracy of image reconstruction.
[0044] The training process for the generative adversarial network based on physical prior constraints in this example is as follows: During the generator training phase, the visible light image is first input into the generator's pre-trained physical prior knowledge learning network to obtain the infrared radiation prior related to the infrared image in the visible light image. The visible light image and the learned infrared prior are concatenated along the channel dimension and fed into the Attention U-Net network. This network consists of a four-level encoder, a four-level decoder, and a bottleneck module. Each encoder level includes a pooling layer, two convolutional layers, two non-linear activation functions, and a downsampling module; each decoder level includes a deconvolutional layer and a convolutional layer; an attention gating module is introduced during skip connections, and the self-attention module is used as the bottleneck module. The generator output includes the generated infrared image and the infrared radiation parameter prediction map. Subsequently, the above parameter map is substituted into the infrared radiation intensity calculation model to obtain the reconstructed infrared image. During training, the pixel-level loss between the generated infrared image and the real infrared image, as well as the physical consistency loss between the reconstructed infrared image and the real infrared image, are calculated, and the generator parameters are updated through backpropagation.
[0045] During the discriminator training phase, real image pairs and generated image pairs are first constructed based on paired samples. For real image pairs, the visible light image is concatenated with the corresponding real infrared image along the channel dimension; for generated image pairs, the same visible light image is concatenated with the generated infrared image output by the generator along the channel dimension. These image pairs are then input into a multi-scale PatchGAN discriminator. The discriminator consists of three PatchGAN branches at different scales, processing the input images at both the original and downsampled scales. Each PatchGAN branch mainly consists of four convolutional layers, four normalization layers, four nonlinear activation layers, and a downsampling module, ultimately outputting a local realism discrimination map. During training, the discrimination results of real image pairs are used as real labels, and the discrimination results of generated image pairs are used as fake labels, thereby updating the discriminator parameters. Simultaneously, the generator needs to be updated synchronously to ensure that the generated infrared image obtains a discrimination result close to that of the real infrared image in the discriminator. Furthermore, the feature maps of the intermediate layers of the discriminator are extracted, and the feature matching loss between real samples and generated samples is calculated.
[0046] In the training process of generative adversarial networks based on physical prior constraints, the model's generation performance is evaluated using a validation set after each fixed-interval training round, and changes in various losses and image quality evaluation metrics are continuously monitored. When the validation set loss gradually converges and the image quality metrics tend to stabilize, it indicates that the model has good generalization ability. Based on the validation results, the current optimal model parameters can be saved for subsequent testing and infrared image generation.
[0047] In this example, the infrared image generation quality evaluation metrics used during the verification and testing phases include Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), Learned Perceptual Image Patch Similarity (LPIPS), and Fréchet Inception Distance (FID). Higher PSNR and SSIM values indicate higher generation quality, while lower LPIPS and FID values indicate higher generation quality.
[0048] Example 4: The method for constructing a physical prior knowledge learning network or the infrared image generation method based on a physical prior constraint generative adversarial network described above can all be implemented using computer software. Therefore, this example provides a system for constructing a physical prior knowledge learning network or an infrared image generation system based on a physical prior constraint generative adversarial network.
[0049] Furthermore, in one specific embodiment, this embodiment provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the above-described method for constructing a physical prior knowledge learning network or the infrared image generation method based on a physical prior constraint generative adversarial network.
[0050] Those skilled in the art will understand that implementing all or part of the processes in the above embodiments can be accomplished by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory.
[0051] Furthermore, in one specific embodiment, this embodiment provides a computer device, which includes a memory and a processor. The memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes the above-described method for constructing a physical prior knowledge learning network or the above-described method for generating infrared images based on physical prior constraints generative adversarial networks.
[0052] This embodiment provides a computer device. The hardware device in this part is a general model and is not shown in the figure. The system includes a processor and a memory, which can be connected by a bus or other means. The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs and modules, as well as corresponding program instructions / modules. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions and modules stored in the memory, so as to realize the above-mentioned methods and steps.
[0053] The memory may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, mobile communication networks, and combinations thereof.
[0054] One or more modules are stored in the memory. When the processor executes, it performs the method steps in the embodiments. In this way, the inventive purpose of the present invention can be achieved through the method, apparatus and process of the present invention. The specific details of the above-mentioned computer device can be understood by referring to the relevant descriptions and effects in the embodiments, and will not be repeated here.
[0055] The above description of the technical solution provided by the present invention through several specific embodiments is intended to highlight the advantages and benefits of the technical solution provided by the present invention. However, the above-described specific embodiments are not intended to limit the present invention. Any reasonable modifications and improvements to the present invention, reasonable combinations of implementation methods and equivalent substitutions based on the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for constructing a physics prior knowledge learning network, characterized in that, Includes the following steps: Step 1: Construct an infrared radiation intensity calculation model, which links infrared radiation intensity with the physical parameters of the target, environment, and atmosphere; Step 2: Preprocess the real infrared image; Step 3: Build the U-Net network. Based on the preprocessed real infrared image and the infrared radiation intensity calculation model, complete the U-Net network training through parameter estimation, image reconstruction and error loss constraints. Step 4: Save the optimal model parameters after training to obtain the physical prior knowledge learning network.
2. The construction method according to claim 1, characterized in that, In step 1, the infrared radiation intensity calculation model is constructed in conjunction with the infrared imaging mechanism. The physical parameters in the model include the target body temperature, target emissivity, atmospheric transmittance, environmental reflected radiation, and atmospheric path radiation.
3. The construction method according to claim 1, characterized in that, Step 3 specifically includes: Construct a U-Net network based on an encoder and decoder structure; During training, preprocessed real infrared images are input into the U-Net network. The U-Net network estimates the parameters of the infrared radiation intensity calculation model pixel by pixel and outputs the predicted images of each parameter through its own output channel. The predicted images of each parameter are calculated pixel by pixel according to the infrared radiation intensity calculation model to obtain the reconstructed infrared image. The mean square error loss is calculated between the original infrared image and the reconstructed infrared image to constrain the training of U-Net.
4. The construction method according to claim 1, characterized in that, The encoder consists of a multi-level feature extraction module, and each level of feature extraction module includes a convolution operation unit, a normalization unit, a nonlinear activation unit, and a downsampling unit in sequence. The decoder includes an upsampling module, a convolution module, and skip connections.
5. An infrared image generation method based on a physical prior constraint generative adversarial network, wherein the method uses a physical prior knowledge learning network obtained by the construction method according to any one of claims 1-4 as a physical prior constraint module to realize infrared image generation, characterized in that, Includes the following steps: Step 1: Build an attention U-Net generator with physical prior constraints. Input the visible light image into the physical prior knowledge learning network and extract the potential infrared radiation features as physical prior features. Then, concatenate the physical prior features with the visible light image in the channel dimension and input them into the attention U-Net backbone network. The backbone network outputs the infrared image and predicts multiple infrared-related physical parameter maps. Step 2: Build a multi-scale PatchGAN discriminator. Stitch the visible light image with the real infrared image and the generated infrared image to obtain image pairs. Input the two sets of image pairs into the PatchGAN discriminator at each scale to obtain the real and fake discrimination results of the local area of the image. Step 3: Training the Infrared Image Generative Adversarial Network Based on Physical Prior Constraints. The infrared image generative adversarial network based on physical prior constraints, consisting of an attention-based U-Net generator with physical prior constraints and a multi-scale PatchGAN discriminator, is trained. The optimizer and learning rate parameters are configured, and the generator and discriminator are jointly constrained by physical consistency loss, pixel-level loss, and feature matching loss to complete the model training and achieve infrared image generation.
6. The infrared image generation method according to claim 5, characterized in that, The physical prior knowledge learning network with parameter freezing extracts the latent infrared radiation features in visible light images that are related to the corresponding infrared images as physical prior features.
7. The infrared image generation method according to claim 5, characterized in that, Each PatchGAN discriminator consists of multiple convolutional layers, normalization layers, and activation functions. Each PatchGAN discriminator outputs a discrimination map through a local receptive field, and each element in the discrimination map corresponds to the true or false score of a local region in the input image.
8. An infrared image generation system based on a physical prior constraint generative adversarial network, characterized in that, The system is used to implement the infrared image generation method based on physical prior constraints generative adversarial networks as described in any one of claims 5-7.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs a method for constructing a physical prior knowledge learning network as described in any one of claims 1-4, or an infrared image generation method based on a physical prior constraint generative adversarial network as described in any one of claims 5-7.
10. A computer device, characterized in that, The device includes a memory and a processor. The memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes a method for constructing a physical prior knowledge learning network as described in any one of claims 1-4 or an infrared image generation method based on a physical prior constraint generative adversarial network as described in any one of claims 5-7.