Zero-shot infrared super-resolution method based on generated blur kernel

Abstract

This invention discloses a zero-shot infrared super-resolution method based on blur kernel generation, which relates to the field of super-resolution reconstruction technology and includes the following steps: preparing a dataset, estimating the degradation kernel, constructing a super-resolution network, training the network model, and calculating reconstruction performance indicators. The invention designs a new blur kernel estimation method and constructs a simple, efficient, and pre-trained super-resolution network. Experimental results demonstrate that the invention demonstrates superior super-resolution effects and high robustness on multiple standard datasets. The invention is suitable for super-resolution reconstruction of various low-resolution infrared images, and has important application value in fields with high image resolution requirements, such as military surveillance, medical imaging, and environmental monitoring.

Description

Technical Field

[0001] The present invention relates to the technical field of super-resolution reconstruction, and in particular to a zero-sample infrared super-resolution method based on generating a blur kernel. Background Art

[0002] In infrared imaging, image resolution directly impacts the accuracy of subsequent analysis and decision-making. Single image super-resolution (SISR) has attracted widespread attention due to its widespread application in computer vision, particularly in the field of infrared imaging. It aims to recover high-resolution (HR) images rich in detail and texture from low-resolution (LR) images. In recent years, methods based on zero-shot learning (ZSL) have provided new insights into super-resolution reconstruction technology. In the field of infrared image super-resolution reconstruction, combining zero-shot learning with methods is of great significance in addressing the scarcity of infrared image data and the simple degradation of existing super-resolution algorithms, significantly improving the detail representation and resolution of infrared images.

[0003] The Chinese patent publication number is "CN117422620A", and its name is "Deep Learning-Based Super-Resolution Reconstruction Method for Infrared Images for Real Scenes". This method establishes an infrared image degradation model and combines channel attention and densely connected super-resolution networks. Low-resolution images with different degradation conditions are used to improve the robustness of the network. Taking into account the scarcity of infrared image data, it is proposed to use complex degradation models to synthesize training data to solve the problem that existing deep learning-based super-resolution methods rely on large amounts of training data. However, the designed degradation model is single and has weak generalization ability in various scenarios. In addition, the model performance is significantly reduced without large-scale data pre-training. Summary of the Invention

[0004] The technical solution of the present invention to solve the above technical problems is to provide a zero-sample infrared super-resolution method based on generating a blur kernel, comprising the following steps:

[0005] S1. Data preprocessing: Randomly select low-resolution infrared images to be processed from an infrared image dataset that supports image formats of multiple infrared imaging devices.

[0006] S2. Degradation kernel estimation: Establish a generative adversarial network model, input noise degradation into the generator, and output the degradation kernel through the generator. The degradation kernel is convolved with the high-resolution image and superimposed with Gaussian noise to generate the degraded low-resolution image I LR′ ;Crop an area from the generated image as the generated patch, and input it into the discriminator together with the real patch cropped from the input image for authenticity judgment. When the discriminator has difficulty distinguishing between the real and the fake, the weight of the generator is determined to be close to the real degradation kernel;

[0007] S3. Build a super-resolution network: Use the designed derived convolutional residual blocks to build a super-resolution network and introduce two skip connections. The first skip connection adds the interpolated image and the output of the second convolutional block. The main branch is used to predict the image residual. The second skip connection adds the output of the first convolutional block and the output of the last derived convolutional residual block.

[0008] S4. Training the network model: Generate a degradation kernel through adversarial training of the generator and discriminator. Convolve the generated degradation kernel with the image to be reconstructed and superimpose noise to generate a degraded image. This image is then fed into the super-resolution network for training. The loss function is optimized to improve performance and generate a reconstructed image.

[0009] S5. Calculation of reconstruction performance indicators: Peak signal-to-noise ratio and structural similarity index are used to objectively evaluate the quality of the reconstructed image.

[0010] Furthermore, in step S2, the generative adversarial network model includes a generator and a discriminator;

[0011] The generator is used to simulate the degradation kernel of the input infrared image, generate a high-dimensional feature map through multiple convolutional layers, and restore the degradation kernel of the original image through deconvolution operations. The generator contains 6 linear full convolutional networks with a stride of 2. The convolution kernels of the first three layers are 7×7, 5×5, and 3×3, the convolution kernels of the next three layers are 1×1, and the last layer performs a blur degradation function with a stride of 2. Using random noise z as the initial input, the blur kernel is extracted through the convolution layer of the generator, and the obtained output is used as the input for the next layer to extract the degradation kernel.

[0012] The discriminator is used to evaluate the authenticity of the degradation kernel output by the generator. It uses a series of convolutional layers and pooling layers to reduce the dimensionality of the input image, and finally uses a binary classifier to determine whether the image is a true degradation kernel. The discriminator adopts a fully convolutional structure and uses a small receptive field of 7×7.

[0013] Furthermore, in step S3, the derived convolution residual block includes two branches: a main branch consisting of a 5×5 derived convolution, an activation function, and a deconvolution operation, and a secondary branch consisting of a residual connection;

[0014] The derived convolution is used to obtain multi-scale output features and greatly reduces the computational complexity and memory usage of the convolutional neural network for extracting quantitative features.

[0015] The deconvolution layer is interpolated by any interpolation operator, and then performs the same convolution operation as the convolution layer to fully mine the information of the image or network features locally;

[0016] The residual connection is used to transfer the gradient of the deep layer directly to the shallow layer, bypassing the main branch, so that the shallow layer parameters can be effectively trained and the gradient vanishing problem can be effectively avoided.

[0017] Furthermore, in step S4, during the adversarial training of the generator and the discriminator, the loss function expression of the entire network is defined as follows:

[0018]

[0019] in, Expressed as:

[0020]

[0021] Among them, α=0.5, β=0.5, γ=5, δ=1, is a regular term used to constrain the degenerate kernel extracted during the G network training process and is composed as follows:

[0022] Indicates that the cumulative sum of degenerate core elements is 1;

[0023] Indicates that m is a constant weighted mask, and the weight increases with the distance from the center. The closer to the boundary, the greater the penalty and the smaller the value;

[0024] Indicates the sparsity of the degenerate kernel and prevents over-smoothing;

[0025] Denote that the center of mass of the degenerate kernel is the geometric center of the kernel.

[0026] Furthermore, in step S4, a composite loss function is designed during training in the super-resolution network, including pixel mean square error loss and total variation regularization term;

[0027] Under the real condition degradation model, the loss function of the real condition degradation scenario is designed as follows:

[0028] loss2=MSE+λ2·loss TV ;

[0029] Among them, the MSE is the pixel mean square error loss, which is defined as follows:

[0030]

[0031] The loss TV is the total variation regularization term, defined as follows:

[0032] loss TV =∑ i,j ((x i,j-1 -x i,j ) 2 +(x i+1,j -xi,j ) 2 );

[0033] The Adam optimization algorithm is used in the training process, and the initial learning rate is selected as 0.001; the network is gradually optimized through an iterative process, and the function value threshold of the loss function is set at around 0.005.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] (1) The present invention provides an efficient degradation kernel estimation scheme that utilizes generative adversarial networks to generate blur kernels that can simulate more realistic physical degradation processes. The degradation kernel is adaptively adjusted based on the input image, and this method obtains a degradation kernel rather than a network. This solves the problem of traditional methods in insufficiently capturing complex blur patterns in blur kernel estimation. The resulting array can be directly applied to any super-resolution algorithm, is highly portable, and is applicable to different sensors and imaging systems, with strong adaptability and versatility.

[0036] (2) This paper utilizes the generated blur kernel to design a zero-shot infrared super-resolution method. This method does not require extensive data pre-training and can perform super-resolution reconstruction using only a single low-resolution image through internal learning. This solves the problem of lack of training data in specific application scenarios, improves the detail representation of super-resolution reconstruction, overcomes the previous limitation of relying on large amounts of annotated data, and significantly improves the reconstruction effect.

[0037] (3) The proposed super-resolution variable rate reconstruction method shows good results compared with traditional super-resolution methods. Experimental results show that in the experiment with a reconstruction factor of 2, the peak signal-to-noise ratio (PSNR) is improved by more than 0.9, and the structural similarity (SSIM) is improved by more than 0.03. In the experiments with reconstruction factors of 3 and 4, the evaluation indicators are also improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0039] Figure 1 This is a flowchart of the steps of the zero-sample infrared super-resolution method based on generating a blur kernel according to the present invention;

[0040] Figure 2 Flowchart of the infrared image degradation kernel estimation method of the present invention;

[0041] Figure 3 generating a structural diagram of the network for the present invention;

[0042] Figure 4 This is a structural diagram of the discriminant network of the present invention;

[0043] Figure 5 FIG1 is a structural diagram of the super-resolution network of the present invention;

[0044] Figure 6 This is a structural diagram of the convolution residual block derived from the present invention;

[0045] Figure 7 This is a training block diagram of the super-resolution network of the present invention. DETAILED DESCRIPTION

[0046] The present invention proposes a zero-sample infrared super-resolution method based on the generation of blur kernels, aiming to design a zero-sample infrared super-resolution method based on the generation of blur kernels to solve the problem that existing super-resolution methods based on deep learning rely on a large amount of training data.

[0047] The zero-sample infrared super-resolution method based on generating blur kernel proposed by the present invention is described below in a specific embodiment:

[0048] In the technical solution of this embodiment, Figure 1 As shown, a zero-shot infrared super-resolution method based on generating a blur kernel includes the following steps:

[0049] S1. Data preprocessing: Randomly select low-resolution infrared images to be processed from an infrared image dataset that supports image formats of multiple infrared imaging devices.

[0050] Specifically, a random selection is made from the KAIST infrared image dataset to input the infrared low-resolution image to be processed. The input data of the generator is multiple 32×32 or 64×64 image blocks cropped from the infrared image. The input image will generate a blur kernel matching the image block through a convolution operation.

[0051] S2. Degradation kernel estimation: Building a generative adversarial network model, such as Figure 2 As shown, the noise degradation is input into the generator, and the degradation kernel is output through the generator. The degradation kernel is convolved with the high-resolution image and superimposed with Gaussian noise to generate the degraded low-resolution image I LR′ ;Crop an area from the generated image as the generated patch, and input it into the discriminator together with the real patch cropped from the input image for authenticity judgment. When the discriminator has difficulty distinguishing between the real and the fake, the weight of the generator is determined to be close to the real degradation kernel;

[0052] Specifically, a generation network (G) is established: Figure 3As shown in Figure 3, the generator is used to simulate the degradation kernel of the input infrared image, generate a high-dimensional feature map through multiple convolutional layers, and restore the degradation kernel of the original image through deconvolution operations.

[0053] The generator G contains 6 linear full convolutional networks with a stride of 2. The convolution kernels of the first three layers are 7×7, 5×5, and 3×3, and the convolution kernels of the last three layers are 1×1. The last layer performs a blur degradation function with a stride of 2. The mathematical expression is:

[0054] D b (I)=(I LR *k)↓2;

[0055] Where * represents the convolution operation, k is the blur kernel of the degradation process, and the blurred degraded image can be obtained by convolving k with the input image;

[0056] Using random noise z as the initial input, each convolution kernel of the generator (G) is used as the convolution kernel of the convolution layer for extracting the blur kernel, and the output is used as the input of the convolution layer for extracting the degradation kernel in the next layer. The mathematical expression of the random noise vector z can be expressed as:

[0057] z=[z1,z2,…,z n ],z i ~P z ;

[0058] Where: z is a vector of dimension n, z i is the noise distribution P z The random variables P are independently sampled in z is a normal distribution, and n is the dimension of the noise vector.

[0059] P z is a normal distribution (Gaussian distribution), that is Each z i They are all sampled from the standard normal distribution, and the corresponding mathematical expressions are:

[0060]

[0061] At this time, the input noise vector z of the generator is a standard normal distribution random vector with dimension n.

[0062] Then, the degraded kernel in the generator is convolved with the input high-resolution image, Gaussian noise is superimposed, and the random perturbation of Gaussian noise is used to interfere with the generated blur kernel to obtain a low-resolution image I LR′ , the formula is as follows:

[0063]

[0064] The degradation kernel extracted by the final generator is a set of small arrays that can be applied to the SR algorithm;

[0065] Establish a discriminant network (D), such as Figure 4 As shown in the figure, the discriminator is used to evaluate the authenticity of the degradation kernel output by the generator. A series of convolutional layers and pooling layers are used to reduce the dimension of the input image, and finally a binary classifier is used to determine whether the image is a real degradation kernel.

[0066] The discriminator uses a probability distribution map D similar to a heat map map To calculate the difference between each pixel, the real patch is judged as true and the generated patch is judged as false, where D map Each value is the probability that each pixel is true. For a real patch, D map The output of should be close to 1; for the generated patch, D map The output should be close to 0.

[0067] A fully convolutional structure is adopted, with a small receptive field of 7×7, no pooling and stride, to ensure that D can make a separate judgment on each local patch and output the probability of each position for the discriminator to carefully distinguish the difference between the generated fake image patches and the real image patches.

[0068] S3. Build a super-resolution network: Use the designed derived convolutional residual blocks to build a super-resolution network and introduce two skip connections. The first skip connection adds the interpolated image and the output of the second convolutional block. The main branch is used to predict the image residual. The second skip connection adds the output of the first convolutional block and the output of the last derived convolutional residual block.

[0069] Specifically, if Figure 5 As shown in the figure, the super-resolution network uses four designed derived convolution residual blocks and introduces two-way skip connections. The first skip connection adds the interpolated image and the output of the second convolution block, and the main branch is used to predict the residual of the image; the second skip connection adds the output of the first convolution block and the output of the last derived convolution residual block.

[0070] like Figure 6 As shown in the figure, the derived convolution residual block is mainly divided into two branches: the main branch composed of 5×5 derived convolution filter, activation function and deconvolution operation, and the secondary branch composed of residual connection. The calculation process is as follows:

[0071] y=deconv(ReLU(conv(x)))+x;

[0072] Among them, ReLU, conv and deconv are activation functions, derived convolution and deconvolution operations respectively. From the input feature x to the output feature y, it is necessary to go through the operation of the main branch of deconv(ReLU(conv(x))) in the above formula, and the residual connection copies x directly backward to the deeper layers of the neural network, and through the secondary branch, the information of x is directly transmitted to the deeper layers of the neural network.

[0073] The derived convolution residual block can be integrated at any position in the network, and the "encoder-decoder" structure composed of derived convolution and deconvolution can be used to fully mine the information of image or network features locally.

[0074] S4. Training the network model: Generate a degradation kernel through adversarial training of the generator and the discriminator. Convolve the generated degradation kernel with the image to be reconstructed and superimpose noise to generate a degraded image. This image is then fed into the super-resolution network for training. The loss function is optimized to improve performance and generate a reconstructed image.

[0075] Specifically, the generator and the discriminator perform adversarial training, the generator tries to generate results as close as possible to the real degradation kernel, while the discriminator tries to distinguish the generated image I LR′ Compared with the real image I LR When the losses of the generator and the discriminator reach a balance, the weight of the generator will be used as the degradation kernel of the input of the super-resolution network to improve the subsequent super-resolution reconstruction effect.

[0076] The GAN was trained for 3000 iterations using the ADAM optimizer (β1 = 0.5; β2 = 0.999), alternating between single optimization steps of G and D, and setting the learning rate to 2e -4 , decaying ×0.1 every 750 times.

[0077] After the generator and discriminator training is completed, the generated blur kernel can be used in the subsequent zero-shot super-resolution reconstruction process.

[0078] S5. Calculation of reconstruction performance indicators: Peak signal-to-noise ratio and structural similarity index are used to objectively evaluate the quality of the reconstructed image.

[0079] Specifically, the corresponding evaluation indicators of the method proposed in the present invention are compared with the existing technical indicators, such as Figure 5 As shown in the figure, it can be seen that the method proposed in the present invention has an increase in computational complexity and parameters compared with the existing method, but the structural similarity and peak signal-to-noise ratio are greatly improved; these indicators show that the present invention can achieve the reconstruction of more accurate high-resolution images while ensuring the network operation speed.

[0080] PSNR and SSIM are used to objectively evaluate the quality of reconstructed images;

[0081] The peak signal-to-noise ratio can be simply defined by the mean square error, and its definition expression is shown as follows:

[0082]

[0083] Where l represents the number of grayscale bits per pixel in the image, typically 8, meaning the grayscale level of the pixels in the image is 256. The unit of peak signal-to-noise ratio (PSNR) is decibel (dB). The PSNR evaluation results are highly similar to subjective evaluation results and are one of the most widely used objective quality evaluation metrics. As can be seen from the definition, the greater the PSNR of the image to be evaluated, the less distortion there is in the image.

[0084] The structural similarity index is a comprehensive representation of the similarity in brightness, contrast, and correlation between the image to be evaluated and the reference image. Its definition is as shown in the formula:

[0085]

[0086] Among them, μ f Represents the grayscale average of the reference image; Represents the grayscale average value of the image to be evaluated; σ f is the variance of the reference image; is the variance of the image to be evaluated; is the covariance between the grayscale values of the reference image and the grayscale values of the image to be evaluated; C1 and C2 are constants set to prevent the denominator from being zero.

[0087] Furthermore, in step S2, the generative adversarial network model includes a generator and a discriminator;

[0088] The generator is used to simulate the degradation kernel of the input infrared image, generate a high-dimensional feature map through multiple convolutional layers, and restore the degradation kernel of the original image through deconvolution operations. The generator contains 6 linear full convolutional networks with a stride of 2. The convolution kernels of the first three layers are 7×7, 5×5, and 3×3, the convolution kernels of the next three layers are 1×1, and the last layer performs a blur degradation function with a stride of 2. Using random noise z as the initial input, the blur kernel is extracted through the convolution layer of the generator, and the obtained output is used as the input for the next layer to extract the degradation kernel.

[0089] The discriminator is used to evaluate the authenticity of the degradation kernel output by the generator. It uses a series of convolutional layers and pooling layers to reduce the dimensionality of the input image, and finally uses a binary classifier to determine whether the image is a true degradation kernel. The discriminator adopts a fully convolutional structure and uses a small receptive field of 7×7.

[0090] Furthermore, in step S3, the derived convolution residual block includes two branches: a main branch consisting of a 5×5 derived convolution, an activation function, and a deconvolution operation, and a secondary branch consisting of a residual connection;

[0091] The derived convolution is used to obtain multi-scale output features and greatly reduces the computational complexity and memory usage of the convolutional neural network for extracting quantitative features.

[0092] The deconvolution layer is interpolated by any interpolation operator, and then performs the same convolution operation as the convolution layer to fully mine the information of the image or network features locally;

[0093] The residual connection is used to transfer the gradient of the deep layer directly to the shallow layer, bypassing the main branch, so that the shallow layer parameters can be effectively trained and the gradient vanishing problem can be effectively avoided.

[0094] Furthermore, in step S4, during the adversarial training of the generator and the discriminator, the loss function of the entire blur kernel estimation network is designed so that during the comparison process, the discriminator compares whether the local feature distributions at different scales are consistent, rather than the global size of the image. Therefore, even if the region extracted from the scaled-down image is relatively large, the discriminator can still learn the similarity between these feature distributions. The loss function expression of the entire network is defined as follows:

[0095]

[0096] in, Expressed as:

[0097]

[0098] Among them, α=0.5, β=0.5, γ=5, δ=1, is a regular term used to constrain the degenerate kernel extracted during the G network training process and is composed as follows:

[0099] Indicates that the cumulative sum of degenerate core elements is 1;

[0100] Indicates that m is a constant weighted mask, and the weight increases with the distance from the center. The closer to the boundary, the greater the penalty and the smaller the value;

[0101] Indicates the sparsity of the degenerate kernel and prevents over-smoothing;

[0102] Denote that the center of mass of the degenerate kernel is the geometric center of the kernel.

[0103] These terms combined together help the model learn a centralized, sparse, and normalized convolution kernel for image processing or generation tasks. Ultimately, the generator continuously adjusts the downsampling kernel of its output image to keep the local features of the image consistent at different scales, thereby achieving optimization results.

[0104] Furthermore, in step S4, training is performed in the super-resolution network, such as Figure 7 As shown in Figure 2, the super-resolution network designed in this paper enriches the training set by performing four rotations (0°, 90°, 180°, 270°) and vertical and horizontal mirroring of each low-resolution (LR) and high-resolution (HR) image pair. To improve robustness and achieve a larger super-resolution scale factor (s) from a smaller LR image, the algorithm performs super-resolution processing step by step, applicable to multiple intermediate scale factors (s1, s2, ..., s m =s). At each intermediate scale factor si, the generated SR image HR i and their scaled / rotated versions are added to the gradually enlarged training set as new HR images. These images are scaled by a gradient factor s i+1 Scaling, generating new LR-HR training example pairs until the desired resolution increase is achieved.

[0105] Design a composite loss function, including pixel mean square error loss and total variation regularization term; under the real conditional degradation model, the loss function of the real conditional degradation scenario is designed as follows:

[0106] loss2=MSE+λ2·loss TV ;

[0107] Among them, the MSE is the pixel mean square error loss, which is defined as follows:

[0108]

[0109] The loss TV is the total variation regularization term, defined as follows:

[0110] loss TV =∑ i,j ((x i,j-1 -x i,j ) 2 +(x i+1,j -x i,j ) 2 );

[0111] The Adam optimization algorithm is used in the training process, and the initial learning rate is selected as 0.001; the network is gradually optimized through an iterative process, and the function value threshold of the loss function is set at around 0.005.

[0112] When the loss function value is less than 0.005, the training of the entire network can be considered to be basically completed; after the training is completed, the network parameters are saved for future use. The super-resolution effect will be dynamically adjusted according to the current image quality during the training phase;

[0113] After training, the network will directly input the input low-resolution image into the optimized super-resolution network to generate a super-resolution output image.

[0114] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A zero-shot infrared super-resolution method based on generating blur kernels, characterized in that: The following steps are involved: S1. Data preprocessing: Randomly select low-resolution infrared images to be processed from an infrared image dataset that supports image formats of multiple infrared imaging devices. S2. Degradation kernel estimation: Establish a generative adversarial network model, input noise degradation into the generator, and output the degradation kernel through the generator. The degradation kernel is convolved with the high-resolution image and superimposed with Gaussian noise to generate the degraded low-resolution image I LR′ ;Crop an area from the generated image as the generated patch, and input it into the discriminator together with the real patch cropped from the input image for authenticity judgment. When the discriminator has difficulty distinguishing between the real and the fake, the weight of the generator is determined to be close to the real degradation kernel; S3. Super-resolution network construction: A super-resolution network is constructed using the designed derived convolutional residual block and introducing two skip connections. The first skip connection adds the interpolated image and the output of the second convolutional block, and the main branch is used to predict the image residual. The second skip connection adds the output of the first convolutional block and the output of the last derived convolutional residual block; S4. Training the network model: Generate a degradation kernel through adversarial training of the generator and the discriminator. Convolve the generated degradation kernel with the image to be reconstructed and superimpose noise to generate a degraded image. This image is then fed into the super-resolution network for training. The loss function is optimized to improve performance and generate a reconstructed image. S5. Calculation of reconstruction performance indicators: Peak signal-to-noise ratio and structural similarity index are used to objectively evaluate the quality of the reconstructed image; In step S4, in the adversarial training of the generator and the discriminator, the loss function expression of the entire network is defined as follows: in, Expressed as: Among them, α=0.5, β=0.5, γ=5, δ=1, is a regular term used to constrain the degenerate kernel extracted during the G network training process and is composed as follows: Indicates that the cumulative sum of degenerate core elements is 1; Indicates that m is a constant weighted mask, and the weight increases with the distance from the center. The closer to the boundary, the greater the penalty and the smaller the value; Indicates the sparsity of the degenerate kernel and prevents over-smoothing; The center of mass of the degenerate core is represented as the geometric center of the core; In the training of super-resolution network, a composite loss function is designed, including pixel mean square error loss and total variation regularization term; Under the real condition degradation model, the loss function of the real condition degradation scenario is designed as follows: loss2=MSE+λ2·loss TV ; Among them, the MSE is the pixel mean square error loss, which is defined as follows: The loss TV is the total variation regularization term, defined as follows: loss TV =∑ i,j ((x i,j-1 -x i,j ) 2 +(x i+1,j -x i,j ) 2 ); The Adam optimization algorithm is used in the training process, and the initial learning rate is selected as 0.001; the network is gradually optimized through an iterative process, and the function value threshold of the loss function is set at around 0.

005.

2. The zero-shot infrared super-resolution method based on blur kernel generation according to claim 1, characterized in that: In step S2, the generative adversarial network model includes a generator and a discriminator; The generator is used to simulate the degradation kernel of the input infrared image, generate a high-dimensional feature map through multiple convolutional layers, and restore the degradation kernel of the original image through deconvolution operations. The generator contains 6 linear full convolutional networks with a stride of 2. The convolution kernels of the first three layers are 7×7, 5×5, and 3×3, the convolution kernels of the next three layers are 1×1, and the last layer performs a blur degradation function with a stride of 2. Using random noise z as the initial input, the blur kernel is extracted through the convolution layer of the generator, and the obtained output is used as the input for the next layer to extract the degradation kernel. The discriminator is used to evaluate the authenticity of the degradation kernel output by the generator. It uses a series of convolutional layers and pooling layers to reduce the dimensionality of the input image, and finally uses a binary classifier to determine whether the image is a true degradation kernel. The discriminator adopts a fully convolutional structure and uses a small receptive field of 7×7.

3. The zero-shot infrared super-resolution method based on blur kernel generation according to claim 1, characterized in that: In step S3, the derived convolution residual block includes two branches: a main branch consisting of a 5×5 derived convolution, an activation function, and a deconvolution operation, and a secondary branch consisting of a residual connection; The derived convolution is used to obtain multi-scale output features and greatly reduces the computational complexity and memory usage of the convolutional neural network for extracting quantitative features; The deconvolution layer is interpolated by any interpolation operator, and then performs the same convolution operation as the convolution layer to fully mine the information of the image or network features locally; The residual connection is used to transfer the gradient of the deep layer directly to the shallow layer, bypassing the main branch, so that the shallow layer parameters can be effectively trained and the gradient vanishing problem can be effectively avoided.

Images