Super-resolution reconstruction method of infrared image

Abstract

The invention provides a super-resolution reconstruction method of an infrared image. The method comprises the following steps: constructing an infrared image super-resolution reconstruction model; obtaining an original infrared image ILR, extracting a shallow feature F0 from the original infrared image ILR, and inputting the original infrared image ILR into an edge detection auxiliary network to extract an edge feature FE of the original infrared image ILR; extracting a deep feature from the shallow feature F0, and fusing the edge feature FE to obtain an edge enhanced deep feature FDEF; and reconstructing a super-resolution infrared image ISR by fusing shallow and deep features. According to the method, the reconstructed infrared image can obtain more useful information, and the optimal visual quality is generated.

Description

Technical Field

[0001] The present invention relates to the technical field of image super-resolution reconstruction, and in particular to a super-resolution reconstruction method for infrared images. Background Art

[0002] Infrared imaging refers to the conversion of infrared radiation energy that is invisible to the naked eye into visible images after being captured by infrared detectors. Infrared imaging technology has the characteristics of good concealment, strong penetration, and high recognition, and is widely used in military, security, and monitoring fields. However, due to the limitations of infrared wavelengths and shooting equipment, the resolution of infrared images is generally low, which seriously limits the application and development of infrared images in various fields. Summary of the invention

[0003] The purpose of the present invention is to provide a super-resolution reconstruction method for infrared images, aiming to solve the problems of blur, lack of details and unclear edges of infrared images captured due to the limitations of infrared wavelength and shooting equipment.

[0004] In a first aspect, the present invention provides a super-resolution reconstruction method for an infrared image, the method comprising:

[0005] Constructing an infrared image super-resolution reconstruction model, the infrared image super-resolution reconstruction model includes an image super-resolution backbone network and an edge detection auxiliary network, the image super-resolution backbone network includes a shallow feature extraction module, a deep feature extraction module and a reconstruction module, and the edge detection auxiliary network includes a bicubic upsampling module, an edge detection network and a convolution module;

[0006] Get the original infrared image I LR , and from the original infrared image I LR The shallow feature F0 is extracted from the original infrared image I LR Input into the edge detection auxiliary network to extract the original infrared image I LR The edge feature F E ;

[0007] Extract deep features from shallow features F0 and fuse edge features F E , and obtain the edge-enhanced deep feature F DEF ;

[0008] Reconstructing super-resolution infrared images by fusing shallow and deep features SR .

[0009] Furthermore, the acquisition of the original infrared image I LR , and from the original infrared image I LR The shallow feature F0 is extracted from the original infrared image ILR Input into the edge detection auxiliary network to extract the original infrared image I LR The edge feature F E The steps include:

[0010] The shallow feature F0 is extracted according to the following formula:

[0011] F0=H SF (I LR )

[0012] The edge feature F is extracted according to the following formula E :

[0013] F E =B_RCF(I LR )

[0014] Among them, H SF (·) represents the shallow feature extraction module, and B_RCF(·) is an edge detection auxiliary network that provides image edge information.

[0015] Furthermore, the deep feature extraction module includes K residual cross-window Transformer modules and a convolution layer with a convolution kernel size of 3×3;

[0016] The deep features are extracted from the shallow features F0, and the edge features F0 are fused at the same time. E , and obtain the edge-enhanced deep feature F DEF The steps include:

[0017] The deep features are extracted block by block from the K RCSTB modules, and the deep intermediate features F1, F2, ..., F K-1 and deep features F K ;

[0018] The deep feature F K With edge feature F E After fusion, it passes through a 3×3 convolution layer and finally outputs F DEF .

[0019] Further,

[0020] The deep features are obtained according to the following formula:

[0021]

[0022] The final output F is obtained according to the following formula DEF :

[0023] F DEF =H CONV (F K +αF E)

[0024] in, represents the i-th RCSTB, H CONV (·) is a convolutional layer.

[0025] Furthermore, the super-resolution infrared image I is reconstructed by fusing shallow and deep features. SR The steps include:

[0026] I SR =H REC (F0+F DEF )

[0027] Among them, H REC (·) represents the reconstruction module, which consists of sub-pixel convolution and 3×3 convolution.

[0028] Furthermore, the method further comprises:

[0029] The RCSTB model consists of L cross-window Transformer layers and a convolutional layer;

[0030] Given the input feature of the i-th RCSTB module is F i,0 , first, extract the intermediate features F from L CSTL layers in sequence i,1 、F i,2 , ..., F i,L :

[0031]

[0032] Then, it passes through another convolution layer before the residual connection. Finally, the output feature of the i-th RCSTB is F i,out :

[0033]

[0034] in, is the jth CSTL layer in the i-th RCSTB module, is the convolutional layer in the i-th RCSTB module.

[0035] Furthermore, the CSTL layer includes a cross-window self-attention module, a multi-layer perceptron, two layers and two residual connections.

[0036] Furthermore, the cross-shaped window self-attention module performs self-attention in parallel by forming a cross-shaped window on the horizontal and vertical stripes. According to the multi-head self-attention mechanism, the input feature X∈R (H×W)×C It is first linearly projected into K heads, and then each head will perform local self-attention within a horizontal or vertical stripe.

[0037] Compared with the prior art, the present invention has the following advantages:

[0038] The present invention designs an edge-enhanced cross-window Transformer infrared image super-resolution reconstruction model ECSSR. The cross-window Transformer layer in the ECSSR can calculate the self-attention in the horizontal and vertical directions in parallel, and obtain a better reconstruction effect without increasing the amount of calculation. In addition, the present invention proposes an edge detection auxiliary network (EDAN) by combining Bicubic and RCF to obtain more accurate edge information. Using this edge information as supplementary information can better restore the edges of super-resolution infrared images. Experimental results show that the proposed method is superior to the current representative methods in both objective indicators and subjective vision. Even when the reconstruction multiple is 8, the edges and textures of the reconstructed image are still clearly visible. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A flowchart of a super-resolution reconstruction method for infrared images proposed in one embodiment of the present invention;

[0040] Figure 2 A schematic diagram of the structure of an ECSSR model according to an embodiment of the present invention;

[0041] Figure 3 A schematic diagram of the structure of a residual cross-window Transformer module according to an embodiment of the present invention;

[0042] Figure 4 A schematic diagram of the structure of a cross-shaped window Transformer layer according to an example of an embodiment of the present invention;

[0043] Figure 5 A schematic diagram of the structure of a cross-shaped window self-attention module according to an example of an embodiment of the present invention;

[0044] Figure 6 This is one of the visual effect comparison diagrams of the ECSSR model and other models in one embodiment of the present invention;

[0045] Figure 7 FIG2 is a second comparison diagram of the visual effects of the ECSSR model and other models in one embodiment of the present invention;

[0046] Figure 8 The third diagram is a comparison of the visual effects of the ECSSR model and other models in one embodiment of the present invention;

[0047] Fig. 9 FIG4 is a fourth comparison diagram of visual effects between the ECSSR model and other models in one embodiment of the present invention;

[0048] Fig.10 FIG5 is a fifth comparison diagram of visual effects between the ECSSR model and other models in one embodiment of the present invention;

[0049] Fig.11 FIG6 is a sixth visual effect comparison diagram of the ECSSR model and other models in one embodiment of the present invention.

[0050] The following specific implementation manner will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be understood by people with general skills in the field to which the present invention belongs. "Including" and similar words used in this article mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects.

[0052] like Figure 1 As shown, an embodiment of the present invention provides a super-resolution reconstruction method for infrared images, the method comprising steps S101 to S104, wherein:

[0053] Step S101: constructing an infrared image super-resolution reconstruction model, wherein the infrared image super-resolution reconstruction model includes an image super-resolution backbone network and an edge detection auxiliary network, wherein the image super-resolution backbone network includes a shallow feature extraction module, a deep feature extraction module and a reconstruction module, and the edge detection auxiliary network includes a bicubic upsampling module, an edge detection network and a convolution module;

[0054] It should be pointed out that infrared images are currently widely used in many fields. However, in the imaging process, due to the interference of the external environment and the limitations of the imaging equipment, infrared images often have problems such as blur, lack of details and unclear edges. In order to obtain high-quality infrared images, this embodiment uses SwinIR as the basic algorithm model, designs a deep residual network with a cross-window Transformer as the main body, introduces a cross-window self-attention module in the network structure to improve the receptive field of the model, and uses local enhanced position encoding to obtain the position information of the image. The model has excellent long-distance context modeling and global information capture capabilities. In addition, this embodiment proposes an edge detection auxiliary network for enhancing image edge information in super-resolution reconstruction tasks, thereby effectively improving the results of super-resolution reconstruction of infrared images.

[0055] like Figure 2 The figure shows the structure of the infrared image super-resolution reconstruction model (ECSSR). The model contains two sub-networks: the image super-resolution backbone network and the edge detection auxiliary network (EDAN). The backbone network consists of three modules: shallow feature extraction module, deep feature extraction module and reconstruction module. EDAN consists of bicubic upsampling, edge detection RCF (Richer Convolutional Features for Edge Detection) network and convolution module. EDAN is used to fully extract the edge features of infrared images and assist the backbone network to reconstruct high-quality infrared images.

[0056] Step S102: acquiring an original infrared image, extracting shallow features from the original infrared image, and inputting the original infrared image into an edge detection auxiliary network to extract edge features of the original infrared image;

[0057] In this step, it is assumed that the input and output of the ECSSR model are I LR and I SR First, a convolution layer with a convolution kernel size of 3×3 is used to form a shallow feature extraction module to extract the LR Extract the shallow feature F0. LR Input to EDAN, extract I LR The edge feature F E .

[0058] F0=HSF (I LR )

[0059] F E =B_RCF(I LR )

[0060] Among them, H SF (·) represents the shallow feature extraction module, and B_RCF(·) is the EDAN that provides image edge information.

[0061] Step S103: extracting deep features from shallow features and fusing edge features to obtain edge-enhanced deep features;

[0062] In the step, deep features are extracted from F0 and edge features F are fused. E , and obtain the edge-enhanced deep feature F DEF .

[0063] F DEF =H DEF (F0,F E )

[0064] Among them, H DEF (·) is the deep feature extraction module, which includes the process of deep feature extraction and edge enhancement.

[0065] In addition, in some embodiments, the deep feature extraction module includes K residual cross-shaped window transformer blocks (RCSTB) and a convolution layer with a convolution kernel size of 3×3. The deep features are extracted block by block from the K RCSTB modules to obtain deep intermediate features F1, F2, ..., F K-1 and deep features F K Then the deep feature F K With edge feature F E After fusion, it passes through a 3×3 convolution layer and finally outputs F DEF .

[0066]

[0067] F DEF =H CONV (F K +αF E )

[0068] in, represents the i-th RCSTB, H CONV (·) is a convolutional layer at the end. In order to reasonably enhance the deep features F KThe edge information is the edge feature F E The balance factor α strength and F K To overlay.

[0069] Step S104: reconstructing a super-resolution infrared image by fusing shallow and deep features;

[0070] Specifically, the reconstructed super-resolution infrared image is obtained according to the following formula:

[0071] I SR =H REC (F0+F DEF )

[0072] Among them, H REC (·) represents the reconstruction module, which consists of sub-pixel convolution and 3×3 convolution. The shallow feature F0 mainly contains low-frequency information, while F DEF Through long skip connections, low-frequency information can be passed to the reconstruction module, allowing the deep feature extraction module to focus on the extraction of high-frequency information and the fusion of edge features.

[0073] In addition, in some embodiments, since the CSWin Transformer performs parallel calculations in the horizontal and vertical directions, the accuracy can be improved without increasing the amount of calculation. Therefore, this embodiment designs a residual cross window Transformer module RCSTB based on the CSWin Transformer, such as Figure 3 shown.

[0074] RCSTB consists of L cross-shaped window transformer layers (Cross-Shaped Window Transformer Layer, CSTL) and a convolutional layer. Given the input feature of the i-th RCSTB is F i,0 , F i,0 This is the above F i First, we extract the intermediate features F from L CSTLs in turn. i,1 、F i,2 , ..., F i,L .

[0075]

[0076] in, is the jth CSTL in the i-th RCSTB. Then, it passes through another convolutional layer before the residual connection. Finally, the i-th RCSTB output feature is F i,out , F i,out This is the above F i+1 .

[0077]

[0078] in, is the convolutional layer in the i-th RCSTB.

[0079] The RCSTB proposed in this embodiment can combine the adaptive filtering characteristics of the Transformer with the spatial invariant filtering characteristics of the convolution to improve the modeling ability of the model. In RCSTB, the residual connection provides a connection from the shallow layer to the deep layer, so that the model can better extract deep features without losing shallow features.

[0080] Furthermore, in some embodiments, the cross-window Transformer layer CSTL is based on the standard multi-head self-attention of the original Transformer. Figure 4 As shown in the figure, CSTL mainly consists of a Cross-ShapedWindow TransformerSelf-Attention (CSWSA) module, a Multilayer Perceptron (MLP), two LayerNorm (LN) layers and two residual connections. Figure X First, after being processed by a LayerNorm layer, it is input into CSWSA, which can be used to extract rich feature information. Then a residual connection is added to fuse the initial input features. Then it passes through an MLP and a LayerNorm layer. The MLP consists of two fully connected layers and a GELU activation function located between the two fully connected layers, which is used for further feature conversion to slow down the convergence of the model. Finally, the input feature X is fused through a residual connection. In CSTL, the LN layer is added before CSWSA and MLP, and residual connections are used to alleviate the rank collapse problem. The whole process is expressed as:

[0081] X=CSWSA(LN(X))+X

[0082] X=MLP(LN(X))+X

[0083] Furthermore, in some embodiments, Figure 5 As shown in the figure, the cross-shaped window self-attention module (CSWSA) is implemented by forming a cross-shaped window on the horizontal and vertical stripes and performing self-attention in parallel. According to the multi-head self-attention mechanism, the input feature X∈R (H×W)×C It is first linearly projected into K heads, and then each head will perform local self-attention within a horizontal or vertical stripe.

[0084] Taking horizontal stripe self-attention as an example, X is evenly divided into non-overlapping horizontal stripes of equal height and size sw, represented as [X 1,X 2 ,...,X M ], each stripe contains sw×W tokens. sw represents the stripe height and can be adjusted to balance the learning ability and computational complexity.

[0085] Formally, assume that Q, K, and V of the k-th head mapping all contain d k dimensions, then the output of the horizontal stripe self-attention of the kth head is defined as:

[0086] X=[X 1 ,X 2 ,...,X M ]

[0087]

[0088] Where X i ∈R (sw×W)×C And M = H / sω, i = 1,...,M, Respectively represent the mapping matrices of Q, K and V of the kth head, d k Set to C / K. Similarly, the vertical stripe self-attention can be derived similarly, and its output for the k-th head is expressed as V_Attention k (X).

[0089] Divide the K heads into two parallel groups equally (each group has K / 2 heads, K is usually an even value). The first group is used for horizontal stripe self-attention, and the second group is used for vertical stripe self-attention. Finally, the outputs of the two parallel groups are fused.

[0090]

[0091] CSWin Attention(x) =Concat(head1,head K )W o

[0092] Where W o ∈R C×C is the projection matrix that maps the self-attention result to the target output dimension (set to C by default). As mentioned above, the key in the design of the self-attention mechanism is to group multiple heads into different groups and apply different self-attention operations accordingly. In other words, by grouping multiple heads, the attention area of ​​each token within a Transformer block is expanded.

[0093] Since self-attention is permutation invariant, it often ignores important position information in two-dimensional images. Therefore, in order to make up for this defect, position encoding is added to Transformer. For a specific input element, the most important position information generally comes from the local area. Therefore, this chapter uses local enhanced position encoding (LEPE) to calculate V by using a deep convolution operator, as shown in the following formula:

[0094]

[0095] Where d represents the dimension of the feature and DWConv(·) is the depthwise convolution used to compute the local enhanced position encoding.

[0096] In summary, the present invention designs an edge-enhanced cross-window Transformer infrared image super-resolution reconstruction model ECSSR. The cross-window Transformer layer in the ECSSR can calculate the self-attention in the horizontal and vertical directions in parallel, and obtain better reconstruction effect without increasing the amount of calculation. In addition, the present invention proposes an edge detection auxiliary network (EDAN) by combining Bicubic and RCF to obtain more accurate edge information. Using this edge information as supplementary information can better restore the edges of super-resolution infrared images. Experimental results show that the proposed method is superior to the current representative methods in both objective indicators and subjective vision. Even when the reconstruction multiple is 8, the edges and textures of the reconstructed image are still clearly visible.

[0097] Test Example 1

[0098] In order to verify the effectiveness of ECSSR, this chapter selects Bicubic, SRCNN, FSRCNN, ESPNN, VDSR, EDSR, RDN, RCAN and SwinIR algorithm models as comparative experiments. Among these algorithm models, Bicubic is a traditional method, while the others are based on deep learning. These algorithm models are trained and tested in the same environment to ensure the fairness of the experimental results.

[0099] As shown in Tables 1, 2, and 3, the PSNR and SSIM results of super-resolution reconstructed images at ×2, ×4, and ×8 reconstruction magnifications are shown. It can be seen that compared with the traditional method Bicubic, the algorithm models based on deep learning have obvious improvements in quantitative indicators. Among them, the ECSSR algorithm model proposed in this chapter has a PSNR improvement of 1.8017dB to 4.1187dB under three reconstruction magnifications, and an SSIM improvement of 0.0107 to 0.0447. Compared with the deep learning-based methods, when the reconstruction magnification is ×2, the PSNR and SSIM of ECSSR reach 46.2480dB and 0.9923 respectively, which is better than other comparison methods. When the reconstruction magnification is ×4, ECSSR also achieves the best overall performance in terms of PSNR and SSIM. Even when the reconstruction magnification is ×8, ECSSR exceeds the second best SwinIR method with a PSNR increase of up to 0.1721dB. Therefore, it can be seen from the indicators of PSNR and SSIM that ECSSR achieves the highest value on the test dataset, which proves the superiority of the proposed method from the quantitative indicators.

[0100] Table 1 Reconstruction results with reconstruction factor of 2

[0101]

[0102] Table 2 Reconstruction results with a reconstruction factor of 4

[0103]

[0104]

[0105] Table 3 Reconstruction results with reconstruction multiple of 8 times

[0106]

[0107] In order to intuitively illustrate that the images reconstructed by the ECSSR algorithm model have better visual effects, the following images are selected from the test set: Figures 6 to 11 The 6 images shown are used to demonstrate the reconstruction effects of each comparison algorithm model, where test image 1 is reconstructed 2 times, test images 2 and 3 are reconstructed 4 times, and test images 5, 6, and 7 are reconstructed 8 times. Then the red box parts in the images are cut out for display.

[0108] When the reconstruction multiple is 2, it can be seen from test image 1 that only the ECSSR algorithm model can accurately restore some texture information on the parking spaces in the image. The lines of the parking spaces in the images reconstructed by SRCNN and FSRCNN are intertwined, the lines of the parking spaces in the images reconstructed by VDSR become curves, the lines of the parking spaces in the images reconstructed by EDSR, RDN and RCAN are in the wrong direction, and the second half of the reconstruction by SwinIR is somewhat misplaced.

[0109] When the reconstruction multiple is 4, as shown in test images 2 and 3, the images reconstructed by SRCNN, FSRCNN, ESPCNN and VDSR are all blurry. In test image 2, the right railing of the reconstructed images of EDSR, RDN, RCAN and SwinIR has different degrees of deformation, and only ECSSR can restore the railing more clearly; in test image 3, the overlapping position of the railing and the branch in the reconstructed images of EDSR, RDN, RCAN and SwinIR are all mistaken for the branch part of the railing and restored, and only ECSSR restores it clearly and accurately with the best visual effect.

[0110] Test images 4, 5, and 6 are all 8 times reconstruction of the image. In this case, the information contained in the input low-resolution image is very limited, so the reconstruction is difficult. It can be seen from test images 4, 5, and 6 that most of the images reconstructed by the compared super-resolution models have very poor visual effects and serious blurring artifacts. In contrast, the ECSSR model proposed in this chapter obtains better results and restores more high-frequency details, such as sharp edges.

[0111] For test image 4, all comparison methods cannot reconstruct the railing part, and cannot even distinguish any information that the image is supposed to express. Only the ECSSR model can reconstruct an image close to the real image. In test image 5, the early algorithm models SRCNN, FSRCNN, ESPCNN, VDSR, and EDSR lost the main structure of the image. Recent methods (such as SwinIR) can restore the main contours, but cannot restore more image edge details. The ECSSR model obtains more refined results. In test image 6, other comparison methods cannot restore the football net structure. In contrast, the ECSSR model performs well. When the scale factor is very large, the LR infrared image contains very limited information. Having lost most of the high-frequency information, it is difficult for the SR method to reconstruct valuable results. Most of the compared methods cannot achieve this goal. However, ECSSR can obtain more useful information and produce the best visual quality.

[0112] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0113] Although the embodiments of the present invention are described in detail above, it is obvious to those skilled in the art that various modifications and variations can be made to these embodiments. However, it should be understood that such modifications and variations are within the scope and spirit of the present invention as described in the claims. Moreover, the present invention described herein may have other embodiments and may be implemented or realized in a variety of ways.

Claims

1. A super-resolution reconstruction method for infrared images, characterized in that: The method comprises: Constructing an infrared image super-resolution reconstruction model, the infrared image super-resolution reconstruction model includes an image super-resolution backbone network and an edge detection auxiliary network, the image super-resolution backbone network includes a shallow feature extraction module, a deep feature extraction module and a reconstruction module, and the edge detection auxiliary network includes a bicubic upsampling module, an edge detection network and a convolution module; Get the original infrared image I LR , and from the original infrared image I LR The shallow feature F0 is extracted from the original infrared image I LR Input into the edge detection auxiliary network to extract the original infrared image I LR The edge feature F E ; Extract deep features from shallow features F0 and fuse edge features F E , and obtain the edge-enhanced deep feature F DEF ; Reconstructing super-resolution infrared images by fusing shallow and deep features SR .

2. The super-resolution reconstruction method of infrared images according to claim 1, characterized in that: The original infrared image I is obtained LR , and from the original infrared image I LR The shallow feature F0 is extracted from the original infrared image I LR Input into the edge detection auxiliary network to extract the original infrared image I LR The edge feature F E The steps include: The shallow feature F0 is extracted according to the following formula: F0=H SF (I LR ) The edge feature F is extracted according to the following formula E : F E =B_RCF(I LR ) Among them, H SF (·) represents the shallow feature extraction module, and B_RCF(·) is an edge detection auxiliary network that provides image edge information.

3. The super-resolution reconstruction method of infrared images according to claim 1, characterized in that: The deep feature extraction module includes K residual cross-window Transformer modules and a convolution layer with a convolution kernel size of 3×3; The deep features are extracted from the shallow features F0, and the edge features F0 are fused at the same time. E , and obtain the edge-enhanced deep feature F DEF Steps include: The deep features are extracted block by block from the K RCSTB modules, and the deep intermediate features F1, F2, ..., F K-1 and deep features F K ; The deep feature F K With edge feature F E After fusion, it passes through a 3×3 convolution layer and finally outputs F DEF .

4. The super-resolution reconstruction method of infrared images according to claim 3, characterized in that: The deep features are obtained according to the following formula: The final output F is obtained according to the following formula DEF : F DEF =H CONV (F K +αF E ) Among them, H RCSTBi (·) represents the i-th RCSTB, H CONV (·) is a convolutional layer.

5. The super-resolution reconstruction method of infrared images according to claim 1, characterized in that: The super-resolution infrared image I is reconstructed by fusing shallow and deep features. SR The steps include: I SR =H REC (F0+F DEF ) Among them, H REC (·) represents the reconstruction module, which consists of sub-pixel convolution and 3×3 convolution.

6. The super-resolution reconstruction method of infrared images according to claim 4, characterized in that: The method further comprises: The RCSTB model consists of L cross-window Transformer layers and a convolutional layer; Given the input feature of the i-th RCSTB module is F i,0 , first, extract the intermediate features F from L CSTL layers in sequence i,1 、F i,2 , ..., F i,L : Then, it passes through another convolution layer before the residual connection. Finally, the output feature of the i-th RCSTB is F i,out : in, is the jth CSTL layer in the i-th RCSTB module, is the convolutional layer in the i-th RCSTB module.

7. The super-resolution reconstruction method of infrared images according to claim 3, characterized in that: The CSTL layer includes a cross-window self-attention module, a multi-layer perceptron, two layers and two residual connections.

8. The super-resolution reconstruction method of infrared images according to claim 7, characterized in that: The cross-shaped window self-attention module performs self-attention in parallel by forming a cross-shaped window on the horizontal and vertical stripes. According to the multi-head self-attention mechanism, the input feature X∈R (H×W)×C It is first linearly projected into K heads, and then each head will perform local self-attention within a horizontal or vertical stripe.

Images