Super-resolution reconstruction method and system for infrared light image multi-scale feature fusion

Abstract

The invention discloses an infrared light image multi-scale feature fusion super-resolution reconstruction method and system, and the method comprises the steps: obtaining an infrared light image, carrying out the shallow feature extraction of the infrared light image according to a first convolution layer with a 3 * 3 convolution kernel, and obtaining a first feature result; the first feature result is input into a multi-scale fusion MSFRSTB module and an alternately-connected SA-CA module, the MSFRSTB module outputs a second feature result, and the SA-CA module outputs a space attention map and a channel attention map; inputting the first feature result, the second feature result, the space attention map and the channel attention map into a second convolution layer with a convolution kernel of 3 * 3, and inputting a convolution result of the second convolution layer into a sub-pixel convolution layer, and the sub-pixel convolution layer reconstructs the first feature result, the second feature result, the space attention map and the channel attention map and upsamples a super-resolution image of a corresponding size to obtain a target image.

Description

Technical Field

[0001] The present invention belongs to the technical field of infrared image processing, and in particular relates to a super-resolution reconstruction method and system for multi-scale feature fusion of infrared images. Background Art

[0002] With the deep integration of cutting-edge technologies such as cloud computing and big data, the power Internet of Things is gradually realizing the comprehensive sharing and intelligent analysis of power data, providing a solid foundation for the efficient operation of the power grid. However, in the process of promoting the construction of the power Internet of Things, an urgent problem to be solved is how to find a balance between cost and benefit, especially in the deployment of infrared monitoring sensors. Although infrared monitoring technology has shown great potential in power equipment fault diagnosis with its long-distance, non-contact, fast and accurate characteristics, low-precision sensors are often difficult to achieve ideal imaging effects due to limitations in installation costs, data transmission efficiency and storage capacity.

[0003] Infrared images, as the core output of infrared monitoring technology, have a resolution that is a key indicator of image detail performance. The level of spatial resolution directly determines the actual physical size represented by each pixel in the image, which in turn affects the richness of image information and the accuracy of diagnosis. Low-precision infrared sensors often produce blurred imaging results due to insufficient spatial resolution, making it difficult to provide clear fault indications for maintenance personnel, and also limiting the effective application of intelligent diagnostic algorithms.

[0004] Existing technologies have made some progress in the field of super-resolution reconstruction of infrared images, but they still face many challenges and defects. First, many super-resolution methods, especially those based on deep learning, have high computational complexity and require a lot of computing resources, which limits their application in edge devices or real-time processing scenarios. Secondly, deep learning methods are highly dependent on training data. For specific application scenarios such as infrared monitoring in the power Internet of Things, it is difficult to obtain sufficient high-resolution infrared image data. In addition, some models perform well on specific data sets, but lack generalization capabilities in new scenarios or conditions, resulting in reduced reconstruction quality. Color distortion problems may also cause color deviations in reconstructed images, affecting diagnostic accuracy. Summary of the invention

[0005] The present invention provides a super-resolution reconstruction method and system for multi-scale feature fusion of infrared images, which are used to solve the technical problem that color distortion may cause color deviation of reconstructed images and affect the accuracy of diagnosis.

[0006] In a first aspect, the present invention provides a super-resolution reconstruction method for infrared images by fusion of multi-scale features, comprising: Acquire an infrared image, perform shallow feature extraction on the infrared image according to the first convolution layer with a convolution kernel of 3×3, and obtain a first feature result ; The first feature result The second feature result is obtained by inputting the multi-scale fusion MSF_RSTB module and the alternating series SA-CA module respectively. , and the SA-CA module outputs a spatial attention map and a channel attention map; The first feature result The second characteristic result , the spatial attention map and the channel attention map are input to a second convolution layer with a convolution kernel of 3×3, and the convolution result of the second convolution layer is input to a sub-pixel convolution layer, and the sub-pixel convolution layer converts the first feature result The second characteristic result , the spatial attention map and the channel attention map are recombined and upsampled to a super-resolution image of corresponding size to obtain a target image , where the target image The expression is: ; In the formula, is the reconstruction function.

[0007] In a second aspect, the present invention provides a super-resolution reconstruction system for infrared image multi-scale feature fusion, comprising: The extraction module is configured to obtain an infrared light image, perform shallow feature extraction on the infrared light image according to a first convolution layer with a convolution kernel of 3×3, and obtain a first feature result ; An output module is configured to output the first feature result The second feature result is obtained by inputting the multi-scale fusion MSF_RSTB module and the alternating series SA-CA module respectively. , and the SA-CA module outputs a spatial attention map and a channel attention map; A fusion module is configured to combine the first feature result The second characteristic result , the spatial attention map and the channel attention map are input to a second convolution layer with a convolution kernel of 3×3, and the convolution result of the second convolution layer is input to a sub-pixel convolution layer, and the sub-pixel convolution layer converts the first feature result The second characteristic result , the spatial attention map and the channel attention map are recombined and upsampled to a super-resolution image of corresponding size to obtain a target image , where the target image The expression is: ; In the formula, is the reconstruction function.

[0008] According to a third aspect, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the super-resolution reconstruction method for multi-scale feature fusion of infrared images according to any embodiment of the present invention.

[0009] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the program instructions are executed by a processor, the processor executes the steps of the super-resolution reconstruction method for multi-scale feature fusion of infrared images of any embodiment of the present invention.

[0010] The super-resolution reconstruction method and system of infrared image multi-scale feature fusion of the present application significantly improves the imaging quality of low-precision infrared sensors by integrating advanced deep learning technology and multi-scale feature extraction strategy. The method utilizes the global modeling capability of the Swin Transformer layer and the local feature extraction capability of the Inception module, combined with the spatial attention (SA) module and the channel attention (CA) module, effectively integrates the local and global information of the image, enhances the model's attention to key areas, and thus restores richer details and texture information during the reconstruction process. In addition, the method uses the ML (MS-SSIM+L1) loss function to balance image brightness, color preservation and edge detail restoration, generating more natural and accurate super-resolution images. This method not only improves the accuracy of power equipment fault diagnosis, but also realizes the efficient operation of the model through lightweight design, making it more suitable for application in resource-constrained environments. Overall, the method shows significant advantages in improving image resolution, enhancing detail performance, improving visual effects, and improving diagnostic accuracy, providing strong technical support for infrared monitoring technology in the power Internet of Things. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0012] Figure 1 A schematic diagram of a framework flow of a super-resolution reconstruction process of infrared image multi-scale feature fusion provided by an embodiment of the present invention; Figure 2 A flowchart of a super-resolution reconstruction method for infrared image multi-scale feature fusion provided by an embodiment of the present invention; Figure 3 A schematic diagram of the structure of the MSF_RSTB module provided in one embodiment of the present invention; Figure 4 A schematic diagram of the structure of a Swin Transformer layer provided in one embodiment of the present invention; Figure 5 A schematic diagram of the structure of an Inception module provided in one embodiment of the present invention; Figure 6 A structural block diagram of a super-resolution reconstruction system for multi-scale feature fusion of infrared images provided by an embodiment of the present invention; Figure 7 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0013] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0014] See also Figure 2 , which shows a flow chart of a super-resolution reconstruction method of infrared image multi-scale feature fusion in the present application.

[0015] like Figure 2 As shown, the super-resolution reconstruction method of infrared image multi-scale feature fusion specifically includes the following steps: Step S101, obtaining an infrared image, performing shallow feature extraction on the infrared image according to a first convolution layer with a convolution kernel of 3×3, and obtaining a first feature result .

[0016] Step S102: The first feature result The second feature result is obtained by inputting the multi-scale fusion MSF_RSTB module and the alternating series SA-CA module respectively. , and the SA-CA module outputs the spatial attention map and the channel attention map.

[0017] In this step, in order to fully retain the shallow and deep information of the features, multiple multi-scale fusion MSF_RSTB modules are superimposed to extract the global features of the image. The structure of the MSF_RSTB module is as follows: Figure 3 shown.

[0018] The MSF_RSTB module is a residual block consisting of 6 Swin Transformer layers (STL) and 1 Inception module. The extracted shallow features are transferred to the MSF_RSTB module for deep feature extraction. The Swin Transformer layer mainly divides the feature map into multiple non-overlapping windows through sliding window multi-head self-attention (SW-MSA). Through the sliding method of the window, the adjacent windows interact with each other, thereby achieving the ability of global modeling. Add an Inception module after the last Swin Transformer layer.

[0019] The MSF_RSTB modules are a first MSF_RSTB module, a second MSF_RSTB module, a third MSF_RSTB module, and a fourth MSF_RSTB module; The first feature result The second feature result is obtained by inputting the multi-scale fusion MSF_RSTB module and the alternating series SA-CA module respectively. include: The first feature result Input to the first MSF_RSTB module, the first MSF_RSTB module outputs the first intermediate feature ; The first intermediate feature Input to the second MSF_RSTB module, the second MSF_RSTB module outputs the second intermediate feature ; The second intermediate feature Input to the third MSF_RSTB module, the third MSF_RSTB module outputs the third intermediate feature , The third intermediate feature Input to the fourth MSF_RSTB module, the fourth MSF_RSTB module outputs the second feature result .

[0020] It should be noted that the MSF_RSTB module includes the Swin Transformer layer and the Inception module; The Swin Transformer layer divides the feature map into multiple non-overlapping windows through sliding window multi-head self-attention, and enables adjacent windows to interact through the window sliding method, thereby performing global modeling, and adding an Inception module after the last Swin Transformer layer. The Inception module is used to obtain edge detail information.

[0021] The internal structure of the Swin Transformer layer (STL) is as follows Figure 4 As shown in the figure, the Swin Transformer layer divides the feature map into multiple non-overlapping windows through sliding window multi-head self-attention. Through the sliding method of the window, adjacent windows interact with each other, so as to perform global modeling including: Given size The Swin Transformer layer first divides the input into non-overlapping The local window reshapes the input into a size of Features, where the total number of windows is , is the height of the feature map, is the width of the feature map, is the number of channels of the feature map, is the window size of the SwinTransformer layer; Then calculate the standard self-attention of each window separately, for a local window feature , query matrix , key matrix Sum Matrix The calculations are: ; ; ; In the formula, To transform the local window feature The projection matrix mapped to the query matrix, To transform the local window feature The projection matrix mapped to the key matrix, To transform the local window feature The projection matrix mapped to the value matrix; Attention Matrix It is calculated by the self-attention mechanism in the local window and is expressed as: ; In the formula, is a mathematical function used to convert a set of real numbers into a set of probability values. is a learnable relative position encoding, is the dimension of multi-head attention.

[0022] Secondly, the multi-layer perceptron (MLP) has two fully connected GELU nonlinear layers for further feature transformation. LayerNorm (LN, layer normalization module) is added before MSA and MLP, and both modules use residual connections. The operation formula is: ; ; The Swin Transformer layer uses window-based multi-head self-attention with regular and shifted window segmentation configurations to achieve cross-window connections, which allows non-overlapping windows to exchange information and increase the receptive field.

[0023] The structure of the Inception (multi-scale feature extraction) module is as follows Figure 5 As shown, the Inception module includes a first parallel convolution circuit, a second parallel convolution circuit, a third parallel convolution circuit, and a fourth parallel convolution circuit; The first parallel convolution circuit includes a convolution layer with a convolution kernel of 1×1; The second parallel convolution circuit includes a convolution layer with a convolution kernel of 1×1, a convolution layer with a convolution kernel of 3×3, and a convolution layer with a convolution kernel of 5×5, which are connected in sequence; The third parallel convolution circuit includes a convolution layer with a convolution kernel of 1×1 and a convolution layer with a convolution kernel of 5×5, which are connected in sequence; The fourth parallel convolution circuit uses average pooling and is followed by a convolution layer with a convolution kernel of 1×1.

[0024] Further, the SA-CA module includes a first spatial attention module, a first channel attention module, a second spatial attention module, and a second channel attention module connected in series in sequence; The first spatial attention module is connected to the first convolutional layer and the first MSF_RSTB module respectively, the first channel attention module is connected to the second MSF_RSTB module, the second spatial attention module is connected to the third MSF_RSTB module, and the second channel attention module is connected to the fourth MSF_RSTB module; Among them, the spatial attention calculation formulas of the first spatial attention module and the second spatial attention module are: ; In the formula, is the output of the spatial attention module, To perform the maximum pooling operation on the feature map, The feature map is obtained after the average pooling operation is performed on the feature map. is the feature map after the maximum pooling operation, The convolution kernel size is Convolution operation; The spatial attention calculation formula of the first channel attention module and the second channel attention module is: ; In the formula, is the output of the channel attention module, is the sigmoid function, This is the result of processing the feature map through a multi-layer perceptron after average pooling. is the feature map obtained by performing an average pooling operation on the feature map. is the feature map, This is the result of processing the feature map through a multi-layer perceptron after performing maximum pooling. is the feature map obtained after performing the maximum pooling operation on the feature map. is the weight matrix from the hidden layer to the output layer, is the weight matrix from the input layer to the hidden layer, is the vector obtained after global average pooling of the feature map, is the vector obtained after performing global maximum pooling on the feature map.

[0025] Getting the target image The objective loss function in the process is: ; ; In the formula, is the target loss function, is the weight coefficient, is the MS-SSIM loss function, is a Gaussian function used for Smoothing is the standard L1 loss function, To reconstruct a high-resolution image, It is a real high-resolution infrared image.

[0026] Step S103: The first feature result The second characteristic result , the spatial attention map and the channel attention map are input to a second convolution layer with a convolution kernel of 3×3, and the convolution result of the second convolution layer is input to a sub-pixel convolution layer, and the sub-pixel convolution layer converts the first feature result The second characteristic result , the spatial attention map and the channel attention map are recombined and upsampled to a super-resolution image of corresponding size to obtain a target image .

[0027] In this step, the target image The expression is: ; In the formula, is the reconstruction function.

[0028] In summary, the method of this application, See also Figure 6 , which shows a structural block diagram of a super-resolution reconstruction system for infrared image multi-scale feature fusion of the present application.

[0029] like Figure 6 As shown, the super-resolution reconstruction system 200 includes an extraction module 210 , an output module 220 and a fusion module 230 .

[0030] The extraction module 210 is configured to obtain an infrared image, perform shallow feature extraction on the infrared image according to a first convolution layer with a convolution kernel of 3×3, and obtain a first feature result. ; Output module 220, configured to output the first feature result The second feature result is obtained by inputting the multi-scale fusion MSF_RSTB module and the alternating series SA-CA module respectively. , and the SA-CA module outputs a spatial attention map and a channel attention map; Fusion module 230, configured to combine the first feature result The second characteristic result , the spatial attention map and the channel attention map are input to a second convolution layer with a convolution kernel of 3×3, and the convolution result of the second convolution layer is input to a sub-pixel convolution layer, and the sub-pixel convolution layer converts the first feature result The second characteristic result , the spatial attention map and the channel attention map are recombined and upsampled to a super-resolution image of corresponding size to obtain a target image .

[0031] Among them, the target image The expression is: ; In the formula, is the reconstruction function.

[0032] It should be understood that Figure 6 Modules and references documented in Figure 2 Therefore, the operations and features described above for the method and the corresponding technical effects are also applicable to Figure 6 The modules in it will not be described in detail here.

[0033] In some other embodiments, the embodiments of the present invention further provide a computer-readable storage medium having a computer program stored thereon, wherein when the program instructions are executed by a processor, the processor executes the super-resolution reconstruction method of multi-scale feature fusion of infrared images in any of the above method embodiments; As an implementation mode, the computer-readable storage medium of the present invention stores computer-executable instructions, and the computer-executable instructions are configured as follows: Acquire an infrared image, perform shallow feature extraction on the infrared image according to the first convolution layer with a convolution kernel of 3×3, and obtain a first feature result ; The first feature result The second feature result is obtained by inputting the multi-scale fusion MSF_RSTB module and the alternating series SA-CA module respectively. , and the SA-CA module outputs a spatial attention map and a channel attention map; The first feature result The second characteristic result , the spatial attention map and the channel attention map are input to a second convolution layer with a convolution kernel of 3×3, and the convolution result of the second convolution layer is input to a sub-pixel convolution layer, and the sub-pixel convolution layer converts the first feature result The second characteristic result , the spatial attention map and the channel attention map are recombined and upsampled to a super-resolution image of corresponding size to obtain a target image , where the target image The expression is: ; In the formula, is the reconstruction function.

[0034] The computer-readable storage medium may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required by at least one function; the data storage area may store data created according to the use of the super-resolution reconstruction system for multi-scale feature fusion of infrared light images, etc. In addition, the computer-readable storage medium may include a high-speed random access memory, and may also include a memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the computer-readable storage medium may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the super-resolution reconstruction system for multi-scale feature fusion of infrared light images via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0035] Figure 7 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Figure 7 As shown, the device includes: a processor 310 and a memory 320. The electronic device may also include: an input device 330 and an output device 340. The processor 310, the memory 320, the input device 330 and the output device 340 may be connected via a bus or other means. Figure 7 The example of the connection via bus is taken. The memory 320 is the above-mentioned computer-readable storage medium. The processor 310 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 320, that is, the super-resolution reconstruction method of infrared light image multi-scale feature fusion of the above-mentioned method embodiment is realized. The input device 330 can receive input digital or character information, and generate key signal input related to user settings and function control of the super-resolution reconstruction system of infrared light image multi-scale feature fusion. The output device 340 may include display devices such as display screens.

[0036] The electronic device can execute the method provided by the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the method provided by the embodiment of the present invention.

[0037] As an implementation mode, the electronic device is applied to a super-resolution reconstruction system for multi-scale feature fusion of infrared images, and is used for a client, and includes: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can: Acquire an infrared image, perform shallow feature extraction on the infrared image according to the first convolution layer with a convolution kernel of 3×3, and obtain a first feature result ; The first feature result The second feature result is obtained by inputting the multi-scale fusion MSF_RSTB module and the alternating series SA-CA module respectively. , and the SA-CA module outputs a spatial attention map and a channel attention map; The first feature result The second characteristic result , the spatial attention map and the channel attention map are input to a second convolution layer with a convolution kernel of 3×3, and the convolution result of the second convolution layer is input to a sub-pixel convolution layer, and the sub-pixel convolution layer converts the first feature result The second characteristic result , the spatial attention map and the channel attention map are recombined and upsampled to a super-resolution image of corresponding size to obtain a target image , where the target image The expression is: ; In the formula, is the reconstruction function.

[0038] Through the description of the above implementation modes, those skilled in the art can clearly understand that each implementation mode can be implemented by means of software plus a necessary general hardware platform, or of course by hardware. Based on such an understanding, the above technical solution can essentially or in other words be embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0039] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A super-resolution reconstruction method for infrared images by multi-scale feature fusion, characterized in that: include: Acquire an infrared image, perform shallow feature extraction on the infrared image according to the first convolution layer with a convolution kernel of 3×3, and obtain a first feature result ; The first feature result The second feature result is obtained by inputting the multi-scale fusion MSF_RSTB module and the alternating series SA-CA module respectively. , and the SA-CA module outputs a spatial attention map and a channel attention map; The first feature result The second characteristic result , the spatial attention map and the channel attention map are input to a second convolution layer with a convolution kernel of 3×3, and the convolution result of the second convolution layer is input to a sub-pixel convolution layer, and the sub-pixel convolution layer converts the first feature result The second characteristic result , the spatial attention map and the channel attention map are recombined and upsampled to a super-resolution image of corresponding size to obtain a target image , where the target image The expression is: ； In the formula, is the reconstruction function.

2. The super-resolution reconstruction method of infrared image multi-scale feature fusion according to claim 1, characterized in that: The MSF_RSTB modules are a first MSF_RSTB module, a second MSF_RSTB module, a third MSF_RSTB module and a fourth MSF_RSTB module; The first feature result The second feature result is obtained by inputting the multi-scale fusion MSF_RSTB module and the alternating series SA-CA module respectively. include: The first feature result Input to the first MSF_RSTB module, the first MSF_RSTB module outputs the first intermediate feature ; The first intermediate feature Input to the second MSF_RSTB module, the second MSF_RSTB module outputs the second intermediate feature ; The second intermediate feature Input to the third MSF_RSTB module, the third MSF_RSTB module outputs the third intermediate feature , The third intermediate feature Input to the fourth MSF_RSTB module, the fourth MSF_RSTB module outputs the second feature result .

3. The super-resolution reconstruction method of infrared image multi-scale feature fusion according to claim 1, characterized in that: The MSF_RSTB module includes a Swin Transformer layer and an Inception module; The Swin Transformer layer divides the feature map into multiple non-overlapping windows through sliding window multi-head self-attention, and enables adjacent windows to interact through the window sliding method, thereby performing global modeling, and adding an Inception module after the last Swin Transformer layer. The Inception module is used to obtain edge detail information.

4. The super-resolution reconstruction method of infrared image multi-scale feature fusion according to claim 3 is characterized in that: The Swin Transformer layer divides the feature map into multiple non-overlapping windows through sliding window multi-head self-attention, and enables adjacent windows to interact with each other through the sliding method of the window, thereby performing global modeling including: Given size The Swin Transformer layer first divides the input into non-overlapping The local window reshapes the input into a size of Features, where the total number of windows is , is the height of the feature map, is the width of the feature map, is the number of channels of the feature map, is the window size of the SwinTransformer layer; Then calculate the standard self-attention of each window separately, for a local window feature , query matrix , key matrix Sum Matrix The calculations are: ； ； ； In the formula, To transform the local window feature The projection matrix mapped to the query matrix, To transform the local window feature The projection matrix mapped to the key matrix, To transform the local window feature The projection matrix mapped to the value matrix; Attention Matrix It is calculated by the self-attention mechanism in the local window and is expressed as: ； In the formula, is a mathematical function used to convert a set of real numbers into a set of probability values. is a learnable relative position encoding, is the dimension of multi-head attention.

5. The super-resolution reconstruction method of infrared image multi-scale feature fusion according to claim 3, characterized in that: The Inception module includes a first parallel convolution circuit, a second parallel convolution circuit, a third parallel convolution circuit and a fourth parallel convolution circuit; The first parallel convolution circuit includes a convolution layer with a convolution kernel of 1×1; The second parallel convolution circuit includes a convolution layer with a convolution kernel of 1×1, a convolution layer with a convolution kernel of 3×3, and a convolution layer with a convolution kernel of 5×5, which are connected in sequence; The third parallel convolution circuit includes a convolution layer with a convolution kernel of 1×1 and a convolution layer with a convolution kernel of 5×5, which are connected in sequence; The fourth parallel convolution circuit uses average pooling and is followed by a convolution layer with a convolution kernel of 1×1.

6. The super-resolution reconstruction method of infrared image multi-scale feature fusion according to claim 1, characterized in that: The SA-CA module includes a first spatial attention module, a first channel attention module, a second spatial attention module and a second channel attention module connected in series; The first spatial attention module is connected to the first convolutional layer and the first MSF_RSTB module respectively, the first channel attention module is connected to the second MSF_RSTB module, the second spatial attention module is connected to the third MSF_RSTB module, and the second channel attention module is connected to the fourth MSF_RSTB module; Among them, the spatial attention calculation formulas of the first spatial attention module and the second spatial attention module are: ； In the formula, is the output of the spatial attention module, To perform the maximum pooling operation on the feature map, The feature map is obtained after the average pooling operation is performed on the feature map. is the feature map after the maximum pooling operation, The convolution kernel size is Convolution operation; The spatial attention calculation formula of the first channel attention module and the second channel attention module is: ； In the formula, is the output of the channel attention module, is the sigmoid function, This is the result of processing the feature map through a multi-layer perceptron after average pooling. is the feature map obtained by performing an average pooling operation on the feature map. is the feature map, This is the result of processing the feature map through a multi-layer perceptron after performing maximum pooling. is the feature map obtained after performing the maximum pooling operation on the feature map. is the weight matrix from the hidden layer to the output layer, is the weight matrix from the input layer to the hidden layer, is the vector obtained after global average pooling of the feature map, is the vector obtained after performing global maximum pooling on the feature map.

7. The super-resolution reconstruction method of infrared image multi-scale feature fusion according to claim 1, characterized in that: in, Getting the target image The objective loss function in the process is: ； ； In the formula, is the target loss function, is the weight coefficient, is the MS-SSIM loss function, is a Gaussian function used for Smoothing is the standard L1 loss function, To reconstruct a high-resolution image, It is a real high-resolution infrared image.

8. A super-resolution reconstruction system for infrared image multi-scale feature fusion, characterized in that: include: The extraction module is configured to obtain an infrared light image, perform shallow feature extraction on the infrared light image according to a first convolution layer with a convolution kernel of 3×3, and obtain a first feature result ; An output module is configured to output the first feature result The second feature result is obtained by inputting the multi-scale fusion MSF_RSTB module and the alternating series SA-CA module respectively. , and the SA-CA module outputs a spatial attention map and a channel attention map; A fusion module is configured to combine the first feature result The second characteristic result , the spatial attention map and the channel attention map are input to a second convolution layer with a convolution kernel of 3×3, and the convolution result of the second convolution layer is input to a sub-pixel convolution layer, and the sub-pixel convolution layer converts the first feature result The second characteristic result , the spatial attention map and the channel attention map are recombined and upsampled to a super-resolution image of corresponding size to obtain a target image , where the target image The expression is: ； In the formula, is the reconstruction function.

9. An electronic device, characterized in that: include: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images