A remote sensing image super-resolution reconstruction method based on feature information distillation network

By using multi-scale information distillation attention blocks of feature information distillation networks and enhanced channel contrast attention mechanisms, the problems of high model complexity and computational resource requirements in remote sensing image super-resolution reconstruction are solved, achieving efficient remote sensing image reconstruction results.

CN115601236BActive Publication Date: 2025-12-30XIAMEN UNIV
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202211181821.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-09-27
Publication Date
2025-12-30
Estimated Expiration
2042-09-27

AI Technical Summary

Technical Problem

Existing deep convolutional neural networks suffer from problems such as excessively large model parameters, high computational resource requirements, overly smoothed edges in generated images, and unstable training in remote sensing image super-resolution reconstruction, making them difficult to apply effectively on resource-constrained hardware systems.

Method used

A super-resolution reconstruction method based on feature information distillation network is adopted. By constructing the multi-scale information distillation attention block (MSIDAB) and enhanced channel contrast attention mechanism (ECCAM) sub-modules, combined with shallow feature extraction module and upsampling module, the high-frequency and low-frequency information of remote sensing images can be effectively restored and fused, reducing model parameters and improving the lightweighting and reconstruction quality of the model.

Benefits of technology

While maintaining high-resolution reconstruction quality, the model complexity was reduced, the semantic feature extraction capability and high-frequency information perception capability of remote sensing images were improved, the problems of excessively large model parameters and high computational resource requirements were solved, and effective application on resource-constrained hardware systems was realized.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115601236B_ABST
    Figure CN115601236B_ABST
Patent Text Reader

Abstract

The application relates to a remote sensing image super-resolution reconstruction method based on a feature information distillation network, and relates to digital image processing. LR The input image is I SR , and a multi-scale information distillation attention network is used to output a super-resolution reconstruction result I The network structure comprises a shallow feature extraction module, a deep feature extraction module and an up-sampling module. The shallow feature extraction module is used for low-frequency information recovery, and the deep feature extraction module is used for high-frequency information recovery. The multi-scale information distillation attention block MSIDAB realizes feature transmission and guarantees information integrity. An MSID sub-module is added to each multi-scale information distillation attention block, so that the network can gradually extract and fuse multi-scale semantic features while reducing model parameters. An ECCAM attention mechanism is introduced, which enhances the perception ability of high-frequency region information in the remote sensing image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to digital image processing, and more particularly to a method for super-resolution reconstruction of remote sensing images based on feature information distillation networks. Background Technology

[0002] With the rapid development of satellite and drone technologies, a large number of remote sensing images are being acquired and used to solve practical problems, such as intelligent transportation planning, geological resource exploration, and natural disaster monitoring. However, due to factors such as ultra-long-distance imaging, camera hardware transmission capabilities, and weather interference, the clarity of the acquired remote sensing images still has significant room for improvement. Considering the high research costs and long hardware iteration development cycles for improving imaging sensors at the physical level, achieving super-resolution (SR) reconstruction of remote sensing images at the algorithmic level is becoming one of the current research hotspots.

[0003] In recent years, with the rapid development of big data technology and computing power, more and more scholars have begun to focus on using deep convolutional neural networks (DCNNs) to solve the problem of image super-resolution reconstruction. In 2016, SRCNN (Super-Resolution Convolutional Neural Network) achieved excellent reconstruction results with only a simple 3-layer network. Subsequently, inspired by residual learning strategies, the deep network VDSR (VeryDeep SR Network) constructed a 20-layer network structure, effectively solving the problem of difficult convergence during training when the network is deep by learning the high-frequency residuals between high-resolution (HR) and low-resolution (LR) images. Thanks to the success of VDSR in image super-resolution reconstruction tasks, more and more deep networks have been proposed by researchers, such as the 69-layer deep network EDSR (Enhanced Deep Super-Resolution Network) and the more than 400-layer deep network RCAN (Residual Channel Attention Network), which improve the quality of image reconstruction.

[0004] While these deep networks significantly improve image reconstruction results, they also suffer from problems such as excessively large model parameters and the need for expensive computational resources, thus limiting their practical application on resource-constrained hardware systems. Compared to natural images, remote sensing images have more complex backgrounds and a wider variety of ground features. The texture structure and high-frequency details between ground objects are more difficult to restore, requiring algorithms with stronger semantic feature extraction capabilities and high-frequency information perception capabilities. Although some algorithms based on deep convolutional neural networks have been proposed for super-resolution reconstruction of remote sensing images, such as the lightweight generative model-based super-resolution reconstruction method for remote sensing images disclosed in document CN113538234A, which is based on Generative Adversarial Networks (GANs) and utilizes the advantages of deep separable convolutions and residual networks to achieve a lightweight network design, GAN-based methods are prone to problems such as overly smoothed edges in the generated remote sensing images and instability during the training process.

[0005] How to provide a superior method for super-resolution reconstruction of remote sensing images is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to address the aforementioned technical problems in existing technologies by providing a remote sensing image super-resolution reconstruction method based on a feature information distillation network. The super-resolution reconstruction network structure is reasonable and effective, achieving a good balance between super-resolution accuracy and model complexity. A multi-scale information distillation (MSID) submodule is added to each MSIDAB block, enabling the network to gradually extract and fuse multi-scale semantic features while reducing model parameters and ensuring lightweight model size. An enhanced channel contrast attention mechanism (ECCAM) submodule is incorporated into the MISDAB block, allowing the network to more effectively focus on important feature information in the image and suppress useless information.

[0007] This invention includes the following steps:

[0008] 1) Construct a super-resolution reconstruction network based on feature information distillation network, which includes a shallow feature extraction module, a deep feature extraction module, and an upsampling module in sequence;

[0009] 2) Let the input low-resolution image be I. LR The shallow feature extraction module outputs shallow feature F0.

[0010] 3) Shallow features F0 are processed by the deep feature extraction module to output deep fusion features F fused ;

[0011] 4) Let the shallow feature F0 and the deep fusion feature F rusedThe images are fused and then upsampled to obtain the final super-resolution reconstruction result of the remote sensing image. SR .

[0012] In step 1), the shallow feature extraction module is used for the restoration of low-frequency information;

[0013] The deep feature extraction module consists of M multi-scale information distillation attention blocks (MSIDABs). The deep feature extraction module is used to restore high-frequency information. The M multi-scale information distillation attention blocks realize the transfer between features, prevent shallow features from being weakened or lost during information transfer, and ensure the integrity of information.

[0014] Each multi-scale information distillation attention block (MSIDAB block) contains a multi-scale information distillation (MSID) submodule, an enhanced contrast-aware channel attention mechanism (ECCAM) submodule, and a 1×1 convolutional layer. The MSID submodule is used to gradually extract and fuse multi-scale semantic features while reducing model parameters. The ECCAM submodule is used to enhance the perception of high-frequency region information in remote sensing images. The 1×1 convolutional layer is used to compress the number of channels in the output feature map.

[0015] The upsampling module is used to project the feature map from a low-resolution space to a high-resolution space.

[0016] In step 2), the input low-resolution image is set to I. LR The formula for outputting the shallow feature F0 after shallow feature extraction is as follows:

[0017] F0 = H SF (I LR (1)

[0018] Among them, H SF (·) represents a convolution operation with a kernel size of 3×3, and F0 represents the extracted shallow feature image.

[0019] In step 3), the shallow feature F0 is processed by the deep feature extraction module to output the deep fusion feature F. fused The deep feature extraction module consists of M MSIDAB blocks, each containing an MSID submodule, an ECCAM submodule, and a 1×1 convolutional layer; its specific steps are as follows:

[0020] (1) The calculation formula for the MSIDAB block is as follows:

[0021] F b,n =H MSIDAB,n (F b,n-1 )=H MSIDAB,n (H MSIDAB,n-1 (…H MSIDAB,1 (F0)···)) (2)

[0022] Among them, H MSIDAB,n (·) represents the nth MSIDAB block operation; F b,n-1 and F b,n These are the input and output of the nth MSIDAB block, respectively;

[0023] (2) The calculation process of the MSID submodule is as follows:

[0024] definition Using the input features of the MSIDAB block, the following features are obtained through multi-scale feature extraction and channel separation mechanisms:

[0025] [F refined-1-i F remaining-1-i ] = Split1(Conv 1-i (F in (3)

[0026] [F refined-2-i F remaining-2-i ] = Split2(Conv 2-i (F remaining-1-i (4)

[0027] [F refined-3-i F remaining-3-i ] = Split3(Conv 3-i (F remaining-2-i (5)

[0028] F refined-4-3 =Conv 4-3 (Sum(F remaining-3-i (6)

[0029] Among them, Conv k-i (·) represents the k-th convolutional layer of size i×i in the MSIDAB block (followed by the LeakyReLU activation function); Split k (·) represents the k-th feature channel separation operation of the MSIDAB block; F refined-j-i and F remaining-j-iThese are the local distillation and preservation features of the j-th branch of the MSIDAB block at the convolutional layer with kernel size i×i, respectively, where the preservation feature is used for the calculation of the next unit; Sum(·) is the feature map summation operation; specifically, the kernel size i×i on each convolutional layer is set to 3×3, 5×5, and 7×7, respectively; the distillations of each multi-scale convolutional layer are fused, and the fused features are cascaded, calculated as follows:

[0030] F MSID =Concat(Sum(F) refined-1-i Sum(F) refined-2-i Sum(F) refined-3-i ), F refined-4-3 ) (7)

[0032] Where Concat(·) and Sum(·) represent the feature map concatenation and summation operations, respectively; F MSID The output characteristics of the MSID submodule within the MSIDAB block;

[0033] (3) The calculation process of the ECCAM submodule is as follows:

[0034] definition For a feature map with a total of C channels, where the size of a single channel is H×W; for the feature map of the k-th channel... Calculate global average pooling (GAP) to obtain the corresponding channel feature descriptors. The calculation is as follows:

[0035]

[0036] in, For F MSID The value at position (i, j) in the k-th feature map. For F MSID The corresponding channel feature descriptors; considering the spatial diversity of remote sensing images, F is analyzed along the channel dimension. MSID Calculate average pooling (AP) to obtain the corresponding spatial feature descriptor D. Spatial The calculation is as follows:

[0037]

[0038] Among them, D Spatial (i, j) is The compressed spatial feature description value at position (i, j); combining the channel and spatial feature descriptions, the enhanced channel contrast information value is calculated as follows:

[0039]

[0040] in, This represents the contrast information value of the k-th enhanced channel; further, a multilayer perceptron structure is used to enhance the feature weights of high-frequency information, calculated as follows:

[0041]

[0042] Among them, f σ [·] and ReLU(·) represent the Sigmoid function and the ReLU activation function, respectively; W D (·) represents a convolution operation with a channel downsampling rate of f and a kernel size of 1×1; W U (·) represents a convolution operation with a channel upsampling rate of r and a kernel size of 1×1; ... represents a feature map channel multiplication operation; This is the output feature map after passing through the ECCAM submodule.

[0043] (4) The convolutional layer with a kernel size of 1×1 is used to compress the number of output feature map channels. The output of the MSIDAB block is calculated as follows:

[0044] F out =H compress (F ECCAM +F MSID )+F in (12)

[0045] Among them, F out H is the final output of the MSIDAB block. compress (·) represents a convolution operation with a kernel size of 1×1, and + represents a feature map addition operation.

[0046] In step 3), the formula for obtaining the deep fusion features is as follows:

[0047] F fused =H fuse (Concat(F b,1 F b,n F b,M (13)

[0048] Wherein, Concat(·) is the feature map concatenation operation, which concatenates and stitches together the outputs of M MSIDAB; F fused For deep fusion features; H fuse(·) Specific operation: First, use a convolutional layer with a kernel size of 1×1 to reduce the dimension of the feature map, then use the LeakyReLU function to reduce redundant parameters, and finally use a convolutional layer with a kernel size of 3×3 to further refine the deep fusion features.

[0049] In step 4), the final remote sensing image super-resolution reconstruction result I is obtained. SR The formula is as follows:

[0050] I SR =H subpixel (H A (F fused +F0)) (14)

[0051] Among them, H A (·) represents a convolutional layer with a kernel size of 3×3, H subpixel (·) represents the upsampling operation of the sub-pixel convolutional layer, I SR This is the final super-resolution reconstructed image.

[0052] Specifically, the entire network is optimized using a loss function, which is defined as follows:

[0053] L(Θ)=||I SR -I HR ||1 (15)

[0054] Among them, I HR Θ represents real high-resolution remote sensing image samples; Θ represents the model parameters of the entire network; ||·||1 represents the L1 loss function.

[0055] Compared with the prior art, the advantages of the present invention are as follows:

[0056] 1. The super-resolution reconstruction network based on feature information distillation proposed in this invention consists of a shallow feature extraction module, a deep feature extraction module, and an upsampling module. Addressing the issue that low-frequency information is easily recovered while high-frequency information is difficult to recover in images, the shallow feature extraction module is used for low-frequency information recovery, and the deep feature extraction module is used for high-frequency information recovery. To prevent shallow features from being weakened or lost during information transmission, this invention proposes a multi-scale information distillation attention block (MSIDAB block) to achieve feature transfer and ensure information integrity. The proposed network structure is reasonable and effective, achieving a good balance between super-resolution accuracy and model complexity.

[0057] 2. To obtain richer semantic information from remote sensing images, this invention adds a multi-scale information distillation (MSID) submodule to each MSIDAB block. This submodule integrates the feature information distillation strategy and the idea of ​​multi-scale feature extraction, so that the network can gradually extract and fuse multi-scale semantic features while reducing model parameters and ensuring lightweight model size.

[0058] 3. This invention incorporates an Enhanced Channel Contrast Attention Mechanism (ECCAM) submodule into the MISDAB block. Traditional channel attention mechanisms only use global mean pooling to obtain feature statistics, which is insufficient for perceiving high-frequency regions (such as structure, texture, and edges). ECCAM uses the channel and spatial contrast values ​​of the image to statistically analyze feature maps, enabling the network to more effectively focus on important features in the image and suppress useless information. Attached Figure Description

[0059] Figure 1 This is a flowchart of an embodiment of the present invention.

[0060] Figure 2 This is a schematic diagram of the structure of the multi-scale information distillation attention block (MSIDAB block) in an embodiment of the present invention.

[0061] Figure 3 This is a schematic diagram of the enhanced channel contrast attention mechanism (ECCAM) submodule structure in an embodiment of the present invention. Detailed Implementation

[0062] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] Figure 1This invention provides an overall flowchart of remote sensing image super-resolution reconstruction based on a feature information distillation network for embodiments of the present invention. The embodiments of the present invention first construct a super-resolution reconstruction network based on a feature information distillation network. This network structure sequentially includes a shallow feature extraction module, a deep feature extraction module, and an upsampling module. The shallow feature extraction module is used for the restoration of low-frequency information. The deep feature extraction module consists of M multi-scale information distillation attention blocks (MSIDABs). The deep feature extraction module is used for the restoration of high-frequency information. The M multi-scale information distillation attention blocks realize the transfer between features, preventing shallow features from being weakened or lost during information transfer, and ensuring the integrity of the information. Each multi-scale information distillation attention block (MSIDAB block) includes a multi-scale information distillation (MSID) sub-module and an enhanced contrast-aware channel attention mechanism. The system consists of an attention mechanism (ECCAM) submodule and a 1×1 convolutional layer. The MSID submodule is used to progressively extract and fuse multi-scale semantic features while reducing model parameters. The ECCAM submodule is used to enhance the perception of high-frequency region information in remote sensing images. The 1×1 convolutional layer is used to compress the number of channels in the output feature map. The upsampling module is used to project the feature map from a low-resolution space to a high-resolution space. In this embodiment of the invention, the input image is I. LR The final super-resolution reconstruction result of the remote sensing image is obtained through the shallow feature extraction module, the deep feature extraction module, and the upsampling module. SR .

[0064] Specifically, the formula for the shallow feature extraction module is as follows:

[0065] F0 = H SF (I LR (1)

[0066] Among them, H SF (·) represents a convolution operation with a kernel size of 3×3, and F0 represents the extracted shallow feature image.

[0067] Specifically, the deep feature extraction module consists of M multi-scale information distillation attention blocks (MSIDAB). The calculation formula is as follows:

[0068] Fb,n =H MSIDAB,n (F b,n-1 )=H MSIDAB,n (H MSIDAB,n-1 (···H MSIDAB,1 (F0)···)) (2)

[0069] Among them, H MSIDAB,n (·) represents the nth MSIDAB block operation. F b,n-1 and F b,n These are the input and output of the nth MSIDAB block, respectively.

[0070] Specifically, each MSIDAB block contains a multi-scale information distillation (MSID) submodule, an enhanced contrast-aware channel attention mechanism (ECCAM) submodule, and a convolutional layer with a kernel size of 1×1. Figure 2 This is a schematic diagram of the multi-scale information distillation attention (MSIDAB block) structure in an embodiment of the present invention.

[0071] Specifically, the MSID submodule is used to gradually extract and fuse multi-scale semantic features while reducing model parameters.

[0072] Specifically, the calculation process for the MSID submodule is as follows:

[0073] Specifically, define Using the input features of the MSIDAB block, the following features are obtained through multi-scale feature extraction and channel separation mechanisms:

[0074] [F refined-1-i F remaining-1-i ] = Split1(Conv 1-i (F in (3)

[0075] [F refined-2-i F remaining-2-i ] = Split2(Conv 2-i (F remaining-1-i (4)

[0076] [F refined-3-i F remaining-3-i ] = Split3(Conv 3-i (F remaining-2-i (5)

[0077] F refined-4-3 =Conv 4-3(Sum(F remaining-3-i (6)

[0078] Among them, Conv k-i (·) represents the k-th convolutional layer of size i×i in the MSIDAB block (followed by the LeakyReLU activation function); Split k (·) represents the k-th feature channel separation operation of the MSIDAB block; F refined-j-i and F remaining-j-i These are the local distillation and preservation features of the j-th branch of the MSIDAB block at the convolutional layer with kernel size i×i, respectively, where the preservation feature is used for the calculation of the next unit; Sum(·) is the feature map summation operation; specifically, the kernel size i×i on each convolutional layer is set to 3×3, 5×5, and 7×7, respectively; the distillations of each multi-scale convolutional layer are fused, and the fused features are cascaded, calculated as follows:

[0079] F MSID =Concat(Sum(F) refined-1-i Sum(F) refined-2-i Sum(F) refined-3-i ), F refined-4-3 ) (7)

[0081] Where Concat(·) and Sum(·) represent the feature map concatenation and summation operations, respectively; F MSID This refers to the output characteristics of the MSID submodule within the MSIDAB block.

[0082] Specifically, Table 1 shows the network parameter settings for the MSID submodule. Among them, Conv3×3, Conv5×5, and Conv7×7 are convolutional layers with kernel sizes of 3×3, 5×5, and 7×7 respectively (followed by the LeakyReLU activation function), Split is the channel splitting operation, and Concat and Sum represent the feature map concatenation and summation operations, respectively.

[0083] Table 1

[0084]

[0085] Specifically, as can be seen from Table 1, due to the channel separation operation, the number of output feature channels gradually decreases, thereby reducing the complexity of the model and the difficulty of training.

[0086] Specifically, the ECCAM submodule is used to enhance the perception capability of high-frequency region information in remote sensing images. Figure 3 This is a schematic diagram of the enhanced channel contrast attention mechanism (ECCAM) submodule structure in an embodiment of the present invention.

[0087] Specifically, the calculation process of the ECCAM submodule is as follows:

[0088] Specifically, define For a feature map with a total of C channels, where the size of a single channel is H×W; for the feature map of the k-th channel... Calculate global average pooling (GAP) to obtain the corresponding channel feature descriptors. The calculation is as follows:

[0089]

[0090] in, For F MSID The value at position (i, j) in the k-th feature map. For F MSID The corresponding channel feature descriptors; considering the spatial diversity of remote sensing images, F is analyzed along the channel dimension. MSID Calculate average pooling (AP) to obtain the corresponding spatial feature descriptor D. Spatial The calculation is as follows:

[0091]

[0092] Among them, D Spatial (i, j) is The compressed spatial feature description value at position (i, j); combining the channel and spatial feature descriptions, the enhanced channel contrast information value is calculated as follows:

[0093]

[0094] in, This represents the contrast information value of the k-th enhanced channel; further, a multilayer perceptron structure is used to enhance the feature weights of high-frequency information, calculated as follows:

[0095]

[0096] Among them, f σ [·] and ReLU(·) represent the Sigmoid function and the ReLU activation function, respectively; W D (·) represents a convolution operation with a channel downsampling rate of r and a kernel size of 1×1; W u (·) represents a convolution operation with a channel upsampling rate of r and a kernel size of 1×1; ... represents a feature map channel multiplication operation; This is the output feature map after passing through the ECCAM submodule.

[0097] Specifically, convolutional layers with a kernel size of 1×1 are used to compress the number of channels in the output feature map.

[0098] Specifically, the output of the MSIDAB block is calculated as follows:

[0099] F out =H compress (F ECCAM +F MSID )+F in (12)

[0100] Among them, F out H is the final output of the MSIDAB block. compress (·) represents a convolution operation with a kernel size of 1×1, and + represents a feature map addition operation.

[0101] Specifically, the final output features of the deep feature extraction module are calculated as follows:

[0102] F fused =H fuse (Concat(F b,1 F b,n F b,M (13)

[0103] Wherein, Concat(·) is the feature map concatenation operation, which concatenates and stitches together the outputs of M MSIDAB; F fused For deep fusion features; H fuse (·) Specific operation: First, use a convolutional layer with a kernel size of 1×1 to reduce the dimension of the feature map, then use the LeakyReLU function to reduce redundant parameters, and finally use a convolutional layer with a kernel size of 3×3 to further refine the deep fusion features.

[0104] Specifically, the formula for the upsampling module is as follows:

[0105] I SR =H subpixel (H A (F fused +F0)) (14)

[0106] Among them, H A (·) represents a convolutional layer with a kernel size of 3×3, H subpixel (·) represents the upsampling operation of the sub-pixel convolutional layer, I SR This is the final super-resolution reconstructed image.

[0107] Specifically, the entire network is optimized using a loss function, which is defined as follows:

[0108] L(Θ)=||I SR -I HR ||1 (15)

[0109] Among them, I HR Θ represents real high-resolution remote sensing image samples; Θ represents the model parameters of the entire network; ||·||1 represents the L1 loss function.

[0110] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined in this invention may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1.A method for remote sensing image super-resolution reconstruction based on feature information distillation network, characterized in that The specific steps are: 1) constructing a super-resolution reconstruction network based on a feature information distillation network, which sequentially includes a shallow feature extraction module, a deep feature extraction module and an upsampling module; 2) let input low-resolution image be I LR , output shallow features F0 through a shallow feature extraction module; 3) the shallow feature F0 outputs a deep fusion feature F through a deep feature extraction module fused The deep feature extraction module is composed of M MSIDAB blocks, each MSIDAB block including an MSID submodule, an ECCAM submodule, and a convolutional layer with a convolutional kernel size of 1x1. The specific steps are as follows: (1) The calculation formula of the MSIDAB block is as follows: F b,n = H MSIDAB,n (F b,n-1 ) = H MSIDAB,n (H MSIDAB,n-1 (··· H MSIDAB,1 (F0)··· )) (2) where H MSIDAB,n (·) is the nth MSIDAB block operation; F b,n-1 and F b,n are the input and output of the nth MSIDAB block, respectively; (2) The calculation process of the MSID sub-module is as follows: Definitions As input features for the MSIDAB block, the features are obtained by multi-scale feature extraction and channel separation mechanism as follows: [F refined-1-i ,F remaining-1-i ] = Splitl(Conv 1-i (F in )) (3) [F refined-2-i ,F remaining-2-i ] = Split2(Conv 2-i (F remaining-1-i )) (4) [F refined-3-i ,F remaining-3-i ] = Split3(Conv 3-i (F remaining-2-i )) (5) F refined-4-3 = Conv 4-3 (Sum(F remaining-3-i )) (6) wherein, Conv k-i (·) is the kth convolutional layer of the MSIDAB block with a convolution kernel size of i×i; Split k (·) is the kth feature channel separation operation of the MSIDAB block; F refined-j-i and F remaining-j-i are the jth local distillation feature and saved feature on the branch of the convolutional layer with a convolution kernel size of i×i of the MSIDAB block, respectively, wherein the saved feature is used for the calculation of the next unit; Sum(·) is a feature map summation operation; specifically, the convolution kernel size values i×i on each convolutional layer are set to 3×3, 5×5, and 7×7, respectively; the distillation of each multi-scale convolutional layer is fused as a whole, and a cascading operation is performed on the fused features, and the calculation is as follows: F MSID = Concat(Sum(F refined-1-i ),Sum(F refined-2-i ),Sum(F refined-3-i ),F refined-4-3 ) (7) where Concat(·) and Sum(·) represent feature map concatenation and summation operation, respectively; F MSID is the output feature of the MSID submodule in the MSIDAB block. (3) The calculation process of the ECCAM sub-module is as follows: Definitions a feature map with a total number of channels C, where the size of a single channel is H x W; a feature map of the k-th channel a global average pooling is computed to obtain the corresponding channel feature descriptor is computed as follows: wherein, is F MSID the value in the k-th feature map at position (i, j), is F MSID the corresponding channel feature descriptor; considering the diversity of spatial distribution of remote sensing images, F MSID the mean pooling is calculated to obtain the corresponding spatial feature descriptor D Spatial is calculated as follows: where D Spatial (i,j) is The compressed spatial feature descriptor value at position (i,j); the enhanced channel contrast information value is calculated as follows in combination with the channel and spatial feature descriptors: wherein, represents the kth enhanced channel contrast information value; further, the feature weight of the enhanced high-frequency information is calculated by using a multi-layer perception structure as follows: wherein f σ [·] and ReLU(·) represent the Sigmoid function and the ReLU activation function, respectively; W D (·) is a convolution operation with a 1×1 convolution kernel size and a channel down-sampling rate of r; W U (·) is a convolution operation with a 1×1 convolution kernel size and a channel up-sampling rate of r; is a feature map channel multiplication operation; is the output feature map after the ECCAM sub-module; (4) The convolution layer with a kernel size of 1*1 is used to compress the output feature map channel number, and the output calculation of the MSIDAB block is as follows: F out = H compress (F ECCAM + F MSID ) + F in (12) where F out is the final output of the MSIDAB block, H compress (·) is a convolution operation with a kernel size of 1x1, and + is a feature map addition operation. 4) let the shallow feature F0 and the deep fusion feature F fused The final remote sensing image super-resolution reconstruction result I is obtained after fusion and up-sampling module SR . 2.The remote sensing image super-resolution reconstruction method based on the feature information distillation network according to claim 1, wherein In step 1), the shallow feature extraction module is used for the recovery of low-frequency information; The deep feature extraction module is composed of M multi-scale information distillation attention blocks, namely MSIDAB blocks; The deep feature extraction module is used for the recovery of high-frequency information, and the M multi-scale information distillation attention blocks realize the transmission between features, prevent the features of the shallow layer from being weakened or lost in the information transmission process, and ensure the integrity of the information; The MSID sub-module is used for gradually extracting and fusing multi-scale semantic features while reducing model parameters, the ECCAM sub-module is used for enhancing the perception ability of high-frequency region information in remote sensing images, and the convolution layer with a kernel size of 1*1 is used to compress the output feature map channel number; The upsampling module is used to project the feature map from a low-resolution space to a high-resolution space. 3.The remote sensing image super-resolution reconstruction method based on the feature information distillation network according to claim 1, wherein In step 2), the input low-resolution image is I LR The formula for outputting the shallow feature F0 through the shallow feature extraction module is as follows: F0 = H SF (I LR ) (1) wherein H SF (·) is a convolution operation with a kernel size of 3x3, F0is the extracted shallow feature image. 4.The remote sensing image super-resolution reconstruction method based on the feature information distillation network according to claim 1, wherein In step 3), the acquisition formula of the deep fusion feature is as follows: F fused = H fuse (Concat(F b,1 ,···,F b,n ,···, F b ,M )) (13) wherein Concat(·) is a feature map concatenation operation, and a total of M outputs of the MSIDABs are spliced by the feature map concatenation operation; F fused is a deep fusion feature; H fuse (·) The specific operation is: first, a convolution layer with a convolution kernel size of 1*1 is used to reduce the feature map dimension, then a LeakyReLU function is used to reduce redundant parameters, and finally a convolution layer with a convolution kernel size of 3*3 is used to further refine the deep fusion feature. 5.The remote sensing image super-resolution reconstruction method based on the feature information distillation network according to claim 1, wherein In step 4), the final remote sensing image super-resolution reconstruction result I is obtained SR The formula is as follows: I SR = H subpixel (H A (F fused + F0)) (14) wherein H A (·) is a convolution layer with a kernel size of 3x3, H subpixel (·) is a sub-pixel convolution layer up-sampling operation, I SR is the final generated super-resolution reconstructed image. 6.The method of claim 5, wherein the feature information distillation network is a feature information distillation network based on a residual network (ResNet). The loss function is used to optimize the entire network, and the loss function is defined as follows: L(Θ) = ||I SR - I HR ‖1 (15) where I HR is a real high-resolution remote sensing image sample; Θ is the model parameter of the entire network; and ||·||1 is an L1 loss function.

Citation Information

Patent Citations

  • Remote sensing image super-resolution reconstruction method based on lightweight generative model

    CN113538234A

  • Remote sensing image super-resolution reconstruction method based on deep convolutional neural network

    CN113222819A

  • Lightweight progressive feature fusion image super-resolution system and method

    CN113256496A