Remote sensing satellite image super-resolution method and system based on hybrid attention mechanism

By adopting a hybrid attention mechanism method in remote sensing satellite image processing, combining shallow and deep feature extraction, the shortcomings of the traditional self-attention mechanism in image detail extraction and artifact removal are solved, and higher quality super-resolution is achieved Rate image reconstruction.

CN119168870BActive Publication Date: 2025-01-17WUHAN UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411661937.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-11-20
Publication Date
2025-01-17
Estimated Expiration
2044-11-20

AI Technical Summary

Technical Problem

The traditional self-attention mechanism has problems such as insufficient feature extraction potential and artifacts in the super-resolution processing of remote sensing satellite images, and it is difficult to effectively restore image details.

Method used

Using a method based on the hybrid attention mechanism, the combination of shallow feature extraction and deep feature extraction is used to perform feature extraction and super-resolution processing using the hybrid attention module to generate super-resolution residual images and merge them with the original image to eliminate artifacts. , improve image detail restoration effect.

Benefits of technology

It significantly improves the super-resolution reconstruction effect of remote sensing satellite low-resolution images, can extract more detailed information and eliminate artifacts, and improves the quality of image detail restoration.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119168870B_ABST
    Figure CN119168870B_ABST
Patent Text Reader

Abstract

The present invention provides a remote sensing satellite image super-resolution method and system based on a hybrid attention mechanism, which belongs to the field of remote sensing image processing technology, including: collecting the original image of the target remote sensing satellite, enlarging the original image to the target size, and obtaining the enlarged original image; performing shallow feature extraction on the original image to obtain a shallow feature image; performing deep feature extraction on the shallow feature image to obtain a deep feature image; performing super-resolution processing on the shallow feature image and the deep feature image to obtain a super-resolution residual image, and combining the super-resolution residual image and the enlarged original image to obtain a super-resolution reconstructed image. The present invention effectively solves the problems of artifact removal and detail restoration in the super-resolution process of remote sensing satellite hyperspectral images, and can generate images with higher resolution, richer information and more accuracy, so as to facilitate visual perception and subsequent high-order visual tasks.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a remote sensing satellite image super-resolution method and system based on a hybrid attention mechanism. Background Art

[0002] In the current field of remote sensing satellite image processing, using deep learning to obtain super-resolution images is one of the most popular directions. Neural networks are usually used to obtain higher-resolution generated images from low-resolution original images that meet usage requirements and are consistent with real scenes. This method has the characteristics of low cost and easy implementation.

[0003] Correspondingly, super-resolution algorithms based on self-attention mechanisms have been a hot topic in recent years. As an alternative to convolutional neural networks, the emerging self-attention mechanism neural network has achieved excellent performance by calculating the autocorrelation of each element in the global receptive field. However, this type of method has the following shortcomings: (1) Traditional attention mechanisms operate from a single dimension, whether in space or channel, which limits the potential for feature extraction and cannot effectively restore more details of the image. (2) Deep learning methods based on self-attention mechanisms usually balance computational cost and model performance by limiting the receptive field, which not only limits the scope of feature extraction but also causes various artifacts. Summary of the invention

[0004] The present invention provides a remote sensing satellite image super-resolution method and system based on a hybrid attention mechanism, which are used to solve the defects in the prior art for super-resolution image processing produced by remote sensing satellites.

[0005] In a first aspect, the present invention provides a remote sensing satellite image super-resolution method based on a hybrid attention mechanism, comprising:

[0006] Acquire an original image of a target remote sensing satellite, and enlarge the original image to a target size to obtain an enlarged original image;

[0007] Performing shallow feature extraction on the original image to obtain a shallow feature image;

[0008] Perform deep feature extraction on the shallow feature image to obtain a deep feature image;

[0009] Super-resolution processing is performed on the shallow feature image and the deep feature image to obtain a super-resolution residual image, and the super-resolution residual image is combined with the enlarged original image to obtain a super-resolution reconstructed image.

[0010] According to a remote sensing satellite image super-resolution method based on a hybrid attention mechanism provided by the present invention, the original image is enlarged to a target size to obtain an enlarged original image, including:

[0011] The interpolation function in the PyTorch software is used to enlarge the original image by bicubic linear interpolation according to a specified multiple to obtain the enlarged original image.

[0012] According to a remote sensing satellite image super-resolution method based on a hybrid attention mechanism provided by the present invention, shallow feature extraction is performed on the original image to obtain a shallow feature image, including:

[0013] Determine the shallow feature extraction module Including convolution kernel 3 3-size convolutional neural network units;

[0014] The original image Input the shallow feature extraction module , get the shallow feature image :

[0015]

[0016] in, , as well as They represent the length, width and number of channels of the enlarged original image respectively.

[0017] According to a remote sensing satellite image super-resolution method based on a hybrid attention mechanism provided by the present invention, deep features are extracted from shallow feature images to obtain deep feature images, including:

[0018] Determine the deep feature extraction module Includes multiple hybrid attention modules and a convolutional layer constitute:

[0019]

[0020]

[0021] in, Indicates A hybrid attention module, Expressed as The feature images generated by the hybrid attention module, represents the total number of hybrid attention modules;

[0022] The shallow feature image Input the deep feature extraction module , get the deep feature image :

[0023] .

[0024] According to a remote sensing satellite image super-resolution method based on a hybrid attention mechanism provided by the present invention, the hybrid attention module includes multiple hybrid attention layers and a convolutional layer, and each hybrid attention module is:

[0025]

[0026]

[0027] in, Representative The first of the hybrid attention modules Mixed attention layers, represents the feature image generated by the hybrid attention layer, Represents the feature image finally obtained by the hybrid attention module, which is composed of the feature map generated by the convolution layer and the input image of the hybrid attention module Obtained through a residual skip link, L Represents the total number of mixed attention layers.

[0028] According to a remote sensing satellite image super-resolution method based on a hybrid attention mechanism provided by the present invention, the hybrid attention layer includes a cascaded spatial attention operation unit and a channel attention unit, and each hybrid attention layer function is:

[0029]

[0030]

[0031]

[0032] in, represents the regularization process, represents a window-slidable spatial self-attention unit, represents the channel attention unit, represents a multilayer perceptron, represents the input image, represents the spatial attention unit generating an image, represents the channel attention unit generating the image, Represents the feature image obtained by the mixed attention layer.

[0033] According to a remote sensing satellite image super-resolution method based on a hybrid attention mechanism provided by the present invention, super-resolution processing is performed on the shallow feature image and the deep feature image to obtain a super-resolution residual image, and the super-resolution residual image and the enlarged original image are combined to obtain a super-resolution reconstructed image, including:

[0034] The shallow feature image and the deep feature image are aggregated through residual jump links, and input into the image super-resolution module to obtain the super-resolution residual image, and then the super-resolution residual image and the enlarged original image are merged to obtain the final super-resolution image :

[0035]

[0036] in, represents the image super-resolution module, Represents the original image interpolation operation.

[0037] In a second aspect, the present invention further provides a remote sensing satellite image super-resolution system based on a hybrid attention mechanism, comprising:

[0038] The enlargement module is used to collect the original image of the target remote sensing satellite and enlarge the original image to the target size to obtain the enlarged original image;

[0039] A shallow feature extraction module, used for performing shallow feature extraction on the original image to obtain a shallow feature image;

[0040] A deep feature extraction module is used to extract deep features from shallow feature images to obtain deep feature images;

[0041] The reconstruction module is used to perform super-resolution processing on the shallow feature image and the deep feature image to obtain a super-resolution residual image, and combine the super-resolution residual image with the enlarged original image to obtain a super-resolution reconstructed image.

[0042] In a third aspect, the present invention also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a remote sensing satellite image super-resolution method based on a hybrid attention mechanism as described above is implemented.

[0043] In a fourth aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a remote sensing satellite image super-resolution method based on a hybrid attention mechanism as described in any one of the above.

[0044] The remote sensing satellite image super-resolution method and system based on the hybrid attention mechanism provided by the present invention can extract more detail information from the original image, better restore the image details, and eliminate the generation of various artifacts, by generating a super-resolution residual image by using the feature information extracted by the hybrid attention neural network and combining the original image interpolation image to generate a super-resolution image, thereby significantly improving the super-resolution reconstruction effect of the remote sensing satellite low-resolution input image. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 It is a flowchart of a remote sensing satellite image super-resolution method based on a hybrid attention mechanism provided by the present invention;

[0047] Figure 2 is a schematic diagram of the processing process of the super-resolution image reconstruction process provided by the present invention;

[0048] Figure 3 It is a schematic diagram of a deep feature extraction module provided by the present invention;

[0049] Figure 4 is a schematic diagram of the hybrid attention layer structure provided by the present invention;

[0050] Figure 5 It is a schematic diagram of window sliding provided by the present invention;

[0051] Figure 6 It is a structural schematic diagram of a remote sensing satellite image super-resolution system based on a hybrid attention mechanism provided by the present invention;

[0052] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] In view of the many deficiencies in the reconstruction process of remote sensing satellite super-resolution images in the prior art, the present invention takes the Venusstar-1 satellite as an example and proposes a remote sensing satellite image super-resolution method based on a hybrid attention mechanism. Since Venusstar-1 can obtain 32-band hyperspectral images, it can capture the fine spectral information of ground objects, with a large spectral range and rich band information. However, it is impossible to obtain high spectral resolution and high spatial resolution at the same time. In order to further expand the use scenarios of the hyperspectral images of the Venusstar-1 satellite, an effective method is needed to improve the resolution of the images obtained, which is conducive to subsequent visual interpretation, ground object classification and other tasks. Super-resolution will be performed on the multispectral data of Venusstar-1, which can improve the ability of image information extraction and analysis, and effectively solve the problems of artifact removal and detail restoration in the super-resolution process of the hyperspectral images of the Venusstar-1 satellite.

[0055] Figure 1 is a flow chart of a remote sensing satellite image super-resolution method based on a hybrid attention mechanism provided by an embodiment of the present invention, such as Figure 1 As shown, including:

[0056] Step 100: collecting an original image of a target remote sensing satellite, and enlarging the original image to a target size to obtain an enlarged original image;

[0057] Step 200: extract shallow features from the original image to obtain a shallow feature image;

[0058] Step 300: extracting deep features from the shallow feature image to obtain a deep feature image;

[0059] Step 400: performing super-resolution processing on the shallow feature image and the deep feature image to obtain a super-resolution residual image, and combining the super-resolution residual image with the enlarged original image to obtain a super-resolution reconstructed image.

[0060] Specifically, Figure 2 As shown, the embodiment of the present invention first interpolates and amplifies the collected Venusstar-1 original image to a desired size; then performs shallow feature extraction on the original image to obtain a shallow feature image; then performs deep feature extraction on the shallow feature image to obtain a deep feature image; super-resolution is performed on the obtained shallow feature image and deep feature image to obtain a super-resolution residual image; finally, the super-resolution residual image and the interpolated image of the original image are combined to obtain a super-resolution reconstructed image.

[0061] The embodiment of the present invention includes a shallow feature extraction module, a deep feature extraction module and an image super-resolution module. The shallow feature extraction module is composed of a convolutional neural network unit, using 3 3 convolution kernel to extract shallow features of the original image, obtain shallow feature images, and directly transfer the shallow feature images to the image super-resolution module; the deep feature extraction module is composed of multiple hybrid attention modules and convolution layers, each hybrid attention module contains multiple hybrid attention layers and a convolution layer, and the hybrid attention layer uses window sliding space self-attention units and channel attention units to achieve attention mixing and cross-window interaction; the image super-resolution module is implemented using convolutional layer neural network units and PixelShuffle functions to process the shallow feature images and deep feature images obtained previously, and finally obtain high-quality super-resolution images.

[0062] The present invention generates a super-resolution residual image by using feature information extracted by a hybrid attention neural network and generates a super-resolution image by combining the original image with an interpolated image. It can extract more detail information from the original image, better restore image details, and eliminate the generation of various artifacts, significantly improving the super-resolution reconstruction effect of low-resolution input images of remote sensing satellites.

[0063] Based on the above embodiment, step 100 includes:

[0064] The interpolation function in the PyTorch software is used to enlarge the original image by bicubic linear interpolation according to a specified multiple to obtain the enlarged original image.

[0065] Specifically, the embodiment of the present invention utilizes the nn.Upsample() interpolation function in the PyTorch software to perform bicubic linear interpolation magnification of the low-resolution original image by a specified multiple.

[0066] Based on the above embodiment, step 200 includes:

[0067] Specifically, the original image ( , as well as Representing the length, width and number of channels of the original image respectively) are input into the trained neural network model. First, the image enters the shallow feature extraction module , this module uses a convolution kernel of 3 The convolutional neural network units of size 3 generate shallow feature images to obtain the low-dimensional feature information of the original image. The objective function is:

[0068]

[0069] is the shallow feature image obtained ( , as well as Represent the length, width and number of channels of the shallow feature image respectively).

[0070] It should be noted that the shallow feature extraction module uses a common convolutional neural network unit with a convolution kernel of 3X3 to generate a shallow feature extraction image, and the shallow feature image will be used as the input of the deep feature extraction module.

[0071] Based on the above embodiment, step 300 includes:

[0072] Specifically, Figure 3 As shown, the shallow feature image Input deep feature extraction module , through this module, deep feature images can be generated , obtain the high-dimensional feature information of the original image, and the objective function is:

[0073]

[0074] This module consists of multiple hybrid attention modules and a convolutional layer The shallow feature images first enter A hybrid attention module is used, and finally a convolution layer is used to generate the final deep feature image. Its function is expressed as:

[0075]

[0076]

[0077] in, Indicates A hybrid attention module, Expressed as The feature images generated by the hybrid attention module, Represents the total number of hybrid attention modules.

[0078] The image input to the deep feature module first enters the hybrid attention module for processing. Each hybrid attention module consists of multiple hybrid attention layers and a convolution layer. This cascade structure can more effectively reuse multi-level features. The function of each hybrid attention module is expressed as:

[0079]

[0080]

[0081] in Representative The first of the hybrid attention modules Mixed attention layers, Represents the feature image generated by the hybrid attention layer. The hybrid attention module finally obtains the feature image The feature map generated by the convolutional layer and the input image of the hybrid attention module Obtained through a residual skip link, L Represents the total number of mixed attention layers.

[0082] like Figure 4 As shown in , the hybrid attention layer contains a basic window sliding spatial self-attention layer, such as Figure 4 As shown in part (a) of the spatial attention, a channel attention unit is added after the spatial self-attention unit, as shown in Figure 4 As shown in the channel attention in part (b), the primary mixed attention feature map is obtained, and the image is regularized and reconstructed to obtain the final mixed attention feature map. In the spatial attention operation, the three vector dimensions of the self-attention mechanism of the input image convolution are (i.e. query, key and value) generation , and Three matrices are used to calculate the autocorrelation of each pixel with all other pixels, generating A new image with three dimensions (i.e., image height, width, and channel, where the superscript indicates a variable value in the sliding image). Each pixel in the new image is composed of all other pixels related to it. The higher the similarity between two pixels, the greater the proportion of the new pixel. During the channel attention operation, each channel is pooled to obtain the eigenvalue of the channel. The eigenvalue sequence composed of all channel eigenvalues ​​is input into the Sigmoid function to obtain the channel weight sequence. The channel weight sequence is multiplied by the input image to obtain a new channel attention feature image.

[0083] The function of each mixed attention layer is expressed as:

[0084]

[0085]

[0086]

[0087] in, represents the regularization process, represents a window-sliding spatial self-attention unit, such as Figure 5 As shown in , two adjacent hybrid attention layers form a hybrid attention layer pair, in which the spatial self-attention operation unit in the second hybrid attention layer performs a window sliding operation, such as Figure 5 As shown in the Layer lThe 4 small windows in the slide operation become Layer l +1 and restore it after the operation is completed to achieve cross-window interaction. It represents the channel attention operation unit. After obtaining the spatial attention feature map and the channel attention feature map, they are linked to the input image through residual jump links to generate a primary mixed attention feature map. The generated image is then regularized and placed in a multi-layer perceptron. Reconstruction in represents the input image, represents the spatial attention unit generating an image, represents the channel attention unit generating the image, Represents the feature image obtained by the mixed attention layer.

[0088] The deep feature extraction module constructed in the embodiment of the present invention includes N Hybrid attention modules and a 3X3 convolution layer. Each hybrid attention module contains M hybrid attention layers and a 3X3 convolution layer. The entire deep feature extraction module uses neural network layers and cascade structures with multiple receptive field ranges to better extract the feature information of the original image at multiple levels and restore more image details.

[0089] The hybrid attention layer consists of a window-sliding spatial self-attention operation unit and a channel attention operation unit. In order to transfer information across windows, two adjacent hybrid attention layers form a pair, and the window sliding operation is performed in the second hybrid attention layer, and the window is restored after the calculation is completed, thereby effectively eliminating seam artifacts. When using hybrid attention for feature extraction, first extract features from the spatial dimension through self-attention, then reconstruct the weights of the obtained feature image in the channel dimension through channel attention, and finally aggregate the generated channel feature image, spatial feature image and input image through residual jump links, regularize and reconstruct, and generate a deep feature image.

[0090] Based on the above embodiment, step 400 includes:

[0091] The shallow feature image and the deep feature image are aggregated through residual jump links and input into the image super-resolution module to obtain a super-resolution residual image, which is then merged with the original image interpolation image to obtain the final super-resolution image. . Its function is:

[0092]

[0093] in represents the image super-resolution module, Represents the original image interpolation operation.

[0094] The remote sensing satellite image super-resolution system based on the hybrid attention mechanism provided by the present invention is described below. The remote sensing satellite image super-resolution system based on the hybrid attention mechanism described below and the remote sensing satellite image super-resolution method based on the hybrid attention mechanism described above can be referenced to each other.

[0095] Figure 6 is a schematic diagram of the structure of a remote sensing satellite image super-resolution system based on a hybrid attention mechanism provided by an embodiment of the present invention. Figure 6 As shown, it includes: an amplification module 61, a shallow feature extraction module 62, a deep feature extraction module 63 and a reconstruction module 64, wherein:

[0096] The enlargement module 61 is used to collect the original image of the target remote sensing satellite, enlarge the original image to the target size, and obtain the enlarged original image; the shallow feature extraction module 62 is used to perform shallow feature extraction on the original image to obtain a shallow feature image; the deep feature extraction module 63 is used to perform deep feature extraction on the shallow feature image to obtain a deep feature image; the reconstruction module 64 is used to perform super-resolution processing on the shallow feature image and the deep feature image to obtain a super-resolution residual image, and combine the super-resolution residual image and the enlarged original image to obtain a super-resolution reconstructed image.

[0097] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7 As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute a remote sensing satellite image super-resolution method based on a hybrid attention mechanism, the method comprising: collecting an original image of a target remote sensing satellite, enlarging the original image to a target size, and obtaining an enlarged original image; performing shallow feature extraction on the original image to obtain a shallow feature image; performing deep feature extraction on the shallow feature image to obtain a deep feature image; performing super-resolution processing on the shallow feature image and the deep feature image to obtain a super-resolution residual image, and combining the super-resolution residual image and the enlarged original image to obtain a super-resolution reconstructed image.

[0098] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0099] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the remote sensing satellite image super-resolution method based on the hybrid attention mechanism provided by the above-mentioned methods. The method includes: collecting the original image of the target remote sensing satellite, enlarging the original image to the target size, and obtaining the enlarged original image; performing shallow feature extraction on the original image to obtain a shallow feature image; performing deep feature extraction on the shallow feature image to obtain a deep feature image; performing super-resolution processing on the shallow feature image and the deep feature image to obtain a super-resolution residual image, and combining the super-resolution residual image and the enlarged original image to obtain a super-resolution reconstructed image.

[0100] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0101] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A remote sensing satellite image super-resolution method based on a hybrid attention mechanism, characterized in that: include: Acquire an original image of a target remote sensing satellite, and enlarge the original image to a target size to obtain an enlarged original image; Performing shallow feature extraction on the original image to obtain a shallow feature image; Perform deep feature extraction on the shallow feature image to obtain a deep feature image; Performing super-resolution processing on the shallow feature image and the deep feature image to obtain a super-resolution residual image, and combining the super-resolution residual image with the enlarged original image to obtain a super-resolution reconstructed image; Performing shallow feature extraction on the original image to obtain a shallow feature image includes: Determine the shallow feature extraction module Including convolution kernel 3 3-size convolutional neural network units; The original image Input the shallow feature extraction module , get the shallow feature image : in, , as well as Respectively represent the length, width and number of channels of the enlarged original image; Perform deep feature extraction on the shallow feature image to obtain a deep feature image, including: Determine the deep feature extraction module Includes multiple hybrid attention modules and a convolutional layer constitute: in, Indicates A hybrid attention module, Expressed as The feature images generated by the hybrid attention module, represents the total number of hybrid attention modules; The shallow feature image Input the deep feature extraction module , get the deep feature image : ; The hybrid attention module consists of multiple hybrid attention layers and a convolutional layer. Each hybrid attention module is: in, Representative The first of the hybrid attention modules Mixed attention layers, represents the feature image generated by the hybrid attention layer, Represents the feature image finally obtained by the hybrid attention module, which is composed of the feature map generated by the convolution layer and the input image of the hybrid attention module Obtained through a residual skip link, L Represents the total number of mixed attention layers; The hybrid attention layer includes cascaded spatial attention units and channel attention units. The function of each hybrid attention layer is: in, represents the regularization process, represents a window-slidable spatial self-attention unit, represents the channel attention unit, represents a multilayer perceptron, represents the input image, represents the spatial attention unit generating an image, represents the channel attention unit generating images, Represents the feature image obtained by the mixed attention layer; Performing super-resolution processing on the shallow feature image and the deep feature image to obtain a super-resolution residual image, and combining the super-resolution residual image with the enlarged original image to obtain a super-resolution reconstructed image, including: The shallow feature image and the deep feature image are aggregated through residual jump links, and input into the image super-resolution module to obtain the super-resolution residual image, and then the super-resolution residual image and the enlarged original image are merged to obtain the final super-resolution image : in, represents the image super-resolution module, Represents the original image interpolation operation.

2. The remote sensing satellite image super-resolution method based on hybrid attention mechanism according to claim 1, characterized in that: Enlarging the original image to a target size to obtain an enlarged original image includes: The interpolation function in the PyTorch software is used to enlarge the original image by bicubic linear interpolation according to a specified multiple to obtain the enlarged original image.

3. A remote sensing satellite image super-resolution system based on a hybrid attention mechanism, based on the remote sensing satellite image super-resolution method based on a hybrid attention mechanism according to claim 1 or 2, characterized in that: include: The enlargement module is used to collect the original image of the target remote sensing satellite and enlarge the original image to the target size to obtain the enlarged original image; A shallow feature extraction module, used for performing shallow feature extraction on the original image to obtain a shallow feature image; A deep feature extraction module is used to extract deep features from shallow feature images to obtain deep feature images; The reconstruction module is used to perform super-resolution processing on the shallow feature image and the deep feature image to obtain a super-resolution residual image, and combine the super-resolution residual image with the enlarged original image to obtain a super-resolution reconstructed image.

4. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the remote sensing satellite image super-resolution method based on the hybrid attention mechanism as described in claim 1 or 2 is implemented.

5. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the remote sensing satellite image super-resolution method based on the hybrid attention mechanism as described in claim 1 or 2 is implemented.

Citation Information

Patent Citations

  • Infrared image super-resolution reconstruction method based on edge enhancement

    CN116071243A

  • Training method and device of image super-resolution network

    CN116385265A