Remote sensing image super-resolution reconstruction method and system based on cross-scale cosine attention
By employing a cross-scale cosine attention-based super-resolution reconstruction method for remote sensing images, combined with deep feature extraction and feature fusion, the problems of high computational cost and insufficient information capture capability in remote sensing image reconstruction are solved, achieving efficient image reconstruction results.
Patent Information
- Application Number
- CN202411403278.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-09
- Publication Date
- 2025-01-14
AI Technical Summary
In existing technologies, super-resolution reconstruction methods for remote sensing images suffer from high computational costs, large memory consumption, and difficulty in deployment on mobile devices when improving image resolution. At the same time, CNN models have difficulty capturing global image information, resulting in poor reconstruction results.
A remote sensing image super-resolution reconstruction method based on cross-scale cosine attention is adopted. The shallow features of the image are obtained through convolution operation, and the deep feature extraction layer and dense feature fusion layer are combined. Cascaded feature distillation blocks and cross-scale Transformer blocks are used for feature extraction and fusion. The cross-scale cosine attention layer is used to improve the feature attention capability, and the image resolution is improved through an upsampling module.
This method improves image resolution and detail richness without increasing network parameters, enhances image reconstruction quality, and reduces computational complexity, making the model feasible on mobile devices.
Smart Images
Figure CN119313562A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of image processing, and relates to a remote sensing image super-resolution reconstruction method and system based on cross-scale cosine attention. Background Art
[0002] Remote sensing images play an important role in the fields of military, agriculture, urban planning and environmental monitoring. Their clarity directly affects the processing effect of subsequent tasks, such as hyperspectral image classification, object recognition, change detection, land cover classification and other applications. However, due to the limitation of sensor resolution and noise interference during the acquisition process, the spatial resolution of satellite remote sensing images is often not high, which limits their practical application effect. Generally speaking, the imaging resolution can be further improved by improving the level of imaging hardware. However, this method is constrained by objective factors such as physical laws, economic costs, and technical difficulties. Therefore, it is particularly important to improve the resolution of remote sensing images through super-resolution (SR) reconstruction from an algorithmic perspective.
[0003] With the development of deep learning, convolutional neural networks have shown great potential in the field of image SR with their powerful fitting ability. Although the reconstruction effect based on CNN has been greatly improved, as the number of proposed network layers becomes more and more complex, such as the residual dense network (RDN) and residual channel attention network (RCAN), the network depth has increased to 100 and 400 layers respectively, resulting in an explosive growth in computational cost and memory consumption, so it cannot be effectively deployed on embedded terminals such as mobile devices.
[0004] In order to solve the above problems, building lightweight SR models has attracted more and more attention. One of the most widely used strategies is to introduce a recursive mechanism. Another is to explore lightweight structures. Although these models reduce the number of model parameters to a certain extent, they also lead to performance degradation and make it difficult to reconstruct high-quality images with rich details. In addition, the fundamental problem of CNN is that its neurons only capture local features of the image, which limits the ability to capture global information of the image. Summary of the invention
[0005] The purpose of the present invention is to solve the problems in the prior art and to provide a remote sensing image super-resolution reconstruction method and system based on cross-scale cosine attention.
[0006] In order to achieve the above object, the present invention adopts the following technical solutions:
[0007] The present invention proposes a remote sensing image super-resolution reconstruction method based on cross-scale cosine attention, comprising:
[0008] Perform convolution operation on the original low-resolution image to obtain shallow features of the image;
[0009] The deep feature extraction layer is used to extract the shallow features of the image to obtain deep features;
[0010] The deep features are fused in the dense feature fusion layer to obtain the fused features, and the fused features are fused with the shallow features of the image after passing through the convolution layer to obtain the features to be reconstructed;
[0011] The features to be reconstructed are processed through the upsampling module to obtain super-resolution reconstruction results.
[0012] Preferably, the deep feature extraction layer includes a plurality of cascade feature distillation blocks, and the specific implementation process of the cascade feature distillation block is:
[0013] The input features first pass through a multi-branch spatial attention module, and then enter two pipelines in parallel, namely the feature distillation pipeline and the feature refinement pipeline. The feature distillation pipeline consists of multiple 1×1 convolutional layers, and the feature refinement pipeline is a cascaded cross-scale Transformer block that connects the iterative distilled features of the operation, adds 1×1 convolutional layers and 3×3 convolutional layers, and finally adds a multi-branch spatial attention module to obtain deep features.
[0014] Preferably, the specific implementation process of the multi-branch spatial attention module is:
[0015] The input features first pass through a 1×1 convolution layer, then through a strided convolution layer and then into three pooling layers of different sizes, and then pass through the convolution layer and upsampling module in sequence. Finally, a sigmoid layer is added to calculate the attention mask, and the residual connection is used to add the features before channel compression directly to the end of the block to obtain the output features.
[0016] Preferably, the cascaded cross-scale Transformer block includes multiple cross-scale Transformer blocks, and the specific process of each cross-scale Transformer block is:
[0017] The input features are passed to the layer normalization layer and the cross-scale cosine attention layer in sequence, and the input features and the output features of the cross-scale cosine attention layer are added element by element. The obtained features are then passed to the layer normalization layer and the ResMLP layer in sequence. Finally, the output of the ResMLP layer is added element by element to the output of the cross-scale cosine attention layer through the residual connection to output the features.
[0018] Preferably, the specific process of the ResMLP layer is as follows:
[0019] The input features pass through the affine transformation layer, layer normalization layer, activation function, layer normalization layer and affine transformation layer in sequence to obtain the output features.
[0020] Preferably, the distilled features extracted by each cascaded feature distillation block are input into the dense feature fusion layer in a residual connection manner to obtain features. The residual connection method realizes efficient feature fusion without increasing network parameters. The residual connection method is used to fuse the shallow features of the image to obtain the feature map to be reconstructed.
[0021] The present invention proposes a remote sensing image super-resolution reconstruction system based on cross-scale cosine attention, comprising:
[0022] The first feature extraction module is used to perform a convolution operation on the original low-resolution image to obtain shallow features of the image;
[0023] The second feature extraction module is used to extract shallow features of the image using a deep feature extraction layer to obtain deep features;
[0024] The third feature extraction module is used to fuse the deep features in the dense feature fusion layer to obtain the fused features, and the fused features are fused with the shallow features of the image after passing through the convolution layer to obtain the features to be reconstructed;
[0025] The feature reconstruction module is used to process the features to be reconstructed through the upsampling module to obtain super-resolution reconstruction results.
[0026] Preferably, the deep feature extraction layer includes a plurality of cascade feature distillation blocks, and the cascade feature distillation blocks are specifically:
[0027] The input features first pass through a multi-branch spatial attention module, and then enter two pipelines in parallel, namely the feature distillation pipeline and the feature refinement pipeline. The feature distillation pipeline consists of multiple 1×1 convolutional layers, and the feature refinement pipeline is a cascaded cross-scale Transformer block that connects the iterative distilled features of the operation, adds 1×1 convolutional layers and 3×3 convolutional layers, and finally adds a multi-branch spatial attention module to obtain deep features.
[0028] A terminal device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of a remote sensing image super-resolution reconstruction method based on cross-scale cosine attention are implemented.
[0029] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a remote sensing image super-resolution reconstruction method based on cross-scale cosine attention.
[0030] Compared with the prior art, the present invention has the following beneficial effects:
[0031] The invention proposes a remote sensing image super-resolution reconstruction method based on cross-scale cosine attention. The original low-resolution image is convolved to obtain shallow features of the image, and the basic contour and some simple texture information of the image can be preliminarily extracted. The shallow features are further extracted by a deep feature extraction layer to obtain deep features, which can capture more complex and abstract image features, such as the structure of the object, the change trend of the texture, etc. The features to be reconstructed are processed by an upsampling module, which can increase the number of pixels of the image, thereby improving the resolution of the image. This makes the reconstructed image clearer and richer in details, and can better meet the demand for high-resolution images. The deep features are fused in the dense feature fusion layer to obtain fused features. This fusion method can fully integrate feature information at different levels. After passing through the convolution layer, the fused features are fused with the shallow features of the image to obtain the features to be reconstructed, and the shallow and deep features are further combined to improve the quality of super-resolution reconstruction.
[0032] Furthermore, the feature distillation pipeline consists of multiple 1×1 convolutional layers, which is mainly responsible for feature extraction and reducing channels with fewer parameters. The feature refinement pipeline is a cascaded cross-scale Transformer block that can focus on the positional relationship and spatial structure between different regions in the image and refine the features layer by layer to obtain more distinguishing information.
[0033] Furthermore, the multi-branch spatial attention module uses three different sizes for maximum pooling in the pooling stage to mine the area where the key information is located, solving the problem that the surrounding unimportant information is given a high weight while the area containing the real key information cannot be paid attention to.
[0034] Furthermore, the cross-scale cosine attention layer effectively utilizes the cross-scale information of the input image without additional high overhead by dividing the input tokens into two parts along the last channel and then calculating the efficient cosine attention mechanism.
[0035] Furthermore, by introducing the affine transformation layer, ResMLP makes network training more stable without increasing additional training costs, alleviating the gradient explosion problem in the process of feature information transmission.
[0036] Furthermore, the efficient cosine attention mechanism uses the cosine function for dot product calculation, which solves the problem that the network's attention is dominated by specific pixels and enhances the network's ability to focus on more key information.
[0037] Furthermore, the distilled features extracted by each cascaded feature distillation block are input into the dense feature fusion layer in the form of residual connection to obtain features. This method achieves efficient feature fusion without increasing network parameters. At the same time, the shallow features are fused using residual connection to obtain the feature map to be reconstructed.
[0038] The invention proposes a remote sensing image super-resolution reconstruction system based on cross-scale cosine attention, which divides the system into a first feature extraction module, a second feature extraction module, a third feature extraction module and a feature reconstruction module to obtain super-resolution reconstruction results and realize remote sensing image super-resolution reconstruction. The modularization concept is adopted to make each module independent of each other, which is convenient for unified management of each module. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.
[0040] Figure 1 This is a flow chart of the remote sensing image super-resolution reconstruction method based on cross-scale cosine attention of the present invention.
[0041] Figure 2 It is a schematic diagram of the principle of a preferred embodiment of the present invention.
[0042] Figure 3 It is a schematic diagram of the network structure of the cascade characteristic distillation module unit of the present invention.
[0043] Figure 4 Schematic diagram of the network structure of the multi-branch spatial attention module unit of the present invention.
[0044] Figure 5 This is a schematic diagram of the overall structure of the cross-scale Transformer block and the cross-scale cosine attention module unit network structure of the present invention.
[0045] Figure 6 Schematic diagram of the ResMLP unit network structure of the present invention.
[0046] Figure 7 Schematic diagram for comparing the super-resolution results of the present invention with those of other methods.
[0047] Figure 8 This is a diagram of the remote sensing image super-resolution reconstruction system based on cross-scale cosine attention of the present invention.
[0048] Fig. 9 The figure is a schematic structural diagram of an electronic device of the present invention. DETAILED DESCRIPTION
[0049] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.
[0050] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0051] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.
[0052] In the description of the embodiments of the present invention, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. indicate an orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the invention is usually placed when in use, it is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.
[0053] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", which does not mean that the structure must be completely horizontal, but can be slightly tilted.
[0054] In the description of the embodiments of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0055] The present invention is further described in detail below in conjunction with the accompanying drawings:
[0056] Example 1
[0057] The present invention proposes a remote sensing image super-resolution reconstruction method based on cross-scale cosine attention. Figure 1 and Figure 2 As shown, the following steps are included:
[0058] S1, perform convolution operation on the original low-resolution image to obtain shallow features of the image;
[0059] S2, using the deep feature extraction layer to extract the shallow features of the image to obtain deep features;
[0060] The deep feature extraction layer includes multiple cascade feature distillation blocks, and the specific implementation process of the cascade feature distillation block is:
[0061] The input features first pass through a multi-branch spatial attention module, and then enter two pipelines in parallel, namely the feature distillation pipeline and the feature refinement pipeline. The feature distillation pipeline consists of multiple 1×1 convolutional layers, and the feature refinement pipeline is a cascaded cross-scale Transformer block that connects the iterative distilled features of the operation, adds 1×1 convolutional layers and 3×3 convolutional layers, and finally adds a multi-branch spatial attention module to obtain deep features.
[0062] The specific implementation process of the multi-branch spatial attention module is:
[0063] The input features first pass through a 1×1 convolution layer, then through a strided convolution layer and then into three pooling layers of different sizes, and then pass through the convolution layer and upsampling module in sequence. Finally, a sigmoid layer is added to calculate the attention mask, and the residual connection is used to add the features before channel compression directly to the end of the block to obtain the output features.
[0064] The cascaded cross-scale Transformer block includes multiple cross-scale Transformer blocks, and the specific process of each cross-scale Transformer block is as follows:
[0065] The input features are passed to the layer normalization layer and the cross-scale cosine attention layer in sequence, and the input features and the output features of the cross-scale cosine attention layer are added element by element. The obtained features are then passed to the layer normalization layer and the ResMLP layer in sequence. Finally, the output of the ResMLP layer is added element by element to the output of the cross-scale cosine attention layer through the residual connection to output the features.
[0066] The specific process of the ResMLP layer is as follows:
[0067] The input features pass through the affine transformation layer, layer normalization layer, activation function, layer normalization layer and affine transformation layer in sequence to obtain the output features.
[0068] The distilled features extracted by each cascaded feature distillation block are input into the dense feature fusion layer in the form of residual connection to obtain features. The residual connection method realizes efficient feature fusion without increasing network parameters. The residual connection method is used to fuse the shallow features of the image to obtain the feature map to be reconstructed.
[0069] S3, the deep features are fused in the dense feature fusion layer to obtain fused features, and the fused features are fused with the shallow features of the image after passing through the convolution layer to obtain the features to be reconstructed;
[0070] S4. Process the features to be reconstructed through an upsampling module to obtain a super-resolution reconstruction result.
[0071] like Figure 3 As shown in the figure, it is a schematic diagram of the unit network structure of the cascaded feature distillation module. The specific implementation process of the cascaded feature distillation block is as follows: the input feature first passes through a multi-branch spatial attention module, and then enters two pipelines in parallel, namely the feature distillation pipeline and the feature refinement pipeline. The feature distillation pipeline consists of multiple 1×1 convolutional layers. The feature refinement pipeline is a cascaded cross-scale Transformer block. After that, the iterative distilled features of these operations are connected together, and 1×1 convolutional layers and 3×3 convolutional layers are added to aggregate information, refine features and ensure the translation equivariance of the network. Finally, a multi-branch spatial attention module is added to better capture the spatial structure and texture information in the image to obtain the output features.
[0072] like Figure 4 As shown in the figure, it is a schematic diagram of the network structure of the multi-branch spatial attention module unit. Specifically: the input feature first passes through a 1×1 convolution layer to compress the feature and reduce the channel, and then passes through a strided convolution layer and then passes into three pooling layers of different sizes to expand the receptive field. The subsequent 3×3 convolution layers are used to further extract the features, and the features are upsampled to adjust the number of channels to make them consistent with the number of channels of the input features. Finally, a sigmoid layer is added to calculate the attention mask, and the residual connection is used to add the features before channel compression directly to the end of the block to obtain the output features.
[0073] like Figure 5 As shown in Figure 1, the overall structure of the cross-scale Transformer block and the network structure of the cross-scale cosine attention module unit are schematically shown. Specifically, the input features are sequentially passed to the layer specification layer and the cross-scale cosine attention layer, and the input features and the output features of the cross-scale cosine attention layer are added element by element. The obtained features are then passed to the layer specification layer and the ResMLP layer in turn. Finally, the output of the ResMLP layer is added element by element to the output of the cross-scale cosine attention layer through the residual connection to output the features. The input label embedding T∈R n×d Along the last channel is divided into two parts: Ta ∈R n×d / 2 and T b ∈R n×d / 2 Then, T a As T o , while T b Relabeled as T with larger label size and overlap l ∈R n'×d' In order to effectively exploit the self-similarity at different scales and transfer the information of larger patches to smaller but self-similar patches, T o and T l Specifically, the query q o ∈R n×d / 2 , key k o ∈R n×d / 2 Sum value v o ∈R n×d / 2 From T o Generate, and query q l ∈R n'×d / 2 , key k l ∈R n'×d / 2 Sum value v l ∈R n'×d / 2 From T l Generated by l and T o Exchange the key-value pairs to obtain the reorganized triples (q o ,k l ,v l ) and (q l ,k o ,v o ) and perform efficient cosine self-attention operation. l The last dimension of query, key, and value in will be reduced from d' to d / 2 through the projection of the attention operation, and T l The attention results are reprojected to the dimension of n'×d' and then rearranged to the dimension of n×d / 2. Finally, the attention results are concatenated to obtain the output of the cross-scale cosine attention layer. The efficient cosine self-attention operation first applies L2 normalization to Q and K, which maps Q and K onto a unit hypersphere and ensures that they have the same norm. The cosine similarity is then obtained by calculating the dot product with the value vector V to obtain the output matrix of the self-attention mechanism.
[0074] like Figure 6As shown in the figure, it is a schematic diagram of the network structure of the ResMLP unit. Specifically, the input features are sequentially passed through the affine transformation layer, layer normalization layer, activation function, layer normalization layer and affine transformation layer to obtain the output features. ResMLP introduces the affine transformation layer to replace the original MLP layer, making the network training more stable without increasing the additional training cost.
[0075] like Figure 7 As shown in the figure, it is a schematic diagram comparing the super-resolution results with the results of other methods. Through careful comparison, it is found that the remote sensing image reconstructed by the model proposed in the present invention is clearer than that of other state-of-the-art methods. Especially when the scale factor is 4, the image details reconstructed by other algorithms are relatively blurred, and there are different degrees of distortion. The local magnified area of the SR image reconstructed by the present invention not only accurately restores the structure, but also has the clearest and sharpest edges, and the visual perception quality is the best. The qualitative comparison between the present invention and other advanced algorithms once again proves the effectiveness of the present invention.
[0076] Example 2
[0077] The present invention proposes a remote sensing image super-resolution reconstruction system based on cross-scale cosine attention, such as Figure 8 As shown, including:
[0078] The first feature extraction module is used to perform a convolution operation on the original low-resolution image to obtain shallow features of the image;
[0079] The second feature extraction module is used to extract shallow features of the image using a deep feature extraction layer to obtain deep features;
[0080] The third feature extraction module is used to fuse the deep features in the dense feature fusion layer to obtain the fused features, and the fused features are fused with the shallow features of the image after passing through the convolution layer to obtain the features to be reconstructed;
[0081] The feature reconstruction module is used to process the features to be reconstructed through the upsampling module to obtain super-resolution reconstruction results.
[0082] Example 3
[0083] See also Fig. 9 As shown, the present invention also provides an electronic device 100 for a remote sensing image super-resolution reconstruction method based on cross-scale cosine attention; the electronic device 100 includes a memory 101, at least one processor 102, a computer program 103 stored in the memory 101 and executable on the at least one processor 102, and at least one communication bus 104.
[0084] The memory 101 can be used to store the computer program 103. The processor 102 implements the steps of the remote sensing image super-resolution reconstruction method based on cross-scale cosine attention described in Example 1 by running or executing the computer program stored in the memory 101 and calling the data stored in the memory 101. The memory 101 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area can store data (such as audio data) created according to the use of the electronic device 100. In addition, the memory 101 can include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices.
[0085] The at least one processor 102 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 102 may be a microprocessor or any conventional processor, etc. The processor 102 is the control center of the electronic device 100, and uses various interfaces and lines to connect various parts of the entire electronic device 100.
[0086] The memory 101 in the electronic device 100 stores a plurality of instructions to implement a remote sensing image super-resolution reconstruction method based on cross-scale cosine attention, and the processor 102 can execute the plurality of instructions to implement:
[0087] Perform convolution operation on the original low-resolution image to obtain shallow features of the image;
[0088] The deep feature extraction layer is used to extract the shallow features of the image to obtain deep features;
[0089] The deep features are fused in the dense feature fusion layer to obtain the fused features, and the fused features are fused with the shallow features of the image after passing through the convolution layer to obtain the features to be reconstructed;
[0090] The features to be reconstructed are processed through the upsampling module to obtain super-resolution reconstruction results.
[0091] Example 4
[0092] If the module / unit integrated in the electronic device 100 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory and read-only memory (ROM, Read-Only Memory).
[0093] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0094] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0095] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0096] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0097] Therefore, the solution proposed in the present invention designs a cascade feature distillation block based on Transformer-CNN by introducing a feature distillation mechanism to extract richer regional features with different perceptual fields and high-frequency information while keeping the model lightweight. Then a multi-branch spatial attention module is embedded in the cascade feature distillation block to further enhance the network's ability to capture key spatial information. In addition, the proposed cross-scale cosine attention layer can effectively calculate the correlation between high-dimensional and low-dimensional features without increasing computational complexity by maintaining a single marker representation, thereby enhancing the model's ability to process features of different scales. The efficient cosine self-attention mechanism solves the problem of network attention being dominated by specific pixels, enhancing the network's ability to focus on more features.
[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.
Claims
1. A remote sensing image super-resolution reconstruction method based on cross-scale cosine attention, characterized in that: include: Perform convolution operation on the original low-resolution image to obtain shallow features of the image; The deep feature extraction layer is used to extract the shallow features of the image to obtain deep features; The deep features are fused in the dense feature fusion layer to obtain the fused features, and the fused features are fused with the shallow features of the image after passing through the convolution layer to obtain the features to be reconstructed; The features to be reconstructed are processed through the upsampling module to obtain super-resolution reconstruction results.
2. The remote sensing image super-resolution reconstruction method based on cross-scale cosine attention according to claim 1 is characterized in that: The deep feature extraction layer includes multiple cascade feature distillation blocks, and the specific implementation process of the cascade feature distillation block is: The input features first pass through a multi-branch spatial attention module, and then enter two pipelines in parallel, namely the feature distillation pipeline and the feature refinement pipeline. The feature distillation pipeline consists of multiple 1×1 convolutional layers, and the feature refinement pipeline is a cascaded cross-scale Transformer block that connects the iterative distilled features of the operation, adds 1×1 convolutional layers and 3×3 convolutional layers, and finally adds a multi-branch spatial attention module to obtain deep features.
3. The remote sensing image super-resolution reconstruction method based on cross-scale cosine attention according to claim 2 is characterized in that: The specific implementation process of the multi-branch spatial attention module is: The input features first pass through a 1×1 convolution layer, then through a strided convolution layer and then into three pooling layers of different sizes, and then pass through the convolution layer and upsampling module in sequence. Finally, a sigmoid layer is added to calculate the attention mask, and the residual connection is used to add the features before channel compression directly to the end of the block to obtain the output features.
4. The remote sensing image super-resolution reconstruction method based on cross-scale cosine attention according to claim 2 is characterized in that: The cascaded cross-scale Transformer block includes multiple cross-scale Transformer blocks, and the specific process of each cross-scale Transformer block is as follows: The input features are passed to the layer normalization layer and the cross-scale cosine attention layer in sequence, and the input features and the output features of the cross-scale cosine attention layer are added element by element. The obtained features are then passed to the layer normalization layer and the ResMLP layer in sequence. Finally, the output of the ResMLP layer is added element by element to the output of the cross-scale cosine attention layer through the residual connection to output the features.
5. The remote sensing image super-resolution reconstruction method based on cross-scale cosine attention according to claim 4 is characterized in that: The specific process of the ResMLP layer is as follows: The input features pass through the affine transformation layer, layer normalization layer, activation function, layer normalization layer and affine transformation layer in sequence to obtain the output features.
6. The remote sensing image super-resolution reconstruction method based on cross-scale cosine attention according to claim 2, characterized in that: The distilled features extracted by each cascaded feature distillation block are input into the dense feature fusion layer in the form of residual connection to obtain features. The residual connection method realizes efficient feature fusion without increasing network parameters. The residual connection method is used to fuse the shallow features of the image to obtain the feature map to be reconstructed.
7. A remote sensing image super-resolution reconstruction system based on cross-scale cosine attention, characterized in that: include: The first feature extraction module is used to perform a convolution operation on the original low-resolution image to obtain shallow features of the image; The second feature extraction module is used to extract shallow features of the image using a deep feature extraction layer to obtain deep features; The third feature extraction module is used to fuse the deep features in the dense feature fusion layer to obtain fused features, and the fused features are fused with the shallow features of the image after passing through the convolution layer to obtain the features to be reconstructed; The feature reconstruction module is used to process the features to be reconstructed through the upsampling module to obtain super-resolution reconstruction results.
8. The remote sensing image super-resolution reconstruction system based on cross-scale cosine attention according to claim 7, characterized in that: The deep feature extraction layer includes a plurality of cascade feature distillation blocks, and the cascade feature distillation blocks are specifically: The input features first pass through a multi-branch spatial attention module, and then enter two pipelines in parallel, namely the feature distillation pipeline and the feature refinement pipeline. The feature distillation pipeline consists of multiple 1×1 convolutional layers, and the feature refinement pipeline is a cascaded cross-scale Transformer block that connects the iterative distilled features of the operation, adds 1×1 convolutional layers and 3×3 convolutional layers, and finally adds a multi-branch spatial attention module to obtain deep features.
9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the remote sensing image super-resolution reconstruction method based on cross-scale cosine attention are implemented as described in any one of claims 1 to 6.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the remote sensing image super-resolution reconstruction method based on cross-scale cosine attention are implemented as described in any one of claims 1 to 6.