Progressive sparse attention multi-scale agricultural remote sensing image super-resolution reconstruction method
The multi-scale agricultural remote sensing image super-resolution reconstruction method using progressive sparse attention solves the problems of high computational cost, limited receptive field, and insufficient multi-scale feature capture in existing technologies, achieving efficient and high-definition agricultural remote sensing image reconstruction and improving the model's processing efficiency and reconstruction quality.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ANQING NORMAL UNIV
- Filing Date
- 2026-01-19
- Publication Date
- 2026-05-05
AI Technical Summary
Existing deep learning-based super-resolution reconstruction methods for agricultural remote sensing images suffer from huge computational overhead, limited receptive field, and insufficient multi-scale feature capture capabilities, making it difficult to meet the demand for efficient and high-definition reconstruction in agricultural scenarios.
A multi-scale agricultural remote sensing image super-resolution reconstruction method with progressive sparse attention is adopted. By constructing a super-resolution reconstruction network, including a shallow feature extraction module, a hybrid attention module, a multi-scale reconstruction module, and an upsampling module, and combining local and global attention mechanisms, the method dynamically focuses on key areas and integrates features from different receptive fields.
It significantly reduces computational and memory overhead, improves processing efficiency, enhances the ability to model long-range dependencies and large-scale structures in agricultural scenarios, improves the overall structural consistency and spectral characteristics of reconstructed images, and takes into account the detailed textures and global contours of land features at different scales.
Smart Images
Figure CN121982525A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image processing technology, and in particular relates to a multi-scale agricultural remote sensing image super-resolution reconstruction method with progressive sparse attention. Background Technology
[0002] Multispectral super-resolution agricultural remote sensing images are a key data source for precision agriculture management tasks such as crop growth monitoring and pest and disease identification. However, constrained by objective factors such as sensor cost and physical diffraction limits, directly acquired remote sensing images often lack sufficient spatial resolution, making it difficult to meet the high-precision analysis requirements for ground feature details. Therefore, remote sensing image super-resolution reconstruction technology has emerged, aiming to recover super-resolution images from low-resolution images. Early reconstruction methods based on interpolation and other traditional methods had limited reconstruction capabilities. In recent years, deep learning-based super-resolution methods have made significant progress due to their powerful feature learning capabilities and have been gradually applied to the field of remote sensing image processing. Existing deep learning-based super-resolution reconstruction methods mainly follow two main architectures: one is based on convolutional neural networks, which extracts features and performs upsampling by stacking convolutional layers; the other is based on Transformer architecture, which uses self-attention mechanisms to model the dependencies within the image. These methods perform well on general natural images, but they still face a series of fundamental challenges when processing agricultural multispectral remote sensing images with unique properties.
[0003] First, the problem of enormous computational overhead is particularly prominent. Agricultural multispectral data typically contains multiple spectral channels, with data dimensionality and information content far exceeding that of ordinary RGB images. This leads to existing complex network models, especially Transformer-type models that rely on global interactions, facing immense computational and memory pressure during processing. The high training and inference costs severely restrict the deployment of models in practical agricultural applications, making it difficult to meet the stringent timeliness requirements of scenarios such as field monitoring. Second, existing methods have limited receptive fields, making it difficult to effectively model long-range dependencies and large-scale structures in agricultural scenes. Agricultural remote sensing images widely contain macroscopic ground features such as regular textures of contiguous farmland, continuous outlines of large ditches or shelterbelts. However, existing methods mostly rely on local operations and lack the ability to effectively model the global semantic consistency and macroscopic structure of images, resulting in insufficient overall structural coherence in the reconstructed images. Finally, the insufficient ability to capture multi-scale features of agricultural scenes is also a key drawback. A typical agricultural remote sensing image simultaneously contains ground features of different scales, such as farmland, crops, and ditches. Traditional super-resolution methods often employ single-scale feature extraction strategies, making it difficult to simultaneously restore local crop details and reconstruct the global farmland pattern, thus limiting their practicality in multi-level agricultural analysis tasks. Solving this problem is challenging because reducing computational complexity often comes at the cost of sacrificing model expressive power, while expanding the receptive field and capturing multi-scale features often further exacerbates the computational burden. Achieving efficient and high-fidelity reconstruction with limited resources remains a long-standing technical challenge. Summary of the Invention
[0004] To address the aforementioned technical problems, this invention proposes a multi-scale agricultural remote sensing image super-resolution reconstruction method with progressive sparse attention, thereby resolving the issues present in the prior art.
[0005] In a first aspect, to achieve the above objectives, the present invention provides a multi-scale agricultural remote sensing image super-resolution reconstruction method with progressive sparse attention. The method is implemented through a constructed super-resolution reconstruction network, which sequentially includes a shallow feature extraction module, a hybrid attention module, a multi-scale reconstruction module, and an upsampling module.
[0006] S1. The shallow feature extraction module performs initial feature extraction and flattening on the input low-resolution multispectral agricultural remote sensing image.
[0007] S2. Input the features obtained in S1 into the hybrid attention module. The hybrid attention module alternately executes the progressive sparse local attention extraction process and the progressive sparse global attention extraction process to obtain deep features that integrate local details and global semantics in an iterative manner.
[0008] S3. After fusing the deep features obtained in S2 with the shallow features obtained in S1, input them into the multi-scale reconstruction module. Extract and fuse features from different receptive fields through parallel multi-scale convolutional paths, and perform detail enhancement and spectral correlation modeling to obtain multi-scale reconstruction features.
[0009] S4. Input the multi-scale reconstruction features obtained in S3 into the upsampling module to improve the resolution and output a super-resolution multispectral agricultural remote sensing image.
[0010] Optionally, the shallow feature extraction module in S1 performs initial feature extraction and flattening processing on the input low-resolution multispectral agricultural remote sensing image, including:
[0011] Data from four channels (R, G, B, and NIR) are extracted from the low-resolution agricultural remote sensing image. Shallow feature extraction is performed on the four-channel data using a 3×3 convolutional layer. The extracted two-dimensional spatial features are flattened into a three-dimensional tensor and then normalized.
[0012] Optionally, the hybrid attention module in S2 performs progressive sparse local attention extraction including:
[0013] After layer normalization of the input features, query, key, and value matrices are generated through linear projection; the value matrices are grouped and convolved to generate local positional encodings; based on window partitioning, scaled dot product attention with the local positional encodings is calculated in each window; within each window, a trainable progressive focus matrix is generated based on the query matrix and key matrix, and interacted with the value matrix to obtain progressive sparse attention features, wherein the parameters of the progressive focus matrix change dynamically during the training process.
[0014] Optionally, in S2, the hybrid attention module alternately executes the progressive sparse local attention extraction process and the progressive sparse global attention extraction process. The difference between the progressive sparse global attention extraction process and the progressive sparse local attention extraction process is that when dividing the input features into windows, an interval block strategy is used instead of a continuous block strategy.
[0015] Optionally, in step S3, after fusing deep and shallow features, the input to the multi-scale reconstruction module includes:
[0016] The deep features are processed by a 3×3 convolution, and then added element-wise to the shallow features.
[0017] Optionally, the multi-scale reconstruction module in S3 extracts and fuses different receptive field features through parallel multi-scale convolutional paths, including:
[0018] The input features are fed into at least three parallel convolutional paths simultaneously. The first path uses a 1×1 convolution, the second path uses a 3×3 convolution with a stride of 2 followed by an upsampling layer, and the third path uses two 3×3 convolutions with a stride of 2 followed by an upsampling layer. The feature maps output from each path are then concatenated and fused using convolution.
[0019] Optionally, the detail enhancement in the multi-scale reconstruction module of S3 includes:
[0020] The features obtained after convolution fusion are processed by a convolutional layer group with residual connection structure, which contains two cascaded 3×3 convolutional layers whose outputs are added to the inputs element by element.
[0021] Optionally, the multi-scale reconstruction module in S3 performs spectral correlation modeling including:
[0022] The enhanced features are grouped by channel, and global average pooling is performed on each group of features. Attention weights between channels are calculated interactively through a fully connected layer. The original features are then weighted and adjusted using these attention weights.
[0023] In a second aspect, the present invention also provides a computer terminal device, comprising:
[0024] One or more processors;
[0025] A memory, coupled to the processor, for storing one or more programs;
[0026] When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the multi-scale agricultural remote sensing image super-resolution reconstruction method with progressive sparse attention in the first aspect described above.
[0027] Thirdly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the steps of the multi-scale agricultural remote sensing image super-resolution reconstruction method with progressive sparse attention described in the first aspect above.
[0028] Compared with the prior art, the present invention has the following advantages and technical effects:
[0029] This invention provides a multi-scale agricultural remote sensing image super-resolution reconstruction method using progressive sparse attention. By employing a progressive sparse attention mechanism, the model can dynamically focus on key regions in the image and optimize attention distribution, thereby significantly reducing computational and memory overhead and improving processing efficiency. Through a hybrid attention mechanism combining local and global attention modules, cross-regional global semantic associations are effectively established, enhancing the model's ability to model long-range dependencies and large-scale structures in agricultural scenes and improving the overall structural consistency of the reconstructed image. By fusing features from different receptive fields through a multi-scale reconstruction module, supplemented by detail enhancement and spectral correlation modeling, the scale diversity of image features is enhanced, taking into account the detailed textures and global contours of ground features at different scales. This significantly improves the spatial resolution of the reconstructed image while maintaining its spectral characteristics and the integrity of ground feature details. Attached Figure Description
[0030] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:
[0031] Figure 1 This is a schematic diagram of the input feature segmentation method for the local aggregation module and the global aggregation module in an embodiment of the present invention, wherein (a) is the local aggregation module and (b) is the global aggregation module;
[0032] Figure 2 This is a flowchart illustrating a multi-scale agricultural remote sensing image super-resolution reconstruction method based on progressive sparse attention, according to an embodiment of the present invention.
[0033] Figure 3 This is a schematic diagram of the architecture of a multi-scale agricultural remote sensing image super-resolution reconstruction method with progressive sparse attention according to an embodiment of the present invention.
[0034] Figure 4 This is a schematic diagram of feature extraction for the progressive sparse local / global attention module in the architecture diagram of an embodiment of the present invention. Detailed Implementation
[0035] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.
[0036] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.
[0037] like Figures 2-3As shown, this embodiment provides a multi-scale agricultural remote sensing image super-resolution reconstruction method with progressive sparse attention, including:
[0038] The method is implemented by constructing a super-resolution reconstruction network, which sequentially includes a shallow feature extraction module, a hybrid attention module, a multi-scale reconstruction module, and an upsampling module.
[0039] S1. The shallow feature extraction module performs initial feature extraction and flattening on the input low-resolution multispectral agricultural remote sensing image.
[0040] S2. Input the features obtained in S1 into the hybrid attention module. The hybrid attention module alternately executes the progressive sparse local attention extraction process and the progressive sparse global attention extraction process to obtain deep features that integrate local details and global semantics in an iterative manner.
[0041] S3. After fusing the deep features obtained in S2 with the shallow features obtained in S1, input them into the multi-scale reconstruction module. Extract and fuse features from different receptive fields through parallel multi-scale convolutional paths, and perform detail enhancement and spectral correlation modeling to obtain multi-scale reconstruction features.
[0042] S4. Input the multi-scale reconstruction features obtained in S3 into the upsampling module to improve the resolution and output a super-resolution multispectral agricultural remote sensing image.
[0043] Furthermore, the shallow feature extraction module in S1 performs initial feature extraction and flattening processing on the input low-resolution multispectral agricultural remote sensing image, including:
[0044] Data from four channels (R, G, B, and NIR) are extracted from the low-resolution agricultural remote sensing image. Shallow feature extraction is performed on the four-channel data using a 3×3 convolutional layer. The extracted two-dimensional spatial features are flattened into a three-dimensional tensor and then normalized.
[0045] Specifically, the implementation process of this embodiment includes:
[0046] Step (1) Extract the R, G, B, and NIR channels from the low-resolution agricultural remote sensing image to form a low-resolution multispectral agricultural remote sensing image with dimensions batchsize×4×h×w, where batchsize is the number of samples used in each iteration of the model, 4 represents the four channels R, G, B, and NIR, and h×w represents the two spatial resolutions of the agricultural remote sensing image. The shallow feature extraction network contains a convolution with a kernel size of 3×3, a stride of 1, and padding of 1. The low-resolution multispectral remote sensing image is fed into the shallow feature extraction network to obtain the shallow features X of the low-resolution image. ShallowAt this point, the dimension is batchsize×240×h×w. The shallow feature is flattened into a three-dimensional feature with a dimension of batchsize×240×hw. The second and third channels are transposed to become a feature with a dimension of batchsize×hw×240. This is then fed into the layer for normalization to generate the flattened feature.
[0047] Furthermore, the progressive sparse local attention extraction performed by the hybrid attention module in S2 includes:
[0048] After layer normalization of the input features, query, key, and value matrices are generated through linear projection; the value matrices are grouped and convolved to generate local positional encodings; based on window partitioning, scaled dot product attention with the local positional encodings is calculated in each window; within each window, a trainable progressive focus matrix is generated based on the query matrix and key matrix, and interacted with the value matrix to obtain progressive sparse attention features, wherein the parameters of the progressive focus matrix change dynamically during the training process.
[0049] Specifically, the implementation process of this embodiment includes:
[0050] like Figure 4 As shown, step (2) feeds the input features into the progressive local sparse attention module. The input features are either the flattened features T from step (1) or the progressive sparse attention local features from step (2), denoted as T. This module first feeds the features into the layer normalization layer, and then transforms the obtained features into a query, key, and value matrix Q through linear projection. w K w V w It contains one fully connected layer with 240 input channels and 720 output channels. Q w K w V w The dimensions of V are all 1×4096×240. w After the dimensions are transformed to 1×240×64×64, the data is fed into a 7×7 grouped convolution with a stride of 1, padding of 3, and 240 groups. The features are then flattened to obtain the local location encoding V. wpart The dimensions are 1×4096×240. Then, the W-MSA module is used. The W-MSA module will convert Q... w K w V w V wpart The window is divided into non-overlapping small windows with a size of m=32 (single window n=64, number of windows (H / 32)×(W / 32)), with a head of h=6 and a single head dimension of d=40. The scaling dot product attention is calculated within each window. And then add relative position offsets, softmax, and V. w Multiply and add Vwpart V wattnpart Within each window, Q will be divided according to the window. w K w The product is fed into a 1×1 convolution and multiplied by the progressive parameters to obtain a trainable progressive focusing PAT matrix S. Ktanhw S Ktanhw The calculation formula is as follows:
[0051] S Ktanhw (1);
[0052] Where t is the parameter obtained after a 1×1 convolution. It is a parameter that changes with the epoch (when all images in a complete dataset have been trained on the neural network once, this process is called an epoch). In this embodiment, S can be seen from the image. Ktanhw The value can only be 0 or 1. This is certainly impossible in reality, but this embodiment assumes that when When choosing a relatively large number, It can be approximated as infinity. The threshold is set to 0.1, with values above 0.1 set to 1 and values below 0.1 set to 0. The formula for how this changes with epochs is as follows:
[0053] (2);
[0054] in It is the initial one. value, It is the final one value, This indicates which epoch is currently being executed during the model training process. This represents the total number of epochs required for training. In this example... Take 1, Take 5000.
[0055] Q after window splitting w K w With S Ktanhw Multiply, we get A PFTw And save it, if its previous W-MSA layer has saved A PFTw Then the current A PFTw Needs to connect with the previous layer A PFTw Multiply to get a new A PFTw . A PFTw Multiply by V and add to V wattnpartAfter element-wise addition, all windows are merged and shifted in reverse to restore the original order. Then, a fully connected layer with 240 input and 240 output channels is applied for channel blending, resulting in a dimension of 4×1024×240. The channel dimension is then transformed to 4×32×32×240, and finally to 1×4096×240. This transformed channel dimension is then added to feature T to obtain feature X. wattn The data is then fed into the convFFN module, which first performs layer normalization, followed by a fully connected layer with 240 input channels and 480 output channels, a GELU activation layer, which transposes the second and third channels and changes the dimensions to 1×480×64×64 before feeding it into a 7×7 grouped convolution with a stride of 1, padding of 3, and 480 groups. This is followed by a GELU activation layer, where the third and fourth channels are flattened and transposed with the second channel, changing the dimensions to 1×4096×480. This is then passed through a fully connected layer with 480 input channels and 240 output channels, and finally combined with feature X. wattn Add them together to obtain feature T. w The dimensions are 1×4096×240. The features are then fed into a normalized layer, and linear projection is used to transform the acquired features into a query, key, and value matrix Q. sw K sw V sw It contains one fully connected layer with 240 input channels and 720 output channels. Q sw K sw V sw The dimensions of V are all 1×4096×240. sw After the dimensions are transformed to 1×240×64×64, the data is fed into a 7×7 grouped convolution with a stride of 1, padding of 3, and 240 groups. The features are then flattened to obtain the local location encoding V. swpart The dimensions are 1×4096×240. The SW-MSA module is then used. During window partitioning, the SW-MSA module first performs a (16,16) cyclic translation on the 2D features, then divides them into non-overlapping 32×32 windows (single window n=64, number of windows (H / 32)×(W / 32)), with a head of h=6 and a single-head dimension of d=40. Scaling dot product attention is calculated within each window. And then add relative position offsets, softmax, and V. sw Multiply and add V swpart V swattnpart Within each window, Q will be divided according to the window. sw K sw The product is fed into a 1×1 convolution and multiplied by the progressive parameters to obtain a trainable progressive focusing PAT matrix S. Ktanhsw S Ktanhsw The calculation formula is the same as formula (1). Qsw K sw With S Ktanhsw Multiply, we get A PFTsw And save it, if its previous SW-MSA layer has saved A PFTsw Then the current A PFTsw Needs to connect with the previous layer A PFTsw Multiply to get a new A PFTsw . A PFTsw With V sw Multiply and add to V swattnpart After element-wise addition, all windows are merged and shifted in reverse to restore the original order; then, a fully connected layer with 240 input and 240 output channels is applied for channel blending, resulting in a dimension of 4×1024×240. The channel dimension is then transformed to 4×32×32×240, and finally to 1×4096×240, and then combined with feature T. w Add them together to obtain feature X. swattn The data is then fed into the convFFN module, which first performs layer normalization, followed by a fully connected layer with 240 input channels and 480 output channels, a GELU activation layer, which transposes the second and third channels and changes the dimensions to 1×480×64×64 before feeding it into a 7×7 grouped convolution with a stride of 1, padding of 3, and 480 groups. This is followed by a GELU activation layer, where the third and fourth channels are flattened and transposed with the second channel, changing the dimensions to 1×4096×480. This is then passed through a fully connected layer with 480 input channels and 240 output channels, and finally combined with feature X. swattn Adding them together, we obtain the progressive local sparse attention feature with a dimension of 1×4096×240.
[0056] Furthermore, in S2, the hybrid attention module alternately executes the progressive sparse local attention extraction process and the progressive sparse global attention extraction process. The difference between the progressive sparse global attention extraction process and the progressive sparse local attention extraction process is that when dividing the input features into windows, an interval block strategy is used instead of a continuous block strategy.
[0057] Specifically, the implementation process of this embodiment includes:
[0058] Step (3): The progressive sparse attention deep features obtained in step (2) are used as input and step (2) is repeated k times to finally obtain the progressive sparse attention deep features. In this embodiment, k is 4.
[0059] The calculation process of step (4) is similar to that of step (2), the only difference being the input feature segmentation method. Step (2) uses continuous segmentation, while step (4) uses interval segmentation, such as... Figure 1 As shown, where Figure 1 (a) is a local aggregation module. Figure 1 (b) represents all aggregation modules. The final result is a progressive sparse hybrid attention feature.
[0060] Further, in step (5), the progressive sparse hybrid attention features obtained in step (4) are used as input to repeat steps (2) to (4) k times to obtain deep features. In this embodiment, k is 4.
[0061] Furthermore, in step S3, after fusing deep and shallow features, the input to the multi-scale reconstruction module includes:
[0062] The deep features are processed by a 3×3 convolution, and then added element-wise to the shallow features.
[0063] Furthermore, the multi-scale reconstruction module in S3 extracts and fuses features from different receptive fields through parallel multi-scale convolutional paths, including:
[0064] The input features are fed into at least three parallel convolutional paths simultaneously. The first path uses a 1×1 convolution, the second path uses a 3×3 convolution with a stride of 2 followed by an upsampling layer, and the third path uses two 3×3 convolutions with a stride of 2 followed by an upsampling layer. The feature maps output from each path are then concatenated and fused using convolution.
[0065] Furthermore, the detail enhancement in the multi-scale reconstruction module of S3 includes:
[0066] The features obtained after convolution fusion are processed by a convolutional layer group with residual connection structure, which contains two cascaded 3×3 convolutional layers whose outputs are added to the inputs element by element.
[0067] Furthermore, the multi-scale reconstruction module in S3 performs spectral correlation modeling, including:
[0068] The enhanced features are grouped by channel, and global average pooling is performed on each group of features. Attention weights between channels are calculated interactively through a fully connected layer. The original features are then weighted and adjusted using these attention weights.
[0069] Specifically, the implementation process of this embodiment includes:
[0070] like Figure 4 As shown, in step (6), the obtained deep features are fed into a 3×3 convolution with a stride of 1 and a padding value of 1, and then accumulated with the shallow features in step (1) to obtain feature X. out5 The data is then fed into the multi-scale reconstruction module. First, X... out5The input is a 1×1 convolution and a ReLU activation layer to obtain the first receptive field feature X. out61 The feature dimensions are 1×80×64×64. Let X... out5 The input is a 3×3 convolution with a stride of 2 and padding of 1, a ReLU activation layer, and a bilinear interpolation upsampling layer to obtain the second receptive field feature X. out62 The feature dimensions are 1×80×64×64. Let X... out5 The feature dimension is changed from 1×240×64×64 to 1×80×32×32 by feeding a 3×3 convolution with a stride of 2 and padding of 1, followed by a bilinear interpolation upsampling layer, resulting in a third receptive field feature X. out63 The feature dimensions are 1×80×64×64. Let X... out61 and X out62 The features are concatenated along the channel dimension, resulting in a feature dimension of 1×160×64×64. This is then fed into a 1×1 convolution to transform the feature dimension to 1×240×64×64. Finally, a ReLU activation layer and a 3×3 convolution with a stride of 1 and padding of 1 are applied to obtain the fused feature X. fusion61 The feature dimensions are 1×240×64×64. X fusion61 and X out63 The features are concatenated along the channel dimension, resulting in a feature dimension of 1×320×64×64. This is then fed into a 1×1 convolution, transforming the feature dimension to 1×240×64×64. Finally, a ReLU activation layer and a 3×3 convolution with a stride of 1 and padding of 1 are applied to obtain the fused feature X. fusion62 The feature dimensions are 1×240×64×64. Then, a 3×3 convolution with a stride of 1 and padding of 1, a ReLU activation layer, and a 1×1 convolution are applied to obtain the fused feature X. fusion6 Then it is sent to the detail enhancement module. In the detail enhancement module, X is first... fusion6 The input is a 3×3 convolution with a stride of 1 and padding of 1, transforming the feature dimension from 1×240×64×64 to 1×240×64×64. This is followed by a ReLU activation layer and another 3×3 convolution with a stride of 1 and padding of 1, then combined with X... fusion6 Adding them together yields the feature X that enhances the details. enhance The feature dimensions are 1×240×64×64. Next, we proceed to the spectral correlation modeling module, which will... enhanceThe data is divided into 4 groups based on channels, and global average pooling is performed within each group. The data is then passed through a fully connected layer with 60 input channels and 15 output channels, a GELU activation layer, and another fully connected layer with 15 input channels and 60 output channels to obtain feature D, which has a dimension of 1×4×60. The second and third dimensions of D are transposed, cross-multiplied by D, and then divided. For each row, perform softmax to obtain attention A. Then, cross-product A and D, and expand the tensor to a dimension of 1×4×60×64×64 using a broadcast mechanism. Finally, reorganize the dimensions to 1×240×64×64, and then combine it with X. enhance After addition, perform grouping and normalization with 4 groups to obtain feature X. G6 Then, a 1×1 convolution is used to change the feature dimension from 1×240×64×64 to 1×60×64×64. A GELU activation layer, a 1×1 convolution, and a Sigmoid activation layer are used to obtain a feature X with a dimension of 1×240×64×64 through weight w. The enhanced feature X is then generated using formula (3). Gw Then, a 1×1 convolution is used to change the feature dimension from 1×240×64×64 to 1×240×64×64. A 3×3 convolution with a stride of 1 and padding of 1 is used to change the feature dimension from 1×240×64×64 to 1×240×64×64. A GELU activation layer and a 3×3 convolution with a stride of 1 and padding of 1 are used to obtain multi-scale reconstructed features.
[0071] X Gw =X G6 *(1+w)(3).
[0072] Step (7) sends the multi-scale reconstruction features obtained in step (6) into the PixelShuffle module for upsampling to obtain a super-resolution image with a specified magnification factor and a dimension of 1×4×256×256.
[0073] In this embodiment, a computer terminal device is provided, including:
[0074] One or more processors;
[0075] A memory, coupled to the processor, for storing one or more programs;
[0076] When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the above-described progressive sparse attention multi-scale agricultural remote sensing image super-resolution reconstruction method.
[0077] In this embodiment, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the above-described progressive sparse attention multi-scale agricultural remote sensing image super-resolution reconstruction method.
[0078] This invention provides a multi-scale agricultural remote sensing image super-resolution reconstruction method using progressive sparse attention. By employing a progressive sparse attention mechanism, the model can dynamically focus on key regions in the image and optimize attention distribution, thereby significantly reducing computational and memory overhead and improving processing efficiency. Through a hybrid attention mechanism combining local and global attention modules, cross-regional global semantic associations are effectively established, enhancing the model's ability to model long-range dependencies and large-scale structures in agricultural scenes and improving the overall structural consistency of the reconstructed image. By fusing features from different receptive fields through a multi-scale reconstruction module, supplemented by detail enhancement and spectral correlation modeling, the scale diversity of image features is enhanced, taking into account the detailed textures and global contours of ground features at different scales. This significantly improves the spatial resolution of the reconstructed image while maintaining its spectral characteristics and the integrity of ground feature details.
[0079] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A multi-scale agricultural remote sensing image super-resolution reconstruction method with progressive sparse attention, characterized in that, The method is implemented by constructing a super-resolution reconstruction network, which sequentially includes a shallow feature extraction module, a hybrid attention module, a multi-scale reconstruction module, and an upsampling module. S1. The shallow feature extraction module performs initial feature extraction and flattening on the input low-resolution multispectral agricultural remote sensing image. S2. Input the features obtained in S1 into the hybrid attention module. The hybrid attention module alternately executes the progressive sparse local attention extraction process and the progressive sparse global attention extraction process to obtain deep features that integrate local details and global semantics in an iterative manner. S3. After fusing the deep features obtained in S2 with the shallow features obtained in S1, input them into the multi-scale reconstruction module. Extract and fuse features from different receptive fields through parallel multi-scale convolutional paths, and perform detail enhancement and spectral correlation modeling to obtain multi-scale reconstruction features. S4. Input the multi-scale reconstruction features obtained in S3 into the upsampling module to improve the resolution and output a super-resolution multispectral agricultural remote sensing image.
2. The method according to claim 1, characterized in that, The shallow feature extraction module in S1 performs initial feature extraction and flattening processing on the input low-resolution multispectral agricultural remote sensing image, including: Data from four channels (R, G, B, and NIR) are extracted from the low-resolution agricultural remote sensing image. Shallow feature extraction is performed on the four-channel data using a 3×3 convolutional layer. The extracted two-dimensional spatial features are flattened into a three-dimensional tensor and then normalized.
3. The method according to claim 1, characterized in that, The hybrid attention module in S2 performs progressive sparse local attention extraction, including: After layer normalization of the input features, query, key, and value matrices are generated through linear projection; the value matrices are grouped and convolved to generate local positional encodings; based on window partitioning, scaled dot product attention with the local positional encodings is calculated in each window; within each window, a trainable progressive focus matrix is generated based on the query matrix and key matrix, and interacted with the value matrix to obtain progressive sparse attention features, wherein the parameters of the progressive focus matrix change dynamically during the training process.
4. The method according to claim 3, characterized in that, In S2, the hybrid attention module alternately executes the progressive sparse local attention extraction process and the progressive sparse global attention extraction process. The difference between the progressive sparse global attention extraction process and the progressive sparse local attention extraction process is that when dividing the input features into windows, the interval block strategy is used instead of the continuous block strategy.
5. The method according to claim 1, characterized in that, The S3 step, which fuses deep and shallow features, is then input into the multi-scale reconstruction module, including: The deep features are processed by a 3×3 convolution, and then added element-wise to the shallow features.
6. The method according to claim 1, characterized in that, The multi-scale reconstruction module in S3 extracts and fuses different receptive field features through parallel multi-scale convolutional paths, including: The input features are fed into at least three parallel convolutional paths simultaneously. The first path uses a 1×1 convolution, the second path uses a 3×3 convolution with a stride of 2 followed by an upsampling layer, and the third path uses two 3×3 convolutions with a stride of 2 followed by an upsampling layer. The feature maps output from each path are then concatenated and fused using convolution.
7. The method according to claim 6, characterized in that, The multi-scale reconstruction module in S3 performs detail enhancements including: The features obtained after convolution fusion are processed by a convolutional layer group with residual connection structure, which contains two cascaded 3×3 convolutional layers whose outputs are added to the inputs element by element.
8. The method according to claim 7, characterized in that, The multi-scale reconstruction module in S3 performs spectral correlation modeling, including: The enhanced features are grouped by channel, and global average pooling is performed on each group of features. Attention weights between channels are calculated interactively through a fully connected layer. The original features are then weighted and adjusted using these attention weights.
9. A computer terminal device, characterized in that, include: One or more processors; A memory, coupled to the processor, for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors perform the steps of the method as described in any one of claims 1-8.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-8.