Multi-scale agricultural remote sensing image super-resolution reconstruction method based on adaptive attention

Abstract

The invention relates to a multi-scale agricultural remote sensing image super-resolution reconstruction method based on adaptive attention. The method comprises the following steps: S1, acquiring shallow layer features of a low-resolution image; s2, performing adaptive attention processing on the shallow layer features to obtain adaptive attention multi-scale features; s3, performing aggregation processing on the self-adaptive attention multi-scale features to obtain global aggregation features, accumulating the global aggregation features and the self-adaptive attention multi-scale features, and performing deep feature extraction to obtain current deep features; s4, repeating the steps S2-S3 for the current deep feature until a target number of times, and obtaining a final deep feature; and S5, accumulating the final deep features and shallow features, and performing up-sampling processing to obtain a super-resolution agricultural remote sensing image. According to the method, the quality of the low-resolution agricultural remote sensing image can be improved, and high-resolution image reconstruction is realized, so that the requirements of the fields of agricultural monitoring, precision agricultural management and the like on the high-resolution remote sensing image are met.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a multi-scale agricultural remote sensing image super-resolution reconstruction method based on adaptive attention. Background Art

[0002] Agricultural remote sensing technology is an important part of the development of modern agriculture. Remote sensing images obtained by satellites or drones can effectively monitor the growth status of crops, the distribution of pests and diseases, soil moisture and other information, thereby realizing precision agricultural management. However, the resolution of remote sensing images directly affects their application effect. High-resolution images can provide clearer and more detailed surface information, which helps to more accurately analyze the various characteristics of agricultural plots. However, due to the technical limitations of remote sensing sensors and the influence of imaging conditions, directly obtaining high-resolution images faces high technical and cost challenges.

[0003] At present, super-resolution technology has been widely studied and applied to improve the resolution of remote sensing images. Traditional super-resolution methods mainly include interpolation methods and reconstruction-based methods. Interpolation methods such as bilinear interpolation and bicubic interpolation, although simple to calculate, often cannot effectively improve the details and quality of the image. The reconstruction-based method reconstructs a high-resolution image by fusing multiple low-resolution images. Although it can improve the image quality to a certain extent, it has high requirements on the consistency of the image sequence and has high computational complexity. In recent years, deep learning technology has shown great potential in super-resolution image reconstruction. Convolutional neural network CNN can capture the high-frequency details of the image through large-scale data training, thereby generating high-quality high-resolution images. For example, SRCNN, VDSR, ESRGAN (and other models have shown excellent performance on a variety of data sets. However, these deep learning models still have some significant deficiencies when processing agricultural remote sensing images:

[0004] Poor model generalization: Existing deep learning models perform inconsistently on different types of agricultural remote sensing images, especially in complex agricultural scenes, where reconstruction distortion and detail loss are prone to occur.

[0005] Lack of optimization for agricultural scenarios: Most super-resolution models fail to fully consider the characteristics and requirements of agricultural remote sensing images and fail to effectively improve image quality in specific agricultural application scenarios. Summary of the invention

[0006] The purpose of this invention is to propose a multi-scale agricultural remote sensing image super-resolution reconstruction method based on adaptive attention to solve the problems existing in the above-mentioned prior art. By utilizing deep learning models and a variety of advanced image processing techniques, it aims to improve the quality of low-resolution agricultural remote sensing images and achieve high-resolution image reconstruction to meet the needs of agricultural monitoring, precision agriculture management and other fields for high-resolution remote sensing images.

[0007] To achieve the above object, the present invention provides the following solutions:

[0008] The multi-scale agricultural remote sensing image super-resolution reconstruction method based on adaptive attention includes:

[0009] S1. Extract shallow features from low-resolution RGB agricultural remote sensing images to obtain shallow features of low-resolution images;

[0010] S2. performing multi-scale adaptive attention processing on the shallow features to obtain adaptive attention multi-scale features;

[0011] S3. Aggregate the adaptive attention multi-scale features to obtain global aggregate features, add the global aggregate features to the adaptive attention multi-scale features, perform deep feature extraction, and obtain current deep features;

[0012] S4. Repeat S2-S3 for the current deep feature until the target number of times to obtain the final deep feature;

[0013] S5. After accumulating the final deep features and the shallow features, upsampling is performed to obtain a super-resolution agricultural remote sensing image.

[0014] Optionally, shallow feature extraction of low-resolution RGB agricultural remote sensing images includes:

[0015] Extract low-resolution RGB agricultural remote sensing images from agricultural remote sensing images;

[0016] Shallow feature extraction is performed on the low-resolution RGB agricultural remote sensing image.

[0017] Optionally, performing multi-scale adaptive attention processing on the shallow features includes:

[0018] The shallow features are first convolved and then copied and passed through a first masked convolutional layer and a second masked convolutional layer respectively to obtain two sets of scale features;

[0019] The two sets of scale features are added and fused, and then global average pooling is performed to obtain channel information features;

[0020] Perform channel information fusion on the channel information features, copy them and input them into a fully connected layer respectively, and perform a softmax operation on the channel dimension to obtain the attention weights of two channels;

[0021] The two channel attention weights are multiplied by the corresponding two sets of scale features, and then the features are added to obtain the adaptive attention features;

[0022] The adaptive attention feature and the shallow feature are added to obtain an adaptive attention multi-scale feature.

[0023] Optionally, aggregating the adaptive attention multi-scale features includes:

[0024] Performing local aggregation processing on the adaptive attention multi-scale features to obtain local aggregation features;

[0025] Perform global aggregation processing on the local aggregation features to obtain global aggregation features.

[0026] Optionally, performing local aggregation processing on the adaptive attention multi-scale features includes:

[0027] Performing spatial dimension processing on the adaptive attention multi-scale feature to obtain a spatial attention feature;

[0028] The spatial attention feature is subjected to feature information suppression processing to obtain X GDFN feature;

[0029] For the X GDFN The features are processed by channel self-attention to obtain channel self-attention features;

[0030] The channel self-attention feature is combined with X GDFN The features are added to obtain the local aggregated features.

[0031] Optionally, obtaining the spatial attention feature includes:

[0032] Divide the adaptive attention multi-scale features into non-overlapping local blocks of size P×P;

[0033] The non-overlapping local blocks are transformed in spatial dimension to obtain feature X;

[0034] Transform the dimension of the feature X and transform the feature X into a query Q through linear projection S , key K S Sum value matrix V S ;

[0035] For the query Q S , key K S Sum value matrix VS Perform multi-head attention processing to obtain

[0036] Calculation query and key The product of and softmax processing is performed to obtain the spatial attention weight;

[0037] The spatial attention weights and value matrix Multiply to obtain preset features;

[0038] The preset features are dimensionally transformed and fully connected, and the features after the fully connected processing are added to the feature X to obtain the spatial attention features.

[0039] Optionally, obtain the X GDFN Features include:

[0040] The spatial attention feature is transformed into a dimension, denoted as X GDFNIN ;

[0041] X GDFNIN After the dimension transformation, it is convolved channel by channel and evenly split into two features X GDFN1 , X GDFN2 ;

[0042] X GDFN1 After GELU activation function and X GDFN2 Multiply and then convolve, and X GDFNIN Add together to get the X GDFN feature.

[0043] Optionally, obtaining the channel self-attention feature includes:

[0044] For the X GDFN After the features are convolved channel by channel, they are evenly divided to obtain the query Q C , key K C , value matrix V C ;

[0045] Q C , K C After matrix multiplication, calculate softmax and then add V C Multiply to obtain the preset dimension features;

[0046] After converting the preset dimensional features, a convolution operation is performed to obtain channel self-attention features.

[0047] Optionally, obtaining preset deep features includes:

[0048] After accumulating the global aggregate feature and the adaptive attention multi-scale feature, the feature X is obtained. ESAIN ;

[0049] For the feature X ESAIN Compress and get feature X ESA1 ;

[0050] For the feature X ESA1 Expand the receptive field and obtain feature X ESA2 ;

[0051] For the feature X ESA2 Perform pooling processing and then convolution processing to obtain spatial dimension correlation features;

[0052] Use bilinear interpolation to restore the dimension of the spatial dimension correlation feature to obtain feature X ESA3 ;

[0053] The feature X ESA1 Perform the first convolution process to obtain feature X ESA4 ;

[0054] The feature X ESA3 and the feature X ESA4 Add them together, perform the second convolution, restore the number of channels, and generate the attention mask through the sigmoid activation function;

[0055] Combine the attention mask with X ESAIN Dot product operation generates preset deep features with long-distance dependencies.

[0056] The beneficial effects of the present invention are:

[0057] Due to the limitations of remote sensing image acquisition hardware equipment such as drones, the resolution of the acquired remote sensing images is low, resulting in loss of image details, blurred features, and difficulty in obtaining information, which limits the accurate identification and monitoring of subtle features or small-scale objects in agricultural applications. The method proposed in the present invention enhances the scale diversity of image features and improves the quality of image super-resolution reconstruction through multi-scale convolution. Through neural architecture search, the optimal network structure is searched in a preset space to obtain adaptive attention. The convolution kernel size can be independently selected according to the characteristics of agricultural image data to achieve a more reasonable receptive field, optimize agricultural scene applications, and improve the generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0059] Figure 1 It is a diagram of a method for partitioning input features of a local aggregation module and a global aggregation module according to an embodiment of the present invention;

[0060] Figure 2 This is a flow chart of a method for super-resolution reconstruction of multi-scale agricultural remote sensing images based on adaptive attention according to an embodiment of the present invention;

[0061] Figure 3 This is an architecture diagram of a multi-scale agricultural remote sensing image super-resolution reconstruction method based on adaptive attention according to an embodiment of the present invention;

[0062] Figure 4 It is an adaptive attention multi-scale network module in the architecture diagram of an embodiment of the present invention;

[0063] Figure 5 This is a masked adaptive convolution module in the architecture diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0065] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0066] like Figure 2-Figure 3 As shown, this embodiment proposes a multi-scale agricultural remote sensing image super-resolution reconstruction method based on adaptive attention, including:

[0067] S1. Extract shallow features from low-resolution RGB agricultural remote sensing images to obtain shallow features of low-resolution images;

[0068] Specifically, in this embodiment, the low-resolution agricultural remote sensing image is mapped to the RGB space, and the shallow features of the low-resolution image are extracted through a shallow feature extraction network;

[0069] S2. Perform multi-scale adaptive attention processing on shallow features to obtain adaptive attention multi-scale features;

[0070] Specifically, in this embodiment, the features of the previous module are fed into the adaptive attention multi-scale network module to obtain adaptive attention multi-scale features;

[0071] S3. Aggregate the adaptive attention multi-scale features to obtain global aggregate features, add the global aggregate features to the adaptive attention multi-scale features, perform deep feature extraction, and obtain preset deep features;

[0072] Specifically, in this embodiment, the adaptive attention features are sent to the local aggregation module to obtain local aggregation features. The local aggregation features are sent to the global aggregation module and accumulated with the input of S2, and then sent to the ESA module to obtain deeper features;

[0073] S4. Repeat the processing of S2-S3 several times for the preset deep features to obtain the final deep features;

[0074] S5. After accumulating the final deep features and shallow features, upsampling is performed to obtain a super-resolution agricultural remote sensing image;

[0075] Specifically, in this embodiment, the deep features are accumulated with the shallow features in S1, and then sent to the PixelShuffle module for upsampling to obtain the final super-resolution agricultural remote sensing image.

[0076] Furthermore, shallow feature extraction of low-resolution RGB agricultural remote sensing images includes:

[0077] A low-resolution RGB agricultural remote sensing image is extracted from the agricultural remote sensing image; a convolution with a convolution kernel size of 3×3 is used to perform shallow feature extraction on the low-resolution RGB agricultural remote sensing image.

[0078] Specifically, in this embodiment, the low-resolution RGB agricultural remote sensing image is sent to the shallow feature extraction module. In this example, the dimension of the low-resolution agricultural remote sensing image is 1×3×64×64. The shallow feature extraction network includes a 3×3 convolution with a step size of 1 and a fill value of 1. The low-resolution RGB image is sent to the shallow feature extraction network to obtain the shallow feature Y of the low-resolution image. shallow , the dimension is 1×64×64×64.

[0079] Furthermore, multi-scale adaptive attention processing of shallow features includes:

[0080] The shallow features are first subjected to a 1×1 convolution and then replicated and then subjected to a 5×5 masked convolution and a 7×7 masked convolution to obtain two sets of scale features.

[0081] The two sets of scale features are added and fused, and then global average pooling is performed to obtain channel information features;

[0082] The channel information features are fused, copied and input into a fully connected layer respectively, and a softmax operation is performed on the channel dimension to obtain the attention weights of the two channels;

[0083] The two channel attention weights are multiplied by the corresponding two sets of scale features, and then the features are added to obtain the adaptive attention features;

[0084] Add the adaptive attention features and shallow features to obtain the adaptive attention multi-scale features.

[0085] Specifically, in this embodiment, the input feature is sent to the adaptive attention multi-scale network module, and the input feature is the shallow feature Y in step (1). shallow Or the deep features in step (4), such as Figure 4 As shown. The adaptive attention multi-scale network contains two convolutions with a kernel size of 1×1, a stride of 1, and a padding value of 0, a convolution kernel size of 5×5 with a mask, a stride of 1, a padding value of 2, and a convolution kernel size of 7×7 with a mask, a stride of 1, and a padding value of 3. Two GELU excitation layers and two ReLU excitation layers. A fully connected layer has 64 input channels and 16 output channels. Two fully connected layers have 16 input channels and 64 output channels. The input features first undergo a 1×1 convolution and then are copied and then undergo a 5×5 masked adaptive convolution and a 7×7 masked adaptive convolution to obtain two sets of scale features. and 5×5 masked convolution is as follows Figure 5 As shown. A 5×5 convolution kernel can be regarded as a 5×5 matrix, and a 3×3 convolution kernel can be regarded as a 3×3 matrix. It can be seen that the 5×5 convolution kernel and the 3×3 convolution can form a certain connection, that is, a 5×5 convolution kernel can be regarded as a 3×3 convolution kernel wrapped with a circle of parameters. The formula is as follows:

[0086] w 5×5 =w 3×3 +w 5× 5\3×3 (1)

[0087] where w 5×5 represents a 5×5 convolution kernel, w 3×3 represents a 3×3 convolution kernel, w5×5\3×3 Represents the parameters of the circle wrapped outside. Transforming the above formula, we get the following formula:

[0088] w k =w 3×3 +αw 5×5\3×3 (2)

[0089] where w k is an adaptive convolution kernel, and α is the neural architecture search operator. If α = 0, the convolution kernel can be regarded as a 3×3 convolution. If α = 1, the convolution kernel can be regarded as a 5×5 convolution. Similarly, a 7×7 convolution kernel can be regarded as a 5×5 convolution kernel with a circle of parameters wrapped around it. The formula of the neural architecture search operator α is as follows:

[0090] α=sigmoid(δt) (3)

[0091] Where t is a trainable parameter, similar to weight, obtained by neural network training. δ is a parameter that changes with epoch (when all images in a complete data set are trained once through the neural network, this process is called an epoch). When δ→∞, it can be seen from the image that the value of α can only be 0 or 1. δ→∞ is definitely impossible in reality, but it can be considered that when δ takes a relatively large number, δ can be approximately regarded as infinity. The formula for the change of δ with epoch is as follows:

[0092]

[0093] where δ start is the initial δ value, δ fin is the final δ value, epoch_now indicates the number of epochs currently being executed during the training process, and num_epochs indicates the total number of epochs required for training. By training the neural network, it is possible to train whether α should be 0 or 1, thereby achieving adaptive convolution. In this example, δ start Take 1, δ fin Take 10000.

[0094] The characteristics and The feature U is obtained by element-wise summing, and then global average pooling is performed to obtain a channel information feature S of 1×64×1×1. After the channel information is fused through a full connection with 64 input channels and 16 output channels, the feature Z is obtained with a dimension of 1×16×1×1. After duplication, it passes through a fully connected layer with 16 input channels and 64 output channels and performs a softmax operation on the channel dimension to obtain two channel attention weights a and b. a and b are respectively related to the corresponding two sets of scale features After multiplication, the features are added to obtain the adaptive attention feature V. The adaptive attention feature is added to the input feature of this step to obtain the adaptive attention multi-scale feature V multi , whose dimensions are 1×64×64×64.

[0095] Furthermore, the aggregation processing of the adaptive attention multi-scale features includes:

[0096] Perform local aggregation processing on the adaptive attention multi-scale features to obtain local aggregation features;

[0097] Perform global aggregation processing on the local aggregation features to obtain global aggregation features.

[0098] The local aggregation processing of adaptive attention multi-scale features includes:

[0099] Perform spatial dimension processing on the adaptive attention multi-scale features to obtain spatial attention features;

[0100] The spatial attention feature is processed by suppressing the feature information to obtain X GDFN feature;

[0101] X GDFN The features are processed by channel self-attention to obtain channel self-attention features;

[0102] Combine the channel self-attention features with X GDFN Features are added to obtain local aggregate features.

[0103] Acquiring spatial attention features includes:

[0104] Divide the adaptive attention multi-scale features into non-overlapping local blocks of size P×P;

[0105] The non-overlapping local blocks are transformed in spatial dimensions to obtain feature X;

[0106] Transform feature X into a query Q by linear projection. S , key K S Sum value matrix V S ;

[0107] For query Q S , key K S Sum value matrix V S Perform multi-head attention processing to obtain

[0108] Calculation query and key The product of and softmax processing is performed to obtain the spatial attention weight;

[0109] The spatial attention weights and the value matrix Multiply to obtain preset features;

[0110] The preset features are transformed in dimension and fully connected, and the features after the fully connected processing are added to the feature X to obtain the spatial attention features.

[0111] Get X GDFN Features include:

[0112] Transform the spatial attention feature into a dimension, denoted as X GDFNIN ;

[0113] X GDFNIN After a 1×1 convolution for dimension transformation, it undergoes a 3×3 channel-by-channel convolution and is evenly split into two features X GDFN1 , X GDFN2 ;

[0114] X GDFN1 After GELU activation function and X GDFN2 Multiply and then go through a 1×1 convolution, and X GDFNIN Add together to get X GDFN feature.

[0115] Obtaining channel self-attention features includes:

[0116] X GDFN The features are fed into a 1×1 convolution and a 3×3 channel-by-channel convolution in turn and then evenly divided to obtain Q C , K C 、V C ;

[0117] Q C , K C After matrix multiplication, calculate softmax and then add V C Multiply to obtain the preset dimension features;

[0118] After converting the preset dimensional features, they are sent into a 1×1 convolution to obtain the channel self-attention features.

[0119] Specifically, in this embodiment, the obtained adaptive attention multi-scale features are fed into the local aggregation module. The module first divides the features into non-overlapping local blocks of size 8×8, and the dimensions of these blocks are aggregated to the spatial dimension. The input features are converted from the dimension 1×64×64×64 to 1×8×8×8×8×64 to obtain the feature X. Then it is converted to 64×64×64 and the obtained features are converted into query, key and value matrices Q through linear projection. S , K S 、V S, contains 1 fully connected layer, with 64 input channels and 192 output channels. S , K S 、V S The dimensions of are 64×64×64. Divide it into 4 multi-head attention features, and get Its dimensions are 64×4×64×16, and the calculation query is and key The product of , and softmax is performed to obtain the spatial attention weight, the dimension is 64×4×64×64, and then Multiply them together to get a feature with a dimension of 64×4×64×16, and then perform a dimension conversion to get a feature with a dimension of 64×8×8×64. After a full connection, the input channel is 64 and the output channel is 64. Then the dimension is changed to 1×8×8×8×8×64, and finally added to the feature X to get the spatial attention feature. The spatial attention feature dimension is converted to 1×64×64×64, recorded as X GDFNIN . It is sent to the GDFN module to suppress features with less information and only allow useful information to pass. This module consists of two convolutions with a kernel size of 1×1, a step size of 1, and a padding value of 0, and a convolution with a kernel size of 3×3, a step size of 1, and a padding value of 1. The input feature first undergoes a 1×1 convolution, and the dimension becomes 1×128×64×64, and then undergoes a 3×3 channel-by-channel convolution and is evenly split into two, resulting in two features X GDFN1 , X GDFN2 , X GDFN1 After GELU activation function and X GDFN2 Multiply and then go through a 1×1 convolution, and X GDFNIN Add them together to get the GDFN feature X GDFN , whose dimensions are 1×64×64×64. Then X GDFN The features are fed into the channel self-attention module. This module consists of two convolutions with a kernel size of 1×1, a stride of 1, and a padding value of 0, and a convolution with a kernel size of 3×3, a stride of 1, and a padding value of 1. First, X GDFN The feature is fed into a 1×1 convolution to obtain a feature with a dimension of 1×192×64×64. It is then fed into a 3×3 channel-by-channel convolution to obtain a feature of 1×192×64×64. The feature is evenly divided to obtain Q C , K C 、V C , the dimensions are 1×64×4×64×16,Q C , K C After matrix multiplication, calculate softmax and then add V CMultiply them together to get a dimension of 1×64×4×16×64. Then perform a dimensional transformation to get 1×64×64×64, and send it to a 1×1 convolution to get the channel self-attention feature, whose dimension is 1×64×64×64. GDFN Features are added to obtain local aggregate features.

[0120] The calculation process of global aggregate features is similar to that of local aggregate features. The only difference is the input feature block method. The local aggregate feature calculation method is continuous block, while the global aggregate feature calculation method is interval block, such as Figure 1 As shown in Figure 1. The module first divides the features into non-overlapping global blocks of size 8×8. The dimensions of these blocks are aggregated to the spatial dimension. The input features are converted from 1×64×64×64 to 1×8×8×8×8×64 to obtain the feature X′. It is then converted to 64×64×64 and converted into query, key and value matrices Q′ through linear projection. S , K′ S , V′ S , contains 1 fully connected layer, with 64 input channels and 192 output channels. Q′ S , K′ S , V′ S The dimensions of are 64×64×64. Divide it into 4 multi-head attention features, and get Its dimensions are 64×4×64×16, and the calculation query is and key The product of , and softmax is performed to obtain the spatial attention weight, the dimension is 64×4×64×64, and then Multiply them together to get a feature with a dimension of 64×4×64×16, and then perform a dimension conversion to get a feature with a dimension of 64×8×8×64. After a full connection, the input channel is 64 and the output channel is 64. Then the dimension is changed to 1×8×8×8×8×64, and finally added to the feature X′ to get the spatial attention feature. The spatial attention feature dimension is converted to 1×64×64×64, recorded as X′ GDFNIN . It is sent to the GDFN module to suppress features with less information and only allow useful information to pass. This module consists of two convolutions with a kernel size of 1×1, a step size of 1, and a padding value of 0, and a convolution with a kernel size of 3×3, a step size of 1, and a padding value of 1. The input feature first undergoes a 1×1 convolution, and the dimension becomes 1×128×64×64, and then undergoes a 3×3 channel-by-channel convolution and is evenly split into two to obtain two features X′ GDFN1 , X′ GDFN2 , X′ GDFN1 After GELU activation function and X′ GDFN2 Multiply and then go through a 1×1 convolution, and X′GDFNIN Add them together to get the GDFN feature X′ GDFN , whose dimensions are 1×64×64×64. Then X′ GDFN The features are fed into the channel self-attention module. This module consists of two convolutions with a kernel size of 1×1, a stride of 1, and a padding value of 0, and a convolution with a kernel size of 3×3, a stride of 1, and a padding value of 1. First, X′ GDFN The feature is fed into a 1×1 convolution to obtain a feature with a dimension of 1×192×64×64. It is then fed into a 3×3 channel-by-channel convolution to obtain a feature of 1×192×64×64. The feature is evenly divided to obtain Q′ C , K′ C , V′ C , the dimensions are 1×64×4×64×16, Q′ C , K′ C After matrix multiplication, calculate softmax and then add V′ C Multiply them together to get a dimension of 1×64×4×16×64. Then perform a dimensional transformation to get 1×64×64×64, and send it to a 1×1 convolution to get the channel self-attention feature, whose dimension is 1×64×64×64. GDFN The features are added together to obtain the global aggregated features.

[0121] Furthermore, obtaining preset deep features includes:

[0122] After accumulating the global aggregation features and the adaptive attention multi-scale features, we get the feature X ESAIN ;

[0123] Use 1×1 convolution to feature X ESAIN Compress and get feature X ESA1 ;

[0124] Use a 3×3 convolution with a stride of 2 to apply to feature X ESA1 Expand the receptive field and obtain feature X ESA2 ;

[0125] Using a 7×7 window and a maximum pooling layer with a step size of 3, feature X ESA2 Perform pooling processing, and then use a 3×3 convolution with a step size of 1 to obtain spatial dimension correlation features;

[0126] Use bilinear interpolation to restore the dimension of spatial dimension correlation features and obtain feature X ESA3 ;

[0127] The feature X ESA1 Send it to a 1×1 convolution to get the feature X ESA4 ;

[0128] The feature X ESA3 and feature X ESA4 Add them together and feed them into a 1×1 convolution. After restoring the number of channels, the attention mask is generated through the sigmoid activation function.

[0129] Combine the attention mask with X ESAIN Dot product operation generates preset deep features with long-distance dependencies.

[0130] Specifically, in this embodiment, the global aggregate feature and the input in S2 are accumulated to form feature X ESAIN The ESA module consists of three 1×1 convolutions with a stride of 1 and a padding of 0, one 3×3 convolution with a stride of 1 and a padding of 1, and one 3×3 convolution with a stride of 2 and a padding of 0. First, a 1×1 convolution is used to compress the feature from the dimension 1×64×64×64 to 1×16×64×64, and the feature X is obtained. ESA1 Then a 3×3 convolution with a step size of 2 is used to expand the receptive field and obtain feature X ESA2 , with a dimension of 1×16×31×31. Pooling is performed using a 7×7 window and a maximum pooling layer with a stride of 3, and then a 3×3 convolution with a stride of 1 is used to obtain a spatial dimension correlation feature with a dimension of 1×16×8×8. Bilinear interpolation is used to restore the feature dimension to 1×16×64×64 to obtain feature X ESA3 . X ESA1 Send it to a 1×1 convolution to get the feature X ESA4 . X ESA3 and X ESA4 Add and send to a 1×1 convolution to restore the number of channels and obtain a feature with a dimension of 1×64×64×64. Generate an attention mask and X through the sigmoid activation function ESAIN Dot product operations generate deeper features with longer distance dependencies.

[0131] S5. After accumulating the final deep features and shallow features, upsampling is performed to obtain a super-resolution agricultural remote sensing image; including:

[0132] The final deep features are sent to a 3×3 convolution with a stride of 1 and a padding value of 1, and then accumulated with S1 and sent to the PixelShuffle module for upsampling to obtain a super-resolution image with a specified magnification. For example, if the magnification is 2, the output image scale is 1×3×128×128. If the magnification is 4, the output image scale is 1×3×256×256, and so on.

[0133] The multi-scale agricultural remote sensing image super-resolution reconstruction method based on adaptive attention proposed in this embodiment utilizes the idea of ​​automatic design of neural network architecture, and automatically explores and optimizes the structure of the neural network through the algorithm to find the best remote sensing image super-resolution reconstruction network architecture for agricultural applications. The method proposed in the present invention first sends the low-resolution agricultural remote sensing image into the shallow feature extraction network to extract the shallow features of the image. Secondly, through the deep feature extraction module. The deep feature extraction module consists of three parts: an adaptive attention multi-scale network module, a local aggregation module and a global aggregation module. The adaptive attention multi-scale network module is divided into an adaptive convolution module and an attention module. Through the adaptive convolution module, the algorithm automatically searches and optimizes the structure of the neural network to extract the best different scale features for agriculture. The attention module strengthens important scale features and suppresses non-important scale features. Generate adaptive attention multi-scale features through the adaptive attention multi-scale network module. Subsequently, local aggregation features are obtained through the local aggregation module, and global aggregation features are obtained through the global aggregation module. The global aggregation features are accumulated with the input features of the deep feature extraction module to obtain deeper features. The deep feature extraction module is repeated n times to obtain deep features. Finally, the deep features and shallow features are accumulated and convolved, and then the PixelShuffle module is used for upsampling to generate high-resolution agricultural remote sensing images. Compared with the traditional remote sensing image reconstruction method, the present invention uses the adaptive attention multi-scale network module in S2, with the help of search operators, and the neural network automatically selects the convolution kernel size and automatically designs the network structure. The neural network architecture can be automatically designed and the network structure can be optimized according to the characteristics of specific agricultural remote sensing image application scenarios. The present invention can also use different information extracted from different receptive fields of agricultural remote sensing images to strengthen important receptive fields, suppress non-important receptive fields, and improve reconstruction quality.

[0134] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A multi-scale agricultural remote sensing image super-resolution reconstruction method based on adaptive attention, characterized in that: include: S1. Perform shallow feature extraction on low-resolution RGB agricultural remote sensing images to obtain shallow features of low-resolution images; S2. performing multi-scale adaptive attention processing on the shallow features to obtain adaptive attention multi-scale features; S3. Aggregate the adaptive attention multi-scale features to obtain global aggregate features, add the global aggregate features to the adaptive attention multi-scale features, perform deep feature extraction, and obtain current deep features; S4. Repeat S2-S3 for the current deep feature until the target number of times, and obtain the final deep feature; S5. After accumulating the final deep features and the shallow features, upsampling is performed to obtain a super-resolution agricultural remote sensing image.

2. The method for super-resolution reconstruction of multi-scale agricultural remote sensing images based on adaptive attention according to claim 1, characterized in that: Shallow feature extraction of low-resolution RGB agricultural remote sensing images includes: Extract low-resolution RGB agricultural remote sensing images from agricultural remote sensing images; Shallow feature extraction is performed on the low-resolution RGB agricultural remote sensing image.

3. The method for super-resolution reconstruction of multi-scale agricultural remote sensing images based on adaptive attention according to claim 1, characterized in that: Performing multi-scale adaptive attention processing on the shallow features includes: The shallow features are first convolved and then copied and passed through a first masked convolutional layer and a second masked convolutional layer respectively to obtain two sets of scale features; The two sets of scale features are added and fused, and then global average pooling is performed to obtain channel information features; Perform channel information fusion on the channel information features, copy them and input them into a fully connected layer respectively, and perform a softmax operation on the channel dimension to obtain the attention weights of two channels; The two channel attention weights are multiplied by the corresponding two sets of scale features, and then the features are added to obtain the adaptive attention features; The adaptive attention feature and the shallow feature are added to obtain an adaptive attention multi-scale feature.

4. The method for super-resolution reconstruction of multi-scale agricultural remote sensing images based on adaptive attention according to claim 1, characterized in that: Aggregating the adaptive attention multi-scale features includes: Performing local aggregation processing on the adaptive attention multi-scale features to obtain local aggregation features; Perform global aggregation processing on the local aggregation features to obtain global aggregation features.

5. The method for super-resolution reconstruction of multi-scale agricultural remote sensing images based on adaptive attention according to claim 4, characterized in that: The local aggregation processing of the adaptive attention multi-scale features includes: Performing spatial dimension processing on the adaptive attention multi-scale feature to obtain a spatial attention feature; The spatial attention feature is subjected to feature information suppression processing to obtain X GDFN feature; For the X GDFN The features are processed by channel self-attention to obtain channel self-attention features; The channel self-attention feature is combined with X GDFN The features are added to obtain the local aggregated features.

6. The method for super-resolution reconstruction of multi-scale agricultural remote sensing images based on adaptive attention according to claim 5, characterized in that: Acquiring the spatial attention feature includes: Divide the adaptive attention multi-scale features into non-overlapping local blocks of size P×P; The non-overlapping local blocks are transformed in spatial dimension to obtain feature X; Transform the dimension of the feature X and transform the feature X into a query Q through linear projection S , key K S Sum value matrix V S ; For the query Q S , key K S Sum value matrix V S Perform multi-head attention processing to obtain Calculation query and key The product of and softmax processing is performed to obtain the spatial attention weight; The spatial attention weights and value matrix Multiply to obtain preset features; The preset features are dimensionally transformed and fully connected, and the features after the fully connected processing are added to the feature X to obtain the spatial attention features.

7. The method for super-resolution reconstruction of multi-scale agricultural remote sensing images based on adaptive attention according to claim 5, characterized in that: Get the X GDFN Features include: The spatial attention feature is transformed into a dimension, denoted as X GDFNIN ; X GDFNIN After the dimension transformation, it is convolved channel by channel and evenly split into two features X GDFN1 , X GDFN2 ; X GDFN1 After GELU activation function and X GDFN2 Multiply and then convolve, and X GDFNIN Add together to get the X GDFN feature.

8. The method for super-resolution reconstruction of multi-scale agricultural remote sensing images based on adaptive attention according to claim 5, characterized in that: Obtaining the channel self-attention feature includes: For the X GDFN After the features are convolved channel by channel, they are evenly divided to obtain the query Q C , key K C , value matrix V C ; Q C , K C After matrix multiplication, calculate softmax and then add V C Multiply to obtain the preset dimension features; After converting the preset dimensional features, a convolution operation is performed to obtain channel self-attention features.

9. The method for super-resolution reconstruction of multi-scale agricultural remote sensing images based on adaptive attention according to claim 1, characterized in that: The deep features of the presets include: After accumulating the global aggregate feature and the adaptive attention multi-scale feature, the feature X is obtained. ESAIN ; For the feature X ESAIN Compress and get feature X ESA1 ; For the feature X ESA1 Expand the receptive field and obtain feature X ESA2 ; For the feature X ESA2 Perform pooling processing and then convolution processing to obtain spatial dimension correlation features; Use bilinear interpolation to restore the dimension of the spatial dimension correlation feature to obtain feature X ESA3 ; The feature X ESA1 Perform the first convolution process to obtain feature X ESA4 ; The feature X ESA3 and the feature X ESA4 Add them together, perform the second convolution, restore the number of channels, and generate the attention mask through the sigmoid activation function; Combine the attention mask with X ESAIN Dot product operation generates preset deep features with long-distance dependencies.

Images