Sequential SAR crop classification method based on multi-scale cross attention

Abstract

The invention relates to the technical field of land coverage remote sensing monitoring, in particular to a time sequence SAR crop classification method based on multi-scale cross attention. According to the method, the complex phenological relationship of crops is considered, the features of time and space dimensions of a time sequence SAR image are fully utilized, an input array is mapped to a high-dimensional feature space through an encoder, and multi-scale local space features are extracted; constructing a multi-scale cross attention module MCMSABlock to extract multi-scale time features of the time sequence SAR image block, taking a weighted result of an original scale as a query matrix of other scales, indexing and weighting, and extracting remote dependency information at different time scales; the decoder adopts a plurality of up-sampling modules to gradually recover the spatial scale, and multi-scale time features are fused while multi-scale local spatial features are reserved by utilizing jump connection. The method is suitable for crop diversity and complex time sequence SAR scenes, the requirement for a memory is reduced while an accurate crop classification result is obtained, and the classification efficiency is improved.

Description

Technical Field

[0001] The invention relates to the technical field of land cover remote sensing monitoring, and in particular to a time series SAR crop classification method based on multi-scale cross attention. Background Art

[0002] The goal of agricultural remote sensing is to use remote sensing data to solve various problems in the agricultural field, such as crop planting mapping, crop yield estimation, etc. Crop planting mapping is the cornerstone of crop yield estimation. Remote sensing technology is used to accurately identify and classify the planting distribution of different crops, thereby providing a scientific basis for agricultural production management, policy formulation and resource allocation.

[0003] Traditional agricultural remote sensing mostly uses time-series optical images, which can obtain multi-temporal observation images of crops during different growth periods, and classify crops by analyzing their texture, shape, spectral changes and other characteristics. However, in areas with frequent clouds and fog, optical images are limited, resulting in poor crop classification based on time-series optical images. In contrast, synthetic aperture radar SAR has the ability to image all day and all weather, and can observe stably under various weather conditions. Crop classification based on time-series SAR images has its unique advantages.

[0004] Traditional agricultural remote sensing is mostly based on machine learning methods, but machine learning methods (such as random forest RF, support vector machine SVM) mostly rely on shallow manual features, which are difficult to handle complex time series changes and have poor modeling effects on complex crop phenological relationships. Deep learning models (such as convolutional neural networks CNN, recurrent neural networks RNN, transformers, etc.) extract features adaptively under the constraints of loss functions and do not rely on shallow manual features. However, existing deep learning models have shortcomings when processing long-term remote sensing images: they cannot handle the long-range dependencies of remote sensing data at different time scales. CNN achieves multi-scale local feature extraction through stacking and lacks the ability to capture long-range dependencies related to crop growth and development; RNN relies on the sequential relationship of time series data; transformer extracts long-range dependencies of uniform scales and ignores multi-scale information in the time dimension. In the context of crop diversity and complex time series SAR scenarios, existing deep learning methods are difficult to obtain accurate crop classification results. Summary of the invention

[0005] In view of the above-mentioned problems and shortcomings, and to solve the limitations of existing deep learning methods in crop classification, the present invention provides a time-series SAR crop classification method based on multi-scale cross-attention, which uses an encoder to map the input array to a high-dimensional feature space to extract multi-scale local spatial features; uses multi-scale cross-attention to extract multi-scale temporal features, indexes and weights them at the original scale, and uses the weighted results as the query matrix of other scale sequences. The final features contain multi-scale global interaction information, which can effectively model the complex phenological relationship of crops; finally, uses a decoder to restore the spatial scale to achieve semantic-level crop classification.

[0006] A temporal SAR crop classification method based on multi-scale cross-attention includes the following steps:

[0007] Step 1: Dataset construction: Divide the samples in the target area dataset into non-overlapping training sets and test sets according to the proportion for training and testing of the constructed neural network model. Each sample includes a time series SAR image block Label Block h represents the height of the array, w represents the width of the array, c represents the number of features of the time series SAR image block sar, and t represents the time series length of the time series SAR image block sar.

[0008] Step 2, construct a neural network model: including an encoder, a multi-scale cross-attention module MCMSABlock, and a decoder. The present invention uses an encoder to map the input array to a high-dimensional feature space and extract multi-scale local spatial features; MCMSABlock extracts multi-scale temporal features, uses the weighted results of the original scale as the query matrix of other scales and indexes and weights them, extracts remote dependency information at different time scales, and models the complex phenological relationship of crops; the decoder uses jump connections to retain multi-scale local spatial features while integrating multi-scale temporal features to gradually restore the spatial scale.

[0009] Step 2.1, build the encoder: The encoder contains L downsampling modules, and the output of the current downsampling module is used as the input of the next downsampling module. The downsampling module includes R convolution blocks stacked in sequence and 1 pooling layer. The convolution block contains convolution, normalization and activation functions in sequence. By stacking downsampling modules, the input array is mapped to a high-dimensional feature space, and multi-scale local spatial features are extracted. For SAR images corresponding to different time steps, a weight-sharing downsampling module is used to extract local spatial features from the input array:

[0010] inder 1 =EnConvBlock 1 (sar)

[0011] EnConvBlock 1 It is the first downsampling module; is the local spatial feature extracted by the first downsampling module; h 1 is the height of the local spatial feature extracted by the first downsampling module; w 1 is the width of the local spatial feature extracted by the first downsampling module; c 1 The number of features of the local spatial features extracted by the first downsampling module.

[0012] inder i =EnConvBlock i (inder i-1 )

[0013] EnConvBlock i is the i-th downsampling module, i∈[2,L]; It is the local spatial feature extracted by the i-1th downsampling module and serves as the input of the i-th downsampling module; is the local spatial feature extracted by the i-th downsampling module; h i is the height of the local spatial feature extracted by the i-th downsampling module; w i is the width of the local spatial feature extracted by the i-th downsampling module; c i is the number of local spatial features extracted by the i-th downsampling module.

[0014] The convolution operation extracts local spatial features by applying convolution kernels to the array; normalization reduces the impact of internal covariate shift and accelerates the model training process by standardizing the distribution of the array; the activation function introduces nonlinear mapping relationships to enhance the ability to process complex data; pooling downsamples local areas in the features, thereby improving the efficiency of the model and its ability to resist overfitting.

[0015] Step 2.2, construct the multi-scale cross attention module MCMSABlock: MCMSABlock includes adding position encoding, normalization, multi-scale cross attention MCMSA, normalization and convolution layers in sequence. MCMSABlock is used to extract multi-scale temporal features, and the weighted results of the original scale are used as the query matrix of other scales. After indexing and weighting again, the multi-scale temporal features are obtained:

[0016] out,att=MCMSABlock(inder L )

[0017] is the local spatial feature extracted by the Lth downsampling module; MCMSABlock is the multi-scale cross attention module constructed by the present invention; att is the attention weight output by MCMSABlock; The multi-scale temporal features extracted by MCMSABlock. Adding position encoding provides the position information of Token (the smallest unit of the input sequence); Multi-scale cross-attention MCMSA first globally interacts all Tokens at the original scale to obtain the attention weight att, and uses the weighted result of the attention weight as the query matrix of other scales, and then indexes and weights it again to extract long-range dependency information at different time scales. The final feature contains multi-scale global interaction information.

[0018] Step 2.3, construct a decoder: The decoder contains L upsampling modules for restoring the spatial scale; a single upsampling module contains upsampling and att weighting, and then processed by a convolution block.

[0019] Specifically, the features in the encoder (inder L-1 ...inder 1 , sar) are weighted in the time dimension and are sequentially combined with (out...output 2 、output 1 ) The upsampling result is skipped and then the result of the skip connection is input into the convolutional layer to change the number of features and obtain the output of the upsampling module:

[0020] output L-1 =DeConvBlock L (out,inder L-1 , att)

[0021] DeConvBlock L is the Lth upsampling module; DeConvBlock is the Lth upsampling module L Output.

[0022] output j-1 =DeConvBlock j (output j ,inder j-1 , att)

[0023] is the output corresponding to the j+1th (j∈[2,L-1]) upsampling module; DeConvBlock j is the jth upsampling module; is the local spatial feature extracted by the j-1th downsampling module; is the jth upsampling module DeConvBlock j The corresponding output.

[0024] output=DeConvBlock 1 (output 1 ,sar,att)

[0025] This is the final neural network model output of the decoder.

[0026] Step 3, training the neural network model constructed in step 2: during the forward propagation of the input array in the neural network model, nonlinear mapping is achieved through linear transformation and nonlinear activation. Linear transformation maps the input data to a new feature space through a weight matrix and a bias vector, and the nonlinear activation function introduces nonlinearity, so that the model has the ability to capture complex data patterns and relationships. Finally, the crop classification prediction result output of the neural network model is obtained. The loss function is used to evaluate the difference LOSS between the output and the label block label. Based on the optimizer, the parameters of the neural network model are continuously iterated, and finally converged on the training set to obtain the trained neural network model.

[0027] LOSS=LOSSFunction(output,label)

[0028] LOSSFunction is a loss function that measures the difference between the output of the crop classification prediction result of the neural network model and the label block label LOSS.

[0029] Step 4: Input the test set into the neural network model trained in step 3 to obtain the crop classification results.

[0030] Furthermore, the construction process of the multi-scale cross attention module MCMSABlock is specifically as follows:

[0031] Step 2.2.1, add position encoding, and extract the local spatial features from step 2.1. L Add positional encoding:

[0032] inder PE =inder L +PE

[0033] PE stands for position encoding; Represents the local spatial features after adding position encoding.

[0034] Step 2.2.2, normalization, add the local spatial feature inder after position encoding in step 2.2.1 PE Normalize:

[0035] inder norm = Norm(inder PE )

[0036] Norm represents the normalization operation; represents the local spatial features after normalization.

[0037] Step 2.2.3, Multi-scale Cross Attention (MCMSA), on the premise of linear computational complexity, retains the global information of the original scale, conducts global interaction, indexing, and weighting with sequences of different scales. After weighting, the features contain multi-scale long-range dependencies:

[0038] inder MCMSA , att = MC<SA(inder norm )

[0039] inder MCMSA is the local spatial feature after being processed by MCMSA.

[0040] The construction process of MC<SA is specifically as follows:

[0041] Step A: For each pixel-corresponding sequence in inder norm , define the query matrix as a learnable parameter Define the key matrix K and the value matrix V as the results after the sequence X is projected by the projection matrix:

[0042] K = XW K , V = XW V

[0043] W K and W V are projection matrices that project the sequence X into a high-dimensional feature space, and the number of features changes from c L to d; Q is the query matrix, K is the key matrix, and V is the value matrix.

[0044] Step B: Calculate the global interaction information of the original scale, calculate the similarity between the query matrix Q and the key matrix K using the dot product, and normalize it using softmax:

[0045]

[0046] attout = attt * V

[0047] is the attention weight of the original scale; is the output after being weighted by attt.

[0048] Step C: Calculate the bond matrix K of scale L(L∈[2,t]) L , value matrix V L :

[0049] K L =pool L (K), V L =pool L (V)

[0050] pool L Represents the pooling operation, which is used to obtain the key matrix of scale L Value Matrix N L Represents the sequence length of scale L after pooling.

[0051] Step D: Calculate the multi-scale global interaction information, taking attout as the query matrix of L scale and the key matrix K of L scale L , value matrix V L To index and weight:

[0052]

[0053] attout MCMSA =attt L *V L

[0054] is the multi-scale cross attention weight; For attt L Weighted output.

[0055] Step E: For the entire inder norm The sequence corresponding to all pixels in the image is processed by step AD, and the original scale sequence is norm Index and weight to get inder norm The corresponding attention weight The weighted result is used as the query matrix of scale L, indexed and weighted, and the inder norM Corresponding multi-scale temporal features

[0056] Step 2.2.4: Normalize the multi-scale temporal features extracted in step 2.2.3. MCMSA Normalization:

[0057] inder norm2 =Norm2(inder MCMSA )

[0058] Norm2 represents the normalization operation; Represents the normalized multi-scale temporal features.

[0059] Step 2.2.5, the convolution layer includes convolution, normalization, and activation function in sequence.

[0060] out = ConvLayer( norm2 )

[0061] ConvLayer is the convolutional layer constructed by the present invention; is the multi-scale temporal feature processed by ConvLayer. The convolution layer ConvLayer is used to restore the number of features.

[0062] Furthermore, the loss function is cross entropy.

[0063] Furthermore, the optimizer is Adam.

[0064] The present invention fully considers the complex phenological relationship of crops, uses an encoder to extract multi-scale local spatial features; uses a multi-scale cross-attention mechanism to extract multi-scale features of time-series SAR image blocks in the time dimension; and uses a decoder to restore the spatial scale. An encoder including a multi-layer downsampling module is used to extract multi-scale local spatial features; multi-scale cross-attention is used to extract multi-scale temporal features, and the weighted results of the attention of the original scale are used as the query matrix of other scales to realize the extraction of remote dependency information of different scales, and effectively model the complex phenological relationship of crops; a decoder including multiple upsampling modules is used to gradually restore the spatial scale, and the attention weights obtained by multi-scale cross-attention are used to weight the local spatial features obtained in the encoder in the time dimension, and the jump connection is used to retain the weighted results of the local spatial features to reduce the loss of information; finally, a neural network model output with the same spatial scale as the input array is obtained. Under the constraint of the loss function, the weights of the neural network model are adjusted by the optimizer to obtain a neural network model that converges on the training set.

[0065] In summary, the present invention maps the input array to a high-dimensional feature space through an encoder to extract multi-scale local spatial features; constructs a multi-scale cross-attention module MCMSABlock to extract multi-scale temporal features of time series SAR image blocks, and extracts remote dependency information at different time scales; uses a decoder with multiple upsampling modules to gradually restore the spatial scale, and uses jump connections to retain multi-scale local spatial features while fusing multi-scale temporal features. The present invention is suitable for crop diversity and complex time series SAR scenes, and can obtain accurate crop classification results while reducing the need for memory and improving classification efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is a schematic diagram of the process of the present invention;

[0067] Figure 2 A schematic diagram of a neural network model constructed for the present invention;

[0068] Figure 3 A schematic diagram of the structure of the multi-scale cross-attention module provided by the present invention;

[0069] Figure 4 A schematic diagram of the structure of the multi-scale cross attention provided by the present invention;

[0070] Figure 5 This is a crop classification result diagram of the embodiment. DETAILED DESCRIPTION

[0071] The present invention is further described in detail below in conjunction with embodiments and drawings.

[0072] The present invention provides a multi-scale cross-attention time series SAR crop classification method. First, a training set and a test set are divided; a time series SAR image block sar in the training set is input into an encoder to obtain multi-scale local spatial features; a multi-scale cross-attention module MCMSABlock is used to extract multi-scale long-range dependencies, the original scale sequence is indexed and weighted, the weighted result is used as a query matrix of other scales, and after indexing and weighting again, global interactive information containing multi-scales is obtained; a decoder is used to restore the spatial scale, and finally a neural network model output with the same spatial scale as the time series SAR image block sar is obtained, and under the constraint of a loss function, an optimizer is used to continuously iterate the parameters of the neural network model to obtain a neural network model that converges on the training set.

[0073] The present invention implements model building and training based on the PyTorch open source framework, which has dynamic computational graphs and convenient debugging functions. In order to accelerate the model training process, the present invention uses NVIDIA's CUDA parallel computing platform, which assigns computing tasks to the GPU, significantly improving the efficiency of model training.

[0074] The specific configuration is as follows: Python version is 3.9.18, and PyTorch version is PyTorch-CUDA version compatible with CUDA11.7.

[0075] Step 1 includes the following steps:

[0076] The time series of Sentinel-1 ascending orbit images in the Western Cape Province of South Africa are used. Each Sentinel-1 image covers an area of ​​24km×24km, including two features: vertical polarization and horizontal polarization, with a spatial resolution of 10m×10m. The Sentinel-1 observation period covers April 2017 to November 2017, with a total of 41 observation images. The labels are obtained through aerial and vehicle surveys from May 2017 to March 2018. The labels include 6 categories, including 5 crop categories and 1 non-crop background category (Background, Wheat, Barely, Canola, Lucerne / Medics, Small grain grazing).

[0077] The images and corresponding labels are divided into 1.28×1.28km square blocks. For Sentinel-1 images with a spatial resolution of 10m, the spatial scale of each square block is 128×128. A time series SAR image block sar and a corresponding label block label constitute a sample. All samples are divided into training sets and test sets in a non-overlapping ratio of 7:3. Label Block The number of crop categories to be classified is class=6.

[0078] Step 2 includes the following steps, such as Figure 2 As shown:

[0079] Step 2.1, build the encoder. The encoder contains 3 downsampling modules. The downsampling module includes 2 convolution blocks and 1 average pooling layer. The convolution block contains convolution, BN and ReLU in sequence. The convolution kernel, stride and padding of the convolution block are 3, 1 and 1 respectively. The kernel, stride and padding of the average pooling layer are 2, 2 and 0 respectively:

[0080] inder 1 =EnConvBlock 1 (sar)

[0081] inder 2 =EnConvBlock 2 (inder 1 )

[0082] inder 3 =EnConvBlock 3 (inder 2 )

[0083] In the above formula, the time series SAR image block Local spatial features

[0084] Step 2.2: Construct a multi-scale cross-attention module MCMSABlock, such as Figure 3 As shown:

[0085] Step 2.2.1, add position coding, for position coding PE:

[0086] PE = Sigmoid (Conv2d (inder 3 ))

[0087] inder PE =inder 3 +PE

[0088] In the above formula, Conv2d represents the convolution operation, and the convolution kernel, stride, and padding are 3, 1, and 1 respectively;

[0089] Step 2.2.2, layer normalization LN:

[0090] inder norm =LN(inder PE )

[0091] In the above formula, LN is layer normalization;

[0092] Step 2.2.3, multi-scale cross attention MCMSA, such as Figure 4 As shown, for inder norm The sequence corresponding to each pixel in Randomly generate query matrix And use the sequence X projection to get the corresponding key matrix Value Matrix

[0093] K=XW K , V = XW V

[0094] Use the dot product attention mechanism to obtain the attention weight attt corresponding to the original scale and the result weighted by the attention weight

[0095]

[0096] attout=attt*V

[0097] Calculate the bond matrix K of scale L (L=2) L , value matrix V L :

[0098] K L =pool L (K), V L =poolL (V)

[0099] The kernel of the average pooling is set to 2 to obtain the corresponding values ​​of different scales. Value Matrix Take attout as the query matrix of scale L, and the key matrix K corresponding to scale L L , value matrix V L To index and weight:

[0100]

[0101] attout MCMSA =attt L *V L

[0102] Get the multi-scale time features corresponding to the sequence X inder norm Corresponding multi-scale temporal features inder norm The corresponding attention weight

[0103] Step 2.2.4, layer normalization LN:

[0104] inder norm2 =LN(inder MCMSA )

[0105] In the above formula, LN represents layer normalization;

[0106] Step 2.2.5, convolutional layer:

[0107] out = ConvLayer( norm2 )

[0108] ConvLayer contains convolution, BN and ReLU, and the convolution kernel, stride and padding are 1, 1 and 0 respectively;

[0109]

[0110] Step 2.3, construct a decoder. The decoder contains 3 upsampling modules. The upsampling module includes upsampling, resampling using att, and weighted and convolution blocks.

[0111] output 2 =DeConvBlock 3 (out,inder 2 , att)

[0112] output 1=DeConvBlock 2 (output 2 ,inder 1 , att)

[0113] output=DeConvBlock 1 (output 1 ,sar,att)

[0114] Output of the three upsampling modules in the decoder

[0115] Specifically, the upsampling module needs to upsample the attention weight att and weight the corresponding local spatial features:

[0116] The attention weights are upsampled by bilinear interpolation to obtain local spatial features at multiple scales. Same spatial scale:

[0117] Using the interpolated attention weight att 1 ,att 2 ,att 3 Multi-scale spatial features 2 、inder 1 , sar are weighted and summed to obtain the weighted multi-scale local spatial features

[0118] The upsampling module needs to deconvolve the output of the previous upsampling module to restore the spatial scale, perform a jump connection between the deconvolution result and the weighted local spatial features, and change the number of features through the convolution layer:

[0119] output 2 =Conv(cat(skip 1 ,ConvTranspose(out)))

[0120] The convolution kernel, stride, and padding of deconvolution are set to 4, 2, and 1; cat means splicing in the feature dimension; Conv includes convolution, BN, and ReLU; the convolution kernel, stride, and padding of convolution are set to 1, 1, and 0.

[0121] output 1 =Conv(cat(skip 2 ,ConvTranspose(output 2 )))

[0122] output = Conv(cat(skip 3 ,ConvTranspose(output 1 )))

[0123] Finally, the neural network model output is obtained with the same spatial scale as the input time series SAR image block sar and the number of features equal to the number of crop classification categories class

[0124] Step 3: Train the neural network model constructed in step 2, and use the cross entropy loss function to evaluate the difference between the output of the neural network model and the label block label:

[0125] LOSS=CrossEntropy(output,label)

[0126] CrossEntropy is the cross entropy loss function:

[0127]

[0128] y c and p c The actual class and predicted probability corresponding to the cth class, class is the number of crop classes to be classified. The Adam optimizer is used to adjust the weights of the neural network model, setting the initial learning rate to 0.001, the maximum number of iterations to 100, and the number of batch samples to 4.

[0129] After the neural network model is trained in step 3, the test set is input to obtain the crop classification results and evaluate them.

[0130] To evaluate the neural network model, the test set was input into the neural network model trained in step 3 to obtain the crop classification prediction results. The quantitative evaluation indicators OA, F1, and IOU were used to evaluate the effect of the neural network model.

[0131]

[0132]

[0133] TP c Indicates the number of pixels that the model correctly predicts as positive for the cth class (c∈[1,class]); FP c Indicates the number of pixels for which the c-th model incorrectly predicts the negative class as the positive class; TN c FN represents the number of pixels that the c-th model correctly predicts as negative. c Represents the number of pixels for which the c-th model incorrectly predicts the positive class as the negative class.

[0134] The crop classification results of this embodiment are as follows Figure 5 As shown, the OA, F1, and IOU on the test set are 78.85%, 67.13%, and 51.78% respectively. It can be seen that the technical effect and accuracy of the present invention meet the industry requirements.

[0135] It can be seen from the above embodiments that the present invention takes into account the complex phenological relationship of crops, makes full use of the characteristics of time and space dimensions of time series SAR images, and constructs a neural network model: uses an encoder to map the input array to a high-dimensional feature space and extracts multi-scale local spatial features; constructs a multi-scale cross-attention module MCMSABlock to extract multi-scale temporal features of time series SAR image blocks, uses the weighted results of the original scale as the query matrix of other scales and indexes and weights them, extracts remote dependency information at different time scales, and models the complex phenological relationship of crops; the decoder uses multiple upsampling modules to gradually restore the spatial scale, and uses jump connections to retain multi-scale local spatial features while fusing multi-scale temporal features. Then the constructed neural network model is trained for crop classification. The present invention is suitable for crop diversity and complex time series SAR scenes, and while obtaining accurate crop classification results, it reduces the need for memory and improves classification efficiency.

Claims

1. A temporal SAR crop classification method based on multi-scale cross-attention, characterized in that: The following steps are involved: Step 1: Dataset construction: Divide the samples in the target area dataset into non-overlapping training sets and test sets according to the proportion for training and testing the constructed neural network model; each sample includes a time series SAR image block Label Block h represents the height of the array, w represents the width of the array, c represents the number of features of the time series SAR image block sar, and t represents the time series length of the time series SAR image block sar; Step 2: Build a neural network model: including encoder, multi-scale cross attention module MCMSABlock, and decoder; Use encoders to map input arrays into high-dimensional feature spaces and extract multi-scale local spatial features; MCMSABlock extracts multi-scale temporal features, uses the weighted results of the original scale as the query matrix of other scales and performs indexing and weighting, extracts remote dependency information at different time scales, and models the complex phenological relationships of crops; The decoder uses skip connections to preserve multi-scale local spatial features while fusing multi-scale temporal features to gradually restore the spatial scale; Step 2.1, build the encoder: the encoder contains L downsampling modules, the output of the current downsampling module is used as the input of the next downsampling module; the downsampling module includes R convolution blocks stacked in sequence and 1 pooling layer, and the convolution block contains convolution, normalization and activation functions in sequence; by stacking downsampling modules, the input array is mapped to a high-dimensional feature space, and multi-scale local spatial features are extracted. For SAR images corresponding to different time steps, a weight-sharing downsampling module is used to extract local spatial features from the input array: inder 1 =EnConvBlock 1 (sar) EnConvBlock 1 It is the first downsampling module; is the local spatial feature extracted by the first downsampling module; h 1 is the height of the local spatial feature extracted by the first downsampling module; w 1 is the width of the local spatial feature extracted by the first downsampling module; c 1 The number of features of the local spatial features extracted by the first downsampling module; inder i =EnConvBlock i (inder i-1 ) EnConvBlock i is the i-th downsampling module, i∈[2,L]; It is the local spatial feature extracted by the i-1th downsampling module and serves as the input of the i-th downsampling module; is the local spatial feature extracted by the i-th downsampling module; h i is the height of the local spatial feature extracted by the i-th downsampling module; w i is the width of the local spatial feature extracted by the i-th downsampling module; c i is the number of features of the local spatial features extracted by the i-th downsampling module; Convolution extracts local spatial features by applying a convolution kernel to an array; Normalization reduces the impact of internal covariate shift and speeds up the model training process by standardizing the distribution of the array; The activation function introduces nonlinear mapping relationships to enhance the ability to process complex data; pooling downsamples local areas in features to improve model efficiency and resistance to overfitting; Step 2.2, construct a multi-scale cross-attention module MCMSABlock: MCMSABlock includes adding position encoding, normalization, multi-scale cross-attention MCMSA, normalization and convolution layers in sequence; MCMSABlock is used to extract multi-scale temporal features, and the weighted results of the original scale are used as the query matrix of other scales. After indexing and weighting again, the multi-scale temporal features are obtained: out, att=MCMSABlock(inner L ) It is the local spatial feature extracted by the Lth downsampling module; att is the attention weight output by MCMSABlock; Multi-scale temporal features extracted for MCMSABlock; Adding position encoding provides the position information of the smallest unit of the Token input sequence; Multi-scale cross attention MCMSA first performs global interaction on all Tokens at the original scale to obtain the attention weight att, uses the weighted result of the attention weight as the query matrix of other scales, and performs indexing and weighting again to extract long-range dependency information at different time scales. The final feature contains multi-scale global interaction information; Step 2.3, construct a decoder: The decoder contains L upsampling modules to restore the spatial scale; A single upsampling module includes upsampling and att weighting, which is then processed by a convolution block; Feature inder in encoder L-1 ...inder 1 , sar is weighted in the time dimension and is sequentially combined with out...output 2 、output 1 The upsampling result is skipped and then input into the convolutional layer to change the number of features, and the output of the upsampling module is obtained: output L-1 =DeConvBlock L (out,inder L-1 ,att) DeConvBlock L is the Lth upsampling module; DeConvBlock is the Lth upsampling module L Output: output j-1 =DeConvBlock j (output j ,inder j-1 ,att) is the output corresponding to the j+1th upsampling module, j∈[2,L-1]; DeConvBlock j is the jth upsampling module; is the local spatial feature extracted by the j-1th downsampling module; is the jth upsampling module DeConvBlock j The corresponding output: output=DeConvBlock 1 (output 1 ,sar,att) The final output of the neural network model obtained by the decoder; Step 3: Train the neural network model constructed in step 2: During the forward propagation of the input array in the neural network model, nonlinear mapping is achieved through linear transformation and nonlinear activation. The linear transformation maps the input data to a new feature space through a weight matrix and a bias vector, and the nonlinear activation function introduces nonlinearity. Finally, the crop classification prediction result output of the neural network model is obtained. The difference LOSS between the output and the label block label is evaluated using the loss function. The parameters of the neural network model are continuously iterated based on the optimizer, and finally converged on the training set to obtain the trained neural network model. LOSS=LOSSFunction(output,label) LOSSFunction is the loss function for measuring LOSS; Step 4: Input the test set into the neural network model trained in step 3 to obtain the crop classification results.

2. The method for crop classification based on multi-scale cross-attention in time series SAR according to claim 1, characterized in that: The construction process of the multi-scale cross attention module MCMSABlock is specifically as follows: Step 2.2.1, add position encoding, and extract the local spatial features from step 2.

1. L Add positional encoding: inside PE =indian L +PE PE stands for position encoding; Represents the local spatial features after adding position encoding; Step 2.2.2, normalization, add the local spatial feature inder after position encoding in step 2.2.1 PE Normalize: inder norm =Norm(inder PE ) Norm represents the normalization operation; Represents the local spatial features after normalization; Step 2.2.3, multi-scale cross attention MCMSA, under the premise of linear computational complexity, retains the global information of the original scale, and globally interacts, indexes and weights sequences of different scales. After weighting, the features contain multi-scale long-range dependencies: Indian MCMSA ,that=MCMSA(inder norm ) inder MCMSA is the local spatial feature after MCMSA processing; The construction process of MCMSA: Step A: For inder norm The sequence corresponding to each pixel in Define the query matrix as a learnable parameter The key matrix K and the value matrix V are defined as the result of projecting the sequence X through the projection matrix: K=XW K ,V=XW V W K , W V is the projection matrix, which projects the sequence X into a high-dimensional feature space. The number of features increases from c L becomes d; Q is the query matrix, K is the key matrix, and V is the value matrix; Step B: Calculate the global interaction information at the original scale, calculate the similarity between the query matrix Q and the key matrix K using the dot product, and normalize with softmax: attout=attt*V is the attention weight of the original scale; is the output after attt weighting; Step C: Calculate the L-scale bond matrix K L , value matrix V L , L∈[2,t]: K L =pool L (K),V L =pool L (V) pool L Represents the pooling operation, which is used to obtain the key matrix of scale L Value Matrix N L Represents the sequence length of scale L after pooling; Step D: Calculate the multi-scale global interaction information, taking attout as the query matrix of L scale and the key matrix K of L scale L , value matrix V L To index and weight: attout MCMSA =attt L *V L is the multi-scale cross attention weight; For attt L Weighted output; Step E: For the entire inder norm The sequence corresponding to all pixels in the image is processed by step AD, and the original scale sequence is norm Index and weight to get inder norm The corresponding attention weight The weighted result is used as the query matrix of scale L, indexed and weighted, and the inder norm Corresponding multi-scale temporal features Step 2.2.4: Normalize the multi-scale temporal features extracted in step 2.2.

3. MCMSA Normalization: in the norm2 =Norm2(inder MCMSA ) Norm2 represents the normalization operation; Represents the multi-scale temporal features after normalization; Step 2.2.5, the convolution layer includes convolution, normalization, and activation function in sequence; out=ConvLayer(inder norm2 ) ConvLayer is the convolution layer; It is the multi-scale temporal feature processed by ConvLayer; the convolution layer ConvLayer is used to restore the number of features.

3. The method for crop classification based on multi-scale cross-attention in temporal SAR according to claim 1, characterized in that: The loss function is cross entropy.

4. The method for crop classification based on multi-scale cross-attention in temporal SAR according to claim 1, characterized in that: The optimizer is Adam.

Images