Crop classification method based on U-Net and Transform

Abstract

The invention discloses a crop classification method based on U-Net and Transform. The method comprises the following steps of 1, obtaining a remote sensing image of an original data set; 2, segmenting the original remote sensing image data into 256 * 256 image blocks, and performing data enhancement on the original remote sensing image data through a data enhancement means; 3, randomly dividing the remote sensing image data subjected to data enhancement processing into a training set, a verification set and a test set, and performing crop classification by using four wave bands of red light, green light, blue light and near-infrared light in the image; 4, constructing a remote sensing semantic segmentation CPSTU-Net; 5, training the built CPSTU-Net by using the training set, and verifying by using the verification set; and 6, performing feature extraction on the test set by using the verified CPSTU-Net, and performing crop classification result prediction. The method is helpful for improving the accurate monitoring of crops so as to improve the grain yield.

Description

Technical Field

[0001] The present invention relates to a crop classification method, and in particular to a crop classification method based on U-Net and Transformer. Background Art

[0002] Food is the most basic means of survival for human beings. With the challenges of population growth and climate change, ensuring food security and sustainable development has gradually become an important global strategic goal. In modern agriculture, the application of precision agriculture technology is of great significance to increasing crop yields and optimizing resource allocation. The development of precision agriculture and smart agriculture has made crop planting more scientific and efficient, which not only increases the yield per unit area, but also reduces the negative impact on the environment.

[0003] Remote sensing technology is a key tool for achieving precision agriculture and smart agriculture. It has the advantages of wide coverage, short revisit cycle, and relatively low acquisition cost. It can provide large-area, continuous monitoring data and promote the management of precision agriculture. As a non-invasive and efficient agricultural monitoring method, remote sensing technology has been widely used in many fields such as crop classification, pest and disease monitoring, and soil testing. Using remote sensing technology to classify crops from multi-temporal remote sensing images can improve the accuracy of agricultural monitoring, optimize farmland management, and thus increase grain production. Crop classification from multi-temporal remote sensing images is the most common crop classification method. However, this method requires obtaining images at multiple time points and often faces problems such as high data acquisition cost and complex processing. In contrast, crop classification methods using remote sensing images at a single time node have the advantages of simple operation and low cost, and thus have become a research hotspot.

[0004] By analyzing high-resolution remote sensing images at a specific time point, the distribution of different crops in large areas of farmland can be quickly and accurately obtained. This not only helps to formulate precise agricultural management strategies, but also effectively reduces the workload and cost of traditional field surveys, and improves the efficiency and automation level of agricultural monitoring. However, due to the limitations of single-phase data in the time dimension, existing single-phase crop classification models often rely on image features at a single time point and cannot fully capture the subtle spectral and spatial differences between different crops. How to extract and distinguish the characteristics of different crops in images at a single time point has become a major challenge facing this field.

[0005] As a breakthrough technology in the field of machine learning, deep learning is superior to traditional machine learning methods in feature extraction and representation, and is therefore often used in remote sensing crop classification. Convolutional neural network (CNN) is one of the most basic deep learning models. It can efficiently extract low-level and high-level features of images through local connections and weight sharing, and is widely used in image classification. Fully convolutional network (FCN) is developed on the basis of CNN, and the fusion of features at different scales further improves the segmentation accuracy. U-Net was originally designed for biomedical image segmentation, but due to its excellent feature extraction and image segmentation capabilities, it has been widely used in the field of remote sensing. U-Net further improves the architecture of FCN and adopts a symmetrical U-shaped structure to retain more detailed information. Zhang et al. proposed ResUNet based on the U-Net model, replacing the basic convolutional blocks in U-Net with residual blocks, and introducing skip connections in each residual block to achieve cross-layer connections, thereby enhancing the model's feature extraction capabilities and alleviating the gradient vanishing problem.

[0006] The CNN-based deep learning model has certain limitations in capturing global information and cannot fully capture global information, which may cause it to lose spatial details when processing high-resolution images. The FCN model improved based on CNN is prone to resolution loss and high computational complexity during upsampling. U-Net and ResUNet have weak extraction capabilities for edge plots and are difficult to clearly extract spatial information when facing complex backgrounds or mixed pixels in remote sensing images. Therefore, it is necessary to design a better method to more fully extract the global features of remote sensing images, so as to achieve more accurate crop classification of remote sensing images.

[0007] The Transformer model was first used in the field of natural language processing (NLP). Due to its excellent performance, researchers introduced its self-attention mechanism into the field of computer vision for image classification and segmentation tasks. SwinTransformer is a Transformer model based on the window attention mechanism. It was proposed by Ze Liu et al. in 2021. It introduces the window attention mechanism and hierarchical architecture. Through layer-by-layer processing and shift window strategy, it reduces computational complexity and enhances global perception. SwinTransformer replaces the multi-head self-attention module (MSA) of the standard Transformer with a multi-head self-attention module based on a shifted window (W-MSA / SW-MSA). The other parts remain unchanged. W-MSA is a method for calculating multi-head self-attention in a window of constant size. It divides the input feature map into non-overlapping windows of constant size, and calculates attention independently in each window. It can process multiple windows in parallel, thereby improving computational efficiency. SW-MSA introduces the concept of shifted windows. The division position of each layer of windows will move half the window size to the lower right so that each pixel can be paid attention to in different windows, enhancing the extraction of global information.

[0008] The transformer-based deep learning semantic segmentation model lacks the perception of local information and is difficult to fully extract the detailed features of remote sensing images. It may still be insufficient when processing image boundary details and consumes a lot of computing resources. It performs poorly on small samples and unbalanced data sets. Therefore, it is necessary to design a better method to more fully extract the detailed information of remote sensing images, so as to achieve more accurate crop classification of remote sensing images. Summary of the invention

[0009] The present invention provides a crop classification method based on U-Net and Transformer. The method adopts CPSTU-Net as a method for single-phase remote sensing image crop classification. Compared with other deep learning methods, the method has stronger feature extraction ability, thereby achieving higher accuracy of crop classification, which is helpful to improve the accurate monitoring of crops and thus increase food yield.

[0010] The objective of the present invention is achieved through the following technical solutions:

[0011] A crop classification method based on U-Net and Transformer includes the following steps:

[0012] Step 1: Obtain remote sensing images of the original data set;

[0013] Step 2: Use a sliding window method to divide the original remote sensing image data into image blocks of 256×256 size, and use data enhancement methods to enhance the original remote sensing image data to increase the number of samples;

[0014] Step 3: Randomly divide the remote sensing image data after data enhancement into training set, validation set and test set, and use the four bands of red light, green light, blue light and near infrared light in the image to classify crops;

[0015] Step 4: Build remote sensing semantic segmentation CPSTU-Net:

[0016] CPSTU-Net adopts an encoder-decoder structure, using ResUNet as the main encoder and SwinTransformer as the auxiliary encoder. The main encoder and the auxiliary encoder transmit information through the feature interaction module (CPRA), and the encoder and decoder are connected through the skip connection layer.

[0017] The input remote sensing image of dimension H×W×4 is given. First, the ResNet50 structure of the main encoder compresses it to obtain its deep features. The auxiliary encoder Swin Transformer splits the input remote sensing image of dimension H×W×4 into 8×8 patches with an overlap rate of 50%, and then projects these patches into one dimension through the linear embedding layer. Subsequently, these patches The tokens are fed into the SwinTransformerblocks with improved self-attention. The auxiliary encoder in the CPSTU-Net framework has four feature extraction levels. The CPRA module fuses the features extracted by each level of the auxiliary encoder with the main encoder through the spatial attention and channel attention mechanisms, increasing the feature interaction between the main encoder and the auxiliary encoder. After the above encoding stage, the feature with a dimension of (H / 32)×(W / 32)×1024 is finally obtained; then the position channel attention module (PC) on the jump connection layer is connected and the encoder and decoder features are cascaded using the depth-separable convolution through the channel attention mechanism and the spatial attention mechanism, respectively, and then through four 3×3 convolutional layers, batch normalization layers and relu layers, the feature F is gradually expanded to F∈R(H / 2)×(W / 2)×64; finally, the feature F is upsampled by 3×3 convolutional layers and linear interpolation to obtain the final prediction template;

[0018] Step 5: Use the training set to train the built CPSTU-Net and use the verification set to verify it;

[0019] Step 6: Use the verified CPSTU-Net to extract features from the test set and predict crop classification results.

[0020] Compared with the prior art, the present invention has the following advantages:

[0021] 1. Multi-scale feature extraction: Traditional CNN-based semantic segmentation methods cannot fully capture global information, while transformer-based semantic segmentation models lack global perception of local features. The CPSTU-Net of the present invention combines the advantages of CNN and Transformer, using ResUNet as the main encoder and Swin Transformer as the auxiliary encoder. This dual encoder structure can fully extract the spatial and spectral features of remote sensing images and enhance the aggregation ability of global and local information.

[0022] 2. Efficient feature fusion: Traditional methods simply add spatial features and local features, and the fusion features are not sufficient. The CPSTU-Net of the present invention introduces the CPRA module and the PC module, and strengthens the information transmission and fusion between the main encoder and the auxiliary encoder through the position attention and channel attention mechanisms, fully extracting the global and detail features of the remote sensing image, thereby improving the accuracy and robustness of feature extraction.

[0023] 3. Sample imbalance processing ability: Traditional methods only use binary cross entropy loss function, which lacks the ability to process sample category imbalance, resulting in low accuracy. The CPSTU-Net of the present invention effectively solves the problem of category imbalance in the data set by combining the Dice loss function and the binary cross entropy loss function. The Dice loss function is particularly suitable for processing sample category imbalance and can more accurately optimize the segmentation performance of the minority class.

[0024] 4. Prediction accuracy: Since CPSTU-Net combines the advantages of CNN and Transformer to extract and fuse features more effectively, it can often achieve higher classification accuracy in crop classification tasks than other traditional methods, which is of great significance for the realization of precision agriculture.

[0025] 5. Application advantages: Compared with traditional crop classification methods, the CPSTU-Net of the present invention has obvious advantages in multi-scale feature extraction, efficient global and local feature fusion, processing of unbalanced samples within a class, and improving prediction accuracy. The single-temporal remote sensing image crop classification realized by CPSTU-Net can quickly and accurately obtain the distribution of different crops in large areas of farmland, significantly reduce the workload and cost of traditional field surveys, and provide reliable support for the realization of precision agriculture and smart agriculture. Therefore, it is suitable for application scenarios that require single-temporal remote sensing images for crop classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is the overall structure diagram of CPSTU-Net;

[0027] Figure 2 It is the CPRA feature aggregation structure diagram;

[0028] Figure 3 This is the PC module structure diagram;

[0029] Figure 4 This is a flow chart of the crop classification method based on U-Net and Transformer;

[0030] Figure 5 This is the comparison result between the present invention and other classification models. DETAILED DESCRIPTION

[0031] The technical solution of the present invention is further described below in conjunction with the accompanying drawings, but is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention should be included in the protection scope of the present invention.

[0032] The present invention provides a crop classification method based on U-Net and Transformer. The method is implemented through the existing deep learning framework Pytorch and corresponding data processing operations. The specific implementation principle is as follows: a new remote sensing semantic segmentation CPSTU-Net is proposed, and the original data after traditional data enhancement technology is input into CPSTU-Net for remote sensing semantic segmentation to perform crop classification. CPSTU-Net adopts an encoder-decoder structure, and the encoder part is composed of a main encoder-CNN-based residual network and an auxiliary encoder-Swin Transformer in parallel. The main encoder and the auxiliary encoder transmit information through a feature interaction module (CPRA), and the model connects the encoder and the decoder through a jump connection layer; a position channel attention module (PC) is designed and attached to the jump connection layer to enhance the feature extraction ability of the model, thereby increasing the segmentation accuracy of the model; finally, a loss function is designed to solve the sample imbalance problem in the data set. The present invention solves the following technical problems:

[0033] 1. In view of the problem that the existing models cannot fully extract the global and detailed features of remote sensing data, a semantic segmentation framework combining CNN and Transformer is designed. The parallel dual encoder structure is adopted to further improve the model's ability to extract the spatial and detailed features of remote sensing images at a single time point, thereby improving the accuracy of crop classification.

[0034] 2. In view of the insufficient interaction between spatial information and local information in the existing model, a CPRA is designed to strengthen the information interaction between the main encoder and the auxiliary encoder, combining the detail information extracted by the main encoder with the global information extracted by the auxiliary encoder, further enhancing the fusion of the two features.

[0035] against

[0036] 3. In order to solve the problem of large consumption of computing resources in the traditional dual attention mechanism, a PC is designed to be attached to the jump connection, which reduces the computing resources required by the traditional dual attention mechanism (DA) module and enhances the feature extraction ability and segmentation performance of the model.

[0037] 4. In response to the sample imbalance problem in the data set, the loss function is modified to a combination of the dice loss function and the binary cross entropy loss function to solve the problem of inaccurate classification results caused by sample imbalance.

[0038] (I) CPSTU-Net remote sensing semantic segmentation:

[0039] CPSTU-Net is a new composite network architecture that combines the U-Net architecture (U-Net) based on convolutional neural networks (CNN) and the swin transformer based on the window self-attention mechanism. This approach combines the advantages of both techniques and can provide better performance and accuracy in tasks such as crop classification of single-temporal remote sensing images.

[0040] The input is a remote sensing image with a dimension of H×W×4, where H and W represent the length and width of the remote sensing image, respectively. First, the ResNet50 structure of the main encoder compresses it to obtain its deep features. The auxiliary encoder Swin Transformer splits the input H×W×4 remote sensing image into 8×8 patches with an overlap rate of 50%, and a total of N patches are split. Then these patches are projected into one dimension through a linear embedding layer. Subsequently, these patch tokens are fed into Swin Transformer blocks with improved self-attention. The auxiliary encoder in the CPSTU-Net framework has a total of four feature extraction levels. The CPRA module fuses the features extracted by each level of the auxiliary encoder with the main encoder through spatial attention and channel attention mechanisms. After the above encoding stage, we finally get features with a dimension of (H / 32)×(W / 32)×1024. Then, the PC module on the skip connection layer is connected and the features of the encoder and decoder are cascaded using the channel attention mechanism and the spatial attention mechanism respectively. Then, the feature F is gradually expanded to F∈R(H / 2)×(W / 2)×64 through four 3×3 convolutional layers, batch normalization layers and relu layers. Finally, the feature F is upsampled by 3×3 convolutional layers and linear interpolation to obtain the final prediction template. The overall framework of the technology is shown in the figure below. Figure 1 shown.

[0041] (II) CPRA feature aggregation:

[0042] Since the main encoder can only obtain information from the local receptive field and has a weak understanding of global context information, and the auxiliary encoder lacks perception of detailed features, it is necessary to interact and aggregate the features obtained by the two. Therefore, the present invention proposes a CPRA module to increase the feature interaction between the main encoder and the auxiliary encoder. The CPRA module is a feature aggregation module based on the combination of spatial attention mechanism and channel attention mechanism. The features output by the main encoder and the features output by the auxiliary encoder are interacted and aggregated through the CPRA module, which enhances the information transmission and fusion between encoders, and enhances the robustness and classification accuracy of the model. The specific structure is as follows Figure 2 As shown, the process can be broken down into the following steps:

[0043] 1. Input feature map B of the module n First, the spatial information is obtained through operations such as average pooling and maximum pooling, and then it is input into the shared MLP and sigmoid is performed to obtain the channel attention map B. n ';

[0044] 2. Multiply the input feature with the channel attention map to obtain the channel prior, input the channel prior into the depth-wise separable convolution module to generate the spatial attention map, and perform channel mixing through 1×1 convolution to enhance the feature representation, and then n 'Multiply it to obtain the refined feature map R n ;

[0045] 3. The main encoder input feature map is A n , after 3x3 convolution, we get A n 'With R n The refined feature map obtained by multiplication is denoted as R n ';

[0046] 4. Finally, the refined feature map R n 'With A n 'The residual structure is combined to form CPRA to obtain the final output T n .

[0047] (III) PC module:

[0048] The PC module uses a deep separable convolution to combine the spatial attention mechanism and the channel attention mechanism. The present invention attaches it to the jump connection layer, which enhances the model feature extraction capability while reducing the consumption of computing resources and enhancing the generalization capability of the model. The PC module includes a channel feature extraction module and a position feature extraction module. The main encoder inputs the feature map A n It is input into the position feature extraction module and the channel feature extraction module to obtain B n and C n , where the channel feature extraction module converts the original feature A∈R C×H×W , C, H, and W represent the number of bands, length, and width used in the original remote sensing image, respectively, and are reshaped into R C×N , and perform matrix multiplication with the transpose of the original feature map. The output is then passed through the softmax layer to obtain a C×C channel attention map X. Next, X is combined with A n Multiply and reshape the feature map into R C×H×W , and then add it to the original input to get the channel attention map C n The position feature extraction module first passes through a convolution module to obtain three local features D1, D2, and D3, with a size of C×H×W. Then D1 and D2 are reshaped to C×N. Matrix multiplication is performed on D1 and D2 and a softmax operation is performed to obtain a feature map S. Next, S is multiplied by D3 and the feature map is reshaped to R. C×H×W , and then add it to the original input to get the channel attention map B n . Then, after deep separable convolution, feature map B is obtained. n ' and C n', finally add the two features to get the final output feature A n '. The algorithm structure diagram is as follows Figure 3 shown.

[0049] (IV) Combination of Dice and binary cross entropy loss function:

[0050] In response to the serious imbalance of sample categories in the data set, Milletari et al. applied the Dice loss function to 3D medical image segmentation in 2016. However, the simple use of the Dice loss function may cause the model to focus too much on the correct prediction of the majority class, while the prediction of the minority class is not accurate enough. Therefore, a loss function combining the binary cross entropy loss function and the Dice loss function is introduced to solve this problem. The Dice loss function directly optimizes the overlapping area of ​​the segmentation results, so that the model can more accurately predict the target area, and the effect is significant when dealing with highly imbalanced data sets. Its calculation formula is as follows:

[0051]

[0052] Among them, p i is the value of the i-th pixel in the predicted mask, g i is the value of the i-th pixel in the true mask and n is the total number of pixels in the image.

[0053] (V) Crop classification based on U-Net and Transformer

[0054] This paper adopts a comprehensive method, mainly relying on CPSTU-Net remote sensing semantic segmentation technology and combining CPRA feature aggregation technology and PC attention technology in this technology, and finally realizes crop classification of single-phase remote sensing images using an improved loss function. Figure 3 As shown, the specific steps are as follows:

[0055] First, the remote sensing images of the original data set are obtained. The data are collected from Sentinel-2 single-phase remote sensing images of Beidahuang, Shuangyashan City, Heilongjiang Province. The geographic coordinate range is 46°28'15”-46°58'39”N, 137°27'50”-132°15'38”E. The data set was collected on August 17, 2021, at the maturity stage of the crops. The main crop types are rice, soybeans, corn, and non-agricultural fields. The spatial resolution is 10m and the original image size is 5505×4280.

[0056] Secondly, the sliding window method is used to divide the original image into image blocks of 256×256, and the original remote sensing data is enhanced through traditional data enhancement methods such as rotation and flipping to increase the number of samples.

[0057] Next, in each experiment, the training set, test set, and validation set were randomly divided into 8:1:1. Finally, there were 143 image blocks for training, 17 image blocks for validation, and 17 image blocks for testing. The four bands of red light, green light, blue light, and near-infrared light in the image were used for crop classification.

[0058] Subsequently, the CPSTU-Net model is used to extract features from the input data and predict the results of crop classification. CPSTU-Net combines Transformer with CNN and adopts a parallel dual encoder structure, which can effectively solve the problem of insufficient extraction of global features and detailed features. In addition, the loss function is modified to a combination of the Dice loss function and the binary cross entropy loss function, which effectively solves the problem of intra-class imbalance in the data set.

[0059] Finally, the trained model is compared with other classification models, and the performance of the model is evaluated using IoU (Intersection over Union), OA (Overall Accuracy) and F1 Score. OA is the proportion of all pixels classified correctly, which is used to measure the classification accuracy of the entire model. mIoU is the average value of Intersection over Union (IoU) of each category, which is used to measure the segmentation performance of the model in each category and comprehensively reflects the overall performance of the model. F1 Score is the harmonic mean of precision and recall, which comprehensively considers precision and recall. The experimental results are shown in Table 1, and the visualization is as follows: Figure 5 shown.

[0060] Table 1

[0061]

[0062] The method of the present invention integrates data processing and deep learning models, providing a comprehensive and effective solution for crop classification of single-temporal remote sensing images.

Claims

1. A crop classification method based on U-Net and Transformer, characterized in that The method comprises the following steps: Step 1: Obtain remote sensing images of the original data set; Step 2: Use a sliding window method to divide the original remote sensing image data into image blocks of 256×256 size, and use data enhancement methods to enhance the original remote sensing image data to increase the number of samples; Step 3: Randomly divide the remote sensing image data after data enhancement into training set, validation set and test set, and use the four bands of red light, green light, blue light and near infrared light in the image to classify crops; Step 4: Build remote sensing semantic segmentation CPSTU-Net: CPSTU-Net adopts an encoder-decoder structure, using ResUNet as the main encoder and Swin Transformer as the auxiliary encoder. The main encoder and the auxiliary encoder transmit information through the feature interaction module, and the encoder and decoder are connected through the skip connection layer. The input remote sensing image is given a dimension of H×W×4, where H and W represent the length and width of the remote sensing image respectively. First, the ResNet50 structure of the main encoder compresses it to obtain its deep features. The auxiliary encoder Swin Transformer splits the input H×W×4 remote sensing image into 8×8 patches with an overlap rate of 50%, and then projects these patches into one dimension through a linear embedding layer. Subsequently, these patch tokens are fed into Swin Transformer blocks with improved self-attention. The auxiliary encoder in the CPSTU-Net framework has four feature extraction levels. The CPRA module fuses the features extracted by each level of the auxiliary encoder with the main encoder through spatial attention and channel attention mechanisms, increasing the feature interaction between the main encoder and the auxiliary encoder. After the above encoding stage, the feature with a dimension of (H / 32)×(W / 32)×1024 is finally obtained; Then, the skip connection layer is used to connect and the position channel attention module on the skip connection layer uses the channel attention mechanism and the spatial attention mechanism to use the depth-separable convolution cascade encoder and decoder features, and then through four 3×3 convolution layers, batch normalization layers and relu layers, the feature F is gradually expanded to F∈R(H / 2)×(W / 2)×64; finally, the feature F is upsampled by 3×3 convolution layers and linear interpolation to obtain the final prediction template; Step 5: Use the training set to train the built CPSTU-Net and use the verification set to verify it; Step 6: Use the verified CPSTU-Net to extract features from the test set and predict crop classification results.

2. The crop classification method based on U-Net and Transformer according to claim 1, characterized in that The feature interaction module is a feature aggregation module based on the combination of spatial attention mechanism and channel attention mechanism. The features output by the main encoder and the features output by the auxiliary encoder are subjected to feature interaction and aggregation through the feature interaction module. The process is decomposed into the following steps: (1) Input feature map B of the module n First, the spatial information is obtained through average pooling and maximum pooling operations, and then it is input into the shared MLP and sigmoid is performed to obtain the channel attention map B. n '; (2) The input feature is multiplied by the channel attention map to obtain the channel prior, and the channel prior is input into the depthwise separable convolution module to generate the spatial attention map and perform channel mixing through 1×1 convolution to enhance the feature representation. n 'Multiply it to obtain the refined feature map R n ; (3) The main encoder input feature map is A n , after 3x3 convolution, we get A n 'With R n The refined feature map obtained by multiplication is denoted as R n '; (4) Finally, the refined feature map R n 'With A n 'The residual structure is combined to form CPRA to obtain the final output T n .

3. The crop classification method based on U-Net and Transformer according to claim 1, characterized in that The position channel attention module adopts deep separable convolution to combine the spatial attention mechanism and the channel attention mechanism, including a channel feature extraction module and a position feature extraction module. The main encoder inputs the feature map A n It is input into the position feature extraction module and the channel feature extraction module to obtain B n and C n , where the channel feature extraction module converts the original feature A∈R C ×H×W , C, H, and W represent the number of bands, length, and width used in the original remote sensing image, respectively, and are reshaped into R C×N , and perform matrix multiplication with the transpose of the original feature map. The output is then passed through the softmax layer to obtain a C×C channel attention map X. Next, X is combined with A n Multiply and reshape the feature map into R C×H×W , and then add it to the original input to get the channel attention map C n The position feature extraction module first obtains three local features D1, D2, and D3 through a convolution module, with a size of C×H×W. Then D1 and D2 are reshaped into C×N. Matrix multiplication is performed on D1 and D2, and a softmax operation is performed to obtain a feature map S. Next, S is multiplied by D3 and the feature map is reshaped into R. C×H×W , and then add it to the original input to get the channel attention map B n , and then through the depth separable convolution to get the feature map B n ' and C n ', finally add the two features to get the final output feature A n '.

4. The crop classification method based on U-Net and Transformer according to claim 1, characterized in that The loss function of the CPSTU-Net is a combination of Dice and binary cross entropy loss functions. The calculation formula of the Dice loss function is as follows: Among them, p i is the value of the i-th pixel in the predicted mask, g i is the value of the i-th pixel in the true mask and n is the total number of pixels in the image.

Images