Lightweight sand dune form type identification model based on partial convolution and Transform

Abstract

The invention relates to a computer visual recognition technology, in particular to a lightweight sand dune form type recognition model based on partial convolution and Transform, which comprises an encoder, a decoder, a feature fusion module and a category attention block, the encoder adopts a partial convolution fast T-shaped block, information interaction between channels is not lost while the parameter quantity is effectively reduced, and the recognition accuracy is improved. The cost is saved; the feature fine degree is ensured; the following sampling operation is carried out after each layer of network; transform is introduced into the decoder, so that local context information and global context information of the feature map can be further extracted, and detail semantic information is considered while global semantic information is extracted; the encoder output features and the decoder output features of each layer need to be aggregated through a feature fusion module, feature refinement fusion is achieved, category feature information is further enhanced through a category attention mechanism, and the sand dune form type recognition effect is improved.

Description

Technical Field

[0001] The present invention relates to computer vision recognition technology, and in particular to a lightweight dune morphological type recognition model based on partial convolution and Transformer. Background Art

[0002] Deserts cover about one-third of the Earth's land surface and are characterized by extreme environmental conditions, including high temperatures, low humidity, and sparse vegetation. Against the backdrop of global warming and intensified droughts, the activation of sand dunes and the expansion of deserts have intensified, leading to serious desertification. Sand dunes are dynamic landforms formed by the action of wind and sand. Sand dunes have a variety of morphological characteristics, including crescent dunes, grid dunes, and composite dunes. Accurate identification of dune types is essential for understanding dune movement, dune migration processes, and the impact of climate change on desert landscapes. For example, accurate mapping of dune morphology is beneficial to infrastructure construction, disaster assessment, and disaster management in arid areas. Therefore, the study of dune morphological type identification has become an important topic in contemporary society. It is of great significance for real-time monitoring of desert conditions and effective desertification control, and has an increasingly important impact on social development and human survival.

[0003] In recent years, automation technology has been used to efficiently identify and classify desert dune morphological types. Deep learning has shown great potential in image recognition and classification, laying a solid foundation for remote sensing data analysis of dune morphological types. In the context of deep learning, the application of convolutional neural networks (CNNs) and other advanced machine learning algorithms provides unprecedented opportunities for automated and accurate dune morphological recognition. The dune morphological type recognition algorithm based on deep learning can extract complex features from remote sensing data, thereby classifying and identifying different desert dune morphological types with high accuracy and efficiency. In addition, the fusion of deep learning and multi-source remote sensing data has enabled technical innovation in dune morphological recognition and improved the accuracy and reliability of classification results. As an important issue in the field of computer vision, the main task of semantic segmentation can be summarized as achieving effective pixel-level segmentation of images. At present, semantic segmentation models based on convolutional neural networks have achieved excellent results in this field and still have high exploration value. Applying semantic segmentation to remote sensing images is beneficial to the interpretation and analysis of remote sensing images. Specifically, by classifying each pixel on the remote sensing image, accurate identification and segmentation of objects can be achieved, including automatic identification and extraction of different types of objects such as buildings, roads, and vegetation, which can provide accurate and reliable data support for urban planning, environmental monitoring, agricultural management and other fields. In fact, although the relevant technologies in the field of semantic segmentation of remote sensing images have been developed to a relatively mature level and have achieved excellent application results, the relevant research in the field of deserts is still relatively weak. Due to factors such as its terrain, lighting, and difficulty in field surveys, the desert urgently needs to realize the automatic extraction and recognition of its morphological types. However, directly applying the existing semantic segmentation model to the desert is often not well adapted, so it is inevitable that various problems such as unsatisfactory recognition results will arise.

[0004] Existing research methods for remote sensing image classification and recognition are very rich, but when applied to deserts, the performance is not satisfactory. This is because the data set for dune morphological type recognition is scarce, and the network model is prone to problems such as low edge recognition accuracy and category imbalance when applied. In view of the difficulty of desert boundary recognition and network calculation redundancy in the process of dune morphological type recognition of remote sensing images, the present invention proposes a lightweight dune morphological type recognition model (Partial Convolution and Transformer Network, PCTNet) based on partial convolution and Transformer for dune morphological type recognition. For the input desert remote sensing feature map, its shallow features are extracted and its number of channels is amplified. There is often redundancy of feature information during deep feature extraction. The encoder of PCTNet adopts partial convolution fast T-block, which effectively reduces the amount of parameters without losing information interaction between channels, saves costs and ensures the fineness of features. At the same time, after each layer of the network, the sampling operation is assisted to expand the receptive field while obtaining multi-scale high-level low-resolution semantic features, so as to learn the detailed semantic information of the feature map. Transformer is introduced into the decoder. The improved dual-branch Transformer based on the self-attention mechanism can further extract local context information and global context information of the feature map, taking into account detailed semantic information while extracting global semantic information. The encoder output features and decoder output features of each layer need to be aggregated through a feature fusion module to further achieve feature refinement and fusion. Finally, in order to solve the problem of unbalanced data volume of each type of dune morphology, the category feature information is further enhanced through the category attention mechanism to improve the recognition effect of dune morphology. Summary of the invention

[0005] The primary purpose of the present invention is to provide a lightweight dune morphology type recognition model based on partial convolution and Transformer, the dune morphology type recognition model comprising:

[0006] (1) Encoder: performs feature extraction on the input image;

[0007] The encoder uses a partial convolution fast T-block, a total of four partial convolution fast T-blocks are used, the first partial convolution fast T-block uses a tiled partial convolution fast T-block with unchanged number of channels, and the remaining three are partial convolution fast T-blocks amplified by 1×1 convolution channels;

[0008] The importance of different positions in the feature map can be evaluated by the Frobenius norm, as shown in formula (2);

[0009]

[0010] Among them, ||F i || represents the Frobenius norm at position i,

[0011] (2) Decoder: Restore the texture details of the feature map output by the encoder and gradually restore its channel dimension; adopt an improved dual-branch Transformer, add a global attention branch and a local attention branch; the Transformer self-attention mechanism is implemented by mapping to three linear layers. Its calculation formula is shown in formula (3). The matrices Query, Key and Value are needed to realize linear transformation during calculation.

[0012]

[0013] Among them, Query and Value represent the encoding vectors of deep semantic feature mapping, Key represents the encoding vector of shallow semantic feature mapping, d represents the number of columns of Query and Key matrices, that is, the vector dimension, which are fused and multiplied. Attention(Query,Key,Value) is the result of residual connection and batch normalization.

[0014] (3) Feature fusion module: For the input desert remote sensing feature map, it is first subjected to a 7×7 convolution operation with a step size of 2 to reduce the feature map size and increase the number of channels, followed by batch normalization and feature activation operations. While completing the shallow feature extraction, it ensures that the subsequent partial convolution blocks obtain input feature maps with sufficient channels to ensure the quality of their output feature maps. The calculation formula is shown in formula (1):

[0015] F shallow =H ReLU (H BN (Conv 7×7 (F0))) (1)

[0016] Among them, F0 represents the original input feature, F shallow Represents the shallow features of the output, H ReLU (·) represents the ReLU activation operation, H BN (·) indicates BN operation, Conv 7×7 Represents a 7×7 convolution kernel;

[0017] (4) Category attention block: For each category, a number of channels k that can cover the feature information is given, and the feature channels of each category are different, so that each can represent a certain category; for the input feature map F Input ∈R h×w×c, assuming there are N categories, it is necessary to convert the number of feature channels to kN through a 1×1 convolution layer to obtain the feature map F′∈R h×w×kN ;

[0018]

[0019] Among them, GMP(·) represents the global maximum pooling, f i, ' j represents the jth feature of the i-th category in F′;

[0020] The above-obtained F′ is subjected to average pooling and maximum pooling operations at the same time, and F″∈R is obtained h×w×N ; In order to obtain significant feature information and retain its detailed feature information, its calculation method is shown in formula (5);

[0021]

[0022] Among them, f i, ' j represents the jth feature of the i-th category in F′, F i ″ represents the feature map of the i-th category;

[0023] The calculation method of a certain position category attention F″′ is shown in formula (6);

[0024] The input feature map F Input Multiply it with the position category attention F″′ to get the final output feature map F Output ∈R h ×w×c , as shown in formula (7);

[0025]

[0026] F Output =F Input F″′ (7).

[0027] Preferably, the partial convolution fast T-block comprises a partial convolution module superimposed with a 1×1 conventional dimensionality-raising convolution.

[0028] Preferably, the input feature of the first partial convolution fast T-block is a feature map whose channels have been expanded after a 7×7 convolution with a step size of 2.

[0029] Preferably, the dual-branch Transformer consists of dual-branch attention, a multi-layer perceptron and two batch normalizations.

[0030] Preferably, the loss function of the recognition model is formed by combining the main loss and the auxiliary loss, and its calculation method is shown in formulas (8)-(11).

[0031]

[0032] L p =L ce +L dice (10)

[0033] L=L p +αL aux (11)

[0034] Among them, L p Indicates the main loss, L dice represents dice loss, L ce represents the cross entropy loss, L aux represents auxiliary loss, L aux =L ce , N and K represent the number of samples and the number of categories respectively, p (n) and They represent the true value and predicted value of the original feature map respectively, and the auxiliary factor α=0.4.

[0035] Preferably, all experiments are implemented based on the open source deep learning framework PyTorch and Python 3.9, and the Ubuntu 20.04 operating system is used during the experimental operation.

[0036] Preferably, the server hardware uses NVIDIA GeForce RTX 4090 GPU with 256GB memory and 24GB video memory, and Intel Xeon Gold 5220 CPU.

[0037] Preferably, the experiment deploys the Adam optimizer to train the experimental model.

[0038] The beneficial effects of the present invention are as follows: the present invention provides a lightweight dune morphology type recognition model PCTNet based on partial convolution and Transformer, wherein the encoder adopts a partial convolution fast T-block, which effectively reduces the amount of parameters without losing information interaction between channels, saves costs and ensures the fineness of features, and at the same time assists with downsampling operations after each layer of the network to expand the receptive field while obtaining multi-scale high-level low-resolution semantic features, thereby learning the detailed semantic information of the feature map, and introducing the Transformer into the decoder. The dual-branch Transformer improved based on the self-attention mechanism can further extract the local context information and global context information of the feature map, taking into account the detailed semantic information while extracting the global semantic information, and the encoder output features and the decoder output features of each layer need to be aggregated through a feature fusion module to further realize feature refinement and fusion; finally, in view of the problem of unbalanced data volume of each type of dune morphology type, the category feature information is further enhanced through the category attention mechanism to improve the dune morphology type recognition effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 PCTNet overall network structure

[0040] Figure 2 Quick T-Block

[0041] Figure 3 Dual-branch Transformer block

[0042] Figure 4 Feature fusion module

[0043] Figure 5 Category Attention Block

[0044] Figure 6 The impact of different lr values ​​on the PA values ​​of the two data sets

[0045] Figure 7 The impact of different bs values ​​on the PA values ​​of the two data sets

[0046] Figure 8 The impact of different epoch values ​​on the PA values ​​of the two data sets

[0047] Fig. 9 Recognition results of different algorithms on the Landsat-8 dataset. Note: (a) morphological data (b) sand ridge data (c) true value (d) FCN (e) SegNet (f) Unet (g) DeepLab V3+ (h) Segmenter (i) SwiftNet (j) SwinUNet (k) SegFormer (l) UNetFormer (m) AFFormer (n) PCTNet

[0048] Fig.10 Recognition results of different algorithms on the GF-1WFV dataset. Note: (a) morphological data (b) sand ridge data (c) true value (d) FCN (e) SegNet (f) Unet (g) DeepLab V3+ (h) Segmenter (i) SwiftNet (j) SwinUNet (k) SegFormer (l) UNetFormer (m) AFFormer (n) PCTNet DETAILED DESCRIPTION

[0049] The protection scope of the present invention is described in detail below in conjunction with specific embodiments. It should be noted that the protection scope of the present invention is not limited by the following embodiments.

[0050] In the following embodiments, the encoder is also referred to as Encoder, and the two have the same meaning.

[0051] In the following embodiments, a decoder is also referred to as a Decoder, and both have the same meaning.

[0052] In the following embodiments, batch normalization is abbreviated as BN because of its full name Batch Normalization.

[0053] Deep Convolutional Neural Networks (DCNN) can achieve tasks such as image classification and recognition. DCNN can learn hierarchical features from raw pixel data better than CNN due to its network depth and parameter sharing. Both use multi-layer convolution to extract features, pooling operations to reduce the dimensionality of data, and fully connected layers to achieve classification. They have achieved advanced performance in various computer vision tasks, such as object recognition, image classification, and facial recognition, and have also been successfully applied to medical imaging, self-driving cars, and natural language processing.

[0054] Transformer is a network model based on the self-attention mechanism proposed in recent years. It continues the encoder-decoder network structure and can learn attention weights from multiple dimensions, thus having better network performance.

[0055] The network structure of the Fully Convolutional Network (FCN) is composed entirely of stacked convolutional layers and pooling layers, without a fully connected layer. In the field of semantic segmentation, FCN is the first to implement an end-to-end convolutional neural network. Unlike image classification and object detection, semantic segmentation requires prediction of the category of each e pixel in the image. In common classification networks, the fully connected layer and softmax at the end of the network to obtain category probability information will turn the probability information into one-dimensional information, thus lacking spatial information and only predicting the category of the entire image. FCN removes the fully connected layer in the convolutional neural network, opening up a new direction for pixel-by-pixel classification of semantic segmentation images. FCN without a fully connected layer has no requirements for the size of the input image, which can be arbitrary. When restoring the size of the output image of the last convolutional layer to make it the same as the original image, FCN uses a deconvolution method to achieve upsampling, so that the spatial information of the original image is retained, so that prediction can be made at each pixel of the upsampled image, thereby achieving pixel-by-pixel classification. In order to further optimize the feature fusion of deep information and shallow information, FCN introduces a skip structure to combine the input features of different network layers, so as to learn more information, which is conducive to the accurate segmentation of the model. However, FCN's segmentation is relatively rough in detail processing, and the upsampling results have the problem of blurring and smoothing. In addition, the fully connected layer is only replaced with a convolutional layer to retain spatial information. The classification of pixels is still independent, and there is still the problem of ignoring their spatial relationships, which lacks globality.

[0056] SegNet also classifies images pixel by pixel. Unlike FCN, its encoder and decoder structures are different. The encoder and decoder of SegNet are symmetrical. The encoder uses the VGG16 network without the fully connected layer to extract features, and the decoder uses deconvolution and upsampling operations to better express the classification feature information and restore the image to the original image size. Finally, softmax is used to obtain the maximum value of different categories to obtain the segmentation result. SegNet has a smaller parameter volume and a lighter network because it removes the fully connected layer. In addition, the encoder of SegNet stores the pooling index when performing maximum pooling, which saves the learning step of the decoder when upsampling. The index can be used directly to implement nonlinear upsampling operations, so that the integrity of high-frequency information details is guaranteed. However, when the target feature is a low-resolution feature map, its low-frequency edge information features are easily ignored and lost when the maximum pooling index is used for de-pooling.

[0057] UNet is also an Encoder-Decoder structure, and its overall structure is U-shaped, such as Figure 2-9As shown in the figure. Among them, the encoder still uses the convolution layer and pooling layer of the classic convolutional neural network. After each downsampling, the number of channels becomes twice the original number; the decoder uses 2×2 deconvolution for upsampling, and the corresponding number of channels becomes half of the original number. The feature fusion is achieved by skipping the feature map with the same level in the encoder, and the feature map with the same number of original channels is obtained. UNet skips the encoder and decoder feature maps at each level to ensure that the decoder can learn more feature information at each level and prevent the feature information from being lost in the encoder. Compared with FCN's feature fusion of pixels, UNet can retain more low-frequency edge detail features through channel feature fusion, and has a certain multi-scale prediction ability. Therefore, the segmentation result is more accurate. Based on the advantages of small sample friendliness, fast training speed and good fuzzy boundary recognition effect, UNet plays an important role in medical image segmentation and has been widely used in various fields.

[0058] The DeepLab series of deep learning semantic segmentation models have been continuously updated and iterated, and have gone through DeepLab V1, DeepLab V2 and DeepLab V3, and have now developed to DeepLab V3+. In order to learn multi-scale information features, DeepLab V3+ introduces DCNN with dilated convolution and Atrous Spatial Pyramid Pooling (ASPP) with dilated convolution in its Encoder. Unlike DeepLab V3, DeepLab V3+ uses the improved Xception as the backbone to replace the ResNet classification network to extract features; and adds a Decoder to implement the Encoder-Decoder structure, so that low-level features can be fused with high-level features, which is conducive to the learning, training and segmentation of boundary feature information. The importance of dilated convolution in DeepLab V3+ is that it can expand the receptive field without changing the size of the feature map, and learn feature information of more scales. The purpose of ASPP is also to extract multi-scale feature information. It achieves feature fusion of information of different scales by connecting multiple dilated convolution results with different dilated convolution rates in series. At the same time, ASPP and Decoder apply deep separable convolution to further improve the overall computational efficiency of the model. Therefore, DeepLab V3+ can simultaneously learn semantic information through high-level features and learn detailed information through low-level features, thereby improving the segmentation accuracy of the model.

[0059] Example 1: Lightweight network structure based on partial convolution and Transformer

[0060] The overall network structure of the model PCTNet described in the present invention is as follows Figure 1 As shown in the figure. The network is divided into the encoder part on the left side of the figure and the decoder part on the right side and the corresponding feature fusion module. In the classic codec structure, the encoder mainly performs feature extraction on the input image, while the decoder restores the texture details of the feature map output by the encoder and gradually restores its channel dimension. For the input desert remote sensing feature map, it is first subjected to a 7×7 convolution operation with a step size of 2 to reduce the feature map size and increase the number of channels, and then batch normalization and feature activation operations are performed. While completing the shallow feature extraction, it ensures that the subsequent partial convolution blocks obtain input feature maps with sufficient number of channels to ensure the quality of its output feature map. Its calculation formula is shown in formula (1).

[0061] F shallow =H ReLU (H BN (Conv 7×7 (F0))) (1)

[0062] Among them, F0 represents the original input feature, F shallow Represents the shallow features of the output, H ReLU (·) represents the ReLU activation operation, H BN (·) indicates BN operation, Conv 7×7 Represents a 7×7 convolution kernel.

[0063] The partial convolution Fast T-shape Block (FTB) used by the encoder of PCTNet is a partial convolution module superimposed with a 1×1 conventional dimensional convolution. The shape of the stacked structure is similar to the English letter T. There is often redundancy of feature information in the process of deep feature extraction, that is, the feature information between different channels is similar. The T-shaped convolution block can effectively reduce the number of parameters while using feature redundancy to achieve an effect similar to that of conventional convolution, and does not lose information interaction between channels. It can not only save costs but also ensure the fineness of features. The feature map will be downsampled after passing through each FTB, and multi-scale high-level low-resolution semantic features will be obtained while expanding the receptive field, so as to learn the detailed semantic information of the feature map. Transformer is introduced in the decoder. The dual-branch transformer block (DBTB) improved based on the self-attention mechanism can further extract the local context information and global context information of the feature map, taking into account the detailed semantic information while extracting the global semantic information. The FTB features and DBTB features of each layer need to be aggregated through a feature fusion module (FFM). In this process, the encoder output is further refined and the features are fused with part of the decoder input. Finally, for each type of dune morphology, in order to achieve high cohesion and low coupling performance, the class feature information is further enhanced through the class attention block (CAB) to improve the recognition effect of dune morphology.

[0064] 1. T-type fast convolution

[0065] In network training, feature maps are used as information carriers throughout the training. However, a large number of studies have shown that many deep feature maps in the network have a certain degree of redundancy, that is, there is a high degree of similarity between some channels in the feature maps. In order to fully and effectively apply these feature information and save computing resources as much as possible, FTB is designed as the feature extraction module in the PCTNet encoder. This module can not only maintain the quality and depth of feature extraction, but also reduce feature redundancy and memory access, greatly saving the computing resources required for network training, and achieving lightweight network while maintaining network performance. Its structure is as follows: Figure 2 shown.

[0066] In PCTNet, a total of four FTBs are used in the encoder part. The first FTB uses a tiled FTB with unchanged number of channels, and the other three are FTBs with 1×1 convolution channel expansion. This is because the input features of the first FTB are feature maps with expanded number of channels after a 7×7 convolution with a step size of 2. In order to avoid feature redundancy caused by excessive span of feature map channel expansion, inputting it into the tiled FTB can further extract the shallow features of the input features without changing the channel dimension, thereby improving the insufficient feature extraction capability of a single 7×7 convolution. The next three FTBs not only extract input features but also increase the dimension of the feature map by expanding the number of channels. Specifically, for each FTB, two T-type convolutions are used for residual concatenation to increase the network depth, improve the feature information extraction capability, and achieve channel number expansion in a progressive manner. The specific implementation of T-type convolution is as follows: Figure 2 As shown in the dotted box, PConv is used to convolve only a portion of the feature channels with low similarity, and then 1×1 convolution is used to not only increase the dimension of the feature map, but also effectively fuse the features of the unconvolved part with the convolution features. T-type convolution can focus more on the central position and channels with higher importance.

[0067] The importance of different positions in the feature map can be evaluated by the Frobenius norm, as shown in formula (2).

[0068]

[0069] Among them, ||F i || represents the Frobenius norm at position i,

[0070] 2. Dual-branch Transformer block

[0071] After the feature map in PCTNet passes through the FTB in the encoder, the low-level high-resolution features such as color, texture, and edge of the dune morphological features are extracted and sent to the decoder in PCTNet to further extract high-level low-resolution features that are conducive to the recognition of dune morphological types. DBTB, as the main network in the decoder, uses the Transformer multi-head self-attention mechanism and the dual-branch form to take into account both global context information and local context information. Its structure is as follows: Figure 3 shown.

[0072] Compared with the traditional Transformer module, we improved its multi-head self-attention module by adding global attention branches and local attention branches, so that its receptive field can have good performance at both global and local scales. DBTB consists of dual-branch attention, multi-layer perceptron and two batch normalizations. It prevents network degradation through two residual connections and enables the network to focus only on the current difference part, further improving its performance.

[0073] In the local branch of DBTB, after the input features are restored to a two-dimensional feature map, feature information is extracted through multiple convolutions of different scales in parallel, so that the local context information of the input features can be better learned; in the global branch, the input features are first converted into a one-dimensional sequence, and then the multi-head self-attention mechanism of the traditional Transformer is used to obtain the global context information of the input features. The feature information obtained by the local branch and the global branch is concatenated (concat) in the channel dimension to output high- and low-frequency fusion features with rich semantic information at different levels. The aggregated high- and low-frequency features are replaced by conventional convolutions through deep separable convolutions, which further reduces the number of parameters and reduces the computational cost, achieving the goal of lightweight network. Deep separable convolution consists of channel-by-channel convolution and point-by-point convolution. In channel-by-channel convolution, the feature channel corresponds to the convolution kernel one by one, and one convolution kernel only performs convolution operations on one feature channel. However, since each feature channel is convolved independently, the feature information of different channels at the same position cannot be effectively utilized and combined. Therefore, point-by-point convolution is used to perform weighted fusion of feature information at the same position of different channels at the depth level, ensuring consistency with conventional convolution and achieving lightweight. The features of the depth-separable convolution layer are batch normalized to improve the network convergence speed and generalization ability, reduce the risk of overfitting, and increase the robustness of the network to ensure the stability of feature data.

[0074] The Transformer self-attention mechanism is implemented by mapping to three linear layers. Its calculation formula is shown in formula (3). During the calculation, matrices Query, Key and Value are required to realize linear transformation.

[0075]

[0076] Among them, Query and Value represent the encoding vector of deep semantic feature mapping, Key represents the encoding vector of shallow semantic feature mapping, and d represents the number of columns of Query and Key matrices, that is, the vector dimension, which are fused and multiplied. Usually, after obtaining Attention(Query,Key,Value), residual connection and batch normalization are performed.

[0077] 3. Feature Fusion Module

[0078] After the above FTB and DBTB extract the low-level high-resolution features and the high-level low-resolution features, it is necessary to effectively fuse the two features. At this time, the low-level high-resolution features and the high-level low-resolution features already have rich feature information. Therefore, with the goal of reducing the number of network parameters, the following is designed: Figure 4 The feature fusion module shown.

[0079] The upstream branch in FFM operates on the low-level high-resolution feature maps in the PCTNet encoder network. The network is lightweighted through a dilated convolution with the smallest possible scale. Dilated convolution, also known as expanded convolution or dilated convolution, replaces the pooling layer and convolution layer, expands the feature receptive field without changing the size of its feature map, and thus eliminates the upsampling and downsampling operations that easily cause feature information loss. The increase in the receptive field will also increase the range of contact with the original features, so it is more conducive to learning global semantic feature information, and it is more suitable to apply dilated convolution to the upstream branch of extracting low-level high-resolution features, while ensuring its original feature information and supplementing the global semantic information of higher layers. On the contrary, it will focus on the local information and detail information of the features, so dilated convolution is also suitable for extracting multi-scale semantic information of features. A dilated convolution with a dilation rate of 2 and a scale of 3×3 is used. The dilation rate is the distance between the values ​​of the convolution kernel when performing the convolution operation. Compared with ordinary convolution, its receptive field changes from 3×3 to 5×5. The downstream branch in FFM operates on the high-level low-resolution feature maps in the PCTNet encoder network. Since the feature information has been extracted through a multi-layer encoding and decoding structure, it has sufficient feature information. At the same time, according to the principle of lightweight, its features are directly Concat-operated without adding other operations. The number of channels of the feature map after Concat processing doubles, and it is necessary to reduce its dimension to restore it to the original number of feature channels. At the same time, the low-level high-resolution feature information and high-level low-resolution feature information of different layers are further integrated to realize the semantic information interaction between hierarchical features.

[0080] 4. Category Attention Block

[0081] For the dune morphology recognition task, which involves multiple dune morphology types, it is inevitable that the data volume of each category is unevenly distributed, causing the network to pay more attention to the category with large data volume and ignore the category with small data volume. Therefore, the category attention mechanism is used to effectively improve the differences between different categories, so that the network can learn the feature information of each channel based on each category, and finally realize the learning of feature information in different categories and accurate recognition of each category.

[0082] The specific network structure of CAB is as follows Figure 5As shown, each category is given a channel number k that can cover the feature information, and the feature channels of each category are different, so they can represent a certain category. Input ∈R h×w×c , assuming there are N categories, it is necessary to convert the number of feature channels to kN through a 1×1 convolution layer to obtain the feature map F′∈R h×w×kN At the same time, it is necessary to perform a random inactivation operation on it, that is, by randomly setting some values ​​to zero, each feature map loses some features and retains some features. The feature map after random inactivation can make all k features in each class N map to learn different discriminant areas. In order to obtain the weight value of each category, firstly perform global maximum pooling on each feature channel to make the feature have higher significance. Then, perform average and maximum calculation operations on the k channels belonging to the same category, while improving the significance of the feature without losing more detail information. Finally, the weight value Weight of each category is obtained, and its calculation method is shown in formula (4).

[0083]

[0084] Among them, GMP(·) represents the global maximum pooling, f i, ' j represents the jth feature of the i-th category in F′.

[0085] At the same time, the F′ obtained above is subjected to average pooling and maximum pooling operations to obtain F″∈R h ×w×N The purpose of doing this is also to obtain significant feature information and retain its detailed feature information. The calculation method is shown in formula (5).

[0086]

[0087] Among them, f i, ' j represents the jth feature of the i-th category in F′, F i ″ represents the feature map of the i-th class.

[0088] The calculation method of a certain position category attention F″′ is shown in formula (6). Input Multiply it with the position category attention F″′ to get the final output feature map F Output ∈R h×w×c , as shown in formula (7).

[0089]

[0090] F Output =F Input F″′ (7)

[0091] 5. Experimental Setup

[0092] All experiments of the proposed PCTNet model are implemented based on the open source deep learning framework PyTorch and Python 3.9. The Ubuntu 20.04 operating system is used during the experimental operation. The server hardware uses an NVIDIA GeForce RTX 4090 GPU with 256GB memory and a graphics card with 24GB video memory, and an Inter Xeon Gold 5220 CPU. The experiment deploys the Adam optimizer to train the experimental model. Compared with other optimizers, Adam has a faster convergence speed and higher accuracy. The specific configuration information of the experimental environment is shown in Table 1.

[0093] Table 1 Experimental environment configuration

[0094]

[0095] During the experiment, the gap between the model's predicted value and the true value needs to be characterized by a loss function. For the design of multi-head attention in the PCTNet network model, its loss function is formed by combining the main loss and the auxiliary loss, and its calculation method is shown in formulas (8)-(11).

[0096]

[0097] L p =L ce +L dice (10)

[0098] L=L p +αL aux (11)

[0099] Among them, L p Indicates the main loss, L dice represents dice loss, L ce represents the cross entropy loss, L aux represents auxiliary loss, L aux =L ce , N and K represent the number of samples and the number of categories respectively, p (n) and They represent the true value and predicted value of the original feature map respectively, and the auxiliary factor α=0.4.

[0100] 6. Experimental results and analysis

[0101] 6.1 Parameter Analysis

[0102] When training the PCTNet model, a series of hyperparameters that affect the training accuracy and recognition performance are involved, including learning rate (lr), batch size (bs) and epochs. Among them, lr can control the learning progress of the model, affecting the loss value and convergence speed of the network. bs refers to the number of data samples used in one training. The memory usage, training speed and stability of the model change with the change of bs. epoch refers to the number of times the entire training data set is traversed once. Appropriate epochs can enable the network to perform its recognition performance well, but too small or too large epochs will cause underfitting or overfitting of training. In order to obtain better network recognition performance, the values ​​of the above hyperparameters are selected by the evaluation index of pixel accuracy (PA).

[0103] When Adam is selected as the optimizer of the PCTNet model, the value of lr is generally set between 0.0001 and 0.001. Therefore, lr is taken at intervals of 0.0002 in this vicinity, and experiments are conducted on the Landsat-8 sand ridgeline dataset and the GF-1WFV sand ridgeline dataset used in this paper. The experimental results are shown in Figure 2. Figure 6 As shown. As the value of lr increases, the PA of PCTNet on both datasets increases first and then decreases. When the selected lr value is too small, the network converges too slowly; and when the selected lr value is too large, its gradient tends to oscillate at the minimum value, so that the network cannot converge. Therefore, selecting an appropriate learning rate can help the network learn and train better and improve the recognition performance of the network. When lr is 0.0008, PCTNet has the best recognition effect on both datasets, and its PA is the maximum value. Therefore, when conducting experiments on two different datasets in the PCTNet model, the lr value is set to 0.0008.

[0104] After obtaining the hyperparameter lr that ensures the maximum PA value, it is necessary to further select the value of the equally important hyperparameter bs. bs, as the amount of data used by a single training network, affects the network's memory usage and training efficiency. A bs that is too small will cause the model training to be time-consuming and inefficient, and difficult to converge, which will lead to underfitting; a bs that is too large will require too much memory space, increase the computational cost, and reduce the generalization ability of the network. Therefore, an appropriate bs can improve training efficiency and help the stability of network convergence. The bs value selection experiment is still carried out on the Landsat-8 dataset and GF-1WFV dataset used by PCTNet. 16, 32, 64 and 128 are selected as the bs values ​​for comparative experiments, and the final bs value is determined by observing the changes in PA. The experimental results are as follows Figure 7 As shown in the figure, the PA values ​​of the two datasets first increase and then decrease with the increase of bs. In the Landsat-8 dataset, when bs is 32, it has the largest PA value; in the GF-1WFV dataset, when bs is 64, it has the largest PA value. Therefore, in the PCTNet model, the hyperparameter bs of the experiment based on the Landsat-8 dataset is set to 32, and the hyperparameter bs of the experiment based on the GF-1WFV dataset is set to 64 to ensure the best recognition performance of the network.

[0105] After obtaining the hyperparameters lr and bs that ensure the maximum PA value, the last step is to select the value of the hyperparameter epoch. Epoch is the number of iterative calculations for the entire data set. Each complete iteration of the training data set will enhance the network's fitting ability. Therefore, too small an epoch will cause insufficient network training and underfitting, while too large an epoch will result in the PA value on the training data set continuing to increase, but the network may have overfitting problems, and its PA value on the test data set will decrease instead of increase. Therefore, determining a suitable epoch value is very important for the recognition accuracy of the network. The experimental values ​​of epoch were selected as 100, 150, 200, and 250, respectively, and comparative experiments were carried out on the Landsat-8 data set and the GF-1WFV data set, and then the epoch value corresponding to the maximum PA value was obtained as the experimental hyperparameter of the PCTNet model. The experimental results are shown in the figure. Figure 8As shown in the figure, the PA values ​​of the two datasets first increase and then decrease with the increase of the epoch value. In the Landsat-8 dataset, when the epoch is 150, it has the largest PA value; in the GF-1WFV dataset, when the epoch is 200, it has the largest PA value. Therefore, in the PCTNet model, the hyperparameter epoch of the experiment based on the Landsat-8 dataset is set to 150, and the hyperparameter epoch of the experiment based on the GF-1WFV dataset is set to 200 to ensure that the network achieves the best recognition accuracy on the test set.

[0106] Based on the above hyperparameters, the PCTNet model we proposed can achieve its optimal performance. The network performance is measured by two types of evaluation indicators. The first type of evaluation indicators is used to evaluate the accuracy of the network, including PA, the harmonic mean of precision and recall (F1), and the mean intersection over union (MIoU). The second type is the model parameter quantity (Params) used to evaluate the network scale. Params can evaluate the memory requirement.

[0107] 6.2 Ablation Experiment

[0108] In order to fully verify the effectiveness of the fast T-block, dual-branch Transformer block, feature fusion module and category attention block in the PCTNet model for dune morphological type recognition, three groups of ablation experiments were conducted on the above modules. The first group of ablation experiments expanded the fast T-block and compared it with the traditional residual block (Bottleneck ResBlock, ResBlock) to verify its lightweight effectiveness. The second group of ablation experiments expanded the dual-branch Transformer block and compared it with the traditional Transformer module to verify the effectiveness of the dual-branch Transformer block for dune morphological type recognition. The third group of ablation experiments added feature fusion modules and category attention blocks to the networks of the above two modules to verify the effectiveness of the proposed modules for dune morphological type recognition of the PCTNet model.

[0109] (1) FTB ablation experiment

[0110] FTB uses partial convolution to maintain a certain recognition accuracy while reducing the number of network parameters. In order to verify the effectiveness of its lightweight, the ablation experiment compares FTB with ResBlock. The experimental results are presented through evaluation indicators such as PA, F1, MIoU and Params, as shown in Table 2. Due to the particularity of the first ResBlock in the ResNet18 network, in order to ensure the universality of the experiment, the Params of this experiment select the second FTB and ResBlock in the network for comparison. In addition, FTB uses less BN than ResBlock, so Params ignores the calculation of the BN layer parameters in the two modules and only compares the number of parameters for the more important convolution module.

[0111] Table 2 Effect of fast T-block on PCTNet recognition performance

[0112]

[0113] Through the experimental results shown in Table 2, it can be observed that FTB only slightly reduces PA, F1 and MIoU compared with ResBlock due to the use of only partial convolution, but obtains a significant reduction in Params. In the Landsat-8 dataset, FTB's PA, F1 and MIoU compared with ResBlock only decreased by 0.07%, 0.09% and 0.05%, respectively. In the GF-1WFV dataset, FTB's PA, F1 and MIoU compared with ResBlock only decreased by 0.06%, 0.06% and 0.05%, respectively. However, in both datasets, FTB's Params decreased by 498.31K compared with ResBlock. Therefore, FTB in the PCTNet model greatly reduces network parameters and improves the computational cost of the network at the cost of only a slight accuracy, which is effective for achieving lightweight networks.

[0114] (2) DBTB ablation experiment

[0115] DBTB is improved on the basis of the original Transformer Block (TB). It replaces the original self-attention mechanism with a dual-branch attention mechanism to obtain local detail information and global semantic information of the feature map, thereby improving the network recognition performance. In order to verify its effectiveness, the ablation experiment compares DBTB with TB. The experimental results are presented through evaluation indicators such as PA, F1, and MIoU. As shown in Table 3, it can be observed that DBTB with a dual-branch attention mechanism has better recognition performance.

[0116] Table 3 Effect of dual-branch Transformer blocks on PCTNet recognition performance

[0117]

[0118] In the Landsat-8 dataset, DBTB improves PA, F1 and MIoU by 0.66%, 0.62% and 0.59% respectively compared with TB. In the GF-1WFV dataset, DBTB improves PA, F1 and MIoU by 0.42%, 0.45% and 0.38% respectively compared with TB. Therefore, DBTB in the PCTNet model improves the accuracy of the network in identifying dune morphological types through the global and local attention mechanisms, which fully verifies the effectiveness of DBTB in the PCTNet model.

[0119] (3) Ablation experiments on each module of PCTNet

[0120] The FTB and DBTB in the PCTNet model have been fully verified for the lightweight model and the accuracy of dune morphological type recognition. Therefore, the network with FTB and DBTB is used as the baseline of the ablation experiment, and the performance of the network with FFM, CAB, and FFM and CAB added at the same time is compared to verify the effectiveness of each module. The experimental results are presented through evaluation indicators such as PA, F1, and MIoU. As shown in Table 4, it can be observed that both FFM and CAB can improve the accuracy of the network in identifying dune morphological types.

[0121] Compared with the baseline, the FFM module in the Landsat-8 dataset has increased PA, F1 and MIoU by 0.29%, 0.44% and 0.26%, respectively, and the PA, F1 and MIoU in the GF-1WFV dataset have increased by 0.35%, 0.34% and 0.32%, respectively. Compared with the baseline, the CAB module in the Landsat-8 dataset has increased PA, F1 and MIoU by 1.80%, 1.96% and 1.62%, respectively, and the PA, F1 and MIoU in the GF-1WFV dataset have increased by 1.19%, 1.19% and 1.06%, respectively. Therefore, it can be fully verified that both FFM and CAB can effectively improve the recognition performance of the network. At the same time, CAB enhances the network's learning ability for categories with small amounts of data, and has a more significant effect than FFM in improving network recognition accuracy.

[0122] Table 4. Impact of related modules on PCTNet recognition performance

[0123]

[0124] At this point, the effectiveness of a single FFM and CAB has been fully verified, and the effectiveness of adding FFM and CAB at the same time needs to be further verified. Compared with the baseline FFM+CAB module, PA, F1, and MIoU in the Landsat-8 dataset increased by 2.08%, 2.21%, and 1.87%, respectively, and PA, F1, and MIoU in the GF-1WFV dataset increased by 1.43%, 1.46%, and 1.29%, respectively. Compared with the FFM+CAB module with only a single FFM added, PA, F1, and MIoU in the Landsat-8 dataset increased by 1.79%, 1.80%, and 1.61%, respectively, and PA, F1, and MIoU in the GF-1WFV dataset increased by 1.08%, 1.12%, and 0.97%, respectively. Compared with the FFM+CAB module with only a single CAB, the PA, F1, and MIoU in the Landsat-8 dataset increased by 0.28%, 0.25%, and 0.19%, respectively, and the PA, F1, and MIoU in the GF-1WFV dataset increased by 0.24%, 0.27%, and 0.23%, respectively. Therefore, it can be concluded that the PCTNet model with both FFM and CAB added has the best accuracy for dune morphological type recognition, which fully verifies the effectiveness of each module in the PCTNet model in improving network performance.

[0125] 6.3 Comparative Experiments of Different Algorithms

[0126] In order to verify the effectiveness of the proposed PCTNet model in the dune morphological type recognition task, it is compared with the existing advanced semantic segmentation models. The experiment compares classic semantic segmentation models, including FCN, SegNet, UNet and DeepLab V3+. Among them, FCN is a milestone in semantic segmentation, enabling images to be classified at the pixel level. SegNet and DeepLab V3+ both achieve excellent performance in natural image semantic segmentation, and UNet is based on medical images, laying a solid foundation for future medical image segmentation research. In addition, based on the characteristics of the PCTNet model, it is compared with some lightweight network models based on Transformer. Among them, Segmenter (Strudel R, Garcia R, Laptev I, et al. Segmenter: Transformer for Semantic Segmentation [C] / / 2021 IEEE / CVF International Conference on Computer Vision (ICCV). 2021: 7242–7252.) applies Transformer to semantic segmentation tasks while keeping the network lightweight; SwiftNet (Orsic M, Segvic S. Efficient semantic segmentation with pyramidal fusion [J]. Pattern Recognition, 2021, 110: 107611.) The upsampling and encoder in the semantic segmentation network are lightweight; SwinUNet (Cao H, Wang YY, Chen J, et al. Swin-Unet: Unet-Like Pure Transformer for a Medical Im- Segmentation[C] / / Computer Vision ECCV 2022 Workshops.Cham:Springer Nature Switzerland,2023.) is a semantic segmentation model for medical image segmentation based on Transformer and combined with UNet structure; SegFormer(Xie EZ,Wang WH,Yu ZD,et al.SegFormer:Simple and Efficient Design for SemanticSegmentation with Transformers[C] / / Advances in Neural Information Processing Systems:Vol 34.New York,NY:CurranAssociates,Inc.,2021:12077–12090.) is a lightweight semantic segmentation model combining Transformer encoder and MLP decoder; UNetForme(Wang LB,Li R,Zhang C,et al.UNetFormer:A UNet-liketransformer for efficient semantic segmentation of remote sensing urban sceneimagery[J].ISPRS Journal of Photogrammetry and Remote Sensing Sensing, 2022, 190: 196–214.) is a network model that uses Transformer as a decoder to achieve semantic segmentation of remote sensing images; AFFormer (Dong B, Wang PC, Wang F. Head-Free Lightweight Semantic Segmentationwith LinearTransformer[C] / / AAAI Conference on Artificial Intelligence. 2023.) combines Transformer and image frequency information to build a lightweight semantic segmentation model.

[0127] The comparative experiments of the above different algorithms were conducted in the same experimental environment and conditions, and were carried out on the Landsat-8 dataset and the GF-1WFV dataset respectively. This study selected a study area containing six types of dune morphology, namely, crescent dunes, grass-shrub dunes, Gobi, ridge-shaped dunes, composite dunes and grid-shaped dunes. The comparative experiments of different algorithms used the intersection over Union (IoU) of the six categories and the overall MIoU as evaluation indicators to verify the effectiveness of our proposed algorithm PCTNet.

[0128] (1) Landsat-8

[0129] Table 5 Recognition results of different algorithms on Landsat-8 dataset

[0130]

[0131] For the Landsat-8 dataset, the recognition results of different algorithms are shown in Table 5. PCTNet achieved the best recognition result on the Landsat-8 dataset, with an MIoU of 82.16%, and achieved the best recognition results on five types of dune morphology: crescent dunes, grass-shrub dunes, Gobi, composite dunes, and grid dunes, with IoUs of 79.40%, 82.98%, 82.03%, 87.49%, and 84.51%, respectively. Compared with the traditional semantic segmentation models FCN, SegNet, UNet and DeepLab V3+, MIoU increased by 5.48%-8.22%, compared with the lightweight semantic segmentation model SwiftNet, MIoU increased by 3.25%, compared with the Transformer-based semantic segmentation model SwinUNet, MIoU increased by 1.34%, compared with the Transformer-based lightweight semantic segmentation models Segmenter, SegFormer, UNetFormer and AFFormer, MIoU increased by 4.07%, 2.14%, 0.81% and 0.43% respectively. Therefore, the excellent performance of PCTNet in dune morphological type recognition can be fully verified. In terms of model scale, the number of parameters of PCTNet is 1.08M. While ensuring the recognition effect, the scale is much smaller than some classic algorithms such as DeepLab V3+ (62.75M), SwiftNet (11.86M) and SwinUNet (41.45M), etc., and it still has advantages over SegFormer (3.81M) and AFFormer (2.34M).

[0132] The visualization results of the recognition performance comparison of different algorithms based on the Landsat-8 dataset are as follows: Fig. 9As shown in the figure, the PCTNet network has the best recognition performance for the dune morphological types of crescent dunes, grass and shrub sand piles, Gobi, composite dunes and grid dunes, and has the highest MIoU. At the same time, it can be observed that the traditional semantic segmentation model can roughly identify the dune morphological types, but there are many noise points in the recognition results. In recent years, advanced Transformer-based network models such as SegFormer, UNetFormer and AFFormer have effectively avoided the problem of excessive noise in the classification results, but there are still certain problems in the boundary delineation of each category, and some category boundaries are relatively blurred. The boundary recognition of PCTNet is obviously clear, and it significantly improves the misclassification of crescent dunes and composite dunes, making the recognition results of the six types of dune morphological types more accurate.

[0133] (2) GF-1WFV

[0134] In order to further verify the effectiveness of the PCTNet network, a comparative experiment of different algorithms was conducted on the GF-1WFV dataset under the same experimental environment, and the recognition results are shown in Table 6. PCTNet achieved the best recognition result on the GF-1WFV dataset, with an MIoU of 82.83%, and achieved the best recognition results on five types of dune morphology, namely, crescent dunes, grass and shrub dunes, Gobi, ridge dunes, and composite dunes, with IoUs of 82.17%, 83.55%, 82.60%, 77.48%, and 87.26%, respectively. Compared with the traditional semantic segmentation models FCN, SegNet, UNet and DeepLab V3+, MIoU is improved by 5.72%-8.19%, compared with the lightweight semantic segmentation model SwiftNet, MIoU is improved by 3.72%, compared with the Transformer-based semantic segmentation model SwinUNet, MIoU is improved by 1.41%, and compared with the Transformer-based lightweight semantic segmentation models Segmenter, SegFormer, UNetFormer and AFFormer, MIoU is improved by 4.50%, 2.31%, 0.86% and 0.62% respectively. Therefore, the excellent performance of our proposed algorithm PCTNet for dune morphological type recognition can be fully verified. In terms of model scale, it is the same as the model scale analysis on the Landsat-8 sand ridge line dataset.

[0135] Table 6 Recognition results of different algorithms on the GF-1WFV dataset

[0136]

[0137]

[0138] Unlike the Landsat-8 remote sensing satellite data with a resolution of 30 meters, the GF-1WFV remote sensing satellite data has a resolution of 16 meters, which allows the network to learn more detailed information. At the same time, the GF-1WFV dataset has a larger amount of data than the Landsat-8 dataset, which allows the network to learn more feature information and thus obtain more detailed recognition results. The visualization results of the recognition performance comparison of different algorithms based on the GF-1WFV dataset are shown in the figure below. Fig.10 As shown in the figure, the PCTNet network has the best recognition performance for crescent dunes, grass and shrub sand piles, Gobi, ridge dunes and composite dunes among the dune morphological types, and has the highest MIoU. Similar to the results of the Landsat-8 sand ridge dataset, advanced Transformer-based network models such as SegFormer, UNetFormer and AFFormer in recent years also have better recognition effects than traditional methods such as FCN, SegNet and UNet, but there are still some problems in delineating the boundaries of each category, and the boundaries of some categories are relatively blurred. However, compared with the results on the Landsat-8 sand ridge dataset, the above problems have been improved to a certain extent because the GF-1WFV sand ridge dataset has richer detailed information. The boundary recognition of PCTNet is obviously clear, and it significantly improves the misclassification of dune morphological types, making the recognition results of the six types of dune morphological types more accurate.

[0139] In fact, based on the experimental performance in the above two data sets, PCTNet, through the combination of partial convolution and dual-branch Transformer, effectively extracts the global and local feature information of the image data while eliminating feature redundancy, achieving excellent recognition performance while maintaining the level of lightweight models. In addition, unlike conventional UNet, the feature fusion module pays more attention to the effective fusion of low-level and high-level feature information through convolution operations, and combines the category attention mechanism at the end of the network to improve the uneven distribution of category data, so that the recognition effect of sand dune morphological types is further improved. From the quantitative analysis results of the model scale, it can be seen that PCTNet has obvious advantages in model lightweight.

[0140] In summary, the present invention provides a dune morphological type recognition model PCTNet, whose encoder adopts partial convolution fast T-block, which effectively reduces the amount of parameters without losing the information interaction between channels, saves costs and ensures the fineness of features. At the same time, after each layer of the network, a downsampling operation is assisted to expand the receptive field while obtaining multi-scale high-level low-resolution semantic features, so as to learn the detailed semantic information of the feature map. Transformer is introduced in the decoder. The dual-branch Transformer improved based on the self-attention mechanism can further extract the local context information and global context information of the feature map, and take into account the detailed semantic information while extracting the global semantic information. The encoder output features and the decoder output features of each layer need to be aggregated through a feature fusion module to further realize feature refinement and fusion; finally, in view of the problem of unbalanced data volume of each type of dune morphological type, the category feature information is further enhanced through the category attention mechanism to improve the recognition effect of dune morphological type.

Claims

1. A lightweight dune morphology recognition model based on partial convolution and Transformer, characterized in that: The dune morphological type recognition model includes: (1) Encoder: performs feature extraction on the input image; The encoder uses a partial convolution fast T-block, a total of four partial convolution fast T-blocks are used, the first partial convolution fast T-block uses a tiled partial convolution fast T-block with unchanged number of channels, and the remaining three are partial convolution fast T-blocks amplified by 1×1 convolution channels; The importance of different positions in the feature map can be evaluated by the Frobenius norm, as shown in formula (2); Among them, ||F i || represents the Frobenius norm at position i, (2) Decoder: Restore the texture details of the feature map output by the encoder and gradually restore its channel dimension; adopt an improved dual-branch Transformer, add a global attention branch and a local attention branch; the Transformer self-attention mechanism is implemented by mapping to three linear layers. Its calculation formula is shown in formula (3). The matrices Query, Key and Value are needed to realize linear transformation during calculation. Among them, Query and Value represent the encoding vectors of deep semantic feature mapping, Key represents the encoding vector of shallow semantic feature mapping, d represents the number of columns of Query and Key matrices, that is, the vector dimension, which are fused and multiplied. Attention(Query,Key,Value) is the result of residual connection and batch normalization. (3) Feature fusion module: For the input desert remote sensing feature map, it is first subjected to a 7×7 convolution operation with a step size of 2 to reduce the feature map size and increase the number of channels, followed by batch normalization and feature activation operations. While completing the shallow feature extraction, it ensures that the subsequent partial convolution blocks obtain input feature maps with sufficient channels to ensure the quality of their output feature maps. The calculation formula is shown in formula (1): F shallow =H ReLU (H BN (Conv. 7×7 (F0))) (1) Among them, F0 represents the original input feature, F shallow Represents the shallow features of the output, H ReLU (·) represents the ReLU activation operation, H BN (·) indicates batch normalization operation, Conv 7×7 Represents a 7×7 convolution kernel; (4) Category attention block: For each category, a number of channels k that can cover the feature information is given, and the feature channels of each category are different, so that each can represent a certain category; for the input feature map F Input ∈R h×w×c , assuming there are N categories, it is necessary to convert the number of feature channels to kN through a 1×1 convolution layer to obtain the feature map F′∈R h×w×kN ; Among them, GMP(·) represents the global maximum pooling, f i, ' j represents the jth feature of the i-th category in F′; The above-obtained F′ is subjected to average pooling and maximum pooling operations at the same time, and F″∈R is obtained h×w×N ; In order to obtain significant feature information and retain its detailed feature information, its calculation method is shown in formula (5); Among them, f i, ' j represents the jth feature of the i-th category in F′, F i ″ represents the feature map of the i-th category; The calculation method of a certain position category attention F″′ is shown in formula (6); The input feature map F Input Multiply it with the position category attention F″′ to get the final output feature map F Output ∈R h×w×c , as shown in formula (7); F Output =F Input F″′ (7)。 2. The dune morphology type recognition model according to claim 1, characterized in that: The partial convolution fast T-block includes a partial convolution module superimposed with a 1×1 conventional dimensionality-enhanced convolution.

3. The dune morphology type recognition model according to claim 1, characterized in that: The input feature of the first partial convolution fast T-block is a feature map whose channels have been expanded after a 7×7 convolution with a step size of 2.

4. The dune morphology type recognition model according to claim 1, characterized in that: The dual-branch Transformer consists of dual-branch attention, a multi-layer perceptron, and two batch normalizations.

5. The dune morphology type recognition model according to claim 1, characterized in that: The loss function of the recognition model is formed by combining the main loss and the auxiliary loss, and its calculation method is shown in formulas (8)-(11): L p =L ce +L dice (10) L=L p +αL aux (11) Among them, L p Indicates the main loss, L dice represents dice loss, L ce represents the cross entropy loss, L aux represents auxiliary loss, L aux =L ce , N and K represent the number of samples and the number of categories respectively, p (n) and They represent the true value and predicted value of the original feature map respectively, and the auxiliary factor α=0.

4.

6. The dune morphology type recognition model according to claim 1, characterized in that: All experiments are implemented based on the open source deep learning framework PyTorch and Python 3.9, and the Ubuntu 20.04 operating system is used during the experimental operation.

7. The dune morphology type recognition model according to claim 1, characterized in that: The server hardware uses NVIDIA GeForce RTX 4090 GPU with 256GB memory and 24GB video memory, and Intel Xeon Gold5220 CPU.

8. The dune morphology type recognition model according to claim 1, characterized in that: The experiment deployed the Adam optimizer to train the experimental model.

Images