A lightweight dune morphological type identification model based on partial convolution and transformer

Abstract

The present application relates to computer vision recognition technology, specifically relates to a kind of light weight dune form type identification model based on partial convolution and Transformer, including encoder, decoder, feature fusion module and class attention block, encoder adopts partial convolution fast T block, effectively reduce the parameter quantity While not losing the information interaction between channels, save cost and ensure the degree of feature refinement;After each layer network, the following sampling operation is assisted;Transformer is introduced in decoder, which can further extract local context information and global context information of feature map, while extracting global semantic information, detail semantic information is also considered;The output features of each layer of encoder and decoder need to be aggregated through a feature fusion module, to realize feature refinement and fusion, and further enhance the class feature information through class attention mechanism, to improve the dune form type identification effect.

Description

Technical Field

[0001] The present invention relates to computer vision recognition technology, and in particular to a lightweight dune morphological type recognition model based on partial convolution and Transformer. Background Art

[0002] Deserts cover approximately one-third of the Earth's land surface and are characterized by extreme environmental conditions, including high temperatures, low humidity, and sparse vegetation. Against the backdrop of global warming and increasing drought, dune activation and desert expansion are accelerating, leading to severe desertification. Sand dunes are dynamic landforms formed by wind-driven sand formations. They exhibit a variety of morphological characteristics, including barchans, grids, and composite dunes. Accurately identifying dune types is crucial for understanding dune movement and migration processes, as well as the impact of climate change on desert landscapes. For example, precise mapping of dune morphology facilitates infrastructure development, disaster assessment, and management in arid regions. Therefore, research on dune morphological type identification has become a crucial topic in contemporary society. It is of great significance for real-time monitoring of desert conditions and effective desertification control, and has an increasingly significant impact on social development and human survival.

[0003] In recent years, automation technology has been used to efficiently identify and classify desert dune morphological types. Deep learning has shown great potential in image recognition and classification, laying a solid foundation for remote sensing data analysis of dune morphological types. In the context of deep learning, the application of Convolutional Neural Network (CNN) and other advanced machine learning algorithms provides unprecedented opportunities for automated and accurate dune morphological identification. Deep learning-based dune morphological type identification algorithms can extract complex features from remote sensing data, enabling high-precision and efficient classification and identification of different desert dune morphological types. In addition, the integration of deep learning with multi-source remote sensing data enables technological innovation in dune morphological identification, improving the accuracy and reliability of classification results. As an important issue in the field of computer vision, the main task of semantic segmentation can be summarized as effectively segmenting images at the pixel level. Currently, semantic segmentation models based on convolutional neural networks have achieved excellent results in this field and still have high exploration value. Applying semantic segmentation to remote sensing images is beneficial to the interpretation and analysis of remote sensing images. Specifically, by classifying each pixel on the remote sensing image, accurate identification and segmentation of ground objects can be achieved, including automatic identification and extraction of different categories of objects such as buildings, roads, and vegetation, which can provide accurate and reliable data support for urban planning, environmental monitoring, and agricultural management. In fact, although the related technology in the field of remote sensing image semantic segmentation has developed relatively maturely and achieved excellent application results, specific research in the desert field is still relatively weak. Deserts need to be automatically extracted and identified due to their topography, lighting, and difficulty in field surveying, among other factors. However, directly applying existing semantic segmentation models to deserts often cannot be well adapted, so it is inevitable that there will be problems such as unsatisfactory identification results.

[0004] Existing research methods for remote sensing image classification and recognition are very rich, but their performance is less than satisfactory when applied to deserts. This is because data sets for dune morphological type recognition are scarce, and network models are prone to problems such as low edge recognition accuracy and class imbalance when applied. To address the difficulties in desert boundary recognition and network computational redundancy in remote sensing image dune morphological type recognition, the present invention proposes a lightweight dune morphological type recognition model (Partial Convolution and Transformer Network, PCTNet) based on partial convolution and transformer for dune morphological type recognition. For the input desert remote sensing feature map, its shallow features are extracted and its number of channels is amplified. There is often redundancy in feature information during deep feature extraction. The encoder of PCTNet uses partial convolution fast T-blocks to effectively reduce the number of parameters without losing information interaction between channels, saving costs and ensuring feature refinement. At the same time, after each layer of the network, a downsampling operation is performed to expand the receptive field while obtaining multi-scale high-level low-resolution semantic features, thereby learning the detailed semantic information of the feature map. The decoder incorporates the Transformer. This improved dual-branch Transformer, based on the self-attention mechanism, further extracts both local and global contextual information from the feature map, taking into account both global semantic information and detailed semantic information. The encoder and decoder output features of each layer are aggregated through a feature fusion module to further refine and fuse the features. Finally, to address the imbalance in data volume within each dune morphology type, a category-based attention mechanism is used to further enhance the category-specific feature information, improving dune morphology recognition. Summary of the Invention

[0005] The primary purpose of the present invention is to provide a lightweight dune morphology recognition model based on partial convolution and Transformer, the dune morphology recognition model comprising:

[0006] (1) Encoder: performs feature extraction on the input image;

[0007] The encoder uses a partial convolution fast T-block, a total of four partial convolution fast T-blocks are used, the first partial convolution fast T-block uses a tiled partial convolution fast T-block with the number of channels unchanged, and the remaining three are partial convolution fast T-blocks amplified by 1×1 convolution channels;

[0008] The importance of different positions in the feature map can be evaluated by the Frobenius norm, as shown in formula (2);

[0009]

[0010] where ||F i || denotes the Frobenius norm at position i,

[0011] (2) Decoder: The texture details of the feature map output by the encoder are restored and the channel dimension is gradually restored. An improved double-branch Transformer is adopted, which adds a global attention branch and a local attention branch. The Transformer self-attention mechanism is realized by mapping to three linear layers, and its calculation formula is shown in formula (3). The matrix Query, Key and Value are used to realize linear transformation during calculation;

[0012]

[0013] where Query and Value represent the encoding vectors of deep semantic feature mapping, Key represents the encoding vector of shallow semantic feature mapping, d represents the column number of Query and Key matrix, i.e. the vector dimension, which is fused and multiplied, and Attention(Query, Key, Value) is the result of residual connection and batch normalization;

[0014] (3) Feature fusion module: For the input desert remote sensing feature map, first make it pass through a 7x7 convolution operation with a step of 2 to reduce the feature map size and increase the channel number, then perform batch normalization and feature activation operation, which completes the shallow feature extraction while ensuring that the subsequent part of the convolution block obtains an input feature map with sufficient channel number, ensuring the quality of the output feature map. Its calculation formula is shown in formula (1):

[0015] F shallow = H ReLU (H BN (Conv 7×7 (F0))) (1)

[0016] where F0 represents the original input feature, F shallow represents the output shallow feature, H ReLU (·) represents the ReLU activation operation, H BN (·) represents the BN operation, Conv 7×7 represents a 7x7 convolution kernel;

[0017] (4) Class attention block: A channel number k that can cover feature information is given to each class, and the feature channels of each class are distinguished, so each class can represent a certain class. For the input feature map F Input ∈R h×w×c, assuming there are N categories, it is necessary to convert the number of feature channels to kN through a 1×1 convolution layer to obtain the feature map F′∈R h×w×kN ;

[0018]

[0019] Among them, GMP(·) represents global maximum pooling, f i, ' j represents the jth feature of the i-th category in F′;

[0020] The F′ obtained above is subjected to average pooling and maximum pooling operations at the same time to obtain F″∈R h×w×N ; In order to obtain significant feature information and retain its detailed feature information, its calculation method is shown in formula (5);

[0021]

[0022] Among them, f i, ' j represents the jth feature of the i-th category in F′, F i ″ represents the feature map of the i-th class;

[0023] The calculation method of a certain position category attention F″′ is shown in formula (6);

[0024] The input feature map F Input Multiply it with the position category attention F″′ to get the final output feature map F Output ∈R h ×w×c , as shown in formula (7);

[0025]

[0026] F Output =F Input F″′ (7).

[0027] Preferably, the partial convolution fast T-block includes a partial convolution module superimposed on a 1×1 conventional dimensionality-raising convolution.

[0028] Preferably, the input feature of the first partial convolution fast T-block is a feature map whose channels have been expanded after a 7×7 convolution with a step size of 2.

[0029] Preferably, the dual-branch Transformer consists of dual-branch attention, a multi-layer perceptron and two batch normalizations.

[0030] Preferably, the loss function of the recognition model is formed by combining the main loss and the auxiliary loss, and its calculation method is shown in formulas (8)-(11).

[0031]

[0032] L p = L ce + L dice (10)

[0033] L = L p + aL aux (11)

[0034] wherein L p represents the main loss, L dice represents the dice loss, L ce represents the cross-entropy loss, L aux represents the auxiliary loss, L aux = L ce , N and K represent the number of samples and the number of classes respectively, p (n) and represent the true value and the predicted value of the original feature map respectively, and the auxiliary factor a = 0.4.

[0035] Preferably, all experiments are implemented based on the open-source deep learning framework PyTorch and Python 3.9, and the Ubuntu 20.04 operating system is used in the experimental operation process.

[0036] Preferably, the server hardware level uses a 256GB memory and a NVIDIA GeForce RTX 4090 GPU with a 24GB video memory, and an Inter Xeon Gold 5220 CPU.

[0037] Preferably, the Adam optimizer is deployed in the experiment to train the experimental model.

[0038] The beneficial effects of the present invention are as follows: the present invention provides a lightweight dune morphology type recognition model PCTNet based on partial convolution and Transformer, wherein the encoder adopts partial convolution fast T-block, which effectively reduces the number of parameters without losing information interaction between channels, saves costs and ensures the fineness of features, and at the same time assists with downsampling operations after each layer of the network to expand the receptive field while obtaining multi-scale high-level low-resolution semantic features, thereby learning the detailed semantic information of the feature map; the Transformer is introduced into the decoder, and the dual-branch Transformer improved based on the self-attention mechanism can further extract the local context information and global context information of the feature map, taking into account the detailed semantic information while extracting the global semantic information; the encoder output features and the decoder output features of each layer need to be aggregated through a feature fusion module to further realize feature refinement and fusion; finally, in response to the problem of unbalanced data volume of each type of dune morphology type, the category feature information is further enhanced through the category attention mechanism to improve the dune morphology type recognition effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 PCTNet overall network structure

[0040] Figure 2 Quick T-block

[0041] Figure 3 Dual-branch Transformer block

[0042] Figure 4 Feature fusion module

[0043] Figure 5 Category Attention Block

[0044] Figure 6 The impact of different lr values ​​on the PA values ​​of the two data sets

[0045] Figure 7 The impact of different bs values ​​on the PA values ​​of the two data sets

[0046] Figure 8 The impact of different epoch values ​​on the PA values ​​of the two datasets

[0047] Figure 9 Recognition results of different algorithms on the Landsat-8 dataset. Note: (a) morphological data (b) sand ridge data (c) ground truth (d) FCN (e) SegNet (f) Unet (g) DeepLab V3+ (h) Segmenter (i) SwiftNet (j) SwinUNet (k) SegFormer (l) UNetFormer (m) AFFormer (n) PCTNet

[0048] Figure 10 Recognition results of different algorithms on the GF-1WFV dataset. Note: (a) morphological data (b) sand ridge data (c) ground truth (d) FCN (e) SegNet (f) Unet (g) DeepLab V3+ (h) Segmenter (i) SwiftNet (j) SwinUNet (k) SegFormer (l) UNetFormer (m) AFFormer (n) PCTNet DETAILED DESCRIPTION

[0049] The protection scope of the present invention will be described in detail below with reference to specific embodiments. It should be noted that the protection scope of the present invention is not limited by the following embodiments.

[0050] In the following embodiments, the encoder is also referred to as Encoder, and the two have the same meaning.

[0051] In the following embodiments, the decoder is also referred to as Decoder, and the two have the same meaning.

[0052] In the following examples, batch normalization is called Batch Normalization, abbreviated as BN.

[0053] Deep Convolutional Neural Networks (DCNNs) are capable of performing tasks such as image classification and recognition. Due to their network depth and parameter sharing, DCNNs are better than CNNs at learning hierarchical features from raw pixel data. Both utilize multiple layers of convolution to extract features, pooling to reduce the dimensionality of the data, and fully connected layers for classification. They have achieved state-of-the-art performance in various computer vision tasks, such as object recognition, image classification, and facial recognition, and have also been successfully applied to fields such as medical imaging, self-driving cars, and natural language processing.

[0054] Transformer is a network model based on the self-attention mechanism proposed in recent years. It continues the encoder-decoder network structure and can learn attention weights from multiple dimensions, resulting in better network performance.

[0055] The fully convolutional neural network (FCN) consists entirely of stacked convolutional and pooling layers, with no fully connected layers. In the field of semantic segmentation, FCN is the first to implement an end-to-end convolutional neural network. Unlike image classification and object detection, semantic segmentation requires predicting the category of every e pixel in an image. In common classification networks, the fully connected layer and softmax function at the end of the network, which are used to obtain category probabilities, convert this probabilistic information into one-dimensional information, lacking spatial information and only predicting the category of the entire image. FCN removes the fully connected layers from convolutional neural networks, opening up a new direction for pixel-by-pixel classification of semantic segmentation images. Without fully connected layers, FCNs have no requirements for the input image size; it can be of any size. When restoring the output image of the last convolutional layer to the same size as the original image, FCN uses deconvolution to achieve upsampling, preserving the spatial information of the original image. This facilitates prediction of each pixel in the upsampled image, thereby achieving pixel-by-pixel classification. To further optimize the fusion of deep and shallow information, FCN introduces a skip architecture that combines input features from different network layers, thereby learning more information and facilitating accurate segmentation. However, FCN segmentation is relatively rough in processing details, and the upsampling results suffer from blurry and smoothing issues. Furthermore, simply replacing fully connected layers with convolutional layers to preserve spatial information still results in independent pixel classification, ignoring spatial relationships and lacking globality.

[0056] SegNet also classifies images pixel by pixel. Unlike FCN, its encoder and decoder structures are different. SegNet's encoder and decoder have symmetrical structures. The encoder uses the VGG16 network without the fully connected layers to extract features, while the decoder uses deconvolution and upsampling to better represent the classification feature information and restore the image to its original size. Finally, softmax is used to obtain the maximum value of different categories to obtain the segmentation result. The removal of the fully connected layers in SegNet reduces the number of parameters and makes the network more lightweight. Furthermore, the encoder of SegNet stores the pooling index during max pooling, eliminating the learning step in the decoder during upsampling. The index can be used directly for nonlinear upsampling, ensuring the integrity of high-frequency information details. However, when the target feature is a low-resolution feature map, the low-frequency edge information features are easily overlooked and lost when depooling using the max pooling index.

[0057] UNet is also an Encoder-Decoder structure, and its overall structure is U-shaped, such as Figures 2-9As shown in the figure, the encoder still uses the convolutional and pooling layers of a classic convolutional neural network. Each downsampling process doubles the number of channels. The decoder uses 2×2 deconvolution for upsampling, halving the number of channels. Feature fusion is achieved through skip connections with feature maps at the same level in the encoder, resulting in a feature map with the same number of channels as the original. UNet uses skip connections between the encoder and decoder feature maps at each level to ensure that the decoder learns more feature information at each level, preventing feature information from being lost in the encoder. Compared to the pixel-based feature fusion of FCN, UNet's channel-wise feature fusion preserves more low-frequency edge details and exhibits multi-scale prediction capabilities, resulting in more accurate segmentation results. Due to its advantages such as small sample size friendliness, fast training speed, and excellent fuzzy boundary recognition, UNet has played an important role in medical image segmentation and has been widely used in various fields.

[0058] The DeepLab series of deep learning semantic segmentation models has undergone continuous updates and iterations, progressing through DeepLab V1, DeepLab V2, and DeepLab V3, and now reaching DeepLab V3+. To learn multi-scale features, DeepLab V3+ introduces a DCNN with dilated convolutions and an Atrous Spatial Pyramid Pooling (ASPP) module in its encoder. Unlike DeepLab V3, DeepLab V3+ uses a modified Xception backbone instead of a ResNet classification network for feature extraction. It also adds a decoder, implementing an encoder-decoder architecture that fuses low-level features with high-level features, facilitating the learning, training, and segmentation of boundary features. The key feature of the dilated convolution in DeepLab V3+ lies in its ability to expand the receptive field while maintaining the size of the feature map, allowing it to learn features at multiple scales. ASPP also aims to extract multi-scale features by concatenating the results of multiple dilated convolutions with different dilated convolution rates to achieve feature fusion at different scales. At the same time, ASPP and the Decoder apply depthwise separable convolution to further improve the overall computational efficiency of the model. As a result, DeepLab V3+ can simultaneously learn semantic information through high-level features and detailed information through low-level features, thereby improving the model's segmentation accuracy.

[0059] Example 1: Lightweight Network Structure Based on Partial Convolution and Transformer

[0060] The overall network structure of the model PCTNet described in the present invention is as follows Figure 1 As shown in Figure 1, the network is divided into the encoder part on the left and the decoder part on the right, as well as the corresponding feature fusion module. In the classic codec structure, the encoder mainly performs feature extraction on the input image, while the decoder restores the texture details of the feature map output by the encoder and gradually restores its channel dimension. For the input desert remote sensing feature map, it first undergoes a 7×7 convolution operation with a stride of 2 to reduce the feature map size and increase the number of channels. Then, batch normalization and feature activation operations are performed. While completing the shallow feature extraction, it ensures that the subsequent partial convolution blocks obtain input feature maps with sufficient channels to ensure the quality of their output feature maps. The calculation formula is shown in Formula (1).

[0061] F shallow =H ReLU (H BN (Conv 7×7 (F0))) (1)

[0062] Among them, F0 represents the original input feature, F shallow Represents the shallow features of the output, H ReLU (·) represents the ReLU activation operation, H BN (·) indicates BN operation, Conv 7×7 Represents a 7×7 convolution kernel.

[0063] The PCTNet encoder uses a partial convolution Fast T-shape Block (FTB), which consists of a partial convolution module stacked with a 1×1 conventional up-convolution. The stacked structure resembles the letter T. Deep feature extraction often involves feature redundancy, where the feature information between different channels is similar. The T-shape convolution block effectively reduces the number of parameters while leveraging feature redundancy to achieve similar results to conventional convolution without losing inter-channel information interaction. This not only saves cost but also ensures feature refinement. The feature map is downsampled after each FTB, expanding the receptive field while acquiring multi-scale, high-level, low-resolution semantic features, thereby learning the detailed semantic information of the feature map. The decoder incorporates a Transformer. The improved Dual-Branch Transformer Block (DBTB), based on the self-attention mechanism, further extracts both local and global contextual information from the feature map, capturing both global and detailed semantic information. The FTB and DBTB features of each layer are aggregated through a Feature Fusion Module (FFM). During this process, the encoder output is further refined and fused with part of the decoder input. Finally, for each dune morphology type, to achieve high cohesion and low coupling, the class feature information is further enhanced through the Class Attention Block (CAB) mechanism, improving dune morphology recognition.

[0064] 1. T-type fast convolution

[0065] In network training, feature maps are used as information carriers throughout the training. However, a large number of studies have shown that many deep feature maps in the network have a certain degree of redundancy, that is, there is a high degree of similarity between some channels in the feature maps. In order to fully and effectively apply these feature information and save computing resources as much as possible, FTB is designed as the feature extraction module in the PCTNet encoder. This module not only maintains the quality and depth of feature extraction, but also reduces feature redundancy and memory access, greatly saving the computing resources required for network training, and achieving lightweight network while maintaining network performance. Its structure is as follows: Figure 2 shown.

[0066] In PCTNet, a total of four FTBs are used in the encoder part. The first FTB uses a tiled FTB with the number of channels unchanged, and the other three are FTBs that have undergone 1×1 convolution channel amplification. This is because the input features of the first FTB are feature maps with amplified channels after a 7×7 convolution with a step size of 2. In order to avoid feature redundancy caused by excessive feature map channel amplification, inputting it into the tiled FTB can further extract the shallow features of the input features without changing the channel dimension, thereby improving the insufficient feature extraction capability of a single 7×7 convolution. The next three FTBs not only extract the input features but also increase the dimension of the feature map by amplifying the number of channels. Specifically, for each FTB, two T-type convolutions are used for residual concatenation to increase the network depth, improve the feature information extraction capability, and achieve channel amplification in a progressive manner. The specific implementation of T-type convolution is as follows: Figure 2 As shown in the dashed box, PConv convolves only a subset of feature channels with low similarity. Then, 1×1 convolution increases the dimension of the feature map while effectively fusing the unconvolved features with the convolved features. T-shaped convolution can focus more on central and more important channels.

[0067] The importance of different positions in the feature map can be evaluated by the Frobenius norm, as shown in formula (2).

[0068]

[0069] Among them, ||F i || represents the Frobenius norm at position i,

[0070] 2. Dual-branch Transformer Block

[0071] After the feature map in PCTNet passes through the FTB in the encoder, the low-level high-resolution features of the dune morphological features such as color, texture, and edges are extracted and sent to the decoder in PCTNet to further extract high-level low-resolution features that are conducive to the recognition of dune morphological types. DBTB, as the main network in the decoder, uses the Transformer multi-head self-attention mechanism and the dual-branch form to take into account both global and local context information. Its structure is as follows: Figure 3 shown.

[0072] Compared to the traditional Transformer module, we have improved its multi-head self-attention module by adding global and local attention branches, enabling it to have a good receptive field at both global and local scales. DBTB consists of a dual-branch attention, a multi-layer perceptron, and two batch normalizations. It uses two residual connections to prevent network degradation and enable the network to focus only on the current difference, further improving its performance.

[0073] In the local branch of DBTB, after the input features are restored to two-dimensional feature maps, feature information is extracted through multiple parallel convolutions at different scales, effectively learning the local context of the input features. In the global branch, the input features are first converted to a one-dimensional sequence, and then the multi-head self-attention mechanism of the traditional Transformer is applied to obtain the global context of the input features. The feature information obtained by the local and global branches is concatenated (concatenated) along the channel dimension to output a fusion of high- and low-frequency features with rich semantic information at different levels. The aggregated high- and low-frequency features are then subjected to depthwise separable convolution instead of conventional convolution, further reducing the number of parameters and computational cost, thereby achieving network lightweighting. Depthwise separable convolution consists of channel-by-channel convolution and point-by-point convolution. In channel-by-channel convolution, feature channels correspond to convolution kernels one-to-one, and each convolution kernel performs convolution operations on only one feature channel. However, since each feature channel is convolved independently, feature information from different channels at the same location cannot be effectively utilized and combined. Therefore, point-by-point convolution is used to perform weighted fusion of feature information at the same location across different channels at the depth level, ensuring consistent results with conventional convolution while achieving lightweight performance. Batch normalization of features passed through the depthwise separable convolution layer improves network convergence speed and generalization, reduces the risk of overfitting, and increases network robustness, ensuring feature data stability.

[0074] The Transformer self-attention mechanism is implemented by mapping to three linear layers. Its calculation formula is shown in formula (3). The matrices Query, Key and Value are required to implement linear transformation during calculation.

[0075]

[0076] Query and Value represent the encoding vectors of the deep semantic feature map, Key represents the encoding vector of the shallow semantic feature map, and d represents the number of columns in the Query and Key matrices, i.e., the vector dimension. These are then fused and multiplied. Typically, after obtaining Attention(Query, Key, Value), residual connections and batch normalization are performed.

[0077] 3. Feature Fusion Module

[0078] After the low-level high-resolution features and high-level low-resolution features are extracted by the FTB and the DBTB, the two kinds of features need to be effectively fused. At this time, the low-level high-resolution features and the high-level low-resolution features already have rich feature information, and therefore, a feature fusion module as shown in Figure 4 is designed with the goal of less network parameter quantity.

[0079] The uplink branch in the FFM operates on the low-level high-resolution feature maps in the PCTNet encoder network. Network lightweight is achieved through a scale as small as possible dilated convolution. Dilated convolution is also called dilated convolution or expanded convolution, which replaces the pooling layer and the convolution layer to expand the feature receptive field without changing the size of the feature map, thereby saving the up-sampling and down-sampling operations that are prone to cause feature information loss. The increase of the receptive field will also increase the range of contact with the original features, and therefore, it is more conducive to learning global semantic feature information, and it is more suitable for applying dilated convolution to the uplink branch for extracting low-level high-resolution features, while ensuring the original feature information and supplementing higher-level global semantic information. On the contrary, it will focus on the local information and detailed information of the features, and therefore, dilated convolution is also suitable for extracting multi-scale semantic information of the features. A dilated convolution with an expansion rate of 2 and a scale of 3x3 is used. The expansion rate is the distance between the values of the convolution kernel during the convolution operation. Compared with the ordinary convolution, the receptive field changes from 3x3 to 5x5. The downlink branch in the FFM operates on the high-level low-resolution feature maps in the PCTNet encoder network. Since the feature information has been extracted through multiple layers of coding and decoding structure, it has sufficient feature information, and at the same time, according to the principle of lightweight, the feature is directly subjected to Concat operation without adding other operations. The channel number of the feature map after Concat processing is doubled, and it needs to be reduced to the original feature channel number, and at the same time, the low-level high-resolution feature information and the high-level low-resolution feature information of different layers are further fused to realize the semantic information interaction between the hierarchical features.

[0080] 4. Class attention block

[0081] For the sand dune morphology type identification task, there are multiple sand dune morphology types, and it is inevitable that the data quantity of each class is unevenly distributed, which causes the network to pay more attention to the class with large data quantity and ignore the class with small data quantity. Therefore, the class attention mechanism is used to effectively improve the difference between different classes, so that the network learns the feature information of each channel based on each class, and finally realizes the learning of feature information in different classes and the accurate identification of each class.

[0082] The specific network structure of the CAB is as shown in Figure 5As shown, each category is given a channel number k that can cover the feature information, and the feature channels of each category are different, so they can represent a certain category. Input ∈R h×w×c , assuming there are N categories, it is necessary to convert the number of feature channels to kN through a 1×1 convolution layer to obtain the feature map F′∈R h×w×kN At the same time, it is necessary to perform a random inactivation operation on it, that is, by randomly setting some values ​​to zero, each feature map loses some features and retains some features. After the random inactivation of the feature map, all k features in each class N can be mapped to learn different discriminant areas. In order to obtain the weight value of each category, first perform global maximum pooling on each feature channel to make the feature more significant. Then, perform the average and maximum calculation operations on the k channels belonging to the same category, while improving the feature significance without losing more detailed information. Finally, the weight value Weight of each category is obtained. Its calculation method is shown in formula (4).

[0083]

[0084] Among them, GMP(·) represents global maximum pooling, f i, ' j represents the jth feature of the i-th category in F′.

[0085] At the same time, the F′ obtained above is subjected to average pooling and maximum pooling operations to obtain F″∈R h ×w×N The purpose of doing this is also to obtain significant feature information and retain its detailed feature information. The calculation method is shown in formula (5).

[0086]

[0087] Among them, f i, ' j represents the jth feature of the i-th category in F′, F i ″ represents the feature map of the i-th class.

[0088] The calculation method of a certain position category attention F″′ is shown in formula (6). Input feature map F Input Multiply it with the position category attention F″′ to get the final output feature map F Output ∈R h×w×c , as shown in formula (7).

[0089]

[0090] F Output =F Input F″′ (7)

[0091] 5. Experimental Setup

[0092] All experiments using our proposed PCTNet model were conducted using the open-source deep learning framework PyTorch and Python 3.9. The experimental environment was Ubuntu 20.04. The server hardware used was an NVIDIA GeForce RTX 4090 GPU with 256GB of RAM and 24GB of video memory, and an Intel Xeon Gold 5220 CPU. The Adam optimizer was deployed to train the experimental model. Compared to other optimizers, Adam has faster convergence and higher accuracy. Table 1 shows the detailed configuration of the experimental environment.

[0093] Table 1 Experimental environment configuration

[0094]

[0095] During the experiment, the gap between the model's predicted value and the true value requires a loss function to characterize it. For the multi-head attention design in the PCTNet network model, its loss function is formed by combining the main loss and auxiliary loss, and its calculation method is shown in formulas (8)-(11).

[0096]

[0097] L p =L ce +L dice (10)

[0098] L=L p +αL aux (11)

[0099] Among them, L p Indicates the main loss, L dice represents dice loss, L ce represents the cross entropy loss, L aux represents auxiliary loss, L aux =L ce , N and K represent the number of samples and the number of categories respectively, p (n) and They represent the true value and predicted value of the original feature map respectively, and the auxiliary factor α = 0.4.

[0100] 6. Experimental Results and Analysis

[0101] 6.1 Parameter Analysis

[0102] Training the PCTNet model involves a series of hyperparameters that influence training accuracy and recognition performance, including the learning rate (LR), batch size (BS), and the number of epochs. LR controls the model's learning progress, affecting the network's loss and convergence rate. BS refers to the number of data samples used in a single training session. Changing BS affects the model's memory usage, training speed, and stability. Epochs represents the number of times the entire training dataset is fully traversed. An appropriate epoch number allows the network to effectively perform its recognition performance, but an excessively small or large epoch number can lead to underfitting or overfitting. To achieve optimal network recognition performance, the values ​​of these hyperparameters are selected using the pixel accuracy (PA) evaluation metric.

[0103] When Adam is selected as the optimizer for the PCTNet model, the value of lr is generally set between 0.0001 and 0.001. Therefore, lr is taken at intervals of 0.0002 in this vicinity, and experiments are conducted on the Landsat-8 sand ridge line dataset and the GF-1WFV sand ridge line dataset used in this paper. The experimental results are shown in the figure. Figure 6 As shown in the figure. As the value of lr increases, the PA of PCTNet on both datasets shows a trend of first increasing and then decreasing. When the selected lr value is too small, the network convergence speed will be too slow; when the selected lr value is too large, its gradient will easily oscillate at the minimum value, so that the network cannot converge. Therefore, selecting an appropriate learning rate can help the network learn and train better and improve the network's recognition performance. When lr is 0.0008, PCTNet has the best recognition effect on both datasets, and its PA is the maximum. Therefore, when conducting experiments on two different datasets in the PCTNet model, the lr value is set to 0.0008.

[0104] After obtaining the hyperparameter lr that ensures the maximum PA value, it is necessary to further select the value of the equally important hyperparameter bs. bs, as the amount of data used for a single training network, affects the network's memory usage and training efficiency. A bs that is too small will cause the model training to be time-consuming and inefficient, and difficult to converge, which will lead to underfitting; a bs that is too large will require too much memory space, increase the computational cost, and reduce the generalization ability of the network. Therefore, an appropriate bs can improve training efficiency and contribute to the stability of network convergence. The bs value selection experiment is still carried out on the Landsat-8 dataset and GF-1WFV dataset used by PCTNet. 16, 32, 64 and 128 are selected as bs values ​​for comparative experiments, and the final bs value is determined by observing the changes in PA. The experimental results are as follows Figure 7 As shown in the figure, the PA values ​​of both datasets first increase and then decrease as bs increases. In the Landsat-8 dataset, the maximum PA value is achieved when bs is 32, while in the GF-1WFV dataset, the maximum PA value is achieved when bs is 64. Therefore, in the PCTNet model, the hyperparameter bs is set to 32 for experiments based on the Landsat-8 dataset and to 64 for experiments based on the GF-1WFV dataset to ensure optimal recognition performance.

[0105] After obtaining the hyperparameters lr and bs that ensure the maximum PA value, the last step is to select the value of the hyperparameter epoch. Epoch is the number of iterative calculations of the entire dataset. Each complete iteration of the training dataset will enhance the network's fitting ability. Therefore, too small an epoch will cause insufficient network training and underfitting, while too large an epoch will show that the PA value on the training dataset continues to increase, but the network may have overfitting problems, and its PA value on the test dataset will decrease instead of increase. Therefore, determining a suitable epoch value is very important for the recognition accuracy of the network. The experimental values ​​of epoch were selected as 100, 150, 200 and 250 in turn for comparative experiments on the Landsat-8 dataset and GF-1WFV dataset, and then the epoch value corresponding to the maximum PA value was obtained as the experimental hyperparameter of the PCTNet model. The experimental results are shown in the figure. Figure 8As shown, the PA values ​​for both datasets first increase and then decrease with increasing epoch values. In the Landsat-8 dataset, the maximum PA value is achieved at epoch 150, while in the GF-1WFV dataset, the maximum PA value is achieved at epoch 200. Therefore, in the PCTNet model, the hyperparameter epoch is set to 150 for experiments based on the Landsat-8 dataset and to 200 for experiments based on the GF-1WFV dataset to ensure that the network achieves optimal recognition accuracy on the test set.

[0106] Based on the aforementioned hyperparameters, our proposed PCTNet model achieves optimal performance. Network performance is measured using two types of metrics. The first type of metrics assesses network accuracy, including PA, the harmonic mean of precision and recall (F1), and mean intersection over union (MIoU). The second type is the number of model parameters (params), which are used to assess network scale and can be used to estimate memory requirements.

[0107] 6.2 Ablation Experiment

[0108] In order to fully verify the effectiveness of the fast T-block, dual-branch Transformer block, feature fusion module, and category attention block in the PCTNet model for dune morphological type recognition, three sets of ablation experiments were conducted on the above modules. The first set of ablation experiments expanded the fast T-block and compared it with the traditional residual block (Bottleneck ResBlock, ResBlock) to verify its lightweight effectiveness. The second set of ablation experiments expanded the dual-branch Transformer block and compared it with the traditional Transformer module to verify the effectiveness of the dual-branch Transformer block for dune morphological type recognition. The third set of ablation experiments added the feature fusion module and category attention block to the network of the above two modules respectively to verify the effectiveness of the proposed modules for dune morphological type recognition in the PCTNet model.

[0109] (1) FTB ablation experiment

[0110] FTB uses partial convolutions to maintain a certain level of recognition accuracy while reducing the network's parameter count. To verify its effectiveness in lightweighting, ablation experiments compared FTB with ResBlock. The experimental results are presented using metrics such as PA, F1, MIoU, and Params, as shown in Table 2. Due to the uniqueness of the first ResBlock in the ResNet-18 network, to ensure universality, the Params in this experiment compare the second FTB and ResBlock in the network. Furthermore, FTB uses less batch normalization (BN) than ResBlock, so Params ignores the calculation of the BN layer parameters in both modules and only compares the parameters of the more important convolutional modules.

[0111] Table 2 Impact of fast T-blocks on PCTNet recognition performance

[0112]

[0113] The experimental results shown in Table 2 show that FTB, due to its use of only partial convolutions, only slightly reduces PA, F1, and MIoU compared to ResBlock, but significantly reduces Params. In the Landsat-8 dataset, FTB's PA, F1, and MIoU compared to ResBlock only decreased by 0.07%, 0.09%, and 0.05%, respectively. In the GF-1WFV dataset, FTB's PA, F1, and MIoU compared to ResBlock only decreased by 0.06%, 0.06%, and 0.05%, respectively. However, in both datasets, FTB's Params decreased by 498.31K compared to ResBlock. Therefore, FTB in the PCTNet model significantly reduces network parameters and improves the network's computational cost at the slight cost of accuracy, making it effective in achieving network lightweighting.

[0114] (2) DBTB ablation experiment

[0115] DBTB improves upon the original Transformer Block (TB) by replacing the original self-attention mechanism with a dual-branch attention mechanism to capture both local detail and global semantic information from feature maps, improving network recognition performance. To validate its effectiveness, ablation experiments compared DBTB with TB. The experimental results are presented using metrics such as performance, F1, and MIoU. As shown in Table 3, DBTB, which uses the dual-branch attention mechanism, achieves superior recognition performance.

[0116] Table 3. Impact of dual-branch Transformer blocks on PCTNet recognition performance

[0117]

[0118] In the Landsat-8 dataset, DBTB improved PA, F1, and MIoU by 0.66%, 0.62%, and 0.59%, respectively, compared to TB. In the GF-1WFV dataset, DBTB improved PA, F1, and MIoU by 0.42%, 0.45%, and 0.38%, respectively. Therefore, DBTB in the PCTNet model improves the network's accuracy in dune morphological classification through its global and local attention mechanisms, fully validating the effectiveness of DBTB in the PCTNet model.

[0119] (3) Ablation experiments on each module of PCTNet

[0120] The FTB and DBTB components of the PCTNet model have been fully validated for their lightweight nature and accurate dune morphology recognition. Therefore, the network with FTB and DBTB was used as the baseline for ablation experiments. The performance of the network with FTB and DBTB was compared with that of a network with FFM, CAB, and both FFM and CAB, verifying the effectiveness of each module. The experimental results are presented using metrics such as PA, F1, and MIoU. As shown in Table 4, both FFM and CAB improve the network's accuracy in dune morphology recognition.

[0121] Compared to the baseline, the FFM module achieved improvements of 0.29%, 0.44%, and 0.26% in PA, F1, and MIoU on the Landsat-8 dataset, and 0.35%, 0.34%, and 0.32% on the GF-1WFV dataset. Compared to the baseline, the CAB module achieved improvements of 1.80%, 1.96%, and 1.62% in PA, F1, and MIoU on the Landsat-8 dataset, and 1.19%, 1.19%, and 1.06% on the GF-1WFV dataset. This demonstrates that both FFM and CAB effectively improve network recognition performance. Furthermore, CAB enhances the network's learning ability for categories with limited data, significantly improving network recognition accuracy compared to FFM.

[0122] Table 4 Impact of related modules on PCTNet recognition performance

[0123]

[0124] While the effectiveness of individual FFMs and CABs has been fully validated, further verification is needed to verify the effectiveness of adding both together. Compared to the baseline, the FFM+CAB module improved PA, F1, and MIoU by 2.08%, 2.21%, and 1.87% in the Landsat-8 dataset, and by 1.43%, 1.46%, and 1.29% in the GF-1WFV dataset. Compared to the FFM+CAB module with only a single FFM, PA, F1, and MIoU improved by 1.79%, 1.80%, and 1.61% in the Landsat-8 dataset, and by 1.08%, 1.12%, and 0.97% in the GF-1WFV dataset. Compared to the FFM+CAB module, which only adds a single CAB, the PA, F1, and MIoU in the Landsat-8 dataset increased by 0.28%, 0.25%, and 0.19%, respectively, and in the GF-1WFV dataset, PA, F1, and MIoU increased by 0.24%, 0.27%, and 0.23%, respectively. Therefore, it can be concluded that the PCTNet model with both FFM and CAB achieves the best accuracy for dune morphological type recognition, fully verifying the effectiveness of each module in improving network performance.

[0125] 6.3 Comparative Experiments of Different Algorithms

[0126] To verify the effectiveness of our proposed PCTNet model in dune morphological type recognition, we compared it with existing state-of-the-art semantic segmentation models. The experiment compared classic semantic segmentation models, including FCN, SegNet, UNet, and DeepLab V3+. FCN was a milestone in semantic segmentation, enabling pixel-level image classification. SegNet and DeepLab V3+ both achieved excellent performance in semantic segmentation of natural images. UNet, developed based on medical images, laid a solid foundation for future research in medical image segmentation. Furthermore, based on the characteristics of the PCTNet model, we compared it with several lightweight Transformer-based network models. Among them, Segmenter (Strudel R, Garcia R, Laptev I, et al. Segmenter: Transformer for Semantic Segmentation [C] / / 2021 IEEE / CVF International Conference on Computer Vision (ICCV). 2021: 7242–7252.) applies Transformer to semantic segmentation tasks while keeping the network lightweight; SwiftNet (Orsic M, Segvic S. Efficient semantic segmentation with pyramidal fusion [J]. Pattern Recognition, 2021, 110: 107611.) performs lightweight processing on upsampling and encoders in the semantic segmentation network; SwinUNet (Cao H, Wang YY, Chen J, et al. Swin-Unet: Unet-Like Pure Transformer for a Medical Im- Segmentation[C] / / Computer Vision ECCV 2022 Workshops.Cham:Springer Nature Switzerland,2023.) is a semantic segmentation model for medical image segmentation based on Transformer and combined with UNet structure; SegFormer(Xie EZ,Wang WH,Yu ZD,et al.SegFormer:Simple and Efficient Design for SemanticSegmentation with Transformers[C] / / Advances in Neural Information Processing Systems:Vol 34.New York,NY:CurranAssociates,Inc.,2021:12077–12090.) is a lightweight semantic segmentation model combining Transformer encoder and MLP decoder; UNetForme(Wang LB,Li R,Zhang C,et al.UNetFormer:A UNet-liketransformer for efficient semantic segmentation of remote sensing urban sceneimagery[J].ISPRS Journal of Photogrammetry and Remote Sensing, 2022, 190: 196–214.) is a network model that uses Transformer as a decoder to achieve semantic segmentation of remote sensing images; AFFormer (Dong B, Wang PC, Wang F. Head-Free Lightweight Semantic Segmentation with Linear Transformer [C] / / AAAI Conference on Artificial Intelligence. 2023.) combines Transformer and image frequency information to build a lightweight semantic segmentation model.

[0127] The comparative experiments of the above different algorithms were carried out under the same experimental environment and conditions, and were carried out on Landsat-8 data set and GF-1 WFV data set respectively. The research area selected in the present study contains six types of dune morphological types, namely crescent dunes, shrub dune, gobi, beam nest dunes, compound dunes and lattice dunes. The Intersection over Union (IoU) of the six categories and the overall MIoU are used as evaluation indexes to verify the effectiveness of the algorithm PCTNet in the comparative experiment of different algorithms.

[0128] (1) Landsat-8

[0129] Table 5 Recognition results of different algorithms on Landsat-8 data set

[0130]

[0131] For Landsat-8 data set, the recognition results of different algorithms are shown in Table 5. PCTNet achieved the best recognition result on Landsat-8 data set, with MIoU of 82.16%, and achieved the best recognition result on five types of dune morphological types, namely crescent dunes, shrub dune, gobi, compound dunes and lattice dunes, with IoU of 79.40%, 82.98%, 82.03%, 87.49% and 84.51% respectively. Compared with the traditional semantic segmentation model FCN, SegNet, UNet and DeepLab V3+, the MIoU is improved by 5.48%-8.22%, compared with the lightweight semantic segmentation model SwiftNet, the MIoU is improved by 3.25%, compared with the semantic segmentation model based on Transformer SwinUNet, the MIoU is improved by 1.34%, compared with the lightweight semantic segmentation model based on Transformer Segmenter, SegFormer, UNetFormer and AFFormer, the MIoU is improved by 4.07%, 2.14%, 0.81% and 0.43% respectively. Therefore, the excellent performance of PCTNet for dune morphological type recognition can be fully verified. In terms of model size, the parameter amount of PCTNet is 1.08M, which is much smaller than some classic algorithms such as DeepLab V3+(62.75M), SwiftNet(11.86M) and SwinUNet(41.45M), and still has an advantage compared with SegFormer(3.81M) and AFFormer(2.34M).

[0132] The visualization results of the recognition performance comparison of different algorithms based on Landsat-8 data set are as follows Figure 9As shown in the figure, the PCTNet network achieves the best recognition performance for barchan dunes, grass-shrub dunes, gobi dunes, composite dunes, and lattice dunes, achieving the highest MIoU. Furthermore, it can be observed that traditional semantic segmentation models can roughly identify dune morphological types, but the recognition results are subject to significant noise. In recent years, advanced Transformer-based network models such as SegFormer, UNetFormer, and AFFormer have effectively avoided the problem of excessive noise in classification results, but still have some issues with delineating the boundaries of each category, with some category boundaries being relatively blurred. PCTNet's boundary recognition is significantly clearer and significantly improves the misclassification of barchan dunes and composite dunes, resulting in more accurate recognition of the six dune morphological types.

[0133] (2)GF-1WFV

[0134] To further verify the effectiveness of the PCTNet network, comparative experiments with different algorithms were conducted on the GF-1WFV dataset under the same experimental environment. The recognition results are shown in Table 6. PCTNet achieved the best recognition results on the GF-1WFV dataset, with an MIoU of 82.83%. It also achieved the best recognition results for five types of dune morphologies: crescent dunes, grass-shrub dunes, gobi dunes, ridge-shaped dunes, and composite dunes, with IoUs of 82.17%, 83.55%, 82.60%, 77.48%, and 87.26%, respectively. Compared with traditional semantic segmentation models FCN, SegNet, UNet, and DeepLab V3+, the MIoU improved by 5.72%-8.19%. Compared with the lightweight semantic segmentation model SwiftNet, the MIoU improved by 3.72%. Compared with the Transformer-based semantic segmentation model SwinUNet, the MIoU improved by 1.41%. Compared with the lightweight Transformer-based semantic segmentation models Segmenter, SegFormer, UNetFormer, and AFFormer, the MIoU improved by 4.50%, 2.31%, 0.86%, and 0.62%, respectively. This fully demonstrates the excellent performance of our proposed algorithm PCTNet for sand dune morphological type recognition. The model scale is the same as that analyzed on the Landsat-8 sand ridge dataset.

[0135] Table 6 Recognition results of different algorithms on the GF-1WFV dataset

[0136]

[0137]

[0138] Unlike the 30-meter resolution of Landsat-8 remote sensing satellite data, the GF-1WFV remote sensing satellite data has a resolution of 16 meters, which gives the network the advantage of learning more detailed information. At the same time, the GF-1WFV dataset has a larger data volume than Landsat-8, which enables the network to learn feature information more fully, thereby obtaining more refined recognition results. The visualization results of the recognition performance comparison of different algorithms based on the GF-1WFV dataset are shown in the figure below. Figure 10 As shown, the PCTNet network achieves the best recognition performance for barchan dunes, grass-shrub dunes, gobi dunes, ridge-shaped dunes, and composite dunes, achieving the highest MIoU. Similar to the results on the Landsat-8 sand ridge dataset, recent advanced Transformer-based network models such as SegFormer, UNetFormer, and AFFormer also outperform traditional methods such as FCN, SegNet, and UNet. However, they still face challenges in delineating the boundaries of each class, with some classes appearing blurred. However, compared to the results on the Landsat-8 sand ridge dataset, the GF-1WFV sand ridge dataset offers a significant improvement in these issues due to its richer detail. PCTNet's boundary recognition is significantly clearer and significantly improves misclassification of dune morphological types, resulting in more accurate recognition of the six dune morphological classes.

[0139] In fact, based on the experimental performance in the two datasets mentioned above, PCTNet, through the combination of partial convolution and a two-branch Transformer, effectively extracts global and local feature information from image data while eliminating feature redundancy, achieving excellent recognition performance while maintaining a lightweight model. Furthermore, unlike conventional UNet, the feature fusion module emphasizes the effective fusion of low-level and high-level feature information through convolution operations. At the end of the network, it incorporates a category attention mechanism to improve the uneven distribution of category data, further enhancing the recognition of dune morphological types. Quantitative analysis of model size shows that PCTNet has a clear advantage in model lightweightness.

[0140] In summary, the present invention provides a dune morphology type recognition model PCTNet, whose encoder adopts partial convolution fast T-block, which effectively reduces the number of parameters without losing information interaction between channels, saves costs and ensures the fineness of features. At the same time, it is assisted by downsampling operation after each layer of the network to expand the receptive field while obtaining multi-scale high-level low-resolution semantic features, thereby learning the detailed semantic information of the feature map. Transformer is introduced in the decoder. The dual-branch Transformer improved based on the self-attention mechanism can further extract the local context information and global context information of the feature map, taking into account the detailed semantic information while extracting the global semantic information. The encoder output features and decoder output features of each layer need to be aggregated through a feature fusion module to further realize feature refinement and fusion; finally, in order to solve the problem of unbalanced data volume of each type of dune morphology type, the category feature information is further enhanced through the category attention mechanism to improve the dune morphology type recognition effect.

Claims

1. A lightweight dune morphology recognition model based on partial convolution and Transformer, characterized by: The dune morphological type identification model includes: (1) Encoder: performs feature extraction on the input image; The encoder uses a partial convolution fast T-block, a total of four partial convolution fast T-blocks are used, the first partial convolution fast T-block uses a tiled partial convolution fast T-block with the number of channels unchanged, and the remaining three are partial convolution fast T-blocks amplified by 1×1 convolution channels; The importance of different positions in the feature map can be evaluated by the Frobenius norm, as shown in formula (2); Among them, ||F i || represents the Frobenius norm at position i, (2) Decoder: Restore the texture details of the feature map output by the encoder and gradually restore its channel dimension; use the improved dual-branch Transformer, add the global attention branch and the local attention branch; the Transformer self-attention mechanism is implemented by mapping to three linear layers, and its calculation formula is shown in formula (3). The matrices Query, Key and Value are required to achieve linear transformation during calculation; Among them, Query and Value represent the encoding vectors of deep semantic feature mapping, Key represents the encoding vector of shallow semantic feature mapping, d represents the number of columns of Query and Key matrices, that is, the vector dimension, which are fused and multiplied. Attention(Query,Key,Value) is the result of residual connection and batch normalization; (3) Feature fusion module: For the input desert remote sensing feature map, it is first subjected to a 7×7 convolution operation with a step size of 2 to reduce the feature map size and increase the number of channels, and then batch normalization and feature activation operations are performed. While completing the shallow feature extraction, it ensures that the subsequent partial convolution blocks obtain input feature maps with sufficient channels to ensure the quality of their output feature maps. Its calculation formula is shown in formula (1): F shallow =H ReLU (H BN (Conv. 7×7 (F0))) (1) Among them, F0 represents the original input feature, F shallow Represents the shallow features of the output, H ReLU (·) represents the ReLU activation operation, H BN (·) represents the batch normalization operation, Conv 7×7 Represents a 7×7 convolution kernel; (4) Category attention block: For each category, a number of channels k that can cover the feature information is given, and the feature channels of each category are different, so that each can represent a certain category; for the input feature map F Input ∈R h×w×c , assuming there are N categories, it is necessary to convert the number of feature channels to kN through a 1×1 convolution layer to obtain the feature map F′∈R h×w×kN ; Among them, GMP(·) represents global maximum pooling, f i, ' j represents the jth feature of the i-th category in F′; The F′ obtained above is subjected to average pooling and maximum pooling operations at the same time to obtain F″∈R h×w×N ; In order to obtain significant feature information and retain its detailed feature information, its calculation method is shown in formula (5); Among them, f i, ' j represents the jth feature of the i-th category in F′, F i ″ represents the feature map of the i-th class; The calculation method of a certain position category attention F″′ is shown in formula (6); The input feature map F Input Multiply it with the position category attention F″′ to get the final output feature map F Output ∈R h×w×c , as shown in formula (7); F Output =F Input F″′ (7)。 2. The dune morphology type recognition model according to claim 1, characterized in that: The partial convolution fast T-block includes a partial convolution module superimposed on a 1×1 conventional dimensionality-raising convolution.

3. The dune morphology identification model according to claim 1, wherein: The input feature of the first partial convolution fast T-block is a feature map whose channels have been expanded after a 7×7 convolution with a step size of 2.

4. The dune morphology identification model according to claim 1, wherein: The dual-branch Transformer consists of dual-branch attention, multi-layer perceptron and two batch normalizations.

5. The dune morphology identification model according to claim 1, wherein: The loss function of the recognition model is formed by combining the main loss and the auxiliary loss, and its calculation method is shown in formulas (8)-(11): L p =L ce +L dice (10) L=L p +αL aux (11) Among them, L p Indicates the main loss, L dice represents dice loss, L ce represents the cross entropy loss, L aux represents auxiliary loss, L aux =L ce , N and K represent the number of samples and the number of categories respectively, p (n) and They represent the true value and predicted value of the original feature map respectively, and the auxiliary factor α = 0.

4.

6. The dune morphology identification model according to claim 1, wherein: All experiments were implemented based on the open source deep learning framework PyTorch and Python 3.9, and the Ubuntu 20.04 operating system was used during the experimental operation.

7. The dune morphology identification model according to claim 1, wherein: The server hardware uses NVIDIA GeForce RTX 4090 GPU with 256GB of memory and 24GB of video memory, and Intel Xeon Gold 5220 CPU.

8. The dune morphology identification model according to claim 1, wherein: The experiment deployed the Adam optimizer to train the experimental model.

Images