Remote sensing semantic segmentation method and system based on improved TransUNet algorithm

Abstract

The invention provides a remote sensing semantic segmentation method and system based on an improved TransUNet algorithm, and relates to the technical field of deep learning and computer vision. The method comprises the following steps: acquiring a high-resolution remote sensing image data set, and performing multi-scale data enhancement; an improved TransUNet neural network model comprising an encoder, a Transform encoder, a context enhancement module CAEM, a decoder, a multi-scale feature fusion module MFFM and a segmentation head is constructed; wherein the CAEM intensifies a global context through multi-granularity pooling and cross-branch attention, and the MFFM fuses multi-scale features at a jump joint and introduces lightweight channel attention; and inputting the processed remote sensing image data set into a pre-trained improved TransUNet neural network model to execute semantic segmentation, and outputting a semantic segmentation result. The method effectively improves the small target recognition rate and the edge segmentation accuracy, and is suitable for fine ground feature interpretation of a complex remote sensing scene.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and computer vision technology, and more specifically, to a remote sensing semantic segmentation method and system based on an improved TransUNet algorithm. Background Technology

[0002] Remote sensing semantic segmentation, a core task in intelligent interpretation of remote sensing images, aims to classify high-resolution remote sensing images acquired by satellite, aerial, or UAV platforms pixel-by-pixel to accurately identify land cover types, such as buildings, roads, water bodies, and vegetation. This technology has wide applications in urban planning, land use monitoring, ecological environment assessment, and disaster emergency response. However, remote sensing images themselves are characterized by high spatial resolution, complex scene structures, significant scale differences, and variable lighting conditions, resulting in ground features exhibiting high spectral similarity, irregular spatial distribution, and blurred edges. Especially in the presence of shadows, clouds, or drastic changes in lighting, ground feature boundary information is easily weakened, further exacerbating the difficulty of semantic segmentation.

[0003] In recent years, deep learning methods have made significant progress in remote sensing semantic segmentation tasks. Among them, TransUNet, as a hybrid model that integrates convolutional neural networks (CNN) and Transformer architecture, extracts local spatial features through CNN and models long-range dependencies with the help of Transformer, thus taking into account both local details and global contextual information to a certain extent. However, in practical applications for remote sensing images, TransUNet still has several key limitations: First, the model's ability to perceive multi-scale ground objects is insufficient, making it difficult to effectively fuse feature representations from different levels, leading to small-scale objects being easily overlooked or misclassified; second, the modeling of contextual information is insufficient, especially in complex background areas, where semantic confusion between categories is quite serious; third, the decoder has limited ability to recover high-resolution spatial details during feature upsampling and reconstruction, resulting in blurred ground object boundaries and unclear category transition areas in the segmentation results, seriously affecting segmentation accuracy and practicality.

[0004] In view of the above, this application is hereby submitted. Summary of the Invention

[0005] This invention aims to provide a remote sensing semantic segmentation method and system based on an improved TransUNet algorithm, in order to solve the technical problems of existing remote sensing image semantic segmentation methods, such as insufficient contextual information modeling, inadequate multi-scale feature fusion, and low accuracy of ground object edge segmentation.

[0006] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution: A remote sensing semantic segmentation method based on an improved TransUNet algorithm includes: Acquire high-resolution remote sensing image datasets and perform multi-scale data augmentation processing; The processed remote sensing image dataset is input into a pre-trained improved TransUNet neural network model to perform semantic segmentation and output semantic segmentation results; wherein, the pre-trained improved TransUNet neural network model includes an encoder, a Transformer encoder, a context-aware enhancement CAEM module, a decoder, a multi-scale feature fusion (MFFM) module, and a segmentation head; The encoder is used to perform progressive downsampling convolution on the input remote sensing image to extract multi-scale spatial feature maps, and to fuse features from different layers until the last downsampled layer features are converted into a feature sequence by linear projection and used as the input of the Transformer encoder. The Transformer encoder is used to capture cross-regional contextual information of the input feature sequence through multi-head self-attention, thereby obtaining global semantically enhanced features. The context-aware enhancement CAEM module is located at the output of the Transformer encoder and is used to perform hierarchical pooling and adaptive weighting adjustment on the global semantic enhancement features to enhance the features of key regions and output fine features. The decoder is used to perform upsampling on the fine features and is connected to the encoder via the multi-scale feature fusion (MFFM) module to perform fusion of the output features of the decoder and encoder at the same scale, thereby obtaining multi-scale fused features. The segmentation head is used to perform convolutional mapping on the multi-scale fused features and output semantic segmentation results.

[0007] Preferably, the multi-scale data augmentation process includes random cropping, horizontal flipping, vertical flipping, brightness perturbation, or random rotation.

[0008] Preferably, the encoder uses a pre-trained ResNet-50 backbone network, which consists of multiple convolutional layers and residual structures; During the encoding process, a pre-trained visual backbone network is first loaded as the basis for feature extraction; Then, multi-level downsampling and convolution operations are performed on the input remote sensing image to extract spatial features layer by layer. The non-linear interaction between features is enhanced by element-wise multiplication to generate a multi-scale spatial feature map containing global semantics and local details. A linear projection is performed on the last downsampled feature map to map it to a feature space of uniform dimension, generating a feature sequence that can be received by the Transformer encoder.

[0009] Preferably, the Transformer encoder comprises 12 encoder layers; The feature sequence generated by the encoder is input into the Transformer encoder; Each layer of the Transformer encoder models long-range dependencies through a multi-head self-attention mechanism, captures cross-regional contextual information, completes global feature reconstruction, and outputs enhanced global semantic features.

[0010] Preferably, the processing procedure of the context-aware enhanced CAEM module is as follows: Global soft pooling and multi-scale adaptive average pooling are performed in parallel on the global semantic enhancement features. After each channel is compressed by 1×1 convolution, the original spatial size is restored by bilinear interpolation upsampling to obtain multi-scale intermediate features. The multi-scale intermediate features are concatenated with the global semantic enhancement features to form an intermediate feature tensor; The intermediate feature tensor is convolutional, batch normalized, and ReLU activated. It is then divided into two sub-branches along the channel dimension. The two sub-branches are added element-wise along the cardinality dimension to obtain the intermediate statistics. Then, an adaptive weighted fusion reconstruction is performed through a cross-branch attention mechanism to output a context-enhanced feature map. The attention weights of the adaptive weighted fusion are generated by performing global average pooling on the intermediate statistics in terms of both height and width, concatenating them, and then performing two 1×1 convolutions and the RadixSoftmax function. After convolutional compression of channels, the context-enhanced feature map outputs fine features with unchanged spatial resolution.

[0011] Preferably, the decoder adopts a symmetrical upsampling structure, and introduces a multi-scale feature fusion (MFFM) module at each skip connection. The processing procedure is as follows: Upsampling is performed layer by layer on the fine features; The multi-scale feature fusion (MFFM) module of each layer receives the upsampling results of the previous layer and the skip connection features of the corresponding encoder layer, and extracts multi-scale context features through multi-branch convolution. The multi-scale context features are sequentially processed through soft pooling, convolution, ReLU activation, convolution, Sigmoid activation, and normalization to output context attention weights. The multi-scale context features are weighted and fused with the context attention weights to obtain multi-scale context weighted features; The multi-scale context-weighted features are then subjected to adaptive average pooling, convolution, and sigmoid activation to obtain fusion attention weights, which are then weighted and fused with the multi-scale context-weighted features to suppress redundant information and output multi-scale fusion features.

[0012] Preferably, in the segmentation head, the number of channels of the final multi-scale fusion feature is mapped to the number of land cover categories through 1×1 convolution to obtain the final feature map; The final feature map is subjected to a Softmax operation to generate a class probability map for each pixel, thus obtaining the semantic segmentation result.

[0013] Preferably, the method further includes optimizing the training of the improved TransUNet neural network model using a total loss function; wherein, the total loss function... The expression is: ; ; ; ; ; ; ; in, , , The weighting factor for the loss term; Cross-entropy loss; The Dice loss is used to address the class imbalance problem. For joint losses; The total number of samples; For the i-th sample in the true category The predicted probability; Let i be the true class label of the i-th sample; True cases are the number of samples that the model predicts to be positive and that are actually positive. False negatives are the number of samples that the model predicts to be negative but are actually positive. False positives are the number of samples that the model predicts to be positive but are actually negative. For boundary loss; For label-aware cross-entropy loss with prior knowledge; The weighted Dice loss is used to assign weights related to prior probabilities to different categories; For all land cover categories; For category Edge pixel precision; For category Edge pixel recall rate; For the sample In category The predicted score; This is a temperature parameter used to control the degree of influence of prior knowledge; , Categories , The prior probability; , Samples The true category label and the predicted category label.

[0014] Preferably, the method further includes employing a multinomial decay learning rate adjustment strategy during model training, wherein the learning rate gradually decreases in a power function form with the number of training iterations, and the calculation formula is as follows: ; in, The current learning rate, The initial learning rate, This represents the current iteration number. This represents the maximum number of iterations. is the power exponent of the polynomial decay.

[0015] This invention also provides a remote sensing semantic segmentation system based on an improved TransUNet algorithm, comprising: The data acquisition unit is used to acquire high-resolution remote sensing image datasets and perform multi-scale data augmentation processing. The semantic segmentation result unit is used to input the processed remote sensing image dataset into the pre-trained TransUNet neural network model to perform semantic segmentation and output the semantic segmentation result.

[0016] The present invention also provides a remote sensing semantic segmentation device based on an improved TransUNet algorithm, including a processor and a memory. The memory stores a computer program that can be executed by the processor to implement the remote sensing semantic segmentation method based on the improved TransUNet algorithm as described above.

[0017] The present invention also provides a computer-readable storage medium storing computer-readable instructions, which, when executed by a processor of the device on which the computer-readable storage medium resides, implement a remote sensing semantic segmentation method based on an improved TransUNet algorithm as described above.

[0018] In summary, compared with the prior art, the present invention has the following beneficial effects: This invention addresses the shortcomings of traditional UNet, which relies solely on convolutional layers and struggles to model long-range dependencies. It introduces a multi-layer Transformer encoder, employing a multi-head self-attention mechanism to capture cross-regional semantic associations of features across a large spatial area. This solves the problem of accurate local segmentation but fragmented global semantics in large-scale remote sensing images. Furthermore, this invention combines the encoder's multi-scale convolutional residual structure, preserving spatial details of shallow features while achieving global integration of high-level semantics through the Transformer, thus balancing macroscopic category distribution with microscopic edge accuracy.

[0019] The CAEM module introduced in this invention enhances the contextual representation of global semantic features by using multi-scale pooling and cross-branch attention weighting, while maintaining the feature resolution. For complex regions with blurred boundaries and mixed categories in remote sensing images, the CAEM module can improve the semantic consistency of pixels within the region, reduce misclassification of different categories within the same region, and the spatial coherence of the segmentation results is significantly better than that of the original TransUNet.

[0020] The MFFM module deployed layer by layer in the decoder of this invention can simultaneously capture contextual information of ground features at different scales through multi-branch convolution and soft pooling operations, adapting to mixed distribution scenes of large and small targets in remote sensing images. MFFM's lightweight channel attention mechanism can adaptively suppress redundant background features and prioritize enhancing the feature responses of ground features, improving the recall rate of small-scale ground features by 10%-15% compared to traditional methods, making it particularly suitable for fine segmentation of high-resolution remote sensing images.

[0021] This invention not only overcomes the long-range dependency of traditional convolutional models, but also solves the problems of detail loss and computational redundancy in pure Transformer models. It achieves comprehensive improvement in core indicators such as semantic consistency, multi-scale recognition, and edge accuracy, and is more suitable for the refined ground cover classification needs of high-resolution remote sensing images. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating a remote sensing semantic segmentation method based on an improved TransUNet algorithm, as provided in Example 1.

[0024] Figure 2 This is a framework diagram of a remote sensing semantic segmentation method based on an improved TransUNet algorithm, provided in Example 1.

[0025] Figure 3 This is a framework diagram of the context-aware enhanced CAEM module provided in Example 1.

[0026] Figure 4 The diagram shows the framework of the Multi-Scale Feature Fusion (MFFM) module provided in Example 1.

[0027] Figure 5 This is a schematic diagram of a remote sensing semantic segmentation device based on an improved TransUNet algorithm, provided in Embodiment 2.

[0028] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0030] Example 1 Embodiment 1 of the present invention provides a remote sensing semantic segmentation method based on an improved TransUNet algorithm, which can be implemented by a remote sensing semantic segmentation device based on an improved TransUNet algorithm (hereinafter referred to as a semantic segmentation device), specifically, executed by one or more processors within the semantic segmentation device.

[0031] In this embodiment, the semantic segmentation device may be an electronic device equipped with a processor, which carries a computer program for the remote sensing semantic segmentation method based on the improved TransUNet algorithm and the computer program can be executed, such as a computer, smartphone, smart tablet, workstation, etc., which are not limited here.

[0032] like Figures 1-2 As shown, a remote sensing semantic segmentation method based on an improved TransUNet algorithm includes steps S1 to S2.

[0033] S1: Acquire a high-resolution remote sensing image dataset and perform multi-scale data augmentation processing.

[0034] First, high-resolution remote sensing image datasets are acquired, including the publicly available Potsdam and Vaihingen remote sensing semantic segmentation datasets, with image resolutions ranging from 0.05 to 0.3 meters per pixel, and containing six types of land cover labels: buildings, roads, water bodies, vegetation, vehicles, and background.

[0035] Specifically, to improve the model's generalization ability, multi-scale data augmentation operations are performed on the original image, including random cropping to 512×512 pixels, horizontal flipping, vertical flipping, brightness perturbation (±20%), and random rotation (±15°), generating diverse training samples. The augmented image is then fed as input into the improved RS-UNet model, which is the improved TransUNet network constructed in this invention.

[0036] S2, input the processed remote sensing image dataset into the pre-trained improved TransUNet neural network model to perform semantic segmentation and output the semantic segmentation result; wherein, the pre-trained improved TransUNet neural network model includes an encoder, a Transformer encoder, a context-aware enhancement CAEM module, a decoder, a multi-scale feature fusion (MFFM) module, and a segmentation head.

[0037] The encoder employs a pre-trained ResNet-50 backbone network, composed of multiple convolutional layers and residual structures. During encoding, the pre-trained visual backbone network is first loaded as the basis for feature extraction; then, multi-layer downsampling (e.g., 4 layers, stride 2) and convolution operations are performed on the input remote sensing image to extract spatial features layer by layer. Element-wise multiplication is used to enhance the non-linear interaction between features (e.g., multiplying shallow high-resolution features with deep semantic features to strengthen the coupling between local texture and global semantics) until the last downsampled feature layer, generating a multi-scale spatial feature map containing both global semantics and local details. For example, the output resolution is the original... Figure 1 Multi-scale feature maps of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 have 256, 512, 1024, and 2048 channels respectively, with each level retaining rich spatial details and semantic information. To adapt to the subsequent Transformer structure, a 1×1 convolutional linear projection is performed on the last layer feature map to map it to a feature space of a uniform dimension (e.g., dimension 768), generating a feature sequence that can be received by the Transformer encoder.

[0038] Subsequently, the flattened feature sequence is input into a Transformer encoder consisting of 12 stacked multi-head self-attention structures. Each layer contains a multi-head self-attention mechanism (12 heads by default) and a feedforward neural network, which captures cross-regional contextual dependencies through global modeling, completes global feature reconstruction, and outputs enhanced global semantic features.

[0039] For example, when there are buildings obscured by shadows in an image, the Transformer can infer their complete outline through semantic associations between distant pixels. After 12 layers of interaction, the output global feature sequence is reshaped back into a two-dimensional spatial form, restoring it to a feature map of H / 32×W / 32×768.

[0040] Next, the global semantic enhancement feature is input into the CAEM module for contextual enhancement.

[0041] The context-aware enhancement CAEM module is located at the output of the Transformer encoder and is used to perform hierarchical pooling and adaptive weighting adjustment on the global semantic enhancement features to enhance the features of key regions and output refined features.

[0042] like Figure 3 As shown, the processing procedure of the context-aware enhanced CAEM module is as follows: Global soft pooling and multi-scale adaptive average pooling are performed in parallel on the global semantic enhancement features, such as three adaptive average pooling branches with kernel sizes of 2×2, 3×3, and 6×6. The input global semantic enhancement features are replicated into four parallel processing paths: the global soft pooling branch aggregates the channel responses at each spatial location exponentially, preserving the gradients of highly activated regions; the three adaptive average pooling branches perform mean downsampling on local regions respectively; then, the features of each path are compressed through 1×1 convolutions, and then upsampled by bilinear interpolation to restore the original spatial size, resulting in multi-scale intermediate features.

[0043] The multi-scale intermediate features are concatenated with the global semantic enhancement features along the channel dimension to form an intermediate feature tensor. This intermediate feature tensor is then subjected to convolution, BN (batch normalization), and ReLU activation, and subsequently divided into two sub-branches along the channel dimension. The two sub-branches are then element-wise summed along the Radix dimension to obtain intermediate statistics. These statistics are then weighted and fused using a cross-branch attention mechanism to reconstruct the context-enhanced feature map. Specifically, the weighted fusion process is as follows: Global average pooling is performed on features H and W, concatenated, and then passed through two 1×1 convolutional layers (FC+BN+ReLU) to generate attention scores, followed by RadixSoftmax (e.g.) Figure 4The weights W are obtained by normalizing with r-softmax, and then weighted and fused with the intermediate statistics to output a context-enhanced feature map.

[0044] After the context-enhanced feature map is further compressed through a 3×3 convolution, the output features retain spatial resolution, effectively enhancing the semantic consistency of complex terrain features (such as dense building clusters).

[0045] The decoding stage then begins. The decoder employs a symmetrical upsampling structure, introducing a multi-scale feature fusion (MFFM) module at each skip connection. The decoder performs upsampling on the fine features and, through the MFFM module, skips to the encoder to fuse the output features of the decoder and encoder at the same scale, obtaining multi-scale fused features.

[0046] For example, a four-level upsampling structure is used, with each level having an upsampling factor of 2, to perform upsampling on the fine features layer by layer. The multi-scale feature fusion (MFFM) module at each layer receives the upsampling result from the previous layer and the skip connection features of the corresponding encoder layer, extracting multi-scale contextual features through multi-branch convolution. For example... Figure 4 As shown, the multi-scale contextual features are sequentially processed through soft pooling, convolution, ReLU activation, convolution again, Sigmoid activation, and softmax normalization to obtain contextual attention weights. These weights are then weighted and fused with the multi-scale contextual features to obtain multi-scale context-weighted features. Next, the multi-scale context-weighted features are input into a lightweight channel attention substructure to obtain fused attention weights.

[0047] In the lightweight channel attention substructure, channel statistics are obtained through adaptive average pooling, fed into a one-dimensional adaptive convolution (kernel size = 3) and activated by Sigmoid to generate fused attention weights, which are then weighted and fused with the multi-scale context-weighted features to enhance key channels (such as road edge response channels), suppress redundant background channels, and output multi-scale fused features.

[0048] This process is repeated at each level of the decoder, restoring spatial details layer by layer, especially in complex boundary areas such as building rooftops and road intersections, significantly improving edge clarity.

[0049] Between each level of the decoder, skip connections concatenate the same-scale feature maps output by the encoder with the decoded features, ensuring effective fusion of high-level semantic information and shallow spatial details. For example, 1 / 4 resolution features output by the encoder contain fine textures (such as roof tiles and lane lines), which are directly passed to the top layer of the decoder through skip connections, avoiding the loss of details caused by multiple upsampling.

[0050] Finally, the high-resolution feature map output by the decoder is fed into the segmentation head. The segmentation head is used to perform convolutional mapping on the multi-scale fused features and output semantic segmentation results. In the segmentation head, the number of channels of the final multi-scale fused features is mapped to the number of land cover categories through 1×1 convolution to obtain the final feature map; the final feature map is then subjected to a Softmax operation to generate a category probability map for each pixel, thus obtaining the semantic segmentation results.

[0051] During the model training phase, the improved TransUNet neural network model is optimized using a total loss function. The total loss function... The weighted sum of cross-entropy loss, Dice loss, and boundary loss is expressed as: ; ; ; ; ; ; ; in, , , The weighting factor for the loss term; Cross-entropy loss; The Dice loss is used to address the class imbalance problem. For joint losses; The total number of samples; For the i-th sample in the true category The predicted probability; Let i be the true class label of the i-th sample; True cases are the number of samples that the model predicts to be positive and that are actually positive. False negatives are the number of samples that the model predicts to be negative but are actually positive. False positives are the number of samples that the model predicts to be positive but are actually negative. For boundary loss; For label-aware cross-entropy loss with prior knowledge; The weighted Dice loss is used to assign weights related to prior probabilities to different categories; For all land cover categories; For category Edge pixel precision; For category Edge pixel recall rate; For the sample In category The predicted score; This is a temperature parameter used to control the degree of influence of prior knowledge; , Categories , The prior probability; , Samples The true category label and the predicted category label.

[0052] During model training, a multinomial decay learning rate adjustment strategy is adopted. The learning rate gradually decreases in the form of a power function with the number of training iterations. The calculation formula is as follows: ; in, The current learning rate, The initial learning rate, This represents the current iteration number. This represents the maximum number of iterations. is the power exponent of the polynomial decay, such as 0.9.

[0053] During the inference phase, the remote sensing image to be segmented is input into the trained model, and a single forward propagation outputs pixel-by-pixel classification results without post-processing. For example, in urban planning scenarios, the model can accurately segment building outlines (even when partially obscured by trees), distinguish between asphalt roads and dirt roads, and identify small bodies of water (such as ponds), achieving an average intersection-union ratio (mIoU) of 89.7% (Potsdam remote sensing dataset) and 87.3% (Vaihingen remote sensing dataset), with a boundary F1-score improvement of over 5%, verifying the high accuracy and robustness of this invention in complex remote sensing scenarios.

[0054] In summary, compared with the prior art, the present invention has the following beneficial effects: This invention, through the aforementioned technical solutions, enables the remote sensing image semantic segmentation model to significantly enhance its responsiveness to multi-scale ground features while maintaining the global modeling advantages of TransUNet: the CAEM module strengthens the global context representation without changing the spatial resolution through multi-granularity pooling and cross-branch attention mechanisms, effectively alleviating the problem of small targets being ignored due to insufficient receptive field; the MFFM module fuses feature responses with different hole rates and convolutional kernel sizes during the decoding stage, and introduces SoftPool weighted aggregation and lightweight channel attention to achieve adaptive weighted fusion of shallow details and high-level semantics, improving the reconstruction accuracy of edge regions; the composite loss function explicitly constrains the prediction error of boundary pixels, enabling the model to focus on the geometric consistency of ground feature outlines while optimizing the overall region classification.

[0055] Example 2 like Figure 5 As shown, the second embodiment of the present invention also provides a remote sensing semantic segmentation system based on an improved TransUNet algorithm, comprising: The data acquisition unit is used to acquire high-resolution remote sensing image datasets and perform multi-scale data augmentation processing. The semantic segmentation result unit is used to input the processed remote sensing image dataset into the pre-trained TransUNet neural network model to perform semantic segmentation and output the semantic segmentation result.

[0056] Example 3 The third embodiment of the present invention also provides a remote sensing semantic segmentation device based on an improved TransUNet algorithm, which includes a memory and a processor. The memory stores a computer program that can be executed by the processor to implement the remote sensing semantic segmentation method based on the improved TransUNet algorithm as described above.

[0057] Example 4 The fourth embodiment of the present invention also provides a computer-readable storage medium storing computer-readable instructions. When the computer-readable instructions are executed by the processor of the device where the computer-readable storage medium is located, the remote sensing semantic segmentation method based on the improved TransUNet algorithm described above is implemented.

[0058] In the several embodiments provided in this invention, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus and method embodiments described above are merely illustrative. For example, the flowcharts in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0059] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0060] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, electronic device, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further restrictions, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0061] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0062] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0063] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0064] The use of "first" and "second" in the embodiments is merely to distinguish similar objects and does not represent a specific ordering of objects. It is understood that "first" and "second" can be interchanged in a specific order or sequence where permitted. It should be understood that the objects distinguished by "first" and "second" can be interchanged where appropriate so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

[0065] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A remote sensing semantic segmentation method based on an improved TransUNet algorithm, characterized in that, include: Acquire high-resolution remote sensing image datasets and perform multi-scale data augmentation processing; The processed remote sensing image dataset is input into a pre-trained improved TransUNet neural network model to perform semantic segmentation and output semantic segmentation results; wherein, the pre-trained improved TransUNet neural network model includes an encoder, a Transformer encoder, a context-aware enhancement CAEM module, a decoder, a multi-scale feature fusion (MFFM) module, and a segmentation head; The encoder is used to perform progressive downsampling convolution on the input remote sensing image to extract multi-scale spatial feature maps, and to fuse features from different layers until the last downsampled layer features are converted into a feature sequence by linear projection and used as the input of the Transformer encoder. The Transformer encoder is used to capture cross-regional contextual information of the input feature sequence through multi-head self-attention, thereby obtaining global semantically enhanced features. The context-aware enhancement CAEM module is located at the output of the Transformer encoder and is used to perform hierarchical pooling and adaptive weighting adjustment on the global semantic enhancement features to enhance the features of key regions and output fine features. The decoder is used to perform upsampling on the fine features and is connected to the encoder via the multi-scale feature fusion (MFFM) module to perform fusion of the output features of the decoder and encoder at the same scale, thereby obtaining multi-scale fused features. The segmentation head is used to perform convolutional mapping on the multi-scale fused features and output semantic segmentation results.

2. The remote sensing semantic segmentation method based on the improved TransUNet algorithm according to claim 1, characterized in that... The multi-scale data augmentation process includes random cropping, horizontal flipping, vertical flipping, brightness perturbation, or random rotation.

3. The remote sensing semantic segmentation method based on the improved TransUNet algorithm according to claim 1, characterized in that... The encoder uses a pre-trained ResNet-50 backbone network, which consists of multiple convolutional layers and residual structures. During the encoding process, a pre-trained visual backbone network is first loaded as the basis for feature extraction; Then, multi-level downsampling and convolution operations are performed on the input remote sensing image to extract spatial features layer by layer. The non-linear interaction between features is enhanced by element-wise multiplication to generate a multi-scale spatial feature map containing global semantics and local details. A linear projection is performed on the last downsampled feature map to map it to a feature space of uniform dimension, generating a feature sequence that can be received by the Transformer encoder.

4. A remote sensing semantic segmentation method based on an improved TransUNet algorithm according to claim 3, characterized in that... The Transformer encoder contains 12 encoder layers; The feature sequence generated by the encoder is input into the Transformer encoder; Each layer of the Transformer encoder models long-range dependencies through a multi-head self-attention mechanism, captures cross-regional contextual information, completes global feature reconstruction, and outputs enhanced global semantic features.

5. A remote sensing semantic segmentation method based on an improved TransUNet algorithm according to claim 4, characterized in that... The processing procedure of the context-aware enhanced CAEM module is as follows: Global soft pooling and multi-scale adaptive average pooling are performed in parallel on the global semantic enhancement features. After each channel is compressed by 1×1 convolution, the original spatial size is restored by bilinear interpolation upsampling to obtain multi-scale intermediate features. The multi-scale intermediate features are concatenated with the global semantic enhancement features to form an intermediate feature tensor; The intermediate feature tensor is convolutional, batch normalized, and ReLU activated. It is then divided into two sub-branches along the channel dimension. The two sub-branches are added element-wise along the cardinality dimension to obtain the intermediate statistics. Then, an adaptive weighted fusion reconstruction is performed through a cross-branch attention mechanism to output a context-enhanced feature map. The attention weights of the adaptive weighted fusion are generated by performing global average pooling on the intermediate statistics in terms of both height and width, concatenating them, and then performing two 1×1 convolutions and the RadixSoftmax function. After convolutional compression of channels, the context-enhanced feature map outputs fine features with unchanged spatial resolution.

6. A remote sensing semantic segmentation method based on an improved TransUNet algorithm according to claim 5, characterized in that... The decoder employs a symmetrical upsampling structure, introducing a multi-scale feature fusion (MFFM) module at each skip connection. The processing procedure is as follows: Upsampling is performed layer by layer on the fine features; The multi-scale feature fusion (MFFM) module of each layer receives the upsampling results of the previous layer and the skip connection features of the corresponding encoder layer, and extracts multi-scale context features through multi-branch convolution. The multi-scale context features are sequentially processed through soft pooling, convolution, ReLU activation, convolution, Sigmoid activation, and normalization to output context attention weights. The multi-scale context features are weighted and fused with the context attention weights to obtain multi-scale context weighted features; The multi-scale context-weighted features are then subjected to adaptive average pooling, convolution, and sigmoid activation to obtain fusion attention weights, which are then weighted and fused with the multi-scale context-weighted features to suppress redundant information and output multi-scale fusion features.

7. A remote sensing semantic segmentation method based on an improved TransUNet algorithm according to claim 6, characterized in that... In the segmentation head, the number of channels of the final multi-scale fused feature is mapped to the number of land cover categories through 1×1 convolution to obtain the final feature map; The final feature map is subjected to a Softmax operation to generate a class probability map for each pixel, thus obtaining the semantic segmentation result.

8. A remote sensing semantic segmentation method based on an improved TransUNet algorithm according to claim 7, characterized in that... It also includes optimizing the training of the improved TransUNet neural network model using a total loss function; wherein, the total loss function The expression is: ； ； ； ； ； ； ； in, , , The weighting factor for the loss term; Cross-entropy loss; The Dice loss is used to address the class imbalance problem. For joint losses; The total number of samples; For the i-th sample in the true category The predicted probability; Let i be the true class label of the i-th sample; True cases are the number of samples that the model predicts to be positive and that are actually positive. False negatives are the number of samples that the model predicts to be negative but are actually positive. False positives are the number of samples that the model predicts to be positive but are actually negative. For boundary loss; For label-aware cross-entropy loss with prior knowledge; The weighted Dice loss is used to assign weights related to prior probabilities to different categories; For all land cover categories; For category Edge pixel precision; For category Edge pixel recall rate; For the sample In category The predicted score; This is a temperature parameter used to control the degree of influence of prior knowledge; , Categories , The prior probability; , Samples The true category label and the predicted category label.

9. A remote sensing semantic segmentation method based on an improved TransUNet algorithm according to claim 7, characterized in that... It also includes employing a multinomial decay learning rate adjustment strategy during model training, where the learning rate gradually decreases as the number of training iterations increases in a power function form. The calculation formula is as follows: ； in, The current learning rate, The initial learning rate, This represents the current iteration number. This represents the maximum number of iterations. is the power exponent of the polynomial decay.

10. A remote sensing semantic segmentation system based on an improved TransUNet algorithm, used to implement the remote sensing semantic segmentation method based on an improved TransUNet algorithm as described in any one of claims 1-9, characterized in that, include: The data acquisition unit is used to acquire high-resolution remote sensing image datasets and perform multi-scale data augmentation processing. The semantic segmentation result unit is used to input the processed remote sensing image dataset into the pre-trained TransUNet neural network model to perform semantic segmentation and output the semantic segmentation result.

Images