A Transformer-based method for extracting buildings from remote sensing images

Abstract

This invention discloses a method for extracting buildings from remote sensing images based on Transformer, belonging to the field of image segmentation technology. The method includes: acquiring remote sensing images and preprocessing them; inputting the preprocessed remote sensing images into a deep learning model for semantic segmentation to obtain building segmentation results. This invention designs an asymmetric network structure within the existing Swing Transformer technology and employs a newly designed multi-branch weighted pyramid pooling module in the skip connections to further mine feature information, overcoming the problem of difficulty in identifying diverse buildings of varying sizes in remote sensing images.

Description

Technical Field

[0001] This invention belongs to the field of image segmentation technology, specifically relating to a method for extracting buildings from remote sensing images based on Transformer. Background Technology

[0002] Building extraction is an important subtask of semantic segmentation in computer vision, with significant implications for military reconnaissance, precision guidance, and civilian applications. Unlike semantic segmentation of natural images, existing methods for building extraction from remote sensing images, which rely on intensive prediction tasks using convolutional neural networks for feature extraction, suffer from the problem of struggling to expand the effective receptive field and establish long-range dependencies. Furthermore, Tranformer-based networks are computationally expensive and prone to overfitting to a small number of remote sensing images. Summary of the Invention

[0003] The purpose of this invention is to provide a method for extracting buildings from remote sensing images based on Transformer. An asymmetric network structure is designed in the original Swing Transformer technology, and a newly designed multi-branch weighted pyramid pooling module is adopted in the skip connections to further mine feature information and overcome the problem of difficulty in identifying buildings of different types and scales in remote sensing images.

[0004] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0005] A method for extracting buildings from remote sensing images based on Transformer, the method comprising:

[0006] Acquire remote sensing images and preprocess them;

[0007] The preprocessed remote sensing images are input into a deep learning model for semantic segmentation to obtain building segmentation results.

[0008] The deep learning model is an asymmetric network structure, comprising a four-layer encoder (SwinTransformer), a multi-branch weighted pyramid pooling module, a three-layer decoder, and a multi-level feature cascade fusion module, wherein:

[0009] The encoder, Swin Transformer, receives preprocessed remote sensing images and outputs feature maps of four sizes, F1, F2, F3, and F4, through a four-layer structure.

[0010] The multi-branch pyramid pooling module has three parts. Each of the three modules takes feature maps F2, F3, and F4 from the output of the last three layers of the encoder's Swing Transformer, processes them, and then outputs feature map F. 22 F 32 F 42 ;

[0011] The first layer structure in the decoder acquires the feature map F. 42 With F 32 Process and output feature map F 32 The second layer structure obtains the feature map F. )2 ′ and F 22 Process and output feature map F 22 The third layer structure obtains the feature map F. 22 The F1 map is processed and output as a feature map F1'.

[0012] The multi-level feature cascade fusion module will combine feature map F 22 ′ and feature map F 32 Feature fusion is performed to obtain feature map F. 22 "Then combine feature map F1' and feature map F 22 "Feature fusion is performed to obtain the final segmentation map, which is then used as the building segmentation result."

[0013] Several alternative methods are provided below, but they are not intended as additional limitations on the overall solution above. They are merely further additions or optimizations. Provided there are no technical or logical contradictions, each alternative method can be combined individually with respect to the overall solution above, or multiple alternative methods can be combined with each other.

[0014] Preferably, the preprocessing includes scaling the remote sensing image to an image size of 512×512.

[0015] Preferably, the four-layer structure of the encoder Swin Transformer is defined as a first-layer structure, a second-layer structure, a third-layer structure, and a fourth-layer structure according to the data flow direction.

[0016] The first layer structure consists of a patch partitioning block, a linear embedding block, and a Swing Transformer block connected sequentially from the data input side to the output side. The second, third, and fourth layer structures each consist of a patch merging layer and a Swing Transformer block connected sequentially from the data input side to the output side.

[0017] The first layer structure outputs feature map F1, the second layer structure outputs feature map F2, the third layer structure outputs feature map F3, and the fourth layer structure outputs feature map F4.

[0018] Preferably, the feature map F is output after processing feature maps F2, F3, and F4. 22 F 32 F 42 ,include:

[0019] For feature map F4, the corresponding multi-branch pyramid pooling module is set to have 4 branches. Pooling operations at different scales are performed on feature map F4 to obtain feature maps with feature sizes of 1, 2, 3, and 6 for each branch. The weights assigned to each branch are 0.1, 0.2, 0.2, and 0.5, respectively. After depthwise separable convolution, the feature maps on each branch are restored to the same size as feature map F4 through bilinear interpolation before being weighted and concatenated. A single convolution operation is then performed on the concatenated feature maps to obtain feature map F. 42 ;

[0020] For feature map F3, the corresponding multi-branch pyramid pooling module is set to have 4 branches. Pooling operations at different scales are performed on feature map F3 to obtain feature maps with feature sizes of 1, 2, 3, and 6 for each branch. The weights assigned to each branch are 0.1, 0.2, 0.2, and 0.5, respectively. After depthwise separable convolution, the feature maps on each branch are restored to the same size as feature map F3 through bilinear interpolation, and then weighted and concatenated. A single convolution operation is performed on the concatenated feature maps to obtain feature map F. 32 ;

[0021] For feature map F2, the corresponding multi-branch pyramid pooling module is set to have 5 branches. Pooling operations at different scales are performed on feature map F2, resulting in feature maps with feature sizes of 1, 4, 8, 16, and 32 for each branch. The weights assigned to each branch are 0.1, 0.1, 0.2, 0.2, and 0.4, respectively. After depthwise separable convolution, the feature maps on each branch are restored to the same size as feature map F2 through bilinear interpolation, and then weighted and concatenated. A single convolution operation is performed on the concatenated feature maps to obtain feature map F. 22 .

[0022] Preferably, the three-layer structure of the decoder is defined as a first-layer structure, a second-layer structure, and a third-layer structure according to the data flow direction, and each layer structure includes a patch expansion layer and a diversion attention module.

[0023] Preferably, the first layer structure in the decoder acquires the feature map F. 42 With F 32 Process and output feature map F 32 The second layer structure obtains the feature map F. 32 ′ and F 22Process and output feature map F 22 The third layer structure obtains the feature map F. 22 The F1-processed feature map F1' is output, including:

[0024] Feature map F 42 The input is fed into the patch expansion layer in the first layer structure. After expansion processing by the patch expansion layer, it is compared with the feature map F. 32 Channel-level concatenation is performed, and the concatenated feature map is input into the split attention module in the first layer structure. The split attention module in the first layer structure outputs the feature map F. 32 ′, and simultaneously feature map F 32 Input to the patch expansion layer in the second-layer structure;

[0025] Feature map F 32 After the expansion process of the patch expansion layer in the second layer structure, and the feature map F 22 Channel-level concatenation is performed, and the concatenated feature map is input into the split attention module in the second layer structure. The split attention module in the second layer structure outputs the feature map F. 22 ′, and simultaneously feature map F 22 Input to the patch expansion layer in the third-layer structure;

[0026] Feature map F 22 After being expanded by the patch expansion layer in the third layer structure, the feature map F1 is concatenated with the feature map F1 along the channel dimension. The concatenated feature map is then input into the split attention module in the third layer structure, and the split attention module in the third layer structure outputs the feature map F1.

[0027] Preferably, the multi-level feature cascade fusion module will fuse the feature map F 22 ′ and feature map F 32 Feature fusion is performed to obtain feature map F. 22 "Then combine feature map F1' and feature map F 22 "Feature fusion is performed to obtain the final segmentation map, which is then used as the building segmentation result, including:"

[0028] First, for feature map F 32 Bilinear interpolation is performed to obtain the feature map F. 22 Feature map F of the same size 32 ", in the feature Figure 2 32 "With feature map F 22 The feature maps F1′ and F1″ are added together and then bilinear interpolation is used to obtain a feature map F1″ with the same size as the feature map F1′. Finally, the feature maps F1″ and F1′ are fused to obtain the final segmentation map.

[0029] This invention provides a method for extracting buildings from remote sensing images based on Transformer. It introduces a new decoder design to form a new asymmetric network structure in the existing SwingTransformer technology. A redesigned multi-branch weighted pyramid pooling module is added between the encoder and decoder to deeply mine feature semantic information, which can effectively improve the accuracy of building extraction from remote sensing images. Attached Figure Description

[0030] Figure 1 This is a flowchart of the Transformer-based remote sensing image building extraction method of the present invention;

[0031] Figure 2 This is a schematic diagram of the structure of the deep learning model of the present invention;

[0032] Figure 3 This is a schematic diagram of the structure of the multi-branch weighted pyramid pooling module of the present invention. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0035] To overcome the shortcomings of existing technologies for remote sensing image segmentation, this embodiment provides a Transformer-based method for extracting buildings from remote sensing images, thereby improving the accuracy of building extraction from remote sensing images.

[0036] like Figure 1 As shown in this embodiment, the method for extracting buildings from remotely sensed images based on Transformer includes the following steps:

[0037] Step 1: Acquire remote sensing images and preprocess them.

[0038] To meet the input requirements of the neural network, this embodiment needs to scale the remote sensing image to a size of 512×512. The preprocessing focus in this embodiment is on image scaling; in other embodiments, other image preprocessing can be performed to improve image quality.

[0039] Step 2: Input the preprocessed remote sensing image into the deep learning model for semantic segmentation to obtain the building segmentation results.

[0040] This embodiment introduces a deep learning model for semantic segmentation, which not only reduces the complexity of the segmentation operation but also improves its accuracy and stability. For example... Figure 2 As shown, the deep learning model in this embodiment is an asymmetric network structure, including a four-layer encoder (Swin Transformer), a multi-branch weighted pyramid pooling module, a three-layer decoder, and a multi-level feature cascade fusion module. The following detailed description of each module illustrates the deep learning model used in this embodiment.

[0041] 1) Encoder Swin Transformer: Receives pre-processed remote sensing images and outputs feature maps of four sizes F1, F2, F3, and F4 through a four-layer structure.

[0042] The encoder in this embodiment uses the Swing Transformer. For ease of description, its four-layer structure is defined as the first layer structure, the second layer structure, the third layer structure, and the fourth layer structure according to the data flow direction.

[0043] The first layer structure, from the data input side to the output side, includes a PatchPartition block, a Linear Embedding block, and a Swin Transformer block connected in sequence. The second, third, and fourth layers, each from the data input side to the output side, include a Patch Merging layer and a Swin Transformer block connected in sequence. The first layer outputs feature map F1, the second layer outputs feature map F2, the third layer outputs feature map F3, and the fourth layer outputs feature map F4.

[0044] The Swin Transformer model employs a hierarchical design, comprising four layers. Each layer reduces the resolution of the input feature map, expanding the receptive field layer by layer, similar to a CNN. At the beginning of the input, a Patch Partition is performed, dividing the image into patches and embedding them into Linear Embedding (dividing into 4x4 patches, stretching them into a number of one-dimensional vectors before embedding). The Patch Merging module primarily reduces the image resolution at the beginning of each layer. The Swin Transformer Block mainly consists of LayerNorm, MLP, Window Attention, and Shifted Window Attention.

[0045] It should be noted that the Swin Transformer is an existing network structure, and its specific structure and workflow will not be described in detail in this embodiment.

[0046] 2) There are three multi-branch pyramid pooling modules. The three multi-branch pyramid pooling modules take the feature maps F2, F3, and F4 output from the last three layers of the encoder SwinTransformer, respectively, process the feature maps F2, F3, and F4, and output feature map 4. 22 F 32 F 42 .

[0047] The multi-branch pyramid pooling module transforms feature maps F2, F3, and F4 from dimensions B (number of batches), L (number of patches), and C (number of channels) into the form B (number of batches), H (height), W (width), and C (number of channels). The overall structure of this module is shown in the attached figure. Figure 3 The module structure is an improvement upon the existing PPM pyramid pooling module. Unlike the PPM pyramid pooling module, the multi-branch pyramid pooling module sets different branches for each layer of the codec, and assigns corresponding weights to the importance of each branch. Furthermore, depthwise separable convolutions are used instead of traditional convolution operations to reduce computational cost. The specific settings of this module are as follows:

[0048] For feature map F4, the corresponding multi-branch pyramid pooling module (Multi-branch PPM Block 1) is set to have 4 branches. Pooling operations at different scales are performed on feature map F4 to obtain feature maps with feature sizes of 1, 2, 3, and 6 for each branch. The weights assigned to each branch are 0.1, 0.2, 0.2, and 0.5, respectively. After each branch's feature map undergoes depthwise separable convolution (or simply depthwise separable convolution), it is restored to the same size as feature map F4 through bilinear upsampling before being weighted and concatenated. A traditional convolution operation is then performed on the concatenated feature map to obtain feature map F. 42 .

[0049] For feature map F3, the corresponding multi-branch pyramid pooling module (Multi-branch PPM Block 2) is set to have 4 branches. Pooling operations at different scales are performed on feature map F3 to obtain feature maps with feature sizes of 1, 2, 3, and 6 for each branch. The weights assigned to each branch are 0.1, 0.2, 0.2, and 0.5, respectively. After depthwise separable convolution, the feature maps on each branch are restored to the same size as feature map F3 through bilinear interpolation before weighted concatenation. A traditional convolution operation is then performed on the concatenated feature maps to obtain feature map F. 32 .

[0050] For feature map F2, the corresponding multi-branch pyramid pooling module (Multi-branch PPM Block 3) is set to have 5 branches. Pooling operations at different scales are performed on feature map F2, resulting in feature maps with feature sizes of 1, 4, 8, 16, and 32 for each branch. The weights assigned to each branch are 0.1, 0.1, 0.2, 0.2, and 0.4, respectively. After depthwise separable convolution, the feature maps on each branch are restored to the same size as feature map F2 through bilinear interpolation, and then weighted and concatenated. A traditional convolution operation is then performed on the concatenated feature maps to obtain feature map F. 22 .

[0051] It should be noted that this embodiment adds weights to each branch, which better reflects important feature information and provides effective feature influence. When weighting and stacking features, the features output by the depthwise separable convolution in each branch can be multiplied by the weights, then restored using bilinear interpolation before concatenation; alternatively, the features output by the depthwise separable convolution can be directly taken, restored using bilinear interpolation, and then weighted and concatenated. Figure 3The weights in this example are for illustrative purposes only and are not intended as specific limitations.

[0052] 3) The first layer structure in the decoder acquires the feature map F. 42 With F 32 Process and output feature map F 32 The second layer structure obtains the feature map F. 32 ′ and F 22 Process and output feature map F 22 The third layer structure obtains the feature map F. 22 The F1 map is processed and output as a feature map F1'.

[0053] The decoder in this embodiment has a three-layer structure defined as a first layer, a second layer, and a third layer according to the data flow direction. Each layer includes a patch expand layer and a shunted transformer block.

[0054] The patch expansion layer uses the patch expansion layer in Swing Unet, while the split attention module is based on the Shunted Transformer Block in Shunted Transformer. The aggregation rate of each layer of the split attention module is set to two sizes. The three decoding layers have different attention and different focus on features. Based on these characteristics, the number of splits set for each layer is different.

[0055] Feature map F 42 The input is fed into the patch expansion layer in the first layer structure. After expansion processing by the patch expansion layer, it is compared with the feature map F. 32 Channel-level concatenation is performed, and the concatenated feature map is input into the split attention module in the first layer structure. The split attention module in the first layer structure outputs the feature map F. 32 ′, and simultaneously feature map F 32 Input to the patch expansion layer in the second layer structure.

[0056] Feature map F 32 After the expansion process of the patch expansion layer in the second layer structure, and the feature map F 22 Channel-level concatenation is performed, and the concatenated feature map is input into the split attention module in the second layer structure. The split attention module in the second layer structure outputs the feature map F. 22 ′, and simultaneously feature map F 22 Input to the patch expansion layer in the third layer structure.

[0057] Feature map F 22After being expanded by the patch expansion layer in the third layer structure, the feature map F1 is concatenated with the feature map F1 along the channel dimension. The concatenated feature map is then input into the split attention module in the third layer structure, and the split attention module in the third layer structure outputs the feature map F1.

[0058] In this embodiment, the patch expansion layer uses a linear mapping method to transform the dimensions of feature maps with dimensions such as B (number of batches), L (number of patches), and C (number of channels) into B, L, 2*C, and then uses a reorganization method to transform the dimensions into B, 4*L, C / 2 for size expansion. The split attention module projects vectors of the form {B,L,C} into tensors of Q, K,V, aggregates K and V through different aggregation rates r, and then performs attention calculations to obtain the output vector that incorporates global context information.

[0059] 4) The multi-level feature cascade fusion module will fuse the feature map F 22 ′ and feature map F 32 Feature fusion is performed to obtain feature map F. 22 "Then combine feature map F1' and feature map F 22 "Feature fusion is performed to obtain the final segmentation map, which is then used as the building segmentation result."

[0060] To ensure the fusion effect, this embodiment first processes the feature map F. 32 Bilinear interpolation is performed to obtain the feature map F. 22 Feature map F of the same size 32 ", in the feature map F 32 "With feature map F 22 The feature maps F1′ and F1″ are added together and then bilinear interpolation is used to obtain a feature map F1″ with the same size as the feature map F1′. Finally, the feature maps F1″ and F1′ are fused to obtain the final segmentation map.

[0061] The feature fusion operation is a conventional feature fusion operation, which will not be described in detail in this embodiment.

[0062] To ensure the effectiveness of deep learning models, they need to be trained in advance. This embodiment provides a training process as follows:

[0063] Step S1: Obtain a remote sensing image training dataset with labeled building segmentation masks, perform random flipping and photometric distortion data augmentation on the remote sensing image training dataset, and then start training in batches.

[0064] This embodiment performs random flipping and photometric distortion data augmentation on the training dataset. Random flipping involves flipping each batch of training images horizontally or vertically with a 50% probability. This increases robustness to our small dataset and improves accuracy for subsequent remote sensing segmentation. Photometric distortion data augmentation adjusts the brightness, chroma, contrast, and saturation of the images, and adds noise, making the dataset more consistent with remote sensing images obtained at various times.

[0065] Step S2: Adjust the size of the input image in the enhanced image training dataset to 512×512 and input it into the network encoder Swin Transformer to obtain the feature maps F1, F2, F3, and F4 of four sizes output from stages 1, 2, 3, and 4 of the encoder Swin Transformer.

[0066] This application uses the Swing Transformer as the backbone network for feature extraction. The Swing Transformer used is pre-trained with weights trained on ADE20K and is trained in batches with a batch size of 8 (i.e., 8 images are processed per batch). The optimizer is AdamW, the initial learning rate is 6e-05, the momentum is 0.9, the learning rate adjustment strategy is a poly strategy, the power is set to 1, and the weight decay is 0.01.

[0067] A 512×512 image is fed into the encoder Swin Transformer. After passing through four encoder layers, feature maps F1, F2, F3, and F4 of four different sizes are output in succession: 128×128, 64×64, 32×32, and 16×16.

[0068] The size of the feature map is determined by the backbone network, the Swing Transformer, which will not be elaborated here. For the original Swing Transformer, only the features extracted from the last layer are used for the final segmentation prediction. However, in this embodiment, all features output by the encoder Swing Transformer are used, and the feature information is deeply mined through a multi-branch weighted pyramid pooling module. These features are then fed into the decoder and sequentially concatenated with the decoded features from the lowest layer. Finally, a multi-scale feature cascade fusion is performed to generate a segmentation prediction mask.

[0069] Step S3: Select the latter three (F2, F3, F4) from the four feature maps of different sizes and process them through the multi-branch weighted pyramid pooling module to obtain the processed feature map F. 22 F 32 F 42 .

[0070] It should be noted that, to reduce computational cost and parameter count, the multi-branch weighted pyramid pooling module employs depthwise separable convolutions, including depthwise convolutions and pointwise convolutions. Depthwise convolutions involve grouping and convolving each channel of the input, while pointwise convolutions are 1×1 convolutions. By processing features at different scales using the multi-branch weighted pyramid pooling module, the semantic information of the features is further extracted, facilitating subsequent processing.

[0071] Step S4: Process the feature maps F1 and F2. 22 F 32 F 42 Each is fed into its corresponding decoder, and the lowest-level processed feature F 42 The network first needs to be doubled in size using a patching expansion layer to restore it to the size of the previous layer. Then, it is processed by a decoder to obtain feature maps, which are then cascaded and fused to obtain the final segmentation prediction mask. The loss is calculated, and backpropagation is performed to update the network parameters, completing the network training.

[0072] For the final three-layer feature F3 ′ 2. F2 ′ 2. F1', first check F 32 Perform bilinear interpolation to obtain F. 22 Same size F 32 ", F 32 "With F 22 The F1′ and F1″ components are added together and then bilinearly interpolated to obtain F1″ of the same size as F1′. F1″ and F1′ are then fused to obtain the final feature map. This feature map is then converted from B, L, C to B, H, W, C format for prediction. Additionally, since the building extraction task only has two categories of labels—buildings and background—the final number of channels is only two.

[0073] The segmentation prediction map and the ground truth label are used to calculate the loss using the CrossEntropyLoss function, as shown in the following formula:

[0074] CELoss=-(ylog(p(x))+(1-y)log(1-p(x)))

[0075] Where y represents whether it is a target, with a value of 1 or 0, and p(x) is the predicted target score.

[0076] It should be noted that the calculation of classification loss, target score loss and bounding box loss is a relatively mature technology in this field, and will not be elaborated here.

[0077] This yields the loss between the predicted and actual values. Before the end of each batch, backpropagation is performed to reduce the loss. Simultaneously, the network parameters are updated, and training begins for the next batch, continuing until all batches of training data have been completed. Finally, the trained weights are obtained, and all updated parameters are saved in the Outputs weights file.

[0078] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0079] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for extracting buildings from remote sensing images based on Transformer, characterized in that, The Transformer-based remote sensing image building extraction method includes: Acquire remote sensing images and preprocess them; The preprocessed remote sensing images are input into a deep learning model for semantic segmentation to obtain building segmentation results. The deep learning model is an asymmetric network structure, comprising a four-layer encoder (SwinTransformer), a multi-branch weighted pyramid pooling module, a three-layer decoder, and a multi-level feature cascade fusion module, wherein: The encoder, Swin Transformer, receives preprocessed remote sensing images and outputs feature maps of four sizes through a four-layer structure. ; The multi-branch weighted pyramid pooling module has three components, each taking a feature map from the output of the last three layers of the encoder's Swing Transformer. For feature maps Output feature map after processing ; The first layer structure in the decoder acquires feature maps. and Process and output feature maps The second layer structure obtains feature maps. and Process and output feature maps The third layer structure obtains feature maps. and Process and output feature maps ; The multi-level feature cascade fusion module will integrate the feature map and feature map Feature fusion is performed to obtain feature maps. Then the feature map and feature map Feature fusion is performed to obtain the final segmentation map, which is then used as the building segmentation result. Among them, the feature map Output feature map after processing ,include: For feature maps The corresponding multi-branch weighted pyramid pooling module is set to have 4 branches for the feature map. Pooling operations at different scales are performed to obtain feature maps with feature sizes of 1, 2, 3, and 6 for each branch, respectively. The weights assigned to each branch are 0.1, 0.2, 0.2, and 0.5, respectively. After depthwise separable convolution, the feature maps on each branch are recovered using bilinear interpolation. The same size feature maps are then weighted and concatenated, and a convolution operation is performed on the concatenated feature maps to obtain the final feature map. ; For feature maps The corresponding multi-branch weighted pyramid pooling module is set to have 4 branches for the feature map. Pooling operations at different scales are performed to obtain feature maps with feature sizes of 1, 2, 3, and 6 for each branch, respectively. The weights assigned to each branch are 0.1, 0.2, 0.2, and 0.5, respectively. After depthwise separable convolution, the feature maps on each branch are recovered using bilinear interpolation. The same size feature maps are then weighted and concatenated, and a convolution operation is performed on the concatenated feature maps to obtain the final feature map. ; For feature maps The corresponding multi-branch weighted pyramid pooling module is set to have 5 branches for the feature map. Pooling operations at different scales are performed to obtain feature maps with feature sizes of 1, 4, 8, 16, and 32 for each branch, respectively. Weights of 0.1, 0.1, 0.2, 0.2, and 0.4 are assigned to each branch. After depthwise separable convolution, the feature maps on each branch are restored using bilinear interpolation. The same size feature maps are then weighted and concatenated, and a convolution operation is performed on the concatenated feature maps to obtain the final feature map. .

2. The method for extracting buildings from remote sensing images based on Transformer as described in claim 1, characterized in that, The preprocessing includes scaling the remote sensing image to a size of [image size missing]. .

3. The method for extracting buildings from remote sensing images based on Transformer as described in claim 1, characterized in that, The encoder Swin Transformer has a four-layer structure, which is defined as the first layer structure, the second layer structure, the third layer structure and the fourth layer structure according to the data flow direction. The first layer structure consists of a patch partitioning block, a linear embedding block, and a SwinTransformer block connected sequentially from the data input side to the output side. The second, third, and fourth layer structures each consist of a patch merging layer and a SwinTransformer block connected sequentially from the data input side to the output side. The first layer structure outputs a feature map. The second layer structure outputs a feature map. The third layer structure outputs feature maps. The fourth layer structure outputs a feature map. .

4. The method for extracting buildings from remote sensing images based on Transformer as described in claim 1, characterized in that, The decoder's three-layer structure is defined as a first-layer structure, a second-layer structure, and a third-layer structure according to the data flow direction, and each layer structure includes a patch expansion layer and a split attention module.

5. The method for extracting buildings from remote sensing images based on Transformer as described in claim 4, characterized in that, The first layer structure in the decoder acquires feature maps. and Process and output feature maps The second layer structure obtains feature maps. and Process and output feature maps The third layer structure obtains feature maps. and Process and output feature maps ,include: Feature map The input is fed into the patch expansion layer in the first layer structure. After expansion processing by the patch expansion layer, it is compared with the feature map. Channel-level concatenation is performed, and the concatenated feature map is input into the split attention module in the first layer structure. The split attention module in the first layer structure outputs the feature map. At the same time, feature map Input to the patch expansion layer in the second-layer structure; Feature map After expansion processing by the patch expansion layer in the second layer structure, and then compared with the feature map... Channel-level concatenation is performed, and the concatenated feature map is input into the split attention module in the second layer structure. The split attention module in the second layer structure outputs the feature map. At the same time, feature map Input to the patch expansion layer in the third-layer structure; Feature map After expansion processing by the patch expansion layer in the third layer structure, and then compared with the feature map... Channel-level concatenation is performed, and the concatenated feature map is input into the split attention module in the third layer structure. The split attention module in the third layer structure outputs the feature map. .

6. The method for extracting buildings from remote sensing images based on Transformer as described in claim 1, characterized in that, The multi-level feature cascade fusion module will integrate the feature map and feature map Feature fusion is performed to obtain feature maps. Then the feature map and feature map Feature fusion is performed to obtain the final segmentation map, which is then used as the building segmentation result, including: First, the feature map Bilinear interpolation is performed to obtain the feature map. Feature maps of the same size In the feature map With feature map Add the results and continue using bilinear interpolation to obtain the feature map. Feature maps of the same size Finally, the feature map With feature map Feature fusion is performed to obtain the final segmentation map.

Images