Remote sensing image building extraction method based on multi-scale interaction and cross decoding

Abstract

The invention relates to a remote sensing image building extraction method based on multi-scale interaction and cross decoding, and belongs to the technical field of remote sensing image information extraction. The method comprises the following steps: S1, acquiring a remote sensing image, labeling the remote sensing image, dividing the remote sensing image into a training set, a verification set and a test set, and performing image data enhancement on the image when the image is input; s2, constructing a building extraction network model MGCC based on multi-scale interaction and cross decoding, wherein the network mainly comprises a multi-scale interaction guide module MSAG and a cross-layer cross decoding module CLCC; s3, training an MGCC model: inputting the training set into the MGCC model, calculating a corresponding loss function, and then adopting an AdamW optimizer and a cosine restart strategy to iteratively train the MGCC model until the model converges; and S4, inputting the test set into the trained MGCC model to obtain an image only having building features. The accuracy of remote sensing image building extraction is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of remote sensing image information extraction, and relates to a remote sensing image building extraction method based on multi-scale interaction and cross decoding. Background Art

[0002] With the development of remote sensing sensors and earth observation technology, it is now possible to quickly obtain high-quality image data over a wide range. Compared with traditional low-resolution images, very high-resolution (VHR) remote sensing images contain more object details, spatial textures, and semantic information, which facilitates the accurate identification of objects such as buildings and roads. Therefore, they are widely used in urban management, smart cities, and virtual reality. However, building extraction based on VHR images faces many challenges. In addition to buildings, VHR images also contain a large amount of redundant information such as trees and roads, which affects the extraction accuracy. In addition, the roof materials, shapes, sizes, and spectral characteristics of buildings are diverse, and they are easily confused with background objects, further increasing the difficulty of extraction.

[0003] In recent years, the application of deep learning methods in building extraction has made significant progress. Building extraction based on deep learning belongs to the task of semantic segmentation, in which the encoder-decoder architecture (such as FCN) provides a basis for image segmentation and can effectively restore the spatiotemporal information of the image. As typical encoder-decoder networks, U-Net and SegNet are effective in processing VHR remote sensing images, but they are still difficult to deal with target extraction in complex scenes. To this end, the attention mechanism is introduced to improve the extraction ability of the model in complex environments. For example, SER-UNet combines the attention mechanism and jump connection to effectively improve the extraction accuracy of building edges; MSSDMPA-Net accurately extracts buildings and road footprints by introducing a dynamic attention module. In addition, multi-scale feature learning helps to improve the model's sensitivity to objects of different sizes and enhance its adaptability in complex environments. MAP-Net retains multi-scale features through parallel path learning and optimizes feature fusion; MDCGA-Net improves the effect of global and local feature extraction by introducing directional attention. The use of global context information further enhances the semantic understanding ability of the model. DPENet combines the dual-path feature extraction of CNN and Transformer to enhance the ability to extract semantic information. Based on the above technological advances, the remote sensing image building extraction method based on deep learning is constantly evolving. Through the combination of multiple mechanisms, the accuracy and efficiency of building extraction have been improved.

[0004] However, the building extraction framework based on the encoder-decoder architecture still has some limitations in the multi-scale feature fusion at the decoder stage. Specifically, on the one hand, due to the lack of interaction between different levels, not just the interaction between high-level and low-level layers, directly fusing high-level and low-level features may lead to information redundancy and conflict, affecting the accuracy of building extraction. On the other hand, in the context fusion process of high-level and low-level features, previous building extraction methods focus on building features through attention mechanisms, but tend to ignore their semantic similarities, which makes the model unable to fully focus on the key semantic features of the building, thereby reducing the accuracy of the extraction results. In summary, there is an urgent need for an optimization method to bridge the semantic gap between these levels, so that multi-scale information can be more reasonably integrated in the decoding process and optimize the semantic capture ability of decoding. In particular, for the needs of building extraction in remote sensing images, it is necessary to design a module that can gradually refine and strengthen building features, which can not only retain semantic consistency but also suppress interference information, and ultimately achieve higher-precision extraction of buildings. Summary of the invention

[0005] In view of this, the object of the present invention is to provide a remote sensing image building extraction method based on multi-scale interaction and cross decoding, so as to improve the accuracy of remote sensing image building extraction.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A remote sensing image building extraction method based on multi-scale interaction and cross decoding specifically comprises the following steps:

[0008] S1: Obtain remote sensing images and annotate them, divide them into training set, validation set and test set, and perform image data enhancement on them when images are input;

[0009] S2: Construct the MGCC model, including the MSAG module and the CLCC module, where MGCC represents the building extraction network based on multi-scale interaction and cross decoding, MSAG represents the multi-scale interaction guidance module, and CLCC represents the cross-layer cross decoding module;

[0010] S3: Training the MGCC model: Input the training set into the MGCC model and calculate the corresponding loss function. Then, the AdamW optimizer and the cosine restart strategy are used to iteratively train the MGCC model until the model converges.

[0011] S4: Input the test set into the trained MGCC model to obtain images with only building features.

[0012] Furthermore, in step S1, the image is enhanced, including random flipping and random rotation.

[0013] Furthermore, in step S2, the constructed MGCC model adopts an encoder-decoder architecture. The encoder part adopts the BFB Vit proposed in BuildFormer to capture global multi-scale information. At the same time, ASPP (Atrous Spatial Pyramid Pooling) is added to the last layer of feature extraction to form the backbone of the model. The output features are F1, F2, F3, F as ; The encoder process is expressed as:

[0014] F1,F2,F3,F as =Backbone(X) (1)

[0015] The decoder part includes the MSAG module and the CLCC module.

[0016] Further, in step S2, the MSAG module is arranged after the encoder performs feature extraction, and is used to shorten the semantic gap between multi-level features and embed multi-scale information into the features; the MSAG module includes three parts: multi-scale fusion, position interactive, and channel interactive;

[0017] The multi-scale fusion is specifically: for the extracted features C represents the channel dimension of the feature map, H represents the height of the feature map, and W represents the width of the feature map. Concatenate them to get Represents the features of the i-th layer concatenation, and then Compress the channel into C through 1×1 convolution i (i=1,2,3), and feed the compressed features into three parallel depth-wise separable convolutions with kernel sizes of 3×3, 5×5, and 7×7 respectively. The parallel output results are added to obtain the multi-scale fusion result. The above process can be expressed as:

[0018] FMS i =B(Conv 3×3 (F ci ))+B(Conv 5×5 (F ci ))+B(Conv 7×7 (F ci )) (2)

[0019] Among them, B(·) represents BatchNorm; Conv k×krepresents the convolution operation with convolution kernel k, which is performed on the first three layers of the encoder to obtain three multi-scale embedded features FMS1, FMS2, and FMS3;

[0020] The position interaction is specifically: after multi-scale fusion, spatial position refinement is performed, firstly, the input feature FMS is respectively processed in the horizontal and vertical directions. i Apply global average pooling and concatenate the output feature maps to generate The above process is described as follows:

[0021] F hwi =[GAP h (FMS i ),GAP w (FMS i )] (3)

[0022] Among them, GAP h (·) represents global average pooling along the horizontal direction, GAP w (·) represents global average pooling along the vertical direction; [,] represents the concatenation operation, followed by a 1×1 convolution for channel reduction. Capturing information from the horizontal and vertical coordinates, r is a multiple of the channel reduction, and the above process can be expressed as:

[0023] F rhwi =BR(Conv(F hwi )) (4)

[0024] Among them, BR(·) represents BatchNorm and ReLU operations, Conv represents 1×1 convolution; then for F rhwi Split horizontally and vertically to get F hi and F wi , and for F hi and F wi Apply 1×1 convolution to restore the channels to C i Then perform Sigmod function activation to obtain the directional weights W along the width and height dimensions hi and W wi , and finally the weight is combined with FMS i Multiply them to get the attention feature F with spatial position information interaction pi , the above process can be expressed as:

[0025] F pi =FMS i σ(Conv(F hi ))*σ(Conv(F wi )) (5)

[0026] Among them, σ represents the Sigmoid activation function;

[0027] The channel interaction is specifically as follows: in the first step of multi-scale fusion, the channel is compressed by 1×1, resulting in a certain degree of loss of information from each layer. This disadvantage is compensated by channel interaction. First, global average pooling is used, and 1×1 convolution and ReLU activation are performed to generate channel attention features, and redundant channel features are adaptively filtered out. Then, the channel is expanded to the output size by 1×1 convolution, and interacts with the attention feature F with spatial position information. pi Multiply them together to get the complete multi-scale interactive output F mpci , the above process can be expressed as:

[0028] F mpci =F pi *Conv(ReLU(Conv(GAP(FMS i )))) (6)

[0029] To this end, the expression for defining MSAG is:

[0030] X=MSAG(X) (7)

[0031] Further, in step S2, the structure of the CLCC module is: the features from the low-level and high-level layers are fed into the multi-scale strip volume, multi-scale alignment is performed, and related Q, K, V features are generated, and then cross-fed into the multi-head self-attention to capture the similar semantic information of the building and perform global long-range dependency perception. At the same time, in order to enhance the ability to pay attention to local details, a C-MLP module is added, where C-MLP represents a convolutional multi-layer perceptron;

[0032] The specific calculation process is: the feature from the lower layer is recorded as F l , The features from the high level are denoted as F h , First, F h Upsample and align channels to match F via 1×1 convolution l , the above process can be expressed as:

[0033] F h =Conv(Up(F hwi )) (8)

[0034] Then F l and F h They are sent to the multi-scale strip convolution block MSSC respectively, and the input features X,X∈RC ×H×W , define MSSC as follows:

[0035] Q,K,V=MSSC(X)=QKV(SC7(X)+SC11 (X)+SC 21 (X)) (9)

[0036] Among them, QKV(X) represents three 1×1 convolutions on X to obtain three outputs, SC k represents a set of serial 1×k and k×1 strip convolutions; for F l and F h The output obtained after being fed into MSSC is expressed as follows:

[0037] Q l ,K l ,V l =MSSC(BN(F l )) (10)

[0038] Q h ,K h ,V h =MSSC(BN(F h )) (11)

[0039] After obtaining the Q, K, V matrices from low-level features and high-level features, the Q matrix is ​​cross-sent into the self-attention module to obtain SA l and SA h , and then perform residual connection to get the output F sa , where the self-attention module is expressed as:

[0040]

[0041] The above process can be expressed as:

[0042] F sa =SA(Q h ,K l ,V l )+SA(Q l ,K h ,V h )+F l +F h (13)

[0043] Finally, F sa The final output D is sent to C-MLP. The above process can be expressed as:

[0044] C_MLP(X)=Conv(DwConv 3×3 (ReLU(Conv(X)))) (14)

[0045] D=F sa +C MLP (BN(F sa )) (15)

[0046] Among them, DwConv 3×3 Represents a depth-separable convolution with a convolution kernel of 3×3;

[0047] To this end, the expression for defining CLCC is:

[0048] X=CLCC(F l ,F h ) (16)

[0049] Further, in step S2, the MGCC model obtains an output after layer-by-layer decoding and refinement, and the network includes a multi-scale output for deep supervision; the last layer of decoding SegHead is expressed as:

[0050] SegHead(X)=Conv(Up(Dropout(CBR(X)))) (17)

[0051] Among them, CBR stands for Conv 3×3 , BatchNorm, ReLU, Up represents upsampling, Conv represents 1×1 convolution, and its output channel is the number of predicted categories. For the input image X, the overall network flow is expressed as:

[0052] F1,F2,F3,F as =Backbone(X) (18)

[0053] F c1 ,F c2 ,F c3 =Concat(F1,F2,F3) (19)

[0054] F1, F2, F3 = MASG (F c1 ),MSAG(F c2 ),MSAG(F c3 ) (20)

[0055] D3=CLCC(F3,F as ) (twenty one)

[0056] D2=CLCC(F2,D3) (22)

[0057] D1=CLCC(F1,D2) (23)

[0058] The output is expressed as:

[0059] R,R1,R2,R3=SegHead(D1),Conv(D1),Conv(D2),Conv(D3) (24)

[0060] Furthermore, in step S3, the loss function is a combination of a cross entropy loss function and a dice loss function, and supervised training is performed on R, R1, R2, and R3 during training.

[0061] The beneficial effects of the present invention are:

[0062] In response to the needs of building extraction from remote sensing images, the model's ability to capture building semantic information is improved by optimizing the multi-scale feature fusion and decoding process. Specifically, the following innovative modules and their optimization effects are included:

[0063] (1) Multi-scale interaction guidance module (MSAG): This module aims to shorten the semantic gap between multi-level features and embed multi-scale information into the features. In remote sensing image scenes, buildings are diverse in scale and morphology. The MSAG module interacts with features across scales and effectively fuses information of different scales through position and space interaction, shortens the semantic information of different levels, and improves the model's ability to recognize building boundaries and shapes, thereby improving the extraction accuracy of buildings.

[0064] (2) Cross-layer Cross-Decoding Module (CLCC): In order to optimize the context fusion of the decoder, the present invention designs a CLCC module, which performs context fusion through cross-layer cross-attention, so that the model can effectively distinguish the semantic information of buildings and background. The CLCC module transmits and refines semantic features layer by layer during the decoding process, so that the model focuses on the semantic features of similar buildings, while suppressing background information with large semantic differences from buildings. The layer-by-layer cross-decoding mechanism of this module ensures the maintenance of semantic consistency, making the building extraction results more refined, which is particularly suitable for complex remote sensing image scenes.

[0065] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0067] Figure 1 This is the overall architecture diagram of the MGCC model;

[0068] Figure 2 This is the MSAG architecture diagram;

[0069] Figure 3 This is a detail picture of MSAG;

[0070] Figure 4 This is the CLCC architecture diagram;

[0071] Figure 5 This is the MSSC architecture diagram;

[0072] Figure 6 The following are the prediction results for the Massachusetts dataset and the WHU dataset. DETAILED DESCRIPTION

[0073] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and the features in the embodiments can be combined with each other without conflict.

[0074] See also Figure 1 to Figure 6 ,This paper proposes a network structure based on multi-scale interaction and cross decoding (MGCC-Net), and the overall architecture is as Figure 1 As shown in the figure, it includes a multi-scale interactive guidance module (MSAG) and a cross-layer cross decoding module (CLCC). The network adopts an encoder-decoder architecture. The encoder part adopts the BFBVit proposed in BuildFormer to capture global multi-scale information. At the same time, ASPP (Atrous Spatial Pyramid Pooling) is added to the last layer of feature extraction to form the Backbone of the present invention. The output features are F1, F2, F3, F as . It is expressed as follows:

[0075] F1,F2,F3,F as =Backbone(X) (1)

[0076] (1) MSAG (Multi-scale interactive guidance module)

[0077] After the encoder extracts features, the semantic gap between multi-level features is shortened by the Multi-Scale Interactive Guidance Module (MSAG), and multi-scale information is embedded into the features. This module mainly consists of three parts: ① Multi-Scale Fusion; ② Position Interactive; ③ Channel Interactive. The overall architecture is as follows Figure 2 , the details are as follows Figure 3 .

[0078] ①Multi-scale fusion

[0079] For the extracted features Splice it together Represents the features of the i-th layer concatenation, and then Compress the channel into C through 1×1 convolution i (i=1,2,3), and feed the compressed features into three parallel depth-wise separable convolutions with kernel sizes of 3×3, 5×5, and 7×7 respectively. The parallel output results are added to obtain the multi-scale fusion result The process is described as:

[0080] FMS i =B(Conv 3×3 (F ci ))+B(Conv 5×5 (F ci ))+B(Conv 7×7 (F ci )) (2)

[0081] Where B(·) represents BatchNorm. Conv k×k Represents a convolution operation with a convolution kernel of k. This operation is performed on the first three layers of the encoder to obtain three multi-scale embedded features FMS1, FMS2, and FMS3.

[0082] ②Position Interactive

[0083] After multi-scale fusion, spatial position refinement is then performed. First, the input feature FMS is fused horizontally and vertically. i Apply global average pooling and concatenate the output feature maps to generate The process is described as follows:

[0084] F hwi =[GAP h (FMS i ),GAP w (FMS i )] (3)

[0085] Among them, [,] represents the concatenation operation, and then 1×1 convolution is applied to reduce the channel to obtain Capturing information from the horizontal and vertical coordinates, r is a multiple of the channel reduction, and the process is expressed as:

[0086]

[0087] Among them, BR(·) represents BatchNorm and ReLU operations, and Conv represents 1×1 convolution. Then, for F rhwi Split horizontally and vertically to get F hi and F wi , and for F hi and F wi Apply 1×1 convolution to restore the channels to C i Then perform Sigmod function activation to obtain the directional weights W along the width and height dimensions hi and W wi , and finally the weight is combined with FMS i Multiply them to get the attention feature F with spatial position information interaction pi , the process can be expressed as:

[0088] F pi =FMS i *σ(Conv(F hi ))*σ(Conv(F wi )) (5)

[0089] Among them, σ represents the Sigmoid activation function.

[0090] ③Channel Interactive

[0091] In the first step of multi-scale fusion, the channel is compressed by 1×1, resulting in a certain degree of loss of information from each layer. This shortcoming is compensated by channel interaction. First, global average pooling is used, and 1×1 convolution and ReLU activation are performed to generate channel attention features, adaptively filter out redundant channel features, and then the channel is expanded to the output size through 1×1 convolution and interacted with the attention feature F with spatial position information. pi Multiply them together to get the complete multi-scale interactive output F mpci The process can be described as:

[0092] F mpci =F pi *Conv(ReLU(Conv(GAP(FMS i )))) (6)

[0093] To this end, a MSAG is defined as follows:

[0094] X=MSAG(X) (7)

[0095] (2) CLCC (cross-layer cross decoding module)

[0096] CLCC architecture, such as Figure 4As shown in the figure, the features from the low and high layers are fed into the multi-scale strip volume for multi-scale alignment, and the relevant Q, K, V features are generated. They are then cross-fed into the multi-head self-attention to capture the similar semantic information of the building and perform global long-range dependency perception. At the same time, in order to enhance the ability to pay attention to local details, a C-MLP module is added.

[0097] The specific process is as follows:

[0098] The features from the lower layer are denoted as F l , The features from the high level are denoted as F h , First, F h Upsample and align channels to match F via 1×1 convolution l The process is described as:

[0099] F h =Conv(Up(F hwi )) (8)

[0100] Then F l and F h They are sent to the multi-scale strip convolution block (MSSC as Figure 5 ), for the input feature X,X∈R C ×H×W , define MSSC as follows:

[0101] Q,K,V=MSSC(X)=QKV(SC7(X)+SC 11 (X)+SC 21 (X)) (9)

[0102] Among them, QKV(X) represents three 1×1 convolutions on X to obtain three outputs, SC k represents a set of serial 1×k and k×1 strip convolutions. l and F h The output obtained after being fed into MSSC is expressed as follows:

[0103] Q l ,K l ,V l =MSSC(BN(F l )) (10)

[0104] Q h ,K h ,V h =MSSC(BN(F h )) (11)

[0105] After obtaining the Q, K, V matrices from low-level features and high-level features, the Q matrix is ​​cross-sent into the self-attention module to obtain SA l and SA h , and then proceed as follows Figure 4 The residual connection shown above gives the output F sa , where the self-attention module is expressed as:

[0106]

[0107] The process is described as:

[0108] F sa =SA(Q h ,K l ,V l )+SA(Q l ,K h ,V h )+F l +F h (13)

[0109] Finally, F sa The final output D is obtained by feeding it into C-MLP. The process is described as:

[0110] C_MLP(X)=Conv(DwConv 3×3 (ReLU(Conv(X)))) (14)

[0111] D=F sa +C MLP (BN(F sa )) (15)

[0112] Among them, DwConv 3×3 It represents a depth-wise separable convolution with a convolution kernel of 3×3.

[0113] To this end, a CLCC is defined as follows:

[0114] X=CLCC(F l ,F h ) (16)

[0115] (3) Overall output

[0116] After layer-by-layer decoding and refinement, the output is obtained, and the network contains a multi-scale output for deep supervision.

[0117] The last layer of decoding SegHead is expressed as:

[0118] SegHead(X)=Conv(Up(Dropout(CBR(X)))) (17)

[0119] Among them, CBR stands for Conv 3×3 , BatchNorm, ReLU, Up represents upsampling, Conv represents 1×1 convolution, and its output channel is the number of predicted categories. For the input image X, the overall network flow is expressed as:

[0120] F1,F2,F3,F as =Sackbone(X) (18)

[0121] F c1 ,F c2 ,F c3 =Concat(F1,F2,F3) (19)

[0122] F1, F2, F3 = MSAG (F c1 ),MSAG(F c2 ),MSAG(F c3 ) (20)

[0123] D3=CLCC(F3,F as ) (twenty one)

[0124] D2=CLCC(F2,D3) (22)

[0125] D1=CLCC(F1,D2) (23)

[0126] The output is expressed as:

[0127] R,R1,R2,R3=SegHead(D1),Conv(D1),Conv(D2),Conv(D3) (24)

[0128] During training, supervised training is performed on R, R1, R2, and R3.

[0129] The network model training is implemented as follows:

[0130] (1) Loss function: A combination of the cross entropy loss function and the dice loss function is used. The weight coefficients of different levels are initially 1, and deep supervision is implemented on R, R1, R2, and R3.

[0131] (2) The model was trained for 240 rounds using the AdamW optimizer and cosine restart strategy on both datasets to ensure model convergence.

[0132] (3) In the Massachusetts dataset, we cropped it to 512×512 size with an overlap ratio of 0.1 and removed the blank parts of the data.

[0133] (4) When training input, adopt data augmentation strategies, including random flipping and random rotation, to increase data diversity.

[0134] Verification experiment:

[0135] In order to evaluate the performance of the MGCC-Net network model, this experiment compares it with two existing cutting-edge methods, including: (1) BCTNet: a Transformer extraction method for dual-branch cross-fusion; (2) SDSNet: a building extraction method based on cross-layer feature information interactive filtering, which achieved better results.

[0136] The F1, IOU, Precision, and Recall of the proposed method on the WHU dataset and the Massachusetts dataset reached 0.9556, 0.9151, 0.9564, 0.955 and 0.8609, 0.7558, 0.8778, 0.8446, respectively. The prediction effect can be seen in Figure 6 .

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A remote sensing image building extraction method based on multi-scale interaction and cross decoding, characterized in that: The method specifically comprises the following steps: S1: Obtain remote sensing images and annotate them, divide them into training set, validation set and test set, and perform image data enhancement on them when images are input; S2: Construct the MGCC model, including the MSAG module and the CLCC module, where MGCC represents the building extraction network based on multi-scale interaction and cross decoding, MSAG represents the multi-scale interaction guidance module, and CLCC represents the cross-layer cross decoding module; S3: Training the MGCC model: Input the training set into the MGCC model and calculate the corresponding loss function. Then, the AdamW optimizer and the cosine restart strategy are used to iteratively train the MGCC model until the model converges. S4: Input the test set into the trained MGCC model to obtain images with only building features.

2. The remote sensing image building extraction method according to claim 1, characterized in that: In step S1, the image is enhanced, including random flipping and random rotation.

3. The remote sensing image building extraction method according to claim 1, characterized in that: In step S2, the constructed MGCC model adopts an encoder-decoder architecture. The encoder part uses the BFB Vit proposed in BuildFormer to capture global multi-scale information. At the same time, ASPP is added to the last layer of feature extraction to form the backbone of the model. The output features are F1, F2, F3, F as ; The encoder process is expressed as: F1,F2,F3,F as =Backbone(X) (1) The decoder part includes the MSAG module and the CLCC module.

4. The remote sensing image building extraction method according to claim 3, characterized in that: In step S2, the MSAG module is arranged after the encoder performs feature extraction, and is used to shorten the semantic gap between multi-level features and embed multi-scale information into the features; the MSAG module includes three parts: multi-scale fusion, position interaction and channel interaction; The multi-scale fusion is specifically: for the extracted feature F i , C represents the channel dimension of the feature map, H represents the height of the feature map, and W represents the width of the feature map. Concatenate them to get Represents the features of the i-th layer concatenation, and then Compress the channel into C through 1×1 convolution i , and feed the compressed features into three parallel depth-separable convolutions, and add the parallel output results to obtain the multi-scale fusion result The above process can be expressed as: FMS i =B(Conv 3×3 (F ci ))+B(Conv 5×5 (F ci ))+B(Conv 7×7 (F ci )) (2) Among them, B(·) represents BatchNorm; Conv k×k represents the convolution operation with convolution kernel k, which is performed on the first three layers of the encoder to obtain three multi-scale embedded features FMS1, FMS2, and FMS3; The position interaction is specifically: after multi-scale fusion, spatial position refinement is performed, firstly, the input feature FMS is respectively processed in the horizontal and vertical directions. i Apply global average pooling and concatenate the output feature maps to generate The above process is described as follows: F hwi =[GAP h (FMS i ),GAP w (FMS i )] (3) Among them, GAP h (·) represents global average pooling along the horizontal direction, GAP w (·) represents global average pooling along the vertical direction; [,] represents the concatenation operation, followed by a 1×1 convolution for channel reduction. Capturing information from the horizontal and vertical coordinates, r is a multiple of the channel reduction, and the above process can be expressed as: F rhwi =BR(Conv(F hwi )) (4) Among them, BR(·) represents BatchNorm and ReLU operations, Conv represents 1×1 convolution; then for F rhwi Split horizontally and vertically to obtain F hi and F wi , and for F hi and F wi Apply 1×1 convolution to restore the channels to C i Then perform Sigmod function activation to obtain the directional weights W along the width and height dimensions hi and W wi , and finally the weight is combined with FMS i Multiply them to get the attention feature F with spatial position information interaction pi , the above process can be expressed as: F pi =FMS i *σ(Conv(F hi ))*σ(Conv(F wi )) (5) Among them, σ represents the Sigmoid activation function; The channel interaction is specifically as follows: first, global average pooling is used, and 1×1 convolution and ReLU activation are performed to generate channel attention features, and redundant channel features are adaptively filtered out. Then, the channel is expanded to the output size through 1×1 convolution and interacted with the attention feature F with spatial position information. pi Multiply them together to get the complete multi-scale interactive output F mpci , the above process can be expressed as: F mpci =F pi *Conv(ReLU(Conv(GAP(FMS i )))) (6) To this end, the expression for defining MSAG is: X=MSAG(X) (7).

5. The remote sensing image building extraction method according to claim 4, characterized in that: In step S2, the structure of the CLCC module is: the features from the low-level and high-level layers are fed into the multi-scale strip volume, multi-scale alignment is performed, and related Q, K, V features are generated, and then cross-fed into the multi-head self-attention to capture the similar semantic information of the building and perform global long-range dependency perception. At the same time, in order to enhance the ability to pay attention to local details, a C-MLP module is added, where C-MLP represents a convolutional multi-layer perceptron; The specific calculation process is: the feature from the lower layer is recorded as F l , The features from the high level are denoted as F h , First, F h Upsample and align channels to match F via 1×1 convolution l , the above process can be expressed as: F h =Conv(Up(F hwi )) (8) Then F l and F h They are sent to the multi-scale strip convolution block MSSC respectively, and the input features X,X∈R C×H×W , define MSSC as follows: Q,K,V=MSSC(X)=QKV(SC7(X)+SC 11 (X)+SC 21 (X)) (9) Among them, QKV(X) represents three 1×1 convolutions on X to obtain three outputs, SC k represents a set of serial 1×k and k×1 strip convolutions; for F l and F h The output obtained after being fed into MSSC is expressed as follows: Q l ,K l ,V l =MSSC(BN(F l )) (10) Q h ,K h ,V h =MSSC(BN(F h )) (11) After obtaining the Q, K, V matrices from low-level features and high-level features, the Q matrix is ​​cross-sent into the self-attention module to obtain SA l and SA h , and then perform residual connection to get the output F sa , where the self-attention module is expressed as: The above process can be expressed as: F sa =SA(Q h ,K l ,V l )+SA(Q l ,K h ,V h )+F l +F h (13) Finally, F sa The final output D is obtained by sending it to the C-MLP convolutional multilayer perceptron. The above process can be expressed as: C_MLP(X)=Conv(DwConv 3×3 (ReLU(Conv(X)))) (14) D=F sa +C MLP (BN(F sa )) (15) Among them, DwConv 3×3 Represents a depth-separable convolution with a convolution kernel of 3×3; To this end, the expression for defining CLCC is: X=CLCC(F l ,F h ) (16)。 6. The remote sensing image building extraction method according to claim 5, characterized in that: In step S2, the MGCC model obtains an output after layer-by-layer decoding and refinement, and the network contains a multi-scale output for deep supervision; the last layer of decoding SegHead is expressed as: SegHead(X)=Conv(Up(Dropout(CBR(X)))) (17) Among them, CBR stands for Conv 3×3 , BatchNorm, ReLU, Up represents upsampling, Conv represents 1×1 convolution, and its output channel is the number of predicted categories. For the input image X, the overall network flow is expressed as: F1,F2,F3,F as =Backbone(X) (18) F c1 ,F c2 ,F c3 =Concat(F1,F2,F3) (19) F1,F2,F3=MSAG(F c1 ),MSAG(F c2 ),MSAG(F c3 ) (20) <h2 style=";text-align:left;direction:ltr">D3=CLCC(F3,F<h2 style=";text-align:left;direction:ltr"> as <h2 style=";text-align:left;direction:ltr"> ) (21) D2=CLCC(F2,D3) (22) D1=CLCC(F1,D2) (23) The output is expressed as: R, R1, R2, R3 = SegHead (D1), Conv (D1), Conv (D2), Conv (D3) (24).

7. The remote sensing image building extraction method according to claim 6, characterized in that: In step S3, the loss function is a combination of a cross entropy loss function and a dice loss function, and supervised training is performed on R, R1, R2, and R3 during training.

Images