Building cross-domain identification method coupling global context and geometric information

Abstract

The invention provides a building cross-domain identification method coupling global context and geometric information, and belongs to the technical field of building identification. Comprising the steps of determining a source domain data set and a target domain data set; training a building cross-domain extraction model 1 considering global context and geometric information; updating parameters of the building cross-domain extraction model 1 to obtain a building cross-domain extraction model 2; inputting a remote sensing image in the target domain data set into the building cross-domain extraction model 2 for prediction to obtain a target domain mask pseudo tag and a target domain AFM pseudo tag; defining a building cross-domain extraction model 2 which does not contain a double-prototype contrast learning module as a building extraction model 3, and training the building extraction model 3 to obtain a building extraction model 4; and inputting a remote sensing image in the target domain data set into the building extraction model 4 for prediction to obtain a building identification result.

Description

Technical Field

[0001] The invention belongs to the technical field of building recognition, and relates to a building cross-domain recognition method that couples global context and geometric information. Background Art

[0002] With the rapid development of remote sensing sensors and artificial intelligence technology, research institutions and researchers need to combine remote sensing data with deep learning to solve the task of identifying and extracting buildings. However, the types of buildings in urban suburbs are single, the number and area are small, making it difficult for building extraction models to learn robust and generalized knowledge from these data. In addition, due to field measurement deviations and urban renewal, there are large deviations between historical building label data and existing high-resolution remote sensing images, resulting in fewer accurate images and label pairs that can be used to train suburban building extraction models, resulting in poor building extraction results in urban suburban areas. Relatively speaking, there are relatively more buildings in urban areas, and the style types and morphological characteristics of buildings are also rich and diverse. Building samples in urban areas can help building extraction models better extract suburban buildings. However, due to factors such as regional weather, terrain or sensor imaging conditions, there are certain differences in style, morphology, spatial layout, context, etc. in remote sensing images of urban areas and suburbs, which are recorded as "domain offset". This makes it difficult for building extraction models trained based on urban remote sensing images to achieve good extraction results and generalization capabilities when applied to building extraction in urban suburbs due to domain offset.

[0003] Existing research on cross-domain building extraction can be mainly divided into pixel alignment-based, feature alignment-based and self-supervision-based cross-domain building extraction methods.

[0004] Inspired by the work of image style transfer, the pixel-aligned cross-domain building extraction method uses the method of aligning the data distribution of the source domain and the target domain dataset at the pixel level of the remote sensing image. This type of method usually consists of two independent sub-networks: an image style transfer sub-network and a building extraction sub-network. Among them, the image style transfer sub-network is usually based on some classic image generation models such as DiscoGAN, StarGAN, StyleGAN, CycleGAN, etc., to convert the style of the remote sensing image in the source domain dataset into an image similar to the style of the target domain, so as to narrow the data distribution difference between the source domain and the target domain, so that a dataset with accurate labels and the same style distribution as the target domain dataset can be obtained; then, a building extraction model, such as CNNs-based or Transformer-based semantic segmentation, is used to train building information on the converted labeled source domain dataset to learn the mapping relationship between the remote sensing image with the target domain data distribution and the building semantics. Since the converted source domain dataset and the target domain dataset have eliminated the difference in style to a certain extent, the building extraction trained on the converted source domain dataset can be used to predict buildings in the target domain images without labeled data.

[0005] The cross-domain building extraction method based on feature alignment is another popular domain adaptation method. It aims to establish a common invariant feature space between the source domain and the target domain through adversarial learning or contrastive learning. In this feature space, the source domain and the target domain remote sensing images share similar shapes, textures, semantics, and contextual information. This type of method generally consists of three modules: feature extractor, building classifier, and domain discriminator (or contrastive learning module). The feature extractor is used to extract abstract deep features from the source domain and the target domain remote sensing images and map them to the common feature space. The building classifier is used to decode the extracted invariant features pixel by pixel to output the prediction results of the building; the domain discriminator can determine whether the input feature belongs to the source domain dataset or the target domain dataset. The adversarial loss generated during the training process can promote the alignment of the source domain and the target domain features, and reduce the impact of the data distribution difference between the source domain and the target domain on the cross-domain semantic segmentation performance. In addition to the domain discriminator, contrastive learning is also a way to eliminate the data distribution difference between the source domain and the target domain. It regards the same category prototype as a positive sample and the prototypes of other categories as negative samples to achieve class-centric distribution alignment.

[0006] In the cross-domain building extraction method based on self-training, the building knowledge of the source domain dataset is directly transferred to the target domain dataset through pseudo-label learning. Specifically, this type of method includes three steps: 1) building extraction model training based on source domain remote sensing images; 2) generating high-confidence pseudo-labels of target domain remote sensing images; 3) fine-tuning the building extraction model using the target domain remote sensing images and pseudo-labels. In order to solve the problem of class imbalance in pseudo-labels, a class confidence normalization method is further introduced.

[0007] However, existing cross-domain building extraction methods often only focus on the semantic information of buildings during knowledge transfer and ignore the use of building geometry and boundary information, resulting in irregular building boundary extraction. In addition, existing CNN-based building extraction methods cannot capture the global context and long-range dependencies of images, resulting in semantic ambiguity and errors in building extraction. Summary of the invention

[0008] The present invention provides a method for cross-domain building recognition by coupling global context and geometric information, comprising the following steps:

[0009] Step 1: Based on the urban remote sensing images and suburban remote sensing images of the same area, determine the source domain dataset (X S ,Y S ) and the target domain dataset (X T );

[0010] Step 2: Using the source domain dataset (X S ,Y S ) and the target domain dataset (X T ) training a building cross-domain extraction model 1 that takes into account global context and geometric information, wherein the building cross-domain extraction model 1 includes a multi-scale global context extraction module, a gravitational field enhanced joint classification module, and a dual prototype contrast learning module;

[0011] Step 3, calculating the overall loss function L of the building cross-domain extraction model, and updating the parameters of the building cross-domain extraction model 1 according to the overall loss function L to obtain the building cross-domain extraction model 2;

[0012] Step 4: The target domain dataset (X T ) is input into the building cross-domain extraction model 2 for prediction, and the target domain mask pseudo label is obtained. and target domain AFM pseudo-labels

[0013] Step 5: define the building cross-domain extraction model 2 without the dual prototype contrast learning module as the building extraction model 3, and train the building extraction model 3 to obtain the building extraction model 4;

[0014] Step 6: The target domain dataset (X T ) is input into the building extraction model 4 for prediction, and the final target domain building mask prediction result is obtained; that is, the building recognition result is obtained.

[0015] Furthermore, in step 1, the source domain dataset (X S ,Y S ) and the target domain dataset (X T The specific process of ) is as follows:

[0016] The urban remote sensing images and suburban remote sensing images of the same area are cropped into image blocks of 512×512 size, and several urban remote sensing image blocks X are obtained respectively. S and suburban remote sensing image blocks X T ;

[0017] Block the urban remote sensing image X S The buildings in the image are labeled with polygons and converted into binary raster labels Y with a value of 0-1. S ;

[0018] Block the urban remote sensing image X S and urban building labels Y S As the source domain dataset (X S ,Y S );

[0019] Suburban remote sensing image block X T As the target domain dataset (X T ).

[0020] Furthermore, the multi-scale global context extraction module includes a hierarchical Transformer module and a multi-scale context fusion module, the hierarchical Transformer module is used to extract multi-scale context features from source domain and target domain images, and the multi-scale context fusion module is used to fuse multi-scale context information to obtain fusion features; wherein the hierarchical Transformer module includes an overlapping image embedding module and four Transformer blocks, the overlapping image embedding module is used to embed the urban remote sensing image block X S and suburban remote sensing image blocks X T Divide into image blocks of size 4×4 and perform feature embedding separately;

[0021] The gravitational field enhanced joint classification module is used to promote the model to learn the semantic information and geometric structure information of the building at the same time, and the gravitational field enhanced joint classification module includes a building mask encoding head H M 、AFM encoder head H A, AFM-enhanced joint attention module, building mask classifier and AFM classifier; Among them, the building mask encoding head H M and AFM encoder head H A Both contain two convolutional layers with a convolution kernel size of 3×3; the building mask classifier and AFM classifier both contain two convolutional layers with convolution kernel sizes of 3×3 and 1×1 respectively;

[0022] The dual prototype contrast learning module is used to eliminate the distribution difference between source domain and target domain data, and includes a building mask prototype contrast learning module and an AFM prototype contrast learning module.

[0023] Furthermore, using the source domain dataset (X S ,Y S ) and the target domain dataset (X T )The specific process of training the building cross-domain extraction model 1 is as follows:

[0024] S2.1. Block the urban remote sensing image X S and suburban remote sensing image blocks X T They are input into the multi-scale global context extraction module to obtain the source domain fusion feature F S And the target domain fusion feature F T ;

[0025] S2.2. Fusion of source domain features F S And the target domain fusion feature F T The results are input into the gravitational field enhanced joint classification module respectively to obtain the mask prediction results of the source domain buildings, the AFM prediction results of the source domain buildings, the mask prediction results of the target domain buildings, and the AFM prediction results of the target domain buildings;

[0026] S2.3. Mask prediction results based on source domain buildings AFM prediction results of source domain buildings Calculate the cross entropy loss function L seg And the L1 loss function L afm ;

[0027] S2.4. Mask prediction results of buildings in the source domain AFM prediction results of source domain buildings Mask prediction results for buildings in the target domain And the AFM prediction results of the target domain building Prototype comparative learning is performed on

[0028] Furthermore, we obtain the source domain fusion feature F S And the target domain fusion feature F T The specific process is as follows:

[0029] S2.1.1. Block X of urban remote sensing image S They are respectively input into a hierarchical Transformer module with four layers of Transformer blocks, and four source domain deep features F corresponding to four different Transformer blocks in the hierarchical Transformer module are obtained. i , where i is the number of Transformer blocks and i∈{1,2,3,4};

[0030] S2.1.2. The four source domain deep features F i They are respectively input into the four 1×1 convolutional layers of the multi-scale context fusion module to obtain four source domain features F with the same number of channels. i ′; and for these four source domain features F i ′ performs bilinear interpolation and upsamples to the same size as the source domain feature F1′ to obtain four upsampled features F i ″;

[0031] S2.1.3, the four upsampled features F i ″Splice at the channel level to obtain the source domain fusion feature F S ;

[0032] S2.1.4, using the same process as steps S2.1.1 to S2.1.3, the suburban remote sensing image block X T In the input level Transformer module and the multi-scale context fusion module, the target domain fusion feature F is obtained T .

[0033] Furthermore, the specific process of obtaining the mask prediction results of the source domain building, the AFM prediction results of the source domain building, the mask prediction results of the target domain building, and the AFM prediction results of the target domain building is as follows:

[0034] S2.2.1. Fusion of source domain features F S Input to the building mask encoding header H M and AFM encoder head H A In the source domain, the fusion feature F S Pass through the building mask encoding head H M and AFM encoder head H A After the convolution operation is performed on the convolutional layers in SM and the source domain AFM feature F SA ;

[0035] S2.2.2.2. The source domain semantic feature F SM and the source domain AFM feature FSA Input into the AFM enhanced joint attention module to obtain the source domain fusion feature F that combines architectural semantic information and geometric information S ' M ;

[0036] S2.2.3. Fusion of source domain building features F S ' M Input into the building mask classifier to obtain the mask prediction result of the source domain building At the same time, the source domain AFM feature F SA Input into the AFM classifier to obtain the AFM prediction results of the source domain buildings

[0037] S2.2.4, using the same process as S2.2.1, S2.2.2 and S2.2.3, based on the target domain fusion feature F T Get the mask prediction result of the target domain building And the AFM prediction results of the target domain building

[0038] Furthermore, the mask prediction results of the source domain buildings in S2.4 are AFM prediction results of source domain buildings Mask prediction results for buildings in the target domain And the AFM prediction results of the target domain building The specific process of prototype contrast learning includes two steps: prototype updating and prototype contrast learning.

[0039] Furthermore, the specific process of prototype update is as follows:

[0040] S2.4.1. Mask prediction results using source domain buildings AFM prediction results of source domain buildings Mask prototype P M and AFM prototype P A Initialize them separately;

[0041] S2.4.2. Use the exponential moving average scheme to transform the mask prediction results of the source domain buildings AFM prediction results of source domain buildings Mask prediction results for the target domain And AFM prediction results Integrated into the mask prototype P M and AFM prototype P A In the prototype update is completed.

[0042] Furthermore, the specific process of prototype contrast learning is as follows:

[0043] S2.4.3. Calculate the mask prediction results of source domain buildings separately With the mask prototype P M Similarities between And the AFM prediction results of the source domain buildings With AFM prototype P A Similarities between

[0044] S2.4.4. Calculate the mask prediction results of the target domain buildings separately With the mask prototype P M Similarities between And the AFM prediction results of the target domain building With AFM prototype P A Similarities between

[0045] S2.4.5. Minimization And the mask prediction results of the source domain buildings The cross entropy loss function between and minimize AFM prediction results of buildings in the source domain The cross entropy loss between

[0046] minimize And the mask prediction results of the target domain buildings The cross entropy loss between and minimize AFM prediction results of target domain buildings The cross entropy loss between

[0047] S2.4.6. Calculate the contrast loss L C , and perform contrastive learning on the updated prototype based on contrastive loss.

[0048] Furthermore, the specific process of obtaining the building cross-domain extraction model 2 is as follows:

[0049] S3.1. For each layer of the building cross-domain extraction model 1, the gradient of its parameters is calculated according to the overall loss function L, and the corresponding parameters in the building cross-domain extraction model 1 are updated one by one according to the gradient direction of the parameter gradient calculated for each layer;

[0050] S3.2, looping the process of S3.1 until all parameters in the building cross-domain extraction model 1 are converged, and a trained building cross-domain extraction model is obtained, that is, a building cross-domain extraction model 2 is obtained;

[0051] S3.3. Save the building cross-domain extraction model 2.

[0052] Furthermore, the specific process of obtaining the building extraction model 4 is as follows:

[0053] S5.1. Using target domain masked pseudo-labels and target domain AFM pseudo-labels Train the building extraction model 3 and calculate the loss function L self ; and use the loss function L self Calculate the gradient of each parameter in the building extraction model 3, and update the corresponding parameters in the building extraction model 3 one by one according to the gradient direction of the calculated gradient of each parameter;

[0054] S5.2, repeat the process of S5.1 until all parameters in the building extraction model 3 converge, and obtain a trained building cross-domain extraction model, that is, obtain a building extraction model 4;

[0055] S5.3. Save the building extraction model 4.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] (1) The present invention proposes a Transformer-based building cross-domain extraction model to extract and transmit the global context information of the building, and adopts a multi-level context fusion module to fuse the multi-level context information of the building, thereby enhancing the building cross-domain extraction model's ability to extract global context information, thereby ensuring the accuracy of building recognition.

[0058] (2) This paper utilizes the hierarchical supervision signals of the regional gravity field (AFM) and the mask to guide the cross-domain building extraction model to transfer building knowledge in a geometrically aware manner, and proposes an AFM-enhanced joint attention module to adaptively embed the AFM into the multi-layer contextual information of the building to enhance the regularity of the building extraction results.

[0059] (3) The present invention proposes a dual prototype contrastive learning strategy, which performs prototype contrastive learning on the mask prediction results and the AFM prediction results respectively, so as to bridge the differences in semantic and geometric information between the source domain and the target domain images.

[0060] In addition to the above-described purposes, features and advantages, the present invention has other purposes, features and advantages. The present invention will be further described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The drawings constituting a part of this application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0062] Figure 1 It is a flow chart of a method for cross-domain identification of buildings coupling global context and geometric information in an embodiment of the present invention;

[0063] Figure 2 is a schematic diagram of a cross-domain building extraction network taking into account global context and geometric information in an embodiment of the present invention;

[0064] Figure 3 is a structural diagram of the HT module in an embodiment of the present invention. DETAILED DESCRIPTION

[0065] In order to make the above-mentioned purposes, features and advantages of the present invention more clear and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings. It should be noted that the drawings of the present invention are all simplified and use non-precise proportions, which are only used to conveniently and clearly assist in explaining the implementation of the present invention; the "number" mentioned in the present invention is not limited to the specific number in the examples in the accompanying drawings; the directions or positional relationships indicated by "front", "middle", "back", "left", "right", "up", "down", "top", "bottom", "middle", etc. mentioned in the present invention are based on the directions or positional relationships shown in the drawings of the present invention, and do not indicate or imply that the devices or components referred to must have a specific direction, nor can they be understood as limitations on the present invention.

[0066] Example:

[0067] See also Figures 1 to 3 As shown, the present invention provides a method for cross-domain building recognition that couples global context and geometric information, comprising the following steps:

[0068] Step 1: Determine the source domain dataset (X S ,Y S ) and the target domain dataset (X T ); the specific process is as follows:

[0069] The urban aerial high-resolution remote sensing images and suburban aerial high-resolution remote sensing images of a certain area are cropped into image blocks of 512*512 size, and several urban remote sensing image blocks X are obtained respectively. S and suburban remote sensing image blocks X T ;

[0070] Block the urban remote sensing image X S The buildings in the image are labeled with polygons and converted into binary raster labels Y with a value of 0 (background) - 1 (building). S ;

[0071] Block the urban remote sensing image X S and urban building labels Y S As the source domain dataset (X S ,Y S );

[0072] Suburban remote sensing image block X T As the target domain dataset (X T ).

[0073] Step 2: Using the source domain dataset (X S ,Y S ) and the target domain dataset (X T ) Train a cross-domain building extraction model 1 that takes into account global context and geometric information;

[0074] The building cross-domain extraction model 1 includes a multi-scale global context extraction module, a gravitational field enhanced joint classification module and a dual prototype contrast learning module;

[0075] The multi-scale global context extraction module includes a hierarchical Transformer module and a multi-scale context fusion module. The hierarchical Transformer module is used to extract multi-scale context features from source domain and target domain images, and the multi-scale context fusion module is used to fuse multi-scale context information to obtain fusion features. The hierarchical Transformer module includes an overlapping image embedding module and four Transformer blocks. The overlapping image embedding module is used to embed the urban remote sensing image block X S and suburban remote sensing image blocks X T Divide into image blocks of size 4×4 and perform feature embedding separately;

[0076] The gravitational field enhanced joint classification module is used to promote the model to learn the semantic information and geometric structure information of the building at the same time, and the gravitational field enhanced joint classification module includes a building mask encoding head H M 、AFM encoder head H A , AFM-enhanced joint attention module, building mask classifier and AFM classifier; Among them, the building mask encoding head H M and AFM encoder head H A Both contain two convolutional layers with a convolution kernel size of 3×3; the building mask classifier and AFM classifier both contain two convolutional layers with convolution kernel sizes of 3×3 and 1×1 respectively;

[0077] The dual prototype contrast learning module is used to eliminate the distribution difference between source domain and target domain data, and includes a building mask prototype contrast learning module and an AFM prototype contrast learning module.

[0078] Furthermore, using the source domain dataset (X S ,Y S ) and the target domain dataset (X T )The specific process of training the building cross-domain extraction model 1 is as follows:

[0079] S2.1. Block the urban remote sensing image X S and suburban remote sensing image blocks X T They are input into the multi-scale global context extraction module to obtain the source domain fusion feature F S And the target domain fusion feature F T The specific process is as follows:

[0080] S2.1.1. Block X of urban remote sensing image S They are respectively input into a hierarchical Transformer module with four layers of Transformer blocks, and four source domain deep features F corresponding to four different Transformer blocks in the hierarchical Transformer module are obtained. i , these features provide not only high-resolution coarse-grained features, but also low-resolution fine-grained features. Where i is the number of Transformer blocks, and i∈{1,2,3,4}.

[0081] S2.1.2. Four source domain deep features F i They are respectively input into the four 1×1 convolutional layers of the multi-scale context fusion module to obtain four source domain features F with the same number of channels. i ′; and for these four source domain features F i ′ performs bilinear interpolation and upsamples to the same size as the source domain feature F1′ to obtain four upsampled features F i ″;

[0082] S2.1.3, the four upsampled features F i ″Splice at the channel level to obtain the source domain fusion feature F S .

[0083] S2.1.4, using the same process as steps S2.1.1 to S2.1.3, the suburban remote sensing image block X T In the input level Transformer module and the multi-scale context fusion module, the target domain fusion feature F is obtained T .

[0084] S2.2. Fusion of source domain features F S And the target domain fusion feature F TThey are input into the joint classification module with gravitational field enhancement respectively to obtain the mask prediction results of source domain buildings, the AFM prediction results of source domain buildings, the mask prediction results of target domain buildings, and the AFM prediction results of target domain buildings. The specific process is as follows:

[0085] S2.2.1. Fusion of source domain features F S Input to the building mask encoding header H M and AFM encoder head H A In the source domain, the fusion feature F S Pass through the building mask encoding head H M and AFM encoder head H A After the convolution operation is performed on the convolutional layers in SM and the source domain AFM feature F SA ;

[0086] S2.2.2.2. The source domain semantic feature F SM and the source domain AFM feature F SA Input into the AFM enhanced joint attention module to obtain the source domain fusion feature F that combines architectural semantic information and geometric information S ' M The specific process is as follows:

[0087] The source domain semantic feature F SM They are respectively input into the two 1×1 convolutional layers in the AFM enhanced joint attention module to obtain the source domain semantic features V∈R C′×H×W and source domain semantic feature K∈R C′×H×W ;

[0088] The source domain AFM feature F SA Input into one of the 1×1 convolutional layers in the AFM-enhanced joint attention module to obtain the source domain AFM feature Q∈R C′×H×W ;

[0089] The source domain semantic feature K∈R C′×H×W Reshape the vector into R C′×HW Size, source domain AFM feature Q∈R C′×H×W Reshape the vector into R HW×C′ Size and matrix multiplication are performed to obtain the attention affinity feature S∈R C′×C′ ;

[0090] The attention affinity feature S∈R C′×C′ and the source domain semantic feature V∈R C′×H×W Perform matrix multiplication to obtain the source domain building fusion feature F that aggregates the building semantic information and geometric information S ' M ∈RC′×H×W ; Source domain building fusion feature F S ' M The feature at each position in the H×W dimension represents the attention affinity feature S∈R C′×C′ and source domain semantic feature V∈R C′×H×W The weighted value of , which can take into account the semantic information and geometric structure information of the building, thereby improving the building extraction performance of the classifier;

[0091] S2.2.3. Fusion of source domain building features F S ' M Input into the building mask classifier to obtain the mask prediction result of the source domain building At the same time, the source domain AFM feature F SA Input into the AFM classifier to obtain the AFM prediction results of the source domain buildings

[0092] S2.2.4, using the same process as S2.2.1, S2.2.2 and S2.2.3, based on the target domain fusion feature F T Get the mask prediction result of the target domain building And the AFM prediction results of the target domain building

[0093] S2.3. Mask prediction results based on source domain buildings AFM prediction results of source domain buildings Calculate the cross entropy loss function L seg And the L1 loss function L afm .

[0094] Cross entropy loss function L seg The expression is as follows:

[0095]

[0096] And L1 loss function L afm The expression is as follows:

[0097]

[0098] Among them, N s is the number of images in the source domain dataset; A S is the true value label of AFM, n={1,2,…,N S}.

[0099] S2.4. Mask prediction results of buildings in the source domain AFM prediction results of source domain buildings Mask prediction results for buildings in the target domain And the AFM prediction results of the target domain building Prototype contrast learning is performed on the source domain building mask prediction results and the mask prototype, the distance between the target domain building mask prediction results and the mask prototype, the distance between the source domain building AFM prediction results and the AFM prototype, and the distance between the target domain building AFM prediction results and the AFM prototype, guiding the model to close the distribution difference between the source domain dataset and the target domain data at the semantic level and the geometric level respectively. Specifically, it mainly includes two steps: prototype updating and prototype contrast learning.

[0100] The specific process of prototype update is as follows:

[0101] S2.4.1. Mask prediction results using source domain buildings AFM prediction results of source domain buildings Mask prototype P M and AFM prototype P A Initialize them separately;

[0102] P M The expression is as follows:

[0103]

[0104] P A The expression is as follows:

[0105]

[0106] Where: k is the index value of the building category 1; is the indicator function, only when hour, otherwise,

[0107] S2.4.2. Use the exponential moving average scheme to transform the mask prediction results of the source domain buildings AFM prediction results of source domain buildings Mask prediction results for the target domain And AFM prediction results Integrated into the mask prototype P M and AFM prototype P A The specific process is as follows: Through the lever moving average method, the mask prototype P M and AFM prototype P A Update them respectively to get the updated mask prototype P′ M and AFM prototype P′ A ;

[0108] P′ M The expression is as follows:

[0109]

[0110] P A The expression of ′ is as follows:

[0111]

[0112] in, is the source domain mask pseudo label of the kth category; is the source domain AFM pseudo label of the kth category; m is the momentum, and the polynomial annealing policy is used to update m to obtain m t .

[0113] m t The expression is as follows:

[0114]

[0115] t∈[0,T];

[0116] Among them, m t is the m value at the tth iteration, T is the total number of training iterations, and α and m0 are both default parameters (in the present invention, α and m0 are both taken as 0.9).

[0117] The specific process of prototype contrast learning is as follows:

[0118] S2.4.3. Calculate the mask prediction results of source domain buildings separately With the mask prototype P M Similarities between And the AFM prediction results of the source domain buildings With AFM prototype P A Similarities between

[0119] The expression is as follows:

[0120]

[0121] The expression is as follows:

[0122]

[0123] S2.4.4. Calculate the mask prediction results of the target domain buildings separately With the mask prototype P M Similarities between And the AFM prediction results of the target domain building With AFM prototype P A Similarities between

[0124] The expression is as follows:

[0125]

[0126] The expression is as follows:

[0127]

[0128] in, is the mask prototype of the kth category, is the AFM prototype of the kth category, τ is the temperature parameter that controls the sharpness of the generated probability distribution, and K is the number of prototype categories.

[0129] S2.4.5. Minimization And the mask prediction results of the source domain buildings The cross entropy loss function between and minimize AFM prediction results of buildings in the source domain The cross entropy loss between

[0130] The expression is as follows:

[0131]

[0132] The expression is as follows:

[0133]

[0134] minimize Mask prediction results of buildings in the target domain The cross entropy loss between and minimize AFM prediction results of target domain buildings The cross entropy loss between

[0135] The expression is as follows:

[0136]

[0137] The expression is as follows:

[0138]

[0139] in, is the mask label of the kth category of the source domain dataset, is the AFM label of the kth category of the source domain dataset, Predict the label for the mask of the kth category of the target domain dataset, Predict labels for the AFM of the kth category in the target domain dataset.

[0140] S2.4.6. Calculate the contrast loss L C , and perform contrastive learning on the updated prototype based on contrastive loss;

[0141] L C The expression is as follows:

[0142]

[0143] Step 3: Calculate the overall loss function L of the building cross-domain extraction model, and update the parameters of the building cross-domain extraction model 1 according to the overall loss function L to obtain the building cross-domain extraction model 2; the specific process is as follows:

[0144] S3.1. For each layer of the building cross-domain extraction model 1, the gradient of its parameters is calculated according to the overall loss function L, and the corresponding parameters in the building cross-domain extraction model 1 are updated one by one according to the gradient direction of the parameter gradient calculated for each layer;

[0145] The expression of L is as follows:

[0146] L=L C +L seg +L afm ;

[0147] S3.2, looping the process of S3.1 until all parameters in the building cross-domain extraction model 1 are converged, and a trained building cross-domain extraction model is obtained, that is, a building cross-domain extraction model 2 is obtained;

[0148] S3.3. Save the building cross-domain extraction model 2.

[0149] Step 4: The target domain dataset (X T ) is input into the building cross-domain extraction model 2 for prediction, and the target domain mask pseudo label is obtained. and target domain AFM pseudo-labels Specifically, the target domain dataset (X T ) is processed by the multi-scale global context extraction module and the gravitational field enhanced joint classification module, and finally the mask classifier outputs the target domain mask pseudo label The target domain AFM pseudo label is output by the AFM classifier

[0150] Step 5: define the building cross-domain extraction model 2 without the dual prototype contrast learning module as the building extraction model 3, and train it to obtain the building extraction model 4. The specific process is as follows:

[0151] S5.1. Using target domain masked pseudo-labels and target domain AFM pseudo-labels Train the building extraction model 3 and calculate the loss function L self ; and use the loss function L self Calculate the gradient of each parameter in the building extraction model 3, and update the corresponding parameters in the building extraction model 3 one by one according to the gradient direction of the calculated gradient of each parameter;

[0152] Loss function L self The expression is as follows:

[0153] L self =L seg-self +L afm-self

[0154]

[0155] Among them, L seg-self is the cross entropy loss L for mask prediction seg , L afm-self L is the L1 loss function afm , N t is the number of remote sensing images in the target domain,

[0156] S5.2, repeat the process of S5.1 until all parameters in the building extraction model 3 converge, and obtain a trained building cross-domain extraction model, that is, obtain a building extraction model 4;

[0157] S5.3. Save the building extraction model 4.

[0158] Step 6: The target domain dataset (X T ) is input into the building extraction model 4 for prediction, and the final target domain building mask prediction result is obtained; that is, the building recognition result is obtained. Preferably, the target domain dataset (X T The specific process of inputting the remote sensing image in ) into the building extraction model 4 for prediction refers to the specific process shown in step 4.

[0159] Experimental example:

[0160] 1) Quantitative analysis

[0161] In order to verify the effectiveness of the proposed method, a comparative experiment was conducted on the urban→suburban cross-domain (C2S) task in the same area, comparing the pure source domain model with eight currently most advanced domain adaptation semantic segmentation methods (AdaptSegNet, AdvEnt, FADA, SDCA, ProCA, MDANet, DSSF and DAFormer). The experimental results are shown in Table 1.

[0162] As can be seen from Table 1, due to the differences in spatial resolution, spatial layout, and building style between urban and suburban areas, the Source-only model achieved the worst building extraction performance, with scores of 60.65%, 43.52%, and 59.38% in F1, IoU, and Acc, respectively. Compared with the Source-only model, the eight UDA comparison methods improved the performance of building extraction to varying degrees by adopting different domain adaptation strategies to eliminate the data distribution differences between the source and target domains and making full use of the high-resolution remote sensing images in the unlabeled target domain dataset. For example, the worst performing comparison method, AdaptSegNet, only obtained scores of 68.95%, 52.62%, and 68.25% in F1, IoU, and Acc, which were only 8.30%, 9.10%, and 8.87% higher than the Source-only model. DAFormer, the best performing method among the comparison methods, achieved 82.12%, 69.66% and 77.55% in F1, IoU and Acc indicators on this task respectively, due to considering the contextual information of the remote sensing scene. Compared with the source-only model, it achieved 21.47%, 26.14% and 18.17% score improvement in F1, IoU and Acc indicators respectively. Furthermore, the model DCANet proposed in this chapter achieved 86.47%, 76.17% and 85.33% in F1, IoU and Acc indicators on the C2S task respectively by considering the multi-scale contextual information of the remote sensing scene and the geometric structure information of the building. Compared with the source-only model, it achieved 25.82%, 32.65% and 25.95% score improvement in F1, IoU and Acc indicators respectively. Compared with the AdaptSegNet model in the comparison method, the scores of F1, IoU and Acc indicators were improved by 17.52%, 23.55% and 17.08% respectively. Compared with the DAFormer model in the comparison method, the scores of F1, IoU and Acc indicators were improved by 4.25%, 6.51% and 7.78% respectively.

[0163] Table 1: Quantitative evaluation results of different methods on the urban→suburban cross-domain (C2S) task

[0164]

[0165]

[0166] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A cross-domain building recognition method that couples global context and geometric information, characterized in that: The following steps are involved: Step 1: Based on the urban remote sensing images and suburban remote sensing images of the same area, determine the source domain dataset (X S ,Y S ) and the target domain dataset (X T ); Step 2: Using the source domain dataset (X S ,Y S ) and the target domain dataset (X T ) training a building cross-domain extraction model 1 that takes into account global context and geometric information, wherein the building cross-domain extraction model 1 includes a multi-scale global context extraction module, a gravitational field enhanced joint classification module, and a dual prototype contrast learning module; Step 3, calculating the overall loss function L of the building cross-domain extraction model, and updating the parameters of the building cross-domain extraction model 1 according to the overall loss function L to obtain the building cross-domain extraction model 2; Step 4: The target domain dataset (X T ) is input into the building cross-domain extraction model 2 for prediction. Get the target domain mask pseudo label Y T and the target domain AFM pseudo label A T ; Step 5: define the building cross-domain extraction model 2 without the dual prototype contrast learning module as the building extraction model 3, and train the building extraction model 3 to obtain the building extraction model 4; Step 6: The target domain dataset (X T ) is input into the building extraction model 4 for prediction, and the final target domain building mask prediction result is obtained; that is, the building recognition result is obtained.

2. The cross-domain building recognition method coupling global context and geometric information according to claim 1 is characterized in that: In step 1, the source domain dataset (X S ,Y S ) and the target domain dataset (X T The specific process of ) is as follows: The urban remote sensing images and suburban remote sensing images of the same area are cropped into image blocks of 512×512 size, and several urban remote sensing image blocks X are obtained respectively. S and suburban remote sensing image blocks X T ; Block X of urban remote sensing image S The buildings in the image are labeled with polygons and converted into binary raster labels Y with a value of 0-1. S ; Block X of urban remote sensing image S and urban building labels Y S As the source domain dataset (X S ,Y S ); Suburban remote sensing image block X T As the target domain dataset (X T ).

3. The cross-domain building recognition method coupling global context and geometric information according to claim 2 is characterized in that: The multi-scale global context extraction module includes a hierarchical Transformer module and a multi-scale context fusion module. The hierarchical Transformer module is used to extract multi-scale context features from source domain and target domain images. The multi-scale context fusion module is used to fuse multi-scale context information to obtain fusion features. The hierarchical Transformer module includes an overlapping image embedding module and four Transformer blocks. The overlapping image embedding module is used to embed the urban remote sensing image block X S and suburban remote sensing image blocks X T Divide into image blocks of size 4×4 and perform feature embedding separately; The gravitational field enhanced joint classification module is used to promote the model to learn the semantic information and geometric structure information of the building at the same time, and the gravitational field enhanced joint classification module includes a building mask encoding head H M 、AFM encoder head H A , AFM-enhanced joint attention module, building mask classifier and AFM classifier; Among them, the building mask encoding head H M and AFM encoder head H A Both contain two convolutional layers with a convolution kernel size of 3×3; the building mask classifier and AFM classifier both contain two convolutional layers with convolution kernel sizes of 3×3 and 1×1 respectively; The dual prototype contrast learning module is used to eliminate the distribution difference between source domain and target domain data, and includes a building mask prototype contrast learning module and an AFM prototype contrast learning module.

4. The method for cross-domain building recognition by coupling global context and geometric information according to claim 3, characterized in that: Using the source domain dataset (X S ,Y S ) and the target domain dataset (X T )The specific process of training the building cross-domain extraction model 1 is as follows: S2.

1. Block X of urban remote sensing image S and suburban remote sensing image blocks X T They are input into the multi-scale global context extraction module to obtain the source domain fusion feature F S And the target domain fusion feature F T ; S2.

2. Fusion of source domain features F S And the target domain fusion feature F T The results are input into the gravitational field enhanced joint classification module respectively to obtain the mask prediction results of the source domain buildings, the AFM prediction results of the source domain buildings, the mask prediction results of the target domain buildings, and the AFM prediction results of the target domain buildings; S2.

3. Mask prediction results based on source domain buildings AFM prediction results of source domain buildings Calculate the cross entropy loss function L seg And the L1 loss function L afm ; S2.

4. Mask prediction results of buildings in the source domain AFM prediction results of source domain buildings Mask prediction results for buildings in the target domain And the AFM prediction results of the target domain building Prototype comparative learning is performed on .

5. The method for cross-domain building recognition by coupling global context and geometric information according to claim 4, characterized in that: Get the source domain fusion feature F S And the target domain fusion feature F T The specific process is as follows: S2.1.

1. Block X of urban remote sensing image S They are respectively input into a hierarchical Transformer module with four layers of Transformer blocks, and four source domain deep features F corresponding to four different Transformer blocks in the hierarchical Transformer module are obtained. i , where i is the number of Transformer blocks and i∈{1,2,3,4}; S2.1.

2. The four source domain deep features F i They are respectively input into the four 1×1 convolutional layers of the multi-scale context fusion module to obtain four source domain features F with the same number of channels. i ′; and for these four source domain features F i ′ performs bilinear interpolation and upsamples to the same size as the source domain feature F1′ to obtain four upsampled features F i ″; S2.1.3, the four upsampled features F i ″Splice at the channel level to obtain the source domain fusion feature F S ; S2.1.4, using the same process as steps S2.1.1 to S2.1.3, the suburban remote sensing image block X T In the input level Transformer module and the multi-scale context fusion module, the target domain fusion feature F is obtained T .

6. The method for cross-domain building recognition coupling global context and geometric information according to claim 4 or 5, characterized in that: The specific process of obtaining the mask prediction results of source domain buildings, the AFM prediction results of source domain buildings, the mask prediction results of target domain buildings, and the AFM prediction results of target domain buildings is as follows: S2.2.

1. Fusion of source domain features F S Input to the building mask encoding header H M and AFM encoder head H A In the source domain, the fusion feature F S Pass through the building mask encoding head H M and AFM encoder head H A After the convolution operation is performed on the convolutional layers in SM and the source domain AFM feature F SA ; S2.2.2.

2. The source domain semantic feature F SM and the source domain AFM feature F SA Input into the AFM enhanced joint attention module to obtain the source domain fusion feature F that combines architectural semantic information and geometric information S ' M ; S2.2.

3. Fusion of source domain building features F S ' M Input into the building mask classifier to obtain the mask prediction result of the source domain building At the same time, the source domain AFM feature F SA Input into the AFM classifier to obtain the AFM prediction results of the source domain buildings S2.2.4, using the same process as S2.2.1, S2.2.2 and S2.2.3, based on the target domain fusion feature F T Get the mask prediction result of the target domain building And the AFM prediction results of the target domain building 7. The method for cross-domain building recognition by coupling global context and geometric information according to claim 6, characterized in that: The mask prediction results of the buildings in the source domain in S2.4 are AFM prediction results of source domain buildings Mask prediction results for buildings in the target domain And the AFM prediction results of the target domain building The specific process of performing prototype contrast learning includes prototype updating, and the specific process of the prototype updating is as follows: S2.4.

1. Mask prediction results using source domain buildings AFM prediction results of source domain buildings Mask prototype P M and AFM prototype P A Initialize them separately; S2.4.

2. Use the exponential moving average scheme to transform the mask prediction results of the source domain buildings AFM prediction results of source domain buildings Mask prediction results for the target domain And AFM prediction results Integrated into the mask prototype P M and AFM prototype P A In the prototype update is completed.

8. The method for cross-domain building recognition by coupling global context and geometric information according to claim 6, characterized in that: The mask prediction results of the buildings in the source domain in S2.4 are AFM prediction results of source domain buildings Mask prediction results for buildings in the target domain And the AFM prediction results of the target domain building The specific process of performing prototype contrast learning also includes prototype contrast learning, and the specific process of the prototype contrast learning is as follows: S2.4.

3. Calculate the mask prediction results of source domain buildings separately With the mask prototype P M Similarities between And the AFM prediction results of the source domain buildings With AFM prototype P A Similarities between S2.4.

4. Calculate the mask prediction results of the target domain buildings separately With the mask prototype P M Similarities between And the AFM prediction results of the target domain building With AFM prototype P A Similarities between S2.4.

5. Minimization And the mask prediction results of the source domain buildings The cross entropy loss function between and minimize AFM prediction results of buildings in the source domain The cross entropy loss between minimize And the mask prediction results of the target domain buildings The cross entropy loss between and minimize AFM prediction results of buildings in the target domain The cross entropy loss between S2.4.

6. Calculate the contrast loss L C , and perform contrastive learning on the updated prototype based on contrastive loss.

9. The method for cross-domain building recognition by coupling global context and geometric information according to any one of claims 1 to 8, characterized in that: The specific process of obtaining the building cross-domain extraction model 2 is as follows: S3.

1. For each layer of the building cross-domain extraction model 1, the gradient of its parameters is calculated according to the overall loss function L, and the corresponding parameters in the building cross-domain extraction model 1 are updated one by one according to the gradient direction of the parameter gradient calculated for each layer; S3.2, looping the process of S3.1 until all parameters in the building cross-domain extraction model 1 are converged, and a trained building cross-domain extraction model is obtained, that is, a building cross-domain extraction model 2 is obtained; S3.

3. Save the building cross-domain extraction model 2.

10. The cross-domain building recognition method coupling global context and geometric information according to claim 9, characterized in that: The specific process of obtaining the building extraction model 4 is as follows: S5.

1. Using target domain masked pseudo-labels and target domain AFM pseudo-labels Train the building extraction model 3 and calculate the loss function L self ; and use the loss function L self Calculate the gradient of each parameter in the building extraction model 3, and update the corresponding parameters in the building extraction model 3 one by one according to the gradient direction of the calculated gradient of each parameter; S5.2, repeat the process of S5.1 until all parameters in the building extraction model 3 converge, and obtain a trained building cross-domain extraction model, that is, obtain a building extraction model 4; S5.

3. Save the building extraction model 4.

Images