A cross-domain building recognition method coupling global context and geometric information

Abstract

The present invention provides a cross-domain building recognition method that couples global context and geometric information, and belongs to the technical field of building recognition. The method comprises the following steps: determining a source domain dataset and a target domain dataset; training a cross-domain building extraction model 1 that takes into account global context and geometric information; updating the parameters of the cross-domain building extraction model 1 to obtain a cross-domain building extraction model 2; inputting a remote sensing image in a target domain dataset into the cross-domain building extraction model 2 for prediction, and obtaining a target domain mask pseudo-label and a target domain AFM pseudo-label; defining the cross-domain building extraction model 2 that does not contain a dual prototype contrast learning module as a building extraction model 3 and training it to obtain a building extraction model 4; inputting the remote sensing image in the target domain dataset into the building extraction model 4 for prediction, and obtaining a building recognition result.

Description

Technical Field

[0001] The present invention belongs to the technical field of building recognition and relates to a cross-domain building recognition method that couples global context and geometric information. Background Art

[0002] With the rapid development of remote sensing sensors and artificial intelligence technologies, research institutions and researchers are seeking to combine remote sensing data with deep learning to address the task of building identification and extraction. However, the limited number and size of buildings in suburban areas make it difficult for building extraction models to learn robust and generalizable knowledge from this data. Furthermore, due to field measurement errors and urban renewal, there is a significant discrepancy between historical building label data and existing high-resolution remote sensing imagery. This results in a limited number of accurate image and label pairs for training suburban building extraction models, leading to poor building extraction performance in suburban areas. In contrast, urban areas have a relatively large number of buildings, with a richer variety of styles, types, and morphological features. Building samples from urban areas can help building extraction models better extract suburban buildings. However, due to factors such as regional weather, topography, and sensor imaging conditions, remote sensing images of buildings in urban and suburban areas exhibit certain differences in style, morphology, spatial layout, and context, known as "domain shift." This makes it difficult for building extraction models trained on urban remote sensing imagery to achieve good extraction performance and generalization when applied to suburban areas.

[0003] Existing research on cross-domain building extraction can be mainly divided into pixel alignment-based, feature alignment-based and self-supervision-based cross-domain building extraction methods.

[0004] Inspired by work on image style transfer, pixel-aligned cross-domain building extraction methods align the data distributions of source and target domain datasets at the pixel level of remote sensing imagery. These methods typically consist of two independent subnetworks: an image style transfer subnetwork and a building extraction subnetwork. The image style transfer subnetwork, often based on classic image generation models such as DiscoGAN, StarGAN, StyleGAN, and CycleGAN, transfers the style of the remote sensing images in the source dataset to images with a similar style to the target domain, narrowing the data distribution gap between the source and target domains. This results in a precisely labeled dataset with the same style distribution as the target dataset. Next, a building extraction model, such as a CNN-based or Transformer-based semantic segmentation model, is trained on the converted labeled source dataset to learn the mapping between remote sensing images with the target domain data distribution and building semantics. Because the converted source and target datasets eliminate some of the stylistic differences, the building extraction model trained on the converted source dataset can be used to predict buildings in unlabeled target domain images.

[0005] Feature alignment-based cross-domain building extraction methods are another popular domain adaptation method. They aim to establish a shared invariant feature space between the source and target domains through adversarial learning or contrastive learning. In this feature space, the source and target remote sensing images share similar shapes, textures, semantics, and contextual information. This type of method generally consists of three modules: a feature extractor, a building classifier, and a domain discriminator (or contrastive learning module). The feature extractor extracts abstract deep features from the source and target remote sensing images and maps them into a shared feature space. The building classifier decodes the extracted invariant features pixel by pixel and outputs building predictions. The domain discriminator determines whether the input features belong to the source or target dataset. The adversarial loss generated during training promotes alignment of source and target domain features, mitigating the impact of data distribution differences between the two domains on cross-domain semantic segmentation performance. In addition to the domain discriminator, contrastive learning is also a way to eliminate data distribution differences between the source and target domains. It achieves class-centric distribution alignment by treating prototypes of the same category as positive samples and prototypes of other categories as negative samples.

[0006] In self-training-based cross-domain building extraction methods, building knowledge from a source domain dataset is directly transferred to a target domain dataset through pseudo-label learning. Specifically, this approach involves three steps: 1) training a building extraction model based on source domain remote sensing imagery; 2) generating high-confidence pseudo-labels for target domain remote sensing imagery; and 3) fine-tuning the building extraction model using target domain remote sensing imagery and pseudo-labels. To address the class imbalance in the pseudo-labels, a class confidence normalization method is further introduced.

[0007] However, existing cross-domain building extraction methods often focus solely on the semantic information of buildings during knowledge transfer, neglecting the use of building geometry and boundary information, resulting in irregular building boundary extraction. Furthermore, existing CNN-based building extraction methods fail to capture the global context and long-range dependencies of the image, leading to semantic ambiguity and errors in building extraction. Summary of the Invention

[0008] The present invention provides a cross-domain building recognition method coupling global context and geometric information, comprising the following steps:

[0009] Step 1: Based on the remote sensing images of urban and suburban areas in the same area, determine the source domain dataset (X S ,Y S ) and the target domain dataset (X T );

[0010] Step 2: Using the source domain dataset (X S ,Y S ) and the target domain dataset (X T ) training a building cross-domain extraction model 1 that takes into account global context and geometric information, wherein the building cross-domain extraction model 1 includes a multi-scale global context extraction module, a gravitational field enhanced joint classification module, and a dual-prototype contrastive learning module;

[0011] Step 3: Calculate the overall loss function L of the building cross-domain extraction model, and update the parameters of the building cross-domain extraction model 1 according to the overall loss function L to obtain the building cross-domain extraction model 2;

[0012] Step 4: The target domain dataset (X T ) is input into the building cross-domain extraction model 2 for prediction, and the target domain mask pseudo label is obtained. and target domain AFM pseudo labels

[0013] Step 5: define the building cross-domain extraction model 2 without the dual prototype contrast learning module as the building extraction model 3, and train the building extraction model 3 to obtain the building extraction model 4;

[0014] Step 6: The target domain dataset (X T ) is input into the building extraction model 4 for prediction to obtain the final target domain building mask prediction result; that is, the building recognition result is obtained.

[0015] Furthermore, in step 1, the source domain dataset (X S ,Y S ) and the target domain dataset (X T The specific process is as follows:

[0016] The urban remote sensing images and suburban remote sensing images of the same area are cropped into image blocks of 512×512 size, and several urban remote sensing image blocks X are obtained respectively. S and suburban remote sensing image blocks X T ;

[0017] Block X of urban remote sensing image S The buildings in the image are labeled with polygons and converted into binary raster labels Y with a value of 0-1. S ;

[0018] Block X of urban remote sensing image S and urban building labels Y S As the source domain dataset (X S ,Y S );

[0019] Suburban remote sensing image block X T As the target domain dataset (X T ).

[0020] Furthermore, the multi-scale global context extraction module includes a hierarchical Transformer module and a multi-scale context fusion module. The hierarchical Transformer module is used to extract multi-scale context features from the source domain and target domain images, and the multi-scale context fusion module is used to fuse the multi-scale context information to obtain fusion features. The hierarchical Transformer module includes an overlapping image embedding module and four Transformer blocks. The overlapping image embedding module is used to integrate the urban remote sensing image block X S and suburban remote sensing image blocks X T Divide the image into 4×4 blocks and perform feature embedding on them respectively;

[0021] The gravitational field enhanced joint classification module is used to promote the model to learn the semantic information and geometric structure information of the building at the same time, and the gravitational field enhanced joint classification module includes a building mask encoding head H M 、AFM encoding head H A, AFM enhanced joint attention module, building mask classifier and AFM classifier; Among them, the building mask encoding head H M and AFM encoder head H A Both contain two convolutional layers with a convolution kernel size of 3×3; the building mask classifier and AFM classifier both contain two convolutional layers with convolution kernel sizes of 3×3 and 1×1 respectively;

[0022] The dual-prototype contrast learning module is used to eliminate the distribution difference between source domain and target domain data, and includes a building mask prototype contrast learning module and an AFM prototype contrast learning module.

[0023] Furthermore, using the source domain dataset (X S ,Y S ) and the target domain dataset (X T The specific process of training the building cross-domain extraction model 1 is as follows:

[0024] S2.1, the urban remote sensing image block X S and suburban remote sensing image blocks X T They are input into the multi-scale global context extraction module to obtain the source domain fusion feature F S and target domain fusion feature F T ;

[0025] S2.2. Fusion of source domain features F S and target domain fusion feature F T The data are input into the gravitational field enhanced joint classification module respectively to obtain the mask prediction results of the source domain buildings, the AFM prediction results of the source domain buildings, the mask prediction results of the target domain buildings, and the AFM prediction results of the target domain buildings;

[0026] S2.3. Mask prediction results based on source domain buildings AFM prediction results of source domain buildings Calculate the cross entropy loss function L seg And the L1 loss function L afm ;

[0027] S2.4 Mask prediction results of buildings in the source domain AFM prediction results of source domain buildings Mask prediction results of target domain buildings And the AFM prediction results of the target domain building Prototype comparative learning is performed on

[0028] Furthermore, the source domain fusion feature F is obtained S and target domain fusion feature F T The specific process is as follows:

[0029] S2.1.1、Transform the urban remote sensing image block X S They are input into the hierarchical Transformer module with four layers of Transformer blocks, and four source domain deep features F corresponding to the four different Transformer blocks in the hierarchical Transformer module are obtained respectively. i , where i is the number of Transformer blocks and i∈{1,2,3,4};

[0030] S2.1.2, the four source domain deep features F i They are input into the four 1×1 convolutional layers of the multi-scale context fusion module to obtain four source domain features F with the same number of channels. i ′; and for these four source domain features F i ' performs bilinear interpolation and upsamples to the same size as the source domain feature F1' to obtain four upsampled features F i ″;

[0031] S2.1.3, the four upsampled features F i ″Splice at the channel level to obtain the source domain fusion feature F S ;

[0032] S2.1.4, using the same process as steps S2.1.1 to S2.1.3, the suburban remote sensing image block X T In the input level Transformer module and the multi-scale context fusion module, the target domain fusion feature F is obtained T .

[0033] Furthermore, the specific process of obtaining the mask prediction results of the source domain building, the AFM prediction results of the source domain building, the mask prediction results of the target domain building, and the AFM prediction results of the target domain building is as follows:

[0034] S2.2.1. Fusion of source domain features F S Input to the building mask encoding header H M and AFM encoder head H A In the source domain, the fusion feature F S Pass through the building mask encoding head H M and AFM encoder head H A After the convolution operation is performed on the convolution layer in the source domain, the semantic features F SM and source domain AFM features F SA ;

[0035] S2.2.2. The source domain semantic feature F SM and source domain AFM features FSA Input into the AFM enhanced joint attention module to obtain the source domain fusion feature F that combines architectural semantic information and geometric information S ' M ;

[0036] S2.2.3. Fusion of source domain building features F S ' M Input into the building mask classifier to obtain the mask prediction result of the source domain building At the same time, the source domain AFM feature F SA Input into the AFM classifier to obtain the AFM prediction results of the source domain buildings

[0037] S2.2.4, using the same process as S2.2.1, S2.2.2 and S2.2.3, based on the target domain fusion feature F T Get the mask prediction results of the target domain building And the AFM prediction results of the target domain building

[0038] Furthermore, the mask prediction results of the source domain buildings in S2.4 are AFM prediction results of source domain buildings Mask prediction results of target domain buildings And the AFM prediction results of the target domain building The specific process of prototype comparative learning includes two steps: prototype updating and prototype comparative learning.

[0039] Furthermore, the specific process of prototype update is as follows:

[0040] S2.4.1. Mask prediction results using source domain buildings AFM prediction results of source domain buildings Mask prototype P M and AFM prototype P A Initialize them separately;

[0041] S2.4.2. Use the exponential moving average scheme to transform the mask prediction results of the source domain buildings AFM prediction results of source domain buildings Mask prediction results of target domain And AFM prediction results Integrated into the mask prototype P M and AFM prototype P A In the prototype update, complete the prototype update.

[0042] Furthermore, the specific process of prototype contrast learning is as follows:

[0043] S2.4.3. Calculate the mask prediction results of the source domain buildings separately With the mask prototype P M Similarities between And the AFM prediction results of the source domain buildings With AFM prototype P A Similarities between

[0044] S2.4.4. Calculate the mask prediction results of the target domain buildings separately With the mask prototype P M Similarities between And the AFM prediction results of the target domain building With AFM prototype P A Similarities between

[0045] S2.4.5 Minimization And the mask prediction results of the source domain buildings The cross entropy loss function between and minimize AFM prediction results of source domain buildings The cross entropy loss between

[0046] minimize And the mask prediction results of the target domain buildings The cross entropy loss between and minimize AFM prediction results of target domain buildings The cross entropy loss between

[0047] S2.4.6. Calculate contrast loss L C , and perform contrastive learning on the updated prototype based on contrastive loss.

[0048] Furthermore, the specific process of obtaining the building cross-domain extraction model 2 is as follows:

[0049] S3.1. Calculate the gradient of the parameters of each layer of the building cross-domain extraction model 1 according to the overall loss function L, and update the corresponding parameters in the building cross-domain extraction model 1 one by one according to the gradient direction of the parameter gradient calculated for each layer;

[0050] S3.2, repeat the process of S3.1 until all parameters in the building cross-domain extraction model 1 converge, and a trained building cross-domain extraction model is obtained, that is, a building cross-domain extraction model 2 is obtained;

[0051] S3.3. Save the building cross-domain extraction model 2.

[0052] Furthermore, the specific process of obtaining the building extraction model 4 is as follows:

[0053] S5.1. Using target domain masked pseudo-labels and target domain AFM pseudo labels Train the building extraction model 3 and calculate the loss function L self ; and use the loss function L self Calculate the gradient of each parameter in the building extraction model 3, and update the corresponding parameters in the building extraction model 3 one by one according to the gradient direction of the calculated gradient of each parameter;

[0054] S5.2. Repeat the process of S5.1 until all parameters in building extraction model 3 converge, thereby obtaining a trained building cross-domain extraction model, i.e., obtaining building extraction model 4.

[0055] S5.3. Save the building extraction model 4.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] (1) The present invention proposes a Transformer-based building cross-domain extraction model to extract and transmit the global context information of buildings, and adopts a multi-level context fusion module to fuse the multi-level context information of buildings, thereby enhancing the ability of the building cross-domain extraction model to extract global context information and ensure the accuracy of building recognition.

[0058] (2) This paper utilizes the hierarchical supervision signals of the area gravity field (AFM) and the mask to guide the cross-domain building extraction model to transfer building knowledge in a geometrically aware manner, and proposes an AFM-enhanced joint attention module to adaptively embed the AFM into the multi-layer contextual information of the building to enhance the regularity of the building extraction results.

[0059] (3) This paper proposes a dual-prototype contrast learning strategy, which performs prototype contrast learning on the mask prediction results and AFM prediction results respectively to bridge the differences in semantic and geometric information between the source domain and target domain images.

[0060] In addition to the above-described objects, features and advantages, the present invention has other objects, features and advantages. The present invention will be further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are intended to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0062] Figure 1 1 is a flow chart of a method for cross-domain building recognition coupling global context and geometric information according to an embodiment of the present invention;

[0063] Figure 2 2 is a schematic diagram of a cross-domain building extraction network that takes into account global context and geometric information in an embodiment of the present invention;

[0064] Figure 3 It is a structural diagram of the HT module in an embodiment of the present invention. DETAILED DESCRIPTION

[0065] In order to make the above-mentioned purposes, features and advantages of the present invention more clear and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings. It should be noted that the drawings of the present invention are all simplified and non-precisely scaled, and are only used to conveniently and clearly assist in explaining the implementation of the present invention; the "numbers" mentioned in the present invention are not limited to the specific quantities in the examples in the accompanying drawings; the directions or positional relationships indicated by "front", "middle", "back", "left", "right", "up", "down", "top", "bottom", "middle", etc. mentioned in the present invention are based on the directions or positional relationships shown in the drawings of the present invention, and do not indicate or imply that the devices or components referred to must have a specific direction, nor can they be understood as limitations on the present invention.

[0066] Example:

[0067] See also Figures 1 to 3 As shown, the present invention provides a cross-domain building recognition method that couples global context and geometric information, including the following steps:

[0068] Step 1: Determine the source domain dataset (X S ,Y S ) and the target domain dataset (X T ); the specific process is as follows:

[0069] The urban aerial high-resolution remote sensing images and suburban aerial high-resolution remote sensing images of a certain area are cropped into image blocks of 512*512 size, and several urban remote sensing image blocks X are obtained respectively. S and suburban remote sensing image blocks X T ;

[0070] Block X of urban remote sensing image S The buildings in the image are labeled with polygons and converted into binary raster labels Y with a value of 0 (background) -1 (building). S ;

[0071] Block X of urban remote sensing image S and urban building labels Y S As the source domain dataset (X S ,Y S );

[0072] Suburban remote sensing image block X T As the target domain dataset (X T ).

[0073] Step 2: Using the source domain dataset (X S ,Y S ) and the target domain dataset (X T ) Train a cross-domain building extraction model 1 that takes into account global context and geometric information;

[0074] The building cross-domain extraction model 1 includes a multi-scale global context extraction module, a gravitational field enhanced joint classification module, and a dual-prototype contrast learning module;

[0075] The multi-scale global context extraction module includes a hierarchical Transformer module and a multi-scale context fusion module. The hierarchical Transformer module is used to extract multi-scale context features from source domain and target domain images, and the multi-scale context fusion module is used to fuse multi-scale context information to obtain fusion features. The hierarchical Transformer module includes an overlapping image embedding module and four Transformer blocks. The overlapping image embedding module is used to integrate the urban remote sensing image block X S and suburban remote sensing image blocks X T Divide the image into 4×4 blocks and perform feature embedding on them respectively;

[0076] The gravitational field enhanced joint classification module is used to promote the model to learn the semantic information and geometric structure information of the building at the same time, and the gravitational field enhanced joint classification module includes a building mask encoding head H M 、AFM encoding head H A , AFM enhanced joint attention module, building mask classifier and AFM classifier; Among them, the building mask encoding head H M and AFM encoder head H A Both contain two convolutional layers with a convolution kernel size of 3×3; the building mask classifier and AFM classifier both contain two convolutional layers with convolution kernel sizes of 3×3 and 1×1 respectively;

[0077] The dual-prototype contrast learning module is used to eliminate the distribution difference between source domain and target domain data, and includes a building mask prototype contrast learning module and an AFM prototype contrast learning module.

[0078] Furthermore, using the source domain dataset (X S ,Y S ) and the target domain dataset (X T The specific process of training the building cross-domain extraction model 1 is as follows:

[0079] S2.1, block the urban remote sensing image X S and suburban remote sensing image blocks X T They are input into the multi-scale global context extraction module to obtain the source domain fusion feature F S and target domain fusion feature F T The specific process is as follows:

[0080] S2.1.1、Transform the urban remote sensing image block X S They are input into the hierarchical Transformer module with four layers of Transformer blocks, and four source domain deep features F corresponding to the four different Transformer blocks in the hierarchical Transformer module are obtained respectively. i , these features provide not only high-resolution coarse-grained features, but also low-resolution fine-grained features. Where i is the number of Transformer blocks, and i∈{1,2,3,4}.

[0081] S2.1.2, the four source domain deep features F i They are input into the four 1×1 convolutional layers of the multi-scale context fusion module to obtain four source domain features F with the same number of channels. i ′; and for these four source domain features F i ' performs bilinear interpolation and upsamples to the same size as the source domain feature F1' to obtain four upsampled features F i ″;

[0082] S2.1.3, the four upsampled features F i ″Splice at the channel level to obtain the source domain fusion feature F S .

[0083] S2.1.4, using the same process as steps S2.1.1 to S2.1.3, the suburban remote sensing image block X T In the input level Transformer module and the multi-scale context fusion module, the target domain fusion feature F is obtained T .

[0084] S2.2. Fusion of source domain features F S and target domain fusion feature F TThe results are input into the joint classification module of gravitational field enhancement to obtain the mask prediction results of the source domain buildings, the AFM prediction results of the source domain buildings, the mask prediction results of the target domain buildings, and the AFM prediction results of the target domain buildings. The specific process is as follows:

[0085] S2.2.1. Fusion of source domain features F S Input to the building mask encoding header H M and AFM encoder head H A In the source domain, the fusion feature F S Pass through the building mask encoding head H M and AFM encoder head H A After the convolution operation is performed on the convolution layer in the source domain, the semantic features F SM and source domain AFM features F SA ;

[0086] S2.2.2. The source domain semantic feature F SM and source domain AFM features F SA Input into the AFM enhanced joint attention module to obtain the source domain fusion feature F that combines architectural semantic information and geometric information S ' M The specific process is as follows:

[0087] The source domain semantic feature F SM They are input into the two 1×1 convolutional layers in the AFM enhanced joint attention module to obtain the source domain semantic features V∈R C′×H×W and source domain semantic features K∈R C′×H×W ;

[0088] The source domain AFM feature F SA Input into one of the 1×1 convolutional layers in the AFM-enhanced joint attention module to obtain the source domain AFM feature Q∈R C′×H×W ;

[0089] The source domain semantic features K∈R C′×H×W Reshape the vector into R C′×HW Size, source domain AFM feature Q∈R C′×H×W Reshape the vector into R HW×C′ Size and matrix multiplication are performed to obtain the attention affinity feature S∈R C′×C′ ;

[0090] The attention affinity feature S∈R C′×C′ and the source domain semantic feature V∈R C′×H×W Perform matrix multiplication to obtain the source domain building fusion feature F that aggregates the building semantic information and geometric information S ' M ∈RC′×H×W ; Source domain building fusion feature F S ' M The feature at each position in the H×W dimension represents the attention affinity feature S∈R C′×C′ and source domain semantic features V∈R C′×H×W The weighted value of , this feature can take into account the semantic information and geometric structure information of the building, thereby improving the building extraction performance of the classifier;

[0091] S2.2.3. Fusion of source domain building features F S ' M Input into the building mask classifier to obtain the mask prediction result of the source domain building At the same time, the source domain AFM feature F SA Input into the AFM classifier to obtain the AFM prediction results of the source domain buildings

[0092] S2.2.4, using the same process as S2.2.1, S2.2.2 and S2.2.3, based on the target domain fusion feature F T Get the mask prediction results of the target domain building And the AFM prediction results of the target domain building

[0093] S2.3. Mask prediction results based on source domain buildings AFM prediction results of source domain buildings Calculate the cross entropy loss function L seg And the L1 loss function L afm .

[0094] Cross entropy loss function L seg The expression is as follows:

[0095]

[0096] And L1 loss function L afm The expression is as follows:

[0097]

[0098] Among them, N s is the number of images in the source domain dataset; A S is the true value label of AFM, n={1,2,…,N S}.

[0099] S2.4 Mask prediction results of buildings in the source domain AFM prediction results of source domain buildings Mask prediction results of target domain buildings And the AFM prediction results of the target domain building Prototype contrast learning is performed on the source domain, narrowing the distance between the source domain building mask prediction results and the mask prototype, the distance between the target domain building mask prediction results and the mask prototype, the distance between the source domain building AFM prediction results and the AFM prototype, and the distance between the target domain building AFM prediction results and the AFM prototype. This guides the model to narrow the distribution differences between the source domain dataset and the target domain data at the semantic and geometric levels. Specifically, it mainly includes two steps: prototype updating and prototype contrast learning.

[0100] The specific process of prototype update is as follows:

[0101] S2.4.1. Mask prediction results using source domain buildings AFM prediction results of source domain buildings Mask prototype P M and AFM prototype P A Initialize them separately;

[0102] P M The expression is as follows:

[0103]

[0104] P A The expression is as follows:

[0105]

[0106] Where: k is the index value of the building category 1; is the indicator function, only when hour, otherwise,

[0107] S2.4.2. Use the exponential moving average scheme to transform the mask prediction results of the source domain buildings AFM prediction results of source domain buildings Mask prediction results of target domain And AFM prediction results Integrated into the mask prototype P M and AFM prototype P A The prototype update is completed in . The specific process is as follows: Through the leveraged moving average method, the mask prototype P M and AFM prototype P A Update them separately to get the updated mask prototype P′ M and AFM prototype P′ A ;

[0108] P′ M The expression is as follows:

[0109]

[0110] P A The expression of ′ is as follows:

[0111]

[0112] in, is the source domain mask pseudo label of the k-th category; is the source domain AFM pseudo label of the kth category; m is the momentum, and the polynomial annealing policy is used to update m to obtain m t .

[0113] m t The expression is as follows:

[0114]

[0115] t∈[0,T];

[0116] Among them, m t is the value of m at the tth iteration, T is the total number of training iterations, and α and m0 are both default parameters (in the present invention, α and m0 are both set to 0.9).

[0117] The specific process of prototype contrast learning is as follows:

[0118] S2.4.3. Calculate the mask prediction results of the source domain buildings separately With the mask prototype P M Similarities between And the AFM prediction results of the source domain buildings With AFM prototype P A Similarities between

[0119] The expression is as follows:

[0120]

[0121] The expression is as follows:

[0122]

[0123] S2.4.4. Calculate the mask prediction results of the target domain buildings separately With the mask prototype P M Similarities between And the AFM prediction results of the target domain building With AFM prototype P A Similarities between

[0124] The expression is as follows:

[0125]

[0126] The expression is as follows:

[0127]

[0128] in, is the mask prototype of the k-th category, is the AFM prototype of the kth category, τ is the temperature parameter that controls the sharpness of the generated probability distribution, and K is the number of prototype categories.

[0129] S2.4.5. Minimization And the mask prediction results of the source domain buildings The cross entropy loss function between and minimize AFM prediction results of source domain buildings The cross entropy loss between

[0130] The expression is as follows:

[0131]

[0132] The expression is as follows:

[0133]

[0134] minimize Mask prediction results of buildings in the target domain The cross entropy loss between and minimize AFM prediction results of target domain buildings The cross entropy loss between

[0135] The expression is as follows:

[0136]

[0137] The expression is as follows:

[0138]

[0139] in, is the mask label of the kth category of the source domain dataset, is the AFM label of the kth category of the source domain dataset, Predict the label for the mask of the kth category of the target domain dataset, Predict labels for the AFM of the kth category in the target domain dataset.

[0140] S2.4.6. Calculate contrast loss L C , and perform contrastive learning on the updated prototype based on contrastive loss;

[0141] L C The expression is as follows:

[0142]

[0143] Step 3: Calculate the overall loss function L of the building cross-domain extraction model, and update the parameters of the building cross-domain extraction model 1 according to the overall loss function L to obtain the building cross-domain extraction model 2; the specific process is as follows:

[0144] S3.1. Calculate the gradient of the parameters of each layer of the building cross-domain extraction model 1 according to the overall loss function L, and update the corresponding parameters in the building cross-domain extraction model 1 one by one according to the gradient direction of the parameter gradient calculated for each layer;

[0145] The expression of L is as follows:

[0146] L=L C +L seg +L afm ;

[0147] S3.2, repeat the process of S3.1 until all parameters in the building cross-domain extraction model 1 converge, and a trained building cross-domain extraction model is obtained, that is, a building cross-domain extraction model 2 is obtained;

[0148] S3.3. Save the building cross-domain extraction model 2.

[0149] Step 4: The target domain dataset (X T ) is input into the building cross-domain extraction model 2 for prediction, and the target domain mask pseudo label is obtained. and target domain AFM pseudo labels Specifically, the target domain dataset (X T Each remote sensing image in the image is processed by the multi-scale global context extraction module and the gravitational field enhanced joint classification module, and finally the mask classifier outputs the target domain mask pseudo label The AFM classifier outputs the target domain AFM pseudo label

[0150] Step 5: Define the building cross-domain extraction model 2 without the dual prototype contrast learning module as the building extraction model 3, and train it to obtain the building extraction model 4. The specific process is as follows:

[0151] S5.1. Using target domain masked pseudo-labels and target domain AFM pseudo labels Train the building extraction model 3 and calculate the loss function L self ; and use the loss function L self Calculate the gradient of each parameter in the building extraction model 3, and update the corresponding parameters in the building extraction model 3 one by one according to the gradient direction of the calculated gradient of each parameter;

[0152] Loss function L self The expression is as follows:

[0153] L self =L seg-self +L afm-self

[0154]

[0155] Among them, L seg-self is the cross entropy loss L for mask prediction seg , L afm-self is the L1 loss function L afm , N t is the number of remote sensing images in the target domain,

[0156] S5.2. Repeat the process of S5.1 until all parameters in building extraction model 3 converge, thereby obtaining a trained building cross-domain extraction model, i.e., obtaining building extraction model 4.

[0157] S5.3. Save the building extraction model 4.

[0158] Step 6: The target domain dataset (X T ) is input into the building extraction model 4 for prediction, and the final target domain building mask prediction result is obtained; that is, the building recognition result is obtained. Preferably, the target domain dataset (X T ) is input into the building extraction model 4 for prediction. Please refer to the specific process shown in step 4.

[0159] Experimental example:

[0160] 1) Quantitative analysis

[0161] To verify the effectiveness of the proposed method, we conducted comparative experiments on the urban-to-suburban cross-domain (C2S) task in the same region, comparing the pure source domain model with eight state-of-the-art domain adaptation semantic segmentation methods (AdaptSegNet, AdvEnt, FADA, SDCA, ProCA, MDANet, DSSF, and DAFormer). The experimental results are shown in Table 1.

[0162] As shown in Table 1, due to differences in spatial resolution, spatial layout, and building style between urban and suburban areas, the source-only model achieved the worst building extraction performance, achieving scores of 60.65%, 43.52%, and 59.38% in F1, IoU, and Acc, respectively. Compared to the source-only model, the eight UDA comparison methods improved building extraction performance to varying degrees by employing different domain adaptation strategies to eliminate data distribution differences between the source and target domains and fully utilizing high-resolution remote sensing imagery in the unlabeled target domain dataset. For example, the worst-performing comparison method, AdaptSegNet, achieved only 68.95%, 52.62%, and 68.25% in F1, IoU, and Acc, respectively, representing improvements of only 8.30%, 9.10%, and 8.87% over the source-only model. DAFormer, the best-performing method among the compared methods, achieved 82.12%, 69.66%, and 77.55% in F1, IoU, and Acc, respectively, by considering the context of the remote sensing scene. Compared to the source-only model, these improvements represent 21.47%, 26.14%, and 18.17% improvements in F1, IoU, and Acc, respectively. Furthermore, the model proposed in this section, DCANet, achieves 86.47%, 76.17%, and 85.33% in F1, IoU, and Acc, respectively, on the C2S task by considering the multi-scale context of the remote sensing scene and the geometric structure of buildings. This represents 25.82%, 32.65%, and 25.95% improvements in F1, IoU, and Acc, respectively, compared to the source-only model. Compared to the AdaptSegNet model in the comparison method, the F1, IoU, and Acc indicators achieved score improvements of 17.52%, 23.55%, and 17.08%, respectively. Compared to the DAFormer model in the comparison method, the F1, IoU, and Acc indicators achieved score improvements of 4.25%, 6.51%, and 7.78%, respectively.

[0163] Table 1: Quantitative evaluation results of different methods on the urban→suburban cross-domain (C2S) task

[0164]

[0165]

[0166] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A cross-domain building recognition method that couples global context and geometric information, characterized by: The following steps are involved: Step 1: Determine the source domain dataset based on urban and suburban remote sensing images of the same area and target domain dataset ; Step 2: Using the source domain dataset and target domain dataset Training a building cross-domain extraction model 1 that takes into account global context and geometric information, wherein the building cross-domain extraction model 1 includes a multi-scale global context extraction module, a gravitational field enhanced joint classification module, and a dual-prototype contrastive learning module; Step 3: Calculate the overall loss function of the building cross-domain extraction model , and according to the overall loss function The parameters of the building cross-domain extraction model 1 are updated to obtain the building cross-domain extraction model 2; Step 4: Target domain dataset The remote sensing image in is input into the building cross-domain extraction model 2 for prediction, and the target domain mask pseudo label is obtained. and target domain AFM pseudo labels ; Step 5: define the building cross-domain extraction model 2 without the dual prototype contrast learning module as the building extraction model 3, and train the building extraction model 3 to obtain the building extraction model 4; Step 6: Target domain dataset The remote sensing image in is input into the building extraction model 4 for prediction, and the final target domain building mask prediction result is obtained; that is, the building recognition result is obtained; The multi-scale global context extraction module includes a hierarchical Transformer module and a multi-scale context fusion module. The hierarchical Transformer module is used to extract multi-scale context features from source domain and target domain images, and the multi-scale context fusion module is used to fuse multi-scale context information to obtain fusion features. The hierarchical Transformer module includes an overlapping image embedding module and four Transformer blocks. The overlapping image embedding module is used to embed urban remote sensing image blocks. and suburban remote sensing image blocks Split into pieces of size image blocks and perform feature embedding respectively; The gravitational field enhanced joint classification module is used to promote the model to learn the semantic information and geometric structure information of the building at the same time, and the gravitational field enhanced joint classification module includes a building mask encoding head , AFM encoder head , AFM-enhanced joint attention module, building mask classifier and AFM classifier; among them, the building mask encoding head and AFM encoder head Both contain two layers of convolution kernel size The building mask classifier and AFM classifier both contain two layers of convolution kernel sizes of and Convolutional layers; The dual-prototype contrast learning module is used to eliminate the distribution difference between source domain and target domain data, and includes a building mask prototype contrast learning module and an AFM prototype contrast learning module.

2. The cross-domain building recognition method coupling global context and geometric information according to claim 1 is characterized in that In step 1, the source domain dataset is determined and target domain dataset The specific process is as follows: The urban remote sensing images and suburban remote sensing images of the same area are cropped into image blocks of 512×512 size, and several urban remote sensing image blocks are obtained respectively. and suburban remote sensing image blocks ; Urban remote sensing image blocks The buildings in the image are labeled with polygons and converted into binary raster labels with a value of 0-1. The binary raster labels are defined as urban building labels. ; Urban remote sensing image blocks and urban building labels As source domain dataset ; Suburban remote sensing image blocks As the target domain dataset .

3. The cross-domain building recognition method coupling global context and geometric information according to claim 1 is characterized in that Leveraging source domain datasets and target domain dataset The specific process of training the building cross-domain extraction model 1 is as follows: S2.

1. Block urban remote sensing images and suburban remote sensing image blocks Input them into the multi-scale global context extraction module respectively to obtain the source domain fusion features Fusion features with the target domain ; S2.

2. Fusion of source domain features Fusion features with the target domain The data are input into the gravitational field enhanced joint classification module respectively to obtain the mask prediction results of the source domain buildings, the AFM prediction results of the source domain buildings, the mask prediction results of the target domain buildings, and the AFM prediction results of the target domain buildings; S2.

3. Mask prediction results based on source domain buildings AFM prediction results of source domain buildings , calculate the cross entropy loss function And L1 loss function ; S2.4 Mask prediction results of buildings in the source domain , AFM prediction results of source domain buildings , Mask prediction results of target domain buildings And the AFM prediction results of the target domain building Prototype comparative learning is performed on 4. The cross-domain building recognition method coupling global context and geometric information according to claim 3 is characterized in that Get the source domain fusion features Fusion features with the target domain The specific process is as follows: S2.1.1、Urban remote sensing image blocks They are input into the hierarchical Transformer module with four layers of Transformer blocks, and four source domain deep features corresponding to the four different Transformer blocks in the hierarchical Transformer module are obtained respectively. ,in, is the number of Transformer blocks, and ; S2.1.

2. Four source domain deep features The four inputs to the multi-scale context fusion module are In the convolutional layer, four source domain features with the same number of channels are obtained ; and for these four source domain features Perform bilinear interpolation and upsample to the source domain feature The same size, four upsampled features are obtained ; S2.1.3, the four upsampled features Splicing is performed at the channel level to obtain source domain fusion features ; S2.1.4, using the same process as steps S2.1.1 to S2.1.3, the suburban remote sensing image blocks In the input level Transformer module and the multi-scale context fusion module, the target domain fusion features are obtained .

5. The cross-domain building recognition method coupling global context and geometric information according to claim 3 or 4, characterized in that: The specific process of obtaining the mask prediction results of the source domain building, the AFM prediction results of the source domain building, the mask prediction results of the target domain building, and the AFM prediction results of the target domain building is as follows: S2.2.

1. Fusion of source domain features Input to the building mask encoding header and AFM encoder head In the source domain, the fusion features Pass through the building mask encoding head respectively and AFM encoder head After the convolution operation is performed on the convolution layer in , the source domain semantic features are obtained respectively. and source domain AFM features ; S2.2.

2. Source domain semantic features and source domain AFM features Input into the AFM enhanced joint attention module to obtain the source domain fusion feature that integrates architectural semantic information and geometric information ; S2.2.

3. Fusion of source domain building features Input into the building mask classifier to obtain the mask prediction result of the source domain building ; At the same time, the source domain AFM features Input into the AFM classifier to obtain the AFM prediction results of the source domain buildings ; S2.2.

4. Using the same process as S2.2.1, S2.2.2 and S2.2.3, based on the target domain fusion feature Get the mask prediction results of the target domain building And the AFM prediction results of the target domain building .

6. The cross-domain building recognition method coupling global context and geometric information according to claim 5 is characterized in that The mask prediction results of the source domain buildings in S2.4 are , AFM prediction results of source domain buildings , Mask prediction results of target domain buildings And the AFM prediction results of the target domain building The specific process of performing prototype comparative learning includes prototype updating, and the specific process of the prototype updating is as follows: S2.4.

1. Mask prediction results using source domain buildings AFM prediction results of source domain buildings Mask prototype and AFM prototypes Initialize them separately; S2.4.

2. Use the exponential moving average scheme to transform the mask prediction results of the source domain buildings , AFM prediction results of source domain buildings , mask prediction results of target domain And AFM prediction results Integrate into mask prototype and AFM prototypes In the prototype update, complete the prototype update.

7. The cross-domain building recognition method coupling global context and geometric information according to claim 5 is characterized in that The mask prediction results of the source domain buildings in S2.4 are , AFM prediction results of source domain buildings , Mask prediction results of target domain buildings And the AFM prediction results of the target domain building The specific process of performing prototype contrast learning also includes prototype contrast learning, and the specific process of the prototype contrast learning is as follows: S2.4.

3. Calculate the mask prediction results of the source domain buildings separately With mask prototype Similarities between And the AFM prediction results of the source domain buildings Prototyping with AFM Similarities between ; S2.4.

4. Calculate the mask prediction results of the target domain buildings separately With mask prototype Similarities between And the AFM prediction results of the target domain building Prototyping with AFM Similarities between ; S2.4.5 Minimization And the mask prediction results of the source domain buildings The cross entropy loss function between and minimize AFM prediction results of source domain buildings The cross entropy loss between ; minimize And the mask prediction results of the target domain buildings The cross entropy loss between and minimize AFM prediction results of target domain buildings The cross entropy loss between ; S2.4.

6. Calculating Contrast Loss , and perform contrastive learning on the updated prototype based on contrastive loss.

8. The cross-domain building recognition method coupling global context and geometric information according to claim 6 or 7, characterized in that: The specific process of obtaining the building cross-domain extraction model 2 is as follows: S3.

1. Each layer of the building cross-domain extraction model 1 is calculated based on the overall loss function Calculate the gradient of its parameters, and update the corresponding parameters in the building cross-domain extraction model 1 one by one according to the gradient direction of the parameter gradient calculated at each layer; S3.2, repeat the process of S3.1 until all parameters in the building cross-domain extraction model 1 converge, and a trained building cross-domain extraction model is obtained, that is, a building cross-domain extraction model 2 is obtained; S3.

3. Save the building cross-domain extraction model 2.

9. The cross-domain building recognition method coupling global context and geometric information according to claim 8, characterized in that: The specific process of obtaining the building extraction model 4 is as follows: S5.

1. Using target domain masked pseudo-labels and target domain AFM pseudo labels Train the building extraction model 3 and calculate the loss function ; and use the loss function Calculate the gradient of each parameter in the building extraction model 3, and update the corresponding parameters in the building extraction model 3 one by one according to the gradient direction of the calculated gradient of each parameter; S5.

2. Repeat the process of S5.1 until all parameters in building extraction model 3 converge, thereby obtaining a trained building cross-domain extraction model, i.e., obtaining building extraction model 4. S5.

3. Save the building extraction model 4.

Images