Building damage assessment method based on cross-domain feature alignment and instance segmentation

Abstract

The present application relates to the technical field of remote sensing image processing, and more particularly to a building damage evaluation method based on cross-domain feature alignment and instance segmentation, a building damage evaluation system extracts target domain and source domain image features through a feature extraction network, obtains domain invariant features through cross-domain feature alignment, extracts building candidate regions through a region proposal network, completes candidate region decoding and damage analysis through a multi-task decoding head, and realizes precise identification and evaluation of damaged buildings. The present application adopts source domain and target domain alternating iterative training, takes an instance segmentation model as the core, introduces an adversarial domain adaptation mechanism and a multi-level classification branch, effectively solves the problem of post-disaster labeling data scarcity, significantly improves the precision, robustness and generalization ability of building damage evaluation, and provides a new technical solution for post-disaster emergency response.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing technology, and in particular relates to a method for assessing building damage based on cross-domain feature alignment and instance segmentation. Background Technology

[0002] Sudden natural disasters often lead to large-scale casualties, economic losses, and social disorder. With the development of remote sensing technology, building damage assessment based on remote sensing imagery has become a major technical means of obtaining large-scale disaster information.

[0003] In the traditional approach phase, researchers primarily relied on manual on-site reconnaissance and visual interpretation, or utilized low- to medium-resolution satellite imagery, combining spectral features (such as the Normalized Differential Vegetation Index (NDVI), texture features (such as the Gray-Level Co-occurrence Matrix (GLCM),) and geometric features (such as LiDAR point cloud height abrupt changes) for analysis. The detection accuracy of these methods is limited by the effectiveness of feature design and image spatial resolution, and they are easily affected by shadows, vegetation, and other ground features in complex scenes, making it difficult to achieve high-precision, instance-level damage identification.

[0004] In recent years, deep learning technology, especially convolutional neural networks (CNNs), has made breakthrough progress in image recognition and segmentation tasks due to its powerful automatic feature extraction and representation capabilities, and has been gradually applied to the field of building damage detection. For example, semantic segmentation networks using encoder-decoder structures (such as U-Net and fully convolutional networks FCN) are used to perform pixel-level classification of remote sensing images, labeling each pixel with a damage category. Some studies have further introduced object detection frameworks (such as Faster R-CNN and the YOLO series) to achieve bounding box localization of damaged buildings. More advanced methods employ instance segmentation models (such as Mask R-CNN), which can simultaneously perform object detection and pixel-level segmentation, outputting the bounding box, category, and precise mask of each independent damaged building, achieving instance-level assessment from "whether there is damage" to "which building is damaged and how," significantly improving the usability of the results. However, directly applying the above deep learning models to building damage assessment in post-earthquake emergency scenarios still faces two key challenges: On the one hand, after a sudden natural disaster, it is often difficult to obtain a sufficient amount of high-quality labeled data for the target disaster area in a short period of time. The model can only be trained on historical disaster areas or simulated datasets with sufficient data. Due to the significant differences in imaging conditions, ground object spectra, building styles and distributions among remote sensing images acquired from different regions, times and sensors, the feature distributions of the source domain and the target domain are inconsistent. When a model trained on the source domain is directly applied to the target domain, the model performance will be significantly degraded due to these distribution differences.

[0005] On the other hand, existing damage assessment methods are mostly based on semantic segmentation patterns. Due to their inherent pixel-level classification mechanism, semantic segmentation methods cannot effectively model and distinguish individual building instances when dealing with the identification task of post-disaster building damage assessment. This results in blurred boundaries in complex scenarios (such as densely built-up areas or areas with ambiguous damage boundaries), making it impossible to accurately separate and quantify the damage status and level of each individual building. This deficiency restricts the decision support capabilities for quantitative disaster assessment and refined allocation of rescue resources. Summary of the Invention

[0006] In view of this, the present invention aims to provide a building damage assessment method based on cross-domain feature alignment and instance segmentation. It constructs a basic architecture with an instance segmentation model at its core, achieving accurate extraction of each independent building instance and overcoming the problems of coarse granularity and instance concatenation in assessment results caused by traditional semantic segmentation methods. Furthermore, it introduces an adversarial domain adaptation mechanism, extracting "domain-invariant features" through adversarial training of the feature extractor and domain discriminator. This enables effective knowledge transfer in target disaster areas with scarce annotations, improving the model's generalization ability and robustness when facing multi-source and cross-regional data, thereby enhancing the accuracy and automation level of building damage assessment. To achieve the above objectives, the technical solution created by this invention is implemented as follows: A building damage assessment method based on cross-domain feature alignment and instance segmentation includes: S1: Obtain the target domain image and its corresponding source domain image, and label the labels in the source domain image to obtain the building damage assessment results for each label in the source domain image; combine the target domain image, the source domain image, and the building damage assessment results into a training set; S2: Construct a building damage assessment system; the building damage assessment system includes a feature extraction network, a region proposal network, and a multi-task decoder: the feature extraction network extracts features from the target domain image and the corresponding source domain image respectively, and then aligns the obtained target domain features with the source domain features to obtain domain-invariant features; the region proposal network extracts building regions from the domain-invariant features to generate candidate regions; the multi-task decoder decodes and performs damage analysis on the obtained candidate regions to obtain the complete identification and assessment results of damaged buildings in the target domain image; S3: Using an iterative training method that alternates between the source domain and the target domain, the building damage assessment system constructed in step S2 is trained using the training set obtained in step S1 to obtain the building damage assessment model. S4: Input the target domain image to be evaluated into the building damage assessment model obtained in step S3, and output the predicted building damage assessment result.

[0007] Furthermore, the feature extraction network in step S2 includes: The feature extractor, ResNet50 combined with the feature pyramid network, extracts features from the target domain image and the source domain image respectively, obtaining target domain features and source domain features respectively; The domain discriminator uses a fully connected layer to perform binary classification of target domain features and source domain features, outputs a probability prediction of the source of the features, and adjusts the feature extractor according to the probability prediction to minimize the difference between the target domain features and the source domain features. At this point, the target domain features are the domain-invariant features.

[0008] Furthermore, the multi-task decoding head in step S2 includes: The hierarchical classifier includes a first-level classifier and a second-level classifier. The first-level classifier determines whether a candidate region is a damaged building. When the first-level classifier determines that a candidate region is a damaged building, the second-level classifier analyzes the degree of damage to the candidate region and outputs the damage level. The bounding box regression head determines the location coordinates of buildings in the candidate region and generates bounding boxes of the buildings in the target domain image based on the location coordinates; The adaptive masking tool generates mask templates for damaged and undamaged buildings based on the classification results of the hierarchical classifier and the building's bounding box.

[0009] Furthermore, both the first-level and second-level classifiers are fully connected networks, while the adaptive masking network is a fully convolutional network.

[0010] Furthermore, in step S3: In the source domain training process: input the source domain image and its annotation, set the domain label to 0, and the total loss function during training is: ; in, This represents the total loss function of the source domain training process. The total loss function includes instance segmentation loss and domain adversarial loss L. dom Instance segmentation loss includes object detection loss L det and mask loss L mask β represents the hyperparameter for balancing the weights of the recognition task; In the target domain training process: Input an unlabeled target domain image, set the domain label to 1, and the total loss function during training is: ; in, This represents the total loss function for the target domain training process; Domain confrontation loss L dom for: ; Where N represents the batch size for training, f id represents the domain-invariant feature of the i-th sample in the batch output by the domain discriminator. i Let σ represent the domain label corresponding to the domain-invariant feature of the i-th sample, σ represent the sigmoid function, and D represent the domain discriminator. Target detection loss L det for: L det =L rpn +L cls +L box ; Among them, L rpn L represents the suggested loss for the region. cls L represents the hierarchical classification loss. box This represents the bounding box regression loss; Mask loss L mask The cross-entropy loss is the binary cross-entropy loss between the true mask composed of the true classification results and the mask template output by the adaptive masking system.

[0011] Furthermore, the hierarchical classification loss L cls for: L cls =L cls1 +α×L cls2 ; L cls1 L represents the classification loss of the first-level classifier. cls2 α represents the classification loss of the second-level classifier, and α represents the balancing weight. The classification loss L of the first-level classifier cls1 The standard multi-class cross-entropy loss is as follows: ; in, This represents the true class that the first-level classifier gives when classifying the candidate region of the i-th sample. This represents the probability corresponding to the true category. This represents the predicted probability output by the first-level classifier when it actually classifies the candidate region of the i-th sample and predicts the j-th category. The classification loss L of the second-level classifier cls2 for: ; Where Ω represents the subset of samples labeled as damaged buildings. This represents the true damage level given by the secondary classifier when classifying the candidate region of the i-th sample. This represents the probability corresponding to the actual level of damage. This represents the predicted probability output by the secondary classifier when it actually classifies the candidate region of the i-th sample and predicts the c-th damage level, where C represents the total number of damage levels.

[0012] Furthermore, the bounding box regression loss L box for: ; in, Represents all positive samples. This represents the true class that the first-level classifier gives when classifying the candidate region of the i-th sample. k represents the bounding box regression parameters for the predicted i-th positive sample. i Indicates the corresponding final category, This represents the regression parameters of the corresponding true bounding box.

[0013] Furthermore, in step S3, the domain discriminator is optimized using the objective function of the following formula: ; Where F represents the feature extractor, D represents the domain discriminator, and D s and D t Let x represent the data distribution in the source domain image and the target domain image, respectively. s and x t These represent the data in the source domain image and the target domain image, respectively.

[0014] Compared with the prior art, the present invention can achieve the following beneficial effects: (1) The building damage assessment method based on cross-domain feature alignment and instance segmentation created in this invention innovatively introduces an adversarial domain adaptation mechanism to enable the model to learn "domain-invariant features", effectively aligning the feature distribution of labeled source domains with unlabeled or scarcely labeled target domains, improving the model's generalization ability and robustness when facing multi-source and cross-regional data, and solving the technical problem that existing methods have poor building damage assessment accuracy when applied across regions due to significant feature distribution shifts between the target domain and the source domain caused by differences in imaging time, season, illumination and sensor, etc. (2) In response to the core problem of difficulty in quickly obtaining a large amount of high-quality labeled data after a disaster, the building damage assessment method based on cross-domain feature alignment and instance segmentation created in this invention realizes the assessment of the target disaster area using historical labeled data. This greatly reduces the dependence on real-time and large-scale manual labeling of the target disaster area, so that the model can be quickly adapted and deployed with little or no labeling, thus meeting the timeliness requirements of emergency rescue. (3) In view of the boundary ambiguity caused by traditional semantic segmentation methods, the building damage assessment method based on cross-domain feature alignment and instance segmentation created in this invention takes the improved Mask R-CNN instance segmentation model as the core, which can output the boundary, location and pixel-level mask of each independent damaged building. Combined with multi-level classification, it further provides fine-grained damage level for each instance, realizes accurate instance-level segmentation, improves the boundary differentiation ability of adjacent buildings, and provides technical support for refined damage level assessment. Attached Figure Description

[0015] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 A schematic diagram of the building damage assessment method based on cross-domain feature alignment and instance segmentation as described in the embodiments of the present invention; Figure 2 A schematic diagram of the building damage assessment system described in an embodiment of the present invention; Figure 3 A schematic diagram of the feature extraction network described in an embodiment of the present invention; Figure 4 This is a schematic diagram of the multi-task decoding head described in an embodiment of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not constitute a limitation thereof. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0017] The invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0018] like Figure 1 As shown in the embodiment of the present invention, the building damage assessment method based on cross-domain feature alignment and instance segmentation includes: S1: Obtain the target domain image and its corresponding source domain image, and label the labels in the source domain image to obtain the building damage assessment results for each label in the source domain image; combine the target domain image, source domain image and building damage assessment results into a training set.

[0019] In this embodiment of the invention, the publicly available benchmark dataset xBD (Ritwik Gupta, Richard Hosfelt, Sandra Sajeev, Nirav Patel, Bryce Goodman, Jigar Doshi, Eric Heim, HowieChoset, and Matthew Gaston. xBD: A Dataset for Assessing Building Damage from Satellite Imagery. arXiv:1911.09296 [cs.CV], 2019.) is used as the data source. During training, disaster events are used as the unit of division. The source domain selects images and annotations of several disaster events, and the target domain selects images of another set of disaster events. It should be noted that the source domain images and the target domain images are remote sensing images of different disaster events.

[0020] S2: Construct a building damage assessment system. The building damage assessment system provided by this invention is as follows: Figure 2 As shown, the system includes a feature extraction network, a region proposal network, and a multi-task decoder. The feature extraction network extracts features from both the target domain image and the corresponding source domain image, and then aligns the obtained target domain features with the source domain features to obtain domain-invariant features. The region proposal network extracts building regions from the domain-invariant features, generating candidate regions. The multi-task decoder decodes the obtained candidate regions and performs damage analysis to obtain a complete identification and evaluation result of damaged buildings in the target domain image.

[0021] In this embodiment of the invention, the feature extraction network outputs multi-scale domain-invariant features, and the Region Proposal Network (RPN, Ren, K. He, R. Girshick and J. Sun, "Faster R-CNN: Towards Real-Time ObjectDetection with Region Proposal Networks," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137-1149, 1 June 2017, doi: 10.1109 / TPAMI.2016.2577031.) generates multi-scale candidate regions that may contain buildings, providing a robust cross-domain feature foundation for subsequent instance segmentation tasks. Furthermore, this embodiment of the invention performs Region of Interest Alignment (ROI Align) on the multi-scale candidate regions output by the RPN, then extracts feature vectors from each aligned candidate region and inputs the feature vectors into the multi-task decoding head.

[0022] To overcome the feature distribution shift caused by differences in imaging conditions between well-labeled source domain images and unlabeled or poorly labeled target domain images, and to achieve effective cross-domain knowledge transfer, this invention first innovatively designs the feature extraction network. Specifically, in some embodiments, the feature extraction network is as follows: Figure 3 As shown, the system includes a feature extractor and a domain discriminator, which work together to support adversarial training. The feature extractor combines ResNet50 with a feature pyramid network to extract features from both the target and source domain images, obtaining target domain features and source domain features respectively. The domain discriminator uses fully connected layers to perform binary classification on the target and source domain features, outputting a probability prediction of the feature source. Based on this probability prediction, the feature extractor is adjusted to minimize the difference between the target and source domain features; the target domain features at this point are the domain-invariant features.

[0023] In the feature extractor provided in this embodiment of the invention, ResNet50 serves as the backbone network, and outputs feature maps with different spatial resolutions through its four residual stages, corresponding to 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image size, respectively. The feature pyramid network takes these multi-scale feature maps as input: first, it applies a 1×1 convolution to each layer of feature maps to unify the number of channels to 256; then, starting from the highest layer, it upsamples the high-level semantic features to the next layer scale through nearest neighbor upsampling, and adds them element-wise to the features of that layer after the 1×1 convolution; finally, it applies a 3×3 convolution to each fusion result to eliminate aliasing and generate multi-scale feature maps.

[0024] In this embodiment of the invention, the feature extraction network is trained using gradient reversal, wherein: the feature extraction network performs an identity transformation during forward propagation to ensure lossless transmission of feature information; during backward propagation, the gradients back propagated from the upstream in the feature extraction network are multiplied by a negative coefficient before continuing backward propagation, that is: ; Where G represents the output of the feature extraction network during forward propagation of the input x, and λ represents the coefficient used to adjust the adversarial strength. This represents the gradient propagated upstream. This represents the gradient calculation during backpropagation.

[0025] To achieve instance-level refined assessment of building damage status and overcome the problems of class imbalance and ambiguous boundaries in semantic segmentation methods, this invention reconstructs the classification prediction head based on the Mask R-CNN instance segmentation framework, designing a multi-level classification branch. Specifically, in some embodiments, the multi-task decoding head is as follows: Figure 4 As shown, the system includes a hierarchical classifier, a bounding box regression head, and an adaptive masking head. The hierarchical classifier comprises a first-level classifier and a second-level classifier. The first-level classifier determines whether a candidate region is a damaged building. In this embodiment, the classification result of the first-level classifier includes "background," "undamaged building," or "damaged building," thereby filtering out damaged targets that require further evaluation. The second-level classifier analyzes the degree of damage to the candidate region and outputs a damage level. In this embodiment, the second-level classifier performs fine-grained level classification of damaged buildings in the candidate region, outputting three damage levels: "mild," "moderate," and "severe." The bounding box regression head determines the location coordinates of buildings in the candidate region and generates bounding boxes for the buildings in the target domain image based on these coordinates. The adaptive masking head generates mask templates for damaged and undamaged buildings in the candidate region based on the building bounding boxes.

[0026] In some embodiments, both the first-level classifier and the second-level classifier are fully connected networks. In this embodiment, the first-level classifier performs a softmax operation on the feature vector output by the feature extraction network and outputs a classification probability, with categories including "background (0)", "undamaged (1)", and "damaged (2)"; the second-level classifier performs a softmax operation on the feature vector output by the feature extraction network and outputs a classification probability for the degree of damage. In the hierarchical classifier provided in this embodiment, the output category probability distribution P final The probability distribution P of each category is in array form. final The first term in the table represents the probability that the output result is "background (0)", and the category probability distribution P is given. finalThe second term in the equation is the probability that the output result is "undamaged (1)", and the category probability distribution P final The subsequent terms in the table represent the probability of each damage category; the final classification result is the category probability distribution P. final The index with the highest probability is used to determine the outcome.

[0027] The adaptive masking terminal is a fully convolutional network. In this embodiment of the invention, the number of channels of the mask template output by the adaptive masking terminal needs to be consistent with the number of final categories that can be predicted, so as to generate a corresponding mask template for each category. For example, the first-level classifier outputs three categories (i.e., "background (0)", "undamaged (1)", and "damaged (2)"), and the second-level classifier outputs three damage levels (i.e., "mild", "moderate", and "severe"). At this time, the number of final categories that can be predicted is 3+3=6, so the number of channels of the mask template output by the adaptive masking terminal is also 6, and each channel in the output mask template corresponds to a mask of one category.

[0028] S3: Using an iterative training method that alternates between the source domain and the target domain, the building damage assessment system constructed in step S2 is trained using the training set obtained in step S1 to obtain the building damage assessment model.

[0029] To achieve end-to-end training of a cross-domain building damage assessment model, this invention designs a complete adversarial joint optimization and training strategy. This strategy involves constructing a unified multi-objective loss function that integrates instance segmentation and domain alignment tasks, designing an iterative training process that alternates between the source and target domains, and introducing an adaptive scheduling mechanism for adversarial training intensity. This systematically drives model parameter optimization, ultimately enabling the model to accurately retain its source domain recognition capability while achieving strong generalization ability to target domain data.

[0030] Specifically, in some embodiments, in step S3: In the source domain training process: input the source domain image and its annotation, set the domain label to 0, and the total loss function during training is: ; in, This represents the total loss function for the source domain training process. The total loss includes instance segmentation loss and domain adversarial loss L. dom Instance segmentation loss includes object detection loss L det and mask loss L mask β represents the hyperparameter for balancing the weights of the recognition task; In the target domain training process: Input an unlabeled target domain image, set the domain label to 1, and the total loss function during training is: ; in, This represents the total loss function for the target domain training process.

[0031] In this embodiment of the invention, the total loss function is calculated during the forward propagation of the training process; and all parameters in the model are updated synchronously during the backpropagation of the training process. This training ensures that the model learns to perform high-precision building detection, hierarchical classification, and segmentation in the source domain.

[0032] Domain confrontation loss L dom for: ; Where N represents the batch size for training, f i d represents the domain-invariant feature of the i-th sample in the batch output by the domain discriminator. i Let σ represent the domain label corresponding to the domain-invariant feature of the i-th sample, σ represent the sigmoid function, and D represent the domain discriminator. Mask loss L mask The binary cross-entropy loss is defined as the difference between the true mask composed of the actual classification results and the mask template output by the adaptive masking system. ; Where M represents the total number of positive sample candidate boxes (i.e. the total number of candidate boxes corresponding to the first-level classifier output categories "undamaged (1)" and "damaged (2)"). This represents the pixel value (0 or 1) at position (u,v) in the m-th positive sample candidate box in the real mask. This represents the pixel value (0 or 1) at position (u,v) in the m-th positive sample candidate box in the mask template. Target detection loss L det for: L det =L rpn +L cls +L box ; Among them, L rpn L represents the suggested loss for the region. cls L represents the hierarchical classification loss. box This represents the bounding box regression loss; Recommended loss for the region L rpn for: ; Where, N rpn p represents the total number of samples. r This represents the true sample label of the r-th sample (foreground is a positive sample, corresponding to p). r A value of 1 indicates a negative background sample, corresponding to p. r (0) λ represents the probability of predicting the r-th sample as a positive sample, λ represents the weight of balancing the classification and regression losses (1 by default in this embodiment), and t r The regression parameters represent the true bounding box of the r-th sample. This represents the regression parameters of the corresponding predicted bounding box. Hierarchical classification loss L cls for: L cls =L cls1 +α×L cls2 ; L cls1 L represents the classification loss of the first-level classifier. cls2 α represents the classification loss of the second-level classifier, and α represents the balancing weight. The classification loss L of the first-level classifier cls1 The standard multi-class cross-entropy loss is as follows: ; in, This represents the true class that the first-level classifier gives when classifying the candidate region of the i-th sample. This represents the probability corresponding to the true category. This represents the predicted probability output by the first-level classifier when it actually classifies the candidate region of the i-th sample and predicts the j-th category.

[0033] The classification loss L of the second-level classifier cls2 for: ; Where Ω represents the subset of samples labeled as damaged buildings. This represents the true damage level given by the secondary classifier when classifying the candidate region of the i-th sample. This represents the probability corresponding to the actual level of damage. This represents the predicted probability output by the secondary classifier when classifying the candidate region of the i-th sample and predicting the c-th damage level, where C represents the total number of damage levels. This loss is calculated only for the subset Ω of samples with the primary true label "damaged"; for background and undamaged samples, the secondary loss is zero. This forces the secondary classifier to focus on learning the subtle differences within damaged samples, avoiding interference from undamaged samples in fine-grained classification learning.

[0034] In some embodiments, the bounding box regression loss L box for: ; in, Represents all positive samples. This represents the true class that the first-level classifier gives when classifying the candidate region of the i-th sample. k represents the bounding box regression parameters for the predicted i-th positive sample. i Indicates the corresponding final category, This represents the regression parameters of the corresponding true bounding box.

[0035] The smoothness in the above formula L1 The function is as follows: ; in, The smoothing parameter is 1 / 9 in this embodiment of the invention.

[0036] In this embodiment of the invention, the total loss function is calculated during the forward propagation of the training process; during the backward propagation of the training process, the feature extraction network is subjected to gradient reversal, and the feature extraction network receives the gradient with reversed sign. The parameters are updated along the direction of maximizing the domain adversarial loss, that is, the target domain features are learned to generate features that are difficult for the domain discriminator to distinguish, thereby aligning the target domain feature distribution with the source domain.

[0037] By repeatedly performing the above alternating training, the feature extraction network is optimized to simultaneously satisfy two objectives: extracting features that are highly discriminative for completing the instance segmentation task, and extracting domain-invariant features that are insensitive to domain changes. The domain discriminator continuously evolves in adversarial training, and the model eventually converges to an equilibrium state.

[0038] In some embodiments, the feature extractor and the domain discriminator in the feature extraction network in step S3 form an adversarial game, the objective of which can be formalized as the following minimax optimization problem, that is, the domain discriminator is optimized by the objective function of the following formula: ; Where F represents the feature extractor, D represents the domain discriminator, and D s and D t Let x represent the data distribution in the source domain image and the target domain image, respectively. s and x t Let L represent the data in the source and target domain images, respectively. The goal of the domain discriminator is to minimize the domain adversarial loss L. dom To accurately distinguish the source of features. In this embodiment of the invention, due to the effect of the gradient reversal layer, the feature extractor actually maximizes and minimizes the domain adversarial loss L when updating parameters. dom By confusing the domain discriminator, this adversarial process continues, thereby forcing the feature extractor to learn to generate features that are insensitive to imaging conditions and imaging differences, i.e., domain-invariant features.

[0039] In this embodiment of the invention, the parameter λ in gradient inversion is also adaptively scheduled. In the early stages of training, the feature distributions differ significantly, so a smaller parameter λ is set. As training progresses and the model gains a preliminary understanding of the source domain task, the parameter λ is gradually increased to strengthen the optimization signal for domain alignment. The adaptive scheduling process is as follows: ; Where, λ epoch Let λ represent the parameter λ during epoch training. start and λ end These represent the initial and target values ​​of the parameters, respectively. N total Φ represents the total training period, and Φ represents the progress function, which can be configured as a linear or S-shaped growth curve.

[0040] In this embodiment of the invention, the environment in which the method is implemented is as follows: the computer GPU is an NVIDIA GeForce RTX 4080, the operating system is Ubuntu 20.04.5 LTS, the CPU processor is an Intel(R) Xeon(R) Platinum 8352V, the Python version is 3.8.10, the PyTorch version is 2.0.0+cu118, the CUDA version is 11.8, and the cuDNN version is 8.7.0.

[0041] S4: Input the target domain image to be evaluated into the building damage assessment model obtained in step S3, and output the predicted building damage assessment result.

[0042] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0043] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for assessing building damage based on cross-domain feature alignment and instance segmentation, characterized in that, include: S1: Obtain the target domain image and its corresponding source domain image, and label the labels in the source domain image to obtain the building damage assessment results for each label in the source domain image; combine the target domain image, the source domain image, and the building damage assessment results into a training set; S2: Construct a building damage assessment system; the building damage assessment system includes a feature extraction network, a region proposal network, and a multi-task decoder: the feature extraction network extracts features from the target domain image and the corresponding source domain image respectively, and then aligns the obtained target domain features with the source domain features to obtain domain-invariant features; The region proposal network extracts building regions from domain-invariant features to generate candidate regions; the multi-task decoder decodes and performs damage analysis on the obtained candidate regions to obtain the complete identification and evaluation results of damaged buildings in the target domain image. S3: Using an iterative training method that alternates between the source domain and the target domain, the building damage assessment system constructed in step S2 is trained using the training set obtained in step S1 to obtain the building damage assessment model. S4: Input the target domain image to be evaluated into the building damage assessment model obtained in step S3, and output the predicted building damage assessment result.

2. The building damage assessment method based on cross-domain feature alignment and instance segmentation according to claim 1, characterized in that, The feature extraction network in step S2 includes: The feature extractor, ResNet50 combined with the feature pyramid network, extracts features from the target domain image and the source domain image respectively, obtaining target domain features and source domain features respectively; The domain discriminator uses a fully connected layer to perform binary classification of target domain features and source domain features, outputs a probability prediction of the source of the features, and adjusts the feature extractor according to the probability prediction to minimize the difference between the target domain features and the source domain features. At this point, the target domain features are the domain-invariant features.

3. The building damage assessment method based on cross-domain feature alignment and instance segmentation according to claim 2, characterized in that, The multi-task decoding head in step S2 includes: The hierarchical classifier includes a first-level classifier and a second-level classifier. The first-level classifier determines whether a candidate region is a damaged building. When the first-level classifier determines that a candidate region is a damaged building, the second-level classifier analyzes the degree of damage to the candidate region and outputs the damage level. The bounding box regression head determines the location coordinates of buildings in the candidate region and generates bounding boxes of the buildings in the target domain image based on the location coordinates; The adaptive masking tool generates mask templates for damaged and undamaged buildings based on the classification results of the hierarchical classifier and the building's bounding box.

4. The building damage assessment method based on cross-domain feature alignment and instance segmentation according to claim 3, characterized in that, Both the first-level and second-level classifiers are fully connected networks, while the adaptive masking network is a fully convolutional network.

5. The building damage assessment method based on cross-domain feature alignment and instance segmentation according to claim 3, characterized in that, In step S3: In the source domain training process: input the source domain image and its annotation, set the domain label to 0, and the total loss function during training is: ； in, This represents the total loss function of the source domain training process. The total loss function includes instance segmentation loss and domain adversarial loss L. dom Instance segmentation loss includes object detection loss L det and mask loss L mask β represents the hyperparameter for balancing the weights of the recognition task; In the target domain training process: Input an unlabeled target domain image, set the domain label to 1, and the total loss function during training is: ； in, This represents the total loss function for the target domain training process; Domain confrontation loss L dom for: ； Where N represents the batch size for training, f i d represents the domain-invariant feature of the i-th sample in the batch output by the domain discriminator. i Let σ represent the domain label corresponding to the domain-invariant feature of the i-th sample, σ represent the sigmoid function, and D represent the domain discriminator. Target detection loss L det for: L det =L rpn +L cls +L box ; Among them, L rpn L represents the suggested loss for the region. cls L represents the hierarchical classification loss. box This represents the bounding box regression loss; Mask loss L mask The cross-entropy loss is the binary cross-entropy loss between the true mask composed of the true classification results and the mask template output by the adaptive masking system.

6. The building damage assessment method based on cross-domain feature alignment and instance segmentation according to claim 5, characterized in that, Hierarchical classification loss L cls for: L cls =L cls1 +α×L cls2 ; L cls1 L represents the classification loss of the first-level classifier. cls2 α represents the classification loss of the second-level classifier, and α represents the balancing weight. The classification loss L of the first-level classifier cls1 The standard multi-class cross-entropy loss is as follows: ； in, This represents the true class that the first-level classifier gives when classifying the candidate region of the i-th sample. This represents the probability corresponding to the true category. This represents the predicted probability output by the first-level classifier when it actually classifies the candidate region of the i-th sample and predicts the j-th category. The classification loss L of the second-level classifier cls2 for: ； Where Ω represents a subset of samples labeled as damaged buildings. This represents the true damage level given by the secondary classifier when classifying the candidate region of the i-th sample. This represents the probability corresponding to the actual level of damage. This represents the predicted probability output by the secondary classifier when it actually classifies the candidate region of the i-th sample and predicts the c-th damage level, where C represents the total number of damage levels.

7. The building damage assessment method based on cross-domain feature alignment and instance segmentation according to claim 5, characterized in that, Bounding box regression loss L box for: ； in, Represents all positive samples. This represents the true class that the first-level classifier gives when classifying the candidate region of the i-th sample. k represents the bounding box regression parameters for the predicted i-th positive sample. i Indicates the corresponding final category, This represents the regression parameters of the corresponding true bounding box.

8. The building damage assessment method based on cross-domain feature alignment and instance segmentation according to claim 5, characterized in that, In step S3, the domain discriminator is optimized using the objective function of the following formula: ； Where F represents the feature extractor, D represents the domain discriminator, and D s and D t Let x represent the data distribution in the source domain image and the target domain image, respectively. s and x t These represent the data in the source domain image and the target domain image, respectively.

Images