Method, device and equipment for assessing building damage degree

Abstract

The present invention provides a method, device, and equipment for assessing the extent of building damage. The method comprises: obtaining visual data reflecting the extent of building damage in a first scenario; obtaining prior knowledge data related to the extent of building damage in a second scenario; inputting the visual data into a first classifier for building damage classification, thereby obtaining a first classification result; the first classifier is trained using a first preset model; inputting the knowledge data into a second classifier for building damage classification, thereby obtaining a second classification result; the second classifier is trained using a second preset model; and inputting the first and second classification results into a fusion model for building damage assessment, thereby obtaining a building damage classification result. The present invention can improve the accuracy of building damage classification.

Description

Technical Field

[0001] The present invention relates to the technical field of image information evaluation and processing, and in particular to a method, device and equipment for evaluating the extent of building damage. Background Art

[0002] Earthquakes cause significant damage to buildings and infrastructure, resulting in enormous property losses and casualties. Building collapse is a leading cause of death and injury in earthquakes, particularly in areas with poor construction standards or inadequate enforcement of building codes. Furthermore, damaged infrastructure complicates rescue efforts and increases the risk of secondary disasters such as fire and environmental contamination. Therefore, assessing the extent of building damage is crucial for post-disaster loss assessment.

[0003] Building damage assessment methods are usually based on macro-level images for structural damage detection, which is less effective in identifying subtle damage features and has poor assessment results. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method, device and equipment for assessing the extent of damage to a building, which can improve the accuracy of the assessment of the extent of damage to the building.

[0005] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0006] A method for assessing the extent of building damage, comprising:

[0007] Obtain visual data reflecting the extent of building damage in the first scene;

[0008] Acquire prior knowledge data related to the extent of building damage in the second scenario;

[0009] Inputting the visual data into a first classifier to classify the degree of building damage to obtain a first classification result, wherein the first classifier is trained by a first preset model;

[0010] Inputting the knowledge data into a second classifier to classify the degree of building damage to obtain a second classification result, wherein the second classifier is trained by a second preset model;

[0011] The first classification result and the second classification result are input into a fusion model to perform building damage degree assessment processing to obtain a building damage degree assessment classification result.

[0012] An embodiment of the present invention further provides a building damage assessment device, comprising:

[0013] An acquisition module, configured to acquire visual data reflecting the extent of damage to the building in a first scenario and prior knowledge data related to the extent of damage to the building in a second scenario;

[0014] The processing module is used to input the visual data into a first classifier to perform building damage degree classification processing to obtain a first classification result, and the first classifier is obtained by training a first preset model; input the knowledge data into a second classifier to perform building damage degree classification processing to obtain a second classification result, and the second classifier is obtained by training a second preset model; input the first classification result and the second classification result into a fusion model to perform building damage degree assessment processing to obtain a building damage degree assessment classification result.

[0015] An embodiment of the present invention also provides a computing device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the building damage assessment method described in the present invention.

[0016] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a program, and when the program is executed by a processor, the building damage degree assessment method of the present invention is implemented.

[0017] The above technical solution of the present invention has at least the following technical effects:

[0018] The above-mentioned building damage assessment method of the present invention obtains visual data reflecting the building damage level in a first scenario; obtains prior knowledge data related to the building damage level in a second scenario; inputs the visual data into a first classifier trained with a first preset model to classify the building damage level, thereby obtaining a first classification result; inputs the knowledge data into a second classifier trained with a second preset model to classify the building damage level, thereby obtaining a second classification result; and inputs the first and second classification results into a fusion model to perform building damage assessment processing, thereby obtaining a building damage assessment classification result. This improves the accuracy of building damage assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of the building damage assessment method of the present invention;

[0020] Figure 2 2 is a schematic diagram of the module architecture of the first classifier of the building damage assessment method of the present invention;

[0021] Figure 32 is a schematic diagram of a training accuracy curve of the first classifier of the building damage assessment method of the present invention;

[0022] Figure 4 2 is a schematic diagram of a training loss curve of a second classifier of the building damage assessment method of the present invention;

[0023] Figure 5 Schematic diagram of evaluation indicators of the building damage assessment model of the present invention and the performance of the non-fusion model;

[0024] Figure 6 Schematic diagram of the building damage assessment device of the present invention. DETAILED DESCRIPTION

[0025] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0026] like Figure 1 As shown, an embodiment of the present invention provides a method for assessing the extent of building damage, comprising:

[0027] Step S1: Obtain visual data reflecting the extent of building damage in a first scene; here, the first scene may be a ground scene of the building after the disaster; the visual data may be image data such as ground photos of the building after the disaster. It should be noted that such visual data does not include geographic location information;

[0028] Step S2: Acquire prior knowledge data related to the extent of building damage in a second scenario. Here, the second scenario may be a ground scene of the building after the disaster. The prior knowledge data includes ground photos of the building after the disaster and geographic location data stored in EXIF ​​metadata, which provides latitude and longitude coordinates. This geographic location data is crucial for understanding the spatial distribution of damage.

[0029] Step S3: inputting the visual data into a first classifier to classify the degree of building damage to obtain a first classification result, wherein the first classifier is trained by a first preset model;

[0030] Step S4: inputting the knowledge data into a second classifier to classify the building damage degree to obtain a second classification result, wherein the second classifier is trained by a second preset model;

[0031] Step S5: Input the first classification result and the second classification result into a fusion model to perform building damage assessment processing to obtain a building damage assessment classification result.

[0032] In this embodiment, Figure 1 As shown in the building damage assessment method,

[0033] First, building image data is obtained, where the image data includes visual data reflecting the extent of damage to the building in a first scenario and prior knowledge data related to the extent of damage to the building in a second scenario;

[0034] Then, the visual data in the acquired building image data is preprocessed by resizing and normalizing the image pixel values, and the knowledge data in the building image data is preprocessed by searching and calculating prior knowledge variables, so as to obtain visual data and knowledge data that are easy to process and use.

[0035] Next, the preprocessed visual data is input into a first classifier for processing to obtain a first classification result. The first classifier is trained using a first preset model, which is an image-based deep learning model. The first classifier is formed by training a certain number of visual image data of buildings with determined damage degree classifications.

[0036] Thirdly, the preprocessed knowledge data is input into a second classifier for processing to obtain a second classification result. The second classifier is trained using a second preset model, which is a machine learning model based on a geographic information system (GIS). The second classifier is formed by training a certain number of building image knowledge data with determined damage degree classifications.

[0037] Finally, the first and second classification results are input into the fusion model for processing to obtain the classification result of the building damage degree. The fusion model provides a dynamic confidence weighting mechanism to integrate the predictions of the first and second classifiers to produce a comprehensive and accurate classification.

[0038] The ground imagery used in this solution includes visual data and prior knowledge of geographic location, enabling the incorporation of broader and more comprehensive information on building damage severity, thereby facilitating large-scale and rapid assessment of disaster damage. By utilizing multimodal fusion methods, the classification approach integrates both visual and non-visual factors, improving the accuracy of building damage assessments and enhancing its applicability across a wider range of scenarios and emergency response phases.

[0039] In an optional embodiment of the present invention, in step S3, the training process of the first classifier includes:

[0040] Step S31, obtaining first training set data, where the first training set includes classification results of building images;

[0041] Step S32, preprocessing the first training set data to obtain first target data;

[0042] Step S33: inputting the first target data into multiple convolution processing layers of the first preset model to obtain a first output result;

[0043] Step S34: inputting the first output result into the fully connected layer of the first preset model to obtain a second output result;

[0044] Step S35, obtaining a predicted loss value according to the second output result and the classification result of the first target data;

[0045] Step S36: Adjust the hyperparameters of the first preset model according to the predicted loss value to obtain a first classifier.

[0046] In this embodiment, the first training set data is a certain number of building image visual data with determined damage classification, which includes damage classification results. The building damage severity classification standards are shown in the following table:

[0047] Table 1 Standards for severity of building damage

[0048] Extent of damage describe Total Damage (TD) The building collapsed or suffered irreparable damage and needed to be completely rebuilt. Severe damage (SD) The load-bearing structure is severely damaged or partially collapsed, requiring extensive repairs or partial demolition. Minor damage (MD) Minor cracks in the load-bearing structure can be restored with minimal repairs. No Damage (ND) There is no visible structural damage and no need for repair or reconstruction.

[0049] Then, the first training set data is preprocessed. Due to the imbalanced data distribution problem in the first training set data, it is necessary to supplement the original post-disaster building image dataset through network search. In order to further enhance the diversity and generalization ability of the visual image dataset, the crisis benchmark dataset can also be used in combination. This dataset summarizes a widely used post-disaster image collection, including building damage images from different disaster events and time periods. The addition of this dataset introduces a wider range of visual information, significantly improving the diversity and overall representativeness of the dataset; the visual data in the acquired building image dataset is preprocessed by resizing and normalizing the image pixel values, and the first The training set data is adjusted to a preset size to obtain first intermediate data; the first intermediate data is normalized to obtain first target data; specifically, when preprocessing the first training set data, the size of the visual image of the data set is first uniformly adjusted to a fixed size, preferably adjusted to 224×224 pixels, to obtain first intermediate data; the image pixel values ​​are then normalized, and the mean of the data set is subtracted from each pixel value, and the pixel values ​​are mapped to a range of 0-1 or -1 to 1 to accelerate the degree of convergence when the data is used; the data set is again divided into a training set, a validation set, and a test set in a ratio of 8:1:1, and the training set is used as the first target data.

[0050] Again, the first target data is input into the convolution processing layer of the first preset model to obtain a first output result; preferably, the first preset model is a convolutional neural network model including 16 weight layers, specifically including 13 convolution layers and 3 fully connected layers, which gradually process the input image, and the extracted visual features become more complex as the layers go deeper. The first preset model uses small 3×3 filters in all convolution layers, so that it can efficiently capture the details in the image while maintaining computational feasibility; the first preset model includes multiple convolution layers, which perform convolution operations on the image by sliding the convolution kernel to extract the features of the image. During the sliding process, the convolution kernel performs numerical computations with each small area of ​​the image. The convolution layer learns operations and generates new feature maps. Each convolution layer learns features at different levels, from simple features such as edges and lines at the bottom layer to more complex and abstract features at the top layer, such as the local shape and texture of objects. The first preset model uses a 3×3 convolution kernel. The depth of the model is increased by stacking multiple convolution layers, so that richer and more advanced features can be learned. After the convolution layer, there is usually a pooling layer. The role of the pooling layer is to downsample the feature map extracted by the convolution layer, reduce the amount of data, reduce the computational complexity of the model, and prevent overfitting to a certain extent. The first preset model mainly uses the maximum pooling method, that is, the maximum value in the pooling window is taken as the pooling result.

[0051] Again, the first output result is input into the fully connected layer of the first preset model to obtain the second output result; after being processed by multiple convolutional layers, the obtained feature map is flattened into a one-dimensional vector and then input into the fully connected layer. The neurons in the fully connected layer are connected to all the neurons in the previous layer. Its function is to perform comprehensive analysis and judgment on the extracted features and map the features to different categories. The first preset model usually contains 3 fully connected layers. The number of neurons in the first two fully connected layers is relatively large, generally 4096. The number of neurons in the last fully connected layer is equal to the number of categories of the classification task. In this embodiment, the number of categories is set to 4. The output of the last fully connected layer is processed by the Softmax function, and the output value is converted into a probability distribution belonging to each category. The category with the highest probability is the model's prediction result for the input image.

[0052] In this embodiment, during the training of the first preset model, an optimizer was used with a learning rate set to 0.0001, as it can adjust the learning rate during training to optimize the model. The batch size was set to 8, and training was performed for 20 epochs.

[0053] To further improve model performance and reduce overfitting, this method also applies data augmentation techniques such as random cropping, horizontal flipping, and rotation, which artificially increase the diversity of the training images. Finally, the selected optimizer is used to update the model's hyperparameters, including the convolution kernel size, learning rate, and batch size. Hyperparameters are updated in the opposite direction of the gradient, gradually reducing the loss value. After multiple training cycles, the first classifier is obtained.

[0054] In an optional embodiment of the present invention, in step S35, obtaining a predicted loss value according to the second output result and the classification result of the first target data includes:

[0055] Step S351: Obtain a predicted loss value through a cross entropy loss function between the second output result and the classification result of the first target data. The cross entropy loss function formula is as follows:

[0056]

[0057] Indicates loss, w i represents the weight assigned to each category i, y i is the true category label, is the predicted probability of category i. Here, if the image belongs to category i, it is 1, otherwise it is 0;

[0058] In this embodiment, due to the imbalance problem of the data set (i.e., there are fewer samples in certain building damage categories), this embodiment applies a weighted category cross entropy loss function, so that the first preset model can better identify minority class images, thereby improving the overall classification accuracy of all damage levels.

[0059] In an optional embodiment of the present invention, in step S3, the training process of the first classifier further includes:

[0060] Step S37, inputting the second output result into the activation mapping module of the first preset model to obtain an output gradient;

[0061] Step S38: Obtain a feature heat map according to the output gradient and the first output result.

[0062] In this embodiment, in order to improve the interpretability of the prediction of the first preset model, the training process of the first classifier also includes: inputting the second output result into the activation mapping module of the first preset model to obtain an output gradient; and obtaining a feature heat map based on the output gradient and the first output result.

[0063] The activation mapping module of the first preset model highlights the areas in the image that the model believes are most relevant to the classification decision. The activation mapping module of the first preset model can help intuitively understand why the model classifies an image as a specific damage category, such as showing a collapsed area in a severely damaged building. The activation mapping module of the first preset model outputs gradients for a specific category (e.g., "severe damage") and compares them with the activation values ​​of the last convolutional layer, thereby visualizing the category-specific decision process in detail. By calculating the importance of feature maps, the activation mapping module of the first preset model assigns weights to these maps and generates a feature heat map that highlights the image areas that are most relevant to the model prediction.

[0064] This process, illustrated below, provides interpretable insights into the most contributing regions.

[0065]

[0066] here, A represents the heat map of category c. k represents the activations (detected features) from the last convolutional layer for feature map k. is the weight of feature map k, derived from the class score of class c relative to the activation A k The ReLU function (ReLU(x)=max(0,x)) ensures that only positive effects are considered when generating the heat map.

[0067] Calculate using the following formula

[0068]

[0069] here, is the gradient of the output score for category c with respect to the activation at position (i, j) in feature map k, and Z is the total number of pixels in the feature map, which is used for normalization.

[0070] Essentially, the first pre-built model highlights parts of the image that have a significant impact on the deep learning model's decision. By tracing gradients back to the last convolutional layer, it identifies regions that are crucial for classification. For example, if an image of a building is classified as "heavily damaged," the first pre-built model might highlight collapsed walls or rubble, indicating the importance of these areas in the model's decision.

[0071] In an optional embodiment of the present invention, in step S4, the second classifier training process includes:

[0072] Step S41, obtaining second training set data, where the second training set data includes classification results of building images with building location information;

[0073] Step S42, preprocessing the second training set data to obtain second target data;

[0074] Step S43: inputting the second target data into the first processing layer of the second preset model, performing random sampling, and obtaining a third output result;

[0075] Step S44: input the third output result into the second processing layer of the second preset model, perform random feature selection and a preset number of node splitting, and obtain a fourth output result;

[0076] Step S45: Adjust the hyperparameters of the second preset model according to the fourth output result and the classification result of the second target data to obtain a second classifier.

[0077] In this embodiment, the second preset model training process includes: first obtaining second training set data, which is a certain number of building image knowledge data with a certain degree of damage classification, including damage degree classification results; then performing preprocessing such as prior knowledge variable search and calculation on the knowledge data in the building image data to obtain second target data; through the first processing layer of the second preset model, randomly sampling with replacement from the second target data to construct multiple different self-service sample sets to obtain a third output result, where the size of each self-service sample set is the same as the size of the original training set, but due to the replacement sampling, each self-service sample set may contain some repeated samples, and some samples may not be sampled; again, the third output result is input into the second processing layer of the second preset model to perform random feature selection and a preset number of node splitting. , obtaining the fourth output result; when splitting each node, randomly select a feature subset, and then select the optimal feature from this feature subset for splitting. The size of the feature subset selected in this embodiment is the square root of the total number of features, which can increase the diversity between decision trees and reduce the correlation between trees; for each node, based on the selected optimal feature and the corresponding splitting criterion (such as information gain, Gini index, etc.), the sample on the node is divided into two child nodes, so that the impurity (such as Gini impurity, information entropy, etc.) of the divided child nodes is reduced as much as possible; repeat this process until a preset stopping condition is reached, such as the number of samples on the node is less than a certain threshold, the depth of the tree reaches a preset value, the decrease in impurity is less than a certain threshold, etc.; in order to prevent the decision tree from overfitting, the generated decision tree needs to be pruned, which can be done in two ways: pre-pruning and post-pruning. Pre-pruning is to stop the growth of the tree in advance according to some conditions during the decision tree generation process; post-pruning is to judge whether some subtrees need to be pruned from the leaf nodes from the bottom up after the decision tree is generated; repeat the above decision tree generation steps to construct multiple decision trees to form a random forest. Each decision tree is trained based on a different bootstrap sample set and feature subset, so there are certain differences between them; for classification tasks, the voting method is usually used, that is, each decision tree makes a classification prediction for the sample, and finally counts the prediction results of all decision trees, and selects the category with the most votes as the prediction result of the random forest; the second preset model hyperparameters include the number of decision trees (n_estimators), the maximum depth of each tree (max_depth), and the number of features considered at each split (max_features). These hyperparameters are fine-tuned through grid search and cross-validation techniques, and finally a robust and accurate second classifier is obtained.

[0078] In an optional embodiment of the present invention, in step S42, preprocessing the second training set data to obtain second target data includes:

[0079] Step S421, extracting elevation data and location information from the second training set data;

[0080] Step S422, obtaining second intermediate data based on the elevation data and the position information;

[0081] Step S423, performing interpolation processing on the second intermediate data to obtain second target data;

[0082] Wherein, the geographic information in the second intermediate data is obtained according to the elevation data, and the normalized vegetation index and earthquake impact data in the second intermediate data are obtained according to the position information.

[0083] In this embodiment, the second training set data is preprocessed, and the elevation (i1), slope (i2), orientation (i3) and terrain undulation (i4) of the corresponding area are calculated by using the elevation data (DEM) in the second training set data to enhance the geographical factors, thereby better representing the geographical background of the disaster area. In addition, the normalized difference vegetation index (NDVI) (i5) of the relevant area is also added. NDVI provides valuable insights into the health and coverage of vegetation, which can reflect the recovery of vegetation after the disaster and its potential impact on the stability of the landscape and buildings. Furthermore, based on the earthquake information and image location information, the distance from each image position to the epicenter (i6), the distance to the central fault zone (i7) and the epicenter intensity (i8) corresponding to each image are calculated, and these three parameters are used as important indicators to measure the impact of earthquakes on buildings. These eight parameters are used as a machine learning prior knowledge data set, as shown in the following table:

[0084] Table 2 Prior knowledge variables of building image data

[0085] variable parameter type <![CDATA[i1]]> altitude Continuous variables <![CDATA[i2]]> slope Continuous variables <![CDATA[i3]]> Slope Continuous variables <![CDATA[i4]]> Topographic relief Continuous variables <![CDATA[i5]]> NDVI Continuous variables <![CDATA[i6]]> Epicenter distance Continuous variables <![CDATA[i7]]> Fault zone distance Continuous variables <![CDATA[i8]]> earthquake intensity discrete variables

[0086] A structured prior knowledge dataset extracted from the geographic locations of images will be used to construct and train a building damage severity assessment model.

[0087] The second intermediate data is then interpolated using synthetic minority oversampling, a method for balancing class labels in machine learning. Synthetic minority oversampling generates synthetic samples by interpolating between minority class instances, resulting in a more balanced training dataset. This improves the representativeness of minority class samples rather than simply replicating existing instances, thereby enhancing the robustness of the training process.

[0088] Specifically, for each minority class instance x, the synthetic minority class oversampling method selects one of its k nearest neighbors and generates a new synthetic sample x by linear interpolation. new, the formula is as follows:

[0089] x new =x+λ×(x neighbor -x)

[0090] Where λ is a random number in the range [0, 1], x represents the feature vector of the original instance from the minority class, and x neighbor Represents the feature vector of one of its selected nearest neighbors. The newly generated synthetic sample x new Located in the feature space connected and x neighbor This process enriches the dataset by effectively diversifying the representation of the minority class, enabling the model to learn more balanced representations of all categories.

[0091] In an optional embodiment of the present invention, in step S5, the first classification result and the second classification result are input into a fusion model to perform building damage assessment processing to obtain a building damage assessment classification result, including:

[0092] Step S51, obtaining a dynamic weight according to the first classification result and the second classification result;

[0093] Step S52: obtaining a building damage degree assessment classification result based on the dynamic weight.

[0094] In this embodiment, the probability outputs of the first classification result and the second classification result are fused to improve the accuracy of the building damage severity classification. This fusion is performed through a dynamic weighting method based on confidence, aiming to integrate the evaluation results of the two models and draw a more comprehensive conclusion. Specifically, the confidence of each model is defined as the highest probability value in its predicted probability distribution, representing the classification choice that the model is most confident in. The dynamic weight of the model is calculated as follows:

[0095]

[0096] Among them, confidence rf represents the highest probability in the output distribution of the second classifier, and confidence dl represents the highest probability among the first classifier outputs. These equations ensure that the weight w rf and w dl The sum of is 1, thus balancing the contributions of the two models.

[0097] The final combined prediction probability combined_prob is obtained by weighted averaging the probability distributions predicted by the two models, and the expression is as follows:

[0098] combined_prob=w rf ·Prf +w dl ·P dl

[0099] Among them, P rf and P dl denote the probability vectors predicted by the second and first classifiers, respectively. The final building damage category is determined by selecting the category with the maximum value in the combined probability vector combined_prob. By leveraging the relative strengths of the two models, this adaptive fusion approach dynamically optimizes their contributions, thereby improving classification robustness and accuracy.

[0100] The specific implementation process of the above method of the present invention is described below:

[0101] 1. Obtain photos of damaged buildings on the ground

[0102] The primary data source consists of images captured during post-disaster field surveys conducted within a specific region following an earthquake. A total of 10,171 images were acquired, documenting the condition of damaged buildings and infrastructure after the earthquake, providing a detailed visual record of damage and debris. Of these, 6,806 images contained geolocation data stored in EXIF ​​metadata, providing latitude and longitude coordinates. This geolocation data is crucial for understanding the spatial distribution of damage. It can be observed that the majority of photos were from the epicenters of magnitude VII and VIII earthquakes. The remaining 3,365 images, while lacking geolocation data, still provide valuable visual evidence for damage assessment. Based on the criteria for building damage severity, the number of photos showing mild damage is relatively high, while the number of images showing more severe damage is relatively low. Secondly, deep learning-based building damage severity assessment faces the problem of data imbalance, a problem also evident in the building damage photo dataset. To address this issue, the original post-disaster building image dataset was supplemented through web searches. After screening, 668 images of intact buildings and 671 images of completely damaged buildings were obtained, referred to as the "Web-Sourced Building Damage Dataset." These additional images introduce necessary diversity into the dataset and help balance the label distribution.

[0103] To further enhance the dataset's diversity and generalization capabilities, we also incorporate a Crisis Benchmark dataset. This dataset compiles a widely used collection of post-disaster images, including images of building damage from various disaster events and time periods. The inclusion of this dataset introduces a wider range of visual information, significantly improving the dataset's diversity and overall representativeness.

[0104] GIS data also plays a vital role in damage assessment in post-disaster scenarios, providing key insights into spatial factors for effective damage identification. In selecting prior knowledge data extracted from earthquake images, three key factors were focused on: (1) geographical factors related to buildings, (2) environmental conditions, and (3) destructive seismic factors associated with earthquakes. Based on the availability of relevant information, digital elevation model (DEM) data was ultimately chosen. ， , Normalized Difference Vegetation Index (NDVI) data, as well as earthquake epicenter location, central fault zone location and epicenter intensity distribution are used as the main data sources for prior knowledge extraction.

[0105] 2. Dataset Reconstruction

[0106] Based on the definition of earthquake damage severity, images from the web-sourced Building Damage Dataset and the Crisis Benchmark Dataset were re-annotated to ensure that all images met the damage level requirements of the research standard. After annotation, images from these three sources were combined to form a more comprehensive and multi-source image dataset. As a result, this dataset not only effectively demonstrates building damage caused by earthquakes but also includes building damage images from different regions and events, thereby enhancing the dataset's diversity.

[0107] Furthermore, this multi-source image dataset helps address the issue of unbalanced label distribution; the original earthquake image dataset was severely skewed, with most images classified as mildly damaged, while images showing complete destruction and no damage were relatively scarce. By introducing new data sources, the constructed dataset achieves a balanced distribution across the four damage levels, providing a more favorable foundation for subsequent modeling work. Finally, the dataset was partitioned into training, validation, and test sets in an 8:1:1 ratio, as shown in the following table:

[0108] Table 3 Distribution of sources and number of images in training, validation and test sets

[0109]

[0110] Maintaining the current data partitioning, and selecting only geotagged images from the training, validation, and test sets, a structured prior knowledge dataset was constructed for machine learning. By using DEM data, the elevation (i1), slope (i2), orientation (i3), and topographic relief (i4) of the corresponding area were calculated to enhance the geographical factors and better represent the geographical context of the disaster area. In addition, the normalized vegetation index (NDVI) (i5) of the relevant area was added. The normalized vegetation index provides valuable insights into vegetation health and cover. This information can reflect the post-disaster vegetation recovery and its potential impact on the stability of the landscape and buildings. Furthermore, based on the earthquake information, the distance from each image location to the epicenter (i6), the distance to the central fault zone (i7), and the epicenter intensity (i8) corresponding to each image were calculated. These three parameters serve as important indicators for measuring the impact of earthquakes on buildings. In general, eight parameters were obtained to form a machine learning prior knowledge dataset.

[0111] Two different datasets were constructed: one is a multi-source image dataset created by integrating multiple post-disaster building images, and the other is a structured prior knowledge dataset extracted from the geographic location of the images. These two datasets will be used to build a building damage severity assessment model.

[0112] 3. Deep Learning Methods for Assessing Building Damage Severity

[0113] Using a dataset of building damage photos, a web-sourced building damage dataset, and a crisis benchmark dataset, a pre-trained first classifier was used to classify building damage severity. Furthermore, an activation mapping module was employed to visualize the model's predictions, enhancing the interpretability of the classification results. This approach provides a deeper understanding of the model's decision-making process, particularly in identifying key damage features in the images.

[0114] The first classifier consists of 13 convolutional layers and 3 fully connected layers. Small 3x3 filters are used in all convolutional layers, which enables it to efficiently capture details in the image while keeping the computation feasible.

[0115] like Figure 2 As shown, in order to adapt the model to the task of the present invention (four damage severity categories), the last fully connected layer is adjusted to output these four categories instead of the original 1,000 categories.

[0116] During training, an optimizer was used with a learning rate of 0.0001, as it allows for model optimization by adjusting the learning rate during training. The batch size was set to 8, and training was performed for 20 epochs. Due to the imbalanced dataset (i.e., fewer samples for certain building damage categories), a weighted categorical cross entropy loss function was applied, which imposes a higher penalty on misclassification of smaller categories, as shown in the following formula:

[0117]

[0118] Here, Indicates loss, w i represents the weight assigned to each class i. (giving greater weight to less represented classes), y i is the true class label (1 if the image belongs to class i, 0 otherwise), is the predicted probability of class i.

[0119] This loss function helps the model better identify minority class images, thereby improving the overall classification accuracy for all damage levels. To further improve model performance and reduce overfitting, data augmentation techniques such as random crops, horizontal flips, and rotations are applied, which artificially increase the diversity of the training images.

[0120] It's worth noting that the model validation process didn't use all 1,428 images in the dataset. Instead, it was specifically validated on 702 images that contained geolocation information. This decision was made to ensure that the deep learning model's results were comparable to those of machine learning-based methods, which also used geotagged images for evaluation.

[0121] To improve the interpretability of the model's predictions, we used gradient-weighted class activation mapping. This method highlights the regions of the image that the model considers most relevant for the classification decision. The activation mapping module can help us intuitively understand why the model classified an image as belonging to a specific damage category, such as showing a collapsed area in a severely damaged building.

[0122] Activation mapping analyzes the gradient of the model's output for a specific class (e.g., "severely damaged") and compares it with the activation values ​​of the last convolutional layer, visualizing the class-specific decision-making process in detail. By calculating the importance of feature maps, activation mapping assigns weights to these maps and generates a heat map that highlights the image regions most relevant to the model's predictions. This process, shown in the formula below, provides interpretable insights into the regions that contribute the most.

[0123]

[0124] here The heatmap Ak representing the activation map of class c represents the activations (detected features) from the last convolutional layer for feature map k, is the weight of feature map k, which comes from the gradient of the category score of category c with respect to the activation Ak. The ReLU function (ReLU(x)=max(0,x)) ensures that only positive effects are considered when generating heat maps.

[0125] Calculate using the following formula

[0126]

[0127] here, is the gradient of the output score for category c with respect to the activation at position (i, j) in feature map k, and Z is the total number of pixels in the feature map, which is used for normalization.

[0128] By tracing gradients back to the last convolutional layer, it identifies regions that are critical for classification. For example, if an image of a building is classified as "severely damaged," the activation mapping module might highlight collapsed walls or rubble, indicating the importance of these areas in the model's decision. This technique improves model interpretability by visually displaying features associated with damage severity.

[0129] 4. Machine Learning Methods for Assessing Building Damage Severity

[0130] In the context of building damage severity assessment using a dataset with structured prior knowledge based on image geolocation, the lack of geolocation information in images limits the ability to enhance the distribution of class and feature labels by integrating data from other sources. To address this issue, a synthetic minority oversampling method was employed, which is highly effective in balancing the distribution of class labels. Subsequently, a random forest model was employed to effectively estimate building damage severity.

[0131] The synthetic minority oversampling method generates synthetic samples by interpolating between existing minority class instances, thereby obtaining a more balanced training dataset. The advantage of this technique is that it improves the representativeness of minority class samples rather than simply copying existing instances, thereby enhancing the robustness of the training process. Mathematically, for each minority class instance x, the synthetic minority oversampling method selects one of its k nearest neighbors and generates a new synthetic sample x through linear interpolation. new .

[0132] x new =x+λ×(x neighbor -x)

[0133] Here, λ is a random number in the range [0, 1], x represents the feature vector of the original instance from the minority class, and xneighbor Represents the feature vector of one of its selected nearest neighbors. The newly generated synthetic sample x new Located in the feature space connecting x and x neighbor This process enriches the dataset by effectively diversifying the representation of the minority class, enabling the model to learn more balanced representations of all categories.

[0134] After addressing the class imbalance, the random forest algorithm was used for injury severity classification. Random forest is an ensemble learning method that improves upon traditional decision trees by introducing bootstrap sampling and random feature selection to reduce variance. It constructs multiple decision trees from subsets of the training data and averages their predictions to determine the final class. Random forests are well-suited for high-dimensional data, resist overfitting, and effectively capture complex feature interactions, making them more robust than single decision trees.

[0135] The random forest model used in our solution was trained using a training dataset of 5,475 examples, with hyperparameters optimized to ensure optimal performance. Key parameters included the number of decision trees (n_estimators), the maximum depth of each tree (max_depth), and the number of features considered at each split (max_features). These parameters were fine-tuned using grid search and cross-validation techniques, resulting in a robust and accurate model.

[0136] Subsequently, a test dataset of 702 samples was used to verify the model's ability to accurately predict damage severity under different disaster characteristics.

[0137] 5. Decision-making layer integrates machine learning and deep learning for injury severity classification

[0138] Given the dimensionality differences between deep learning and machine learning methods, decision-level fusion was chosen to effectively combine their outputs. Specifically, the probabilistic outputs of a deep learning model (VGG16) and a machine learning model (Random Forest) were fused to improve the accuracy of building damage severity classification. This fusion was performed using a confidence-based dynamic weighting method, aiming to synthesize the evaluation results of the two models and draw more comprehensive conclusions.

[0139] Specifically, the confidence of each model is defined as the highest probability value in its predicted probability distribution, representing the classification choice that the model is most confident in. The dynamic weight of the model is calculated as follows:

[0140]

[0141] Here, confidence rfrepresents the highest probability in the random forest model output distribution, while confidence dl represents the highest probability in the output of the VGG16 model. These equations ensure that the weight w rf and w dl The sum of is 1, thus balancing the contributions of the two models.

[0142] The final combined prediction probability combined_prob is obtained by weighted averaging the probability distributions predicted by the two models, and the expression is as follows:

[0143] combined_prob=w rf ·P rf +w dl ·P dl

[0144] Here, P rf and P dl denote the probability vectors predicted by the Random Forest and VGG16 models, respectively. The final building damage category is determined by selecting the category with the maximum value in the combined probability vector combined_prob. By leveraging the relative strengths of the two models, this adaptive fusion approach dynamically optimizes their contributions, thereby improving classification robustness and accuracy.

[0145] 6. Evaluation Metrics

[0146] To evaluate the performance of deep learning, machine learning, and decision-level fusion models in building damage severity assessment, four key metrics were used: accuracy, precision, recall, and F1 score. These metrics provide valuable insights into each model's classification performance at each damage level. Accuracy measures the overall proportion of correctly classified instances, while precision assesses the reliability of the model's predictions for the positive class. Recall assesses the model's ability to identify the actual positive class in all real-world cases, while the F1 score balances precision and recall, providing a comprehensive perspective on model performance. The mathematical representation of these metrics is as follows:

[0147]

[0148] Among them, TP and TN represent true positives and true negatives, respectively, while FP and FN represent false positives and false negatives, respectively.

[0149] The proposed framework was comprehensively evaluated by analyzing the performance of the base model and the decision-level fusion model. The performance of each model was assessed using key metrics such as accuracy, precision, recall, and F1-score, providing a clear comparison of their strengths and weaknesses across four damage levels. This analysis provides a deeper understanding of the classification performance of each model.

[0150] Deep learning classification results: Figure 3 As shown in Figure 2, the accuracy of the model is stable and continues to rise during the training process, reaching about 90% on the training set and stabilizing at 85% on the validation set. This shows that the model has a good learning effect and strong generalization ability without obvious overfitting. In addition, as Figure 4 As shown, both the training loss and the validation loss show a continuous downward trend, which further demonstrates that the model can effectively learn the building damage characteristics during the training process.

[0151] When evaluated on a test set of 702 geotagged images, the model demonstrated high classification accuracy across all damage categories, as summarized in Table 4. Classification accuracy for the severe and complete damage categories was particularly high, reaching 98.72% for complete damage and 83.62% for severe damage. These results demonstrate the model's strong ability to identify critical damage levels. However, some confusion persists between minor and severe damage, reflecting the challenges of distinguishing features between adjacent damage levels.

[0152] Table 4 Deep learning performance

[0153] category Sample size TP TN FP FN Accuracy (%) Accuracy (%) Recall rate (%) F1 score (%) ND 60 44 624 18 16 95.16 70.97 73.33 72.13 MD 355 301 267 80 54 80.91 79.00 84.79 81.79 SD 259 189 398 45 70 83.62 80.77 72.97 76.67 TD 28 22 671 3 6 98.72 88.00 78.57 83.02

[0154] Precision, recall, and F1 scores further demonstrate the model's balanced performance across categories. For example, the complete damage class achieves the highest precision (88.00%), demonstrating the model's ability to effectively reduce false positives in the critical classification task. While the severe damage class exhibits a relatively low recall (72.97%), its precision is high (80.77%), demonstrating the model's robustness in identifying truly severe damage cases. On the other hand, the minor damage class achieves an F1 score of 81.79% and a relatively high recall (84.79%), demonstrating the model's ability to effectively detect subtle damage features. However, classification performance in the no damage class is slightly lacking, with a precision of 70.97% and an F1 score of 72.13%. This may be due to the model occasionally mistaking background features or minor irregularities due to low image quality for damage. To further improve the model's ability to distinguish across all damage levels, the dataset can be optimized by collecting more high-quality, no-damage images.

[0155] The heatmap visually demonstrates the areas that the deep learning model focuses on when classifying damage severity. In the no-damage category, the model primarily focuses on unaffected structural areas, such as intact walls and roofs, but occasionally pays attention to irrelevant areas, such as background vegetation. In the minor-damage category, the model is able to successfully highlight local cracks and surface defects, demonstrating its ability to identify subtle damage features. However, in some cases, the heatmap shows scattered attention to no-damage areas, which may indicate that there are still certain limitations in feature differentiation. In the severe-damage category, the model accurately identifies the main structural damage, including large cracks and partial collapses, and focuses on critical failure areas. However, in some cases, attention may extend to surrounding debris or peripheral areas, affecting classification accuracy. For the complete-damage category, the model shows strong and concentrated attention to areas of complete structural failure (such as collapsed walls and roofs), which is able to effectively capture the extent of the damage.

[0156] These results confirm the model's ability to localize features across different damage severity levels. However, the attentional overlap between the minor and severe damage classes suggests that further optimization is needed in feature extraction to improve classification accuracy, especially in the presence of ambiguity. Overall, the Grad-CAM visualization results not only validate the model's practical value in identifying and localizing damage in post-earthquake scenarios, but also reveal potential areas for improvement.

[0157] The second classifier was optimized using grid search (n_estimators=163, max_depth=16, and max_features=3) and evaluated on a test set containing position information, ultimately achieving an accuracy of 62.11%. Although this machine learning model performed worse than the deep learning model, it is important to note that random forests rely solely on non-visual priors and cannot incorporate additional visual information during training. This limitation inherently reduces its classification accuracy, but the model still performs well in classification tasks with slightly corrupted categories.

[0158] As shown in Table 5, the model achieved a precision of 63.12%, a recall of 81.97%, and an F1 score of 71.32% for the minor damage category. These results indicate that the model is highly effective in identifying low-severity damage, likely due to the high proportion of minor damage samples in the training set, which enables the model to effectively capture the characteristics of this category.

[0159] Furthermore, the model performed robustly in identifying non-damaged buildings, achieving a classification accuracy of 92.45%. However, its precision was only 62.07%, and its recall was a low 30.00%, indicating significant room for improvement. The low recall rate means that many non-damaged instances are misclassified as minor damage, which may be due to the random forest model's reliance on handcrafted features and its inability to learn more complex visual representations like deep learning models. This limitation suggests that further feature engineering optimization is needed to improve classification accuracy and robustness when conducting damage assessment based on non-visual data.

[0160] Table 5 Machine Learning Performance

[0161]

[0162]

[0163] Table 6 summarizes the classification performance of the multi-model decision-level fusion method, which combines the output of the VGG16 and random forest classifiers. The fusion model achieves the highest overall accuracy of 77.92% on the test dataset, demonstrating the effectiveness of combining visual and non-visual information for damage assessment.

[0164] Table 6 Fusion model performance

[0165] category Sample size TP TN FP FN Accuracy (%) Accuracy (%) Recall rate (%) F1 score (%) ND 60 48 621 21 12 95.30 69.57 80.00 74.42 MD 355 327 232 115 28 79.63 73.98 92.11 82.06 SD 259 151 429 14 108 82.62 91.52 58.30 71.23 TD 28 21 669 5 7 98.29 80.77 75.00 77.78

[0166] Among the three models evaluated, the fusion model achieved the highest overall accuracy of 77.92%, slightly surpassing the deep learning model (76.92%) and significantly outperforming the machine learning model (62.11%). This result emphasizes the feasibility of integrating visual and non-visual information in the decision-making stage.

[0167] like Figure 5 As shown, the performance of the two basic classifiers differed significantly across injury categories. The deep learning model performed well in identifying high-severity injuries, exceeding 70% for all evaluation metrics. In contrast, the machine learning model performed poorly in classifying high-severity injuries, tending to predict samples as mild. This performance difference is closely related to the characteristics of the data: vision-based deep learning models are better at identifying injury severity because visual cues directly reflect the extent of the injury, which closely aligns with real-world injury assessment scenarios, which typically rely on visual inspection. Non-visual machine learning methods, on the other hand, are limited by their inability to incorporate visual cues. Furthermore, during training, machine learning models are unable to augment the data with additional non-visual prior knowledge sources, leading to potential data imbalance. Despite these challenges, non-visual machine learning methods still have considerable potential for improvement and can provide complementary insights into the spatial distribution of injuries with faster inference speed, making them valuable in specific applications.

[0168] Despite achieving the highest overall accuracy, the fusion model lagged behind the deep learning model in the severe and complete injury categories. Specifically, the fusion model achieved a recall of 58.30% and an F1 score of 71.23% in the severe injury category, compared to the deep learning model's recall of 72.97% and F1 score of 76.67%. In the complete injury category, the fusion model achieved a recall of 75.00% and an F1 score of 77.78%, lower than the deep learning model's recall of 78.57% and F1 score of 83.02%. These differences suggest that noise in the fusion model, likely introduced by the machine learning component, impaired its performance in the high-severity category. While the deep learning model benefits from direct visual injury analysis, the machine learning model relies on non-visual information, which can introduce potential inaccuracies, particularly when classifying severe or complete injuries. This noise appears to be one of the primary reasons why the fusion model performs worse than the deep learning model in these categories.

[0169] Despite this, the fusion model performed well in identifying minor damage, achieving a recall of 92.11% and an F1 score of 82.06%, comparable to the deep learning model's recall of 84.79% and F1 score of 81.79%. Furthermore, the fusion model outperformed the machine learning model in all categories, demonstrating effective synergy between the two base classifiers. The fusion model also surpassed the machine learning model in precision, recall, and F1 score for undamaged buildings, achieving a recall of 80.00% and an F1 score of 74.42%, compared to the machine learning model's recall of 30.00% and F1 score of 40.45%.

[0170] These results highlight the strengths of each model across different injury categories. Deep learning models excel in classifying high-severity injuries due to their ability to leverage visual cues, while fusion models achieve balanced performance across all categories by combining visual and non-visual information. While currently limited by data imbalance and an inability to integrate visual features, machine learning models hold potential for further advancement in specific scenarios requiring fast inference or analysis of spatial distributions.

[0171] In summary, while the fusion model provided reliable performance, especially in identifying mild and moderate injuries, there is still significant room for improvement in classifying high-severity injuries. Future research should focus on increasing the diversity and number of high-severity injury samples, addressing data imbalance, especially with respect to non-visual information, and enhancing the feature set of the fusion model to improve its overall classification performance.

[0172] The present invention uniquely combines structured disaster-related prior knowledge with post-disaster imagery, and adopts the advantages of two types of basic classifiers. Vision-based deep learning methods act as visual interpreters, assessing building damage by identifying structural damage in post-disaster scenarios. This type of model is particularly suitable for refined damage assessment, and can use computer vision technology to extract complex image features with high precision and robustness. In addition, the present invention further combines heat maps to enhance the interpretability of the model and provide strong support for decision-making in disaster response. Knowledge-based non-visual structured data is used to predict the severity of damage to unknown buildings, which is particularly suitable for rapid assessment. When visual data is difficult to obtain or unclear, machine learning models can integrate prior knowledge and geospatial data, use spatial information to clarify the extent of damage, and provide support for accurate decision-making.

[0173] The decision-level fusion model combines the strengths of deep learning in image analysis with the contextual understanding capabilities of machine learning, achieving high accuracy in large-scale assessments and providing richer context in complex scenarios. This hybrid model provides a flexible and adaptable solution for post-disaster building damage assessment.

[0174] As shown in Table 7, the multimodal framework proposed in this paper is adaptable to a variety of disaster scenarios. When visual information is available, deep learning methods can be used preferentially for refined and comprehensive damage assessment. These models use advanced computer vision techniques to accurately identify structural damage and are particularly suitable for detecting high-severity damage in post-disaster imagery. However, when visual data is missing or of low quality (such as image blur or scene occlusion), machine learning methods that rely on non-visual structured data and geospatial information can still provide reliable predictions. This ensures that damage severity can be quickly assessed even under data-limited conditions.

[0175] Table 7 Prior knowledge variables used in the present invention

[0176]

[0177] The fusion of visual and non-visual data enables the framework to provide rapid screening and in-depth damage assessment in a complementary manner. This is particularly important for disaster management, where efficient emergency response and detailed damage analysis are crucial for effective decision-making. The framework's flexibility allows it to adapt to varying levels of information availability, enabling it to be useful in both large-scale hazard assessments, where high-resolution imagery is scarce, and localized scenarios, where spatial data alone can provide valuable insights.

[0178] The multimodal framework established by this invention enables both refined damage assessment and rapid screening, making it adaptable to diverse scenarios, including those where visual data is incomplete or unavailable. This multimodal approach expands data sources, increases the flexibility of damage assessment, and enhances its applicability in real-world disaster scenarios.

[0179] like Figure 6 As shown, an embodiment of the present invention further provides a building damage assessment device 60, comprising:

[0180] An acquisition module 61 is configured to acquire visual data reflecting the extent of damage to the building in a first scenario and prior knowledge data related to the extent of damage to the building in a second scenario;

[0181] The processing module 62 is used to input the visual data into a first classifier to perform building damage degree classification processing to obtain a first classification result, and the first classifier is obtained by training a first preset model; input the knowledge data into a second classifier to perform building damage degree classification processing to obtain a second classification result, and the second classifier is obtained by training a second preset model; input the first classification result and the second classification result into a fusion model to perform building damage degree assessment processing to obtain a building damage degree assessment classification result.

[0182] Optionally, the training process of the first classifier includes:

[0183] Acquire first training set data, where the first training set includes classification results of building images;

[0184] Preprocessing the first training set data to obtain first target data;

[0185] Inputting the first target data into multiple convolution processing layers of the first preset model to obtain a first output result;

[0186] Inputting the first output result into the fully connected layer of the first preset model to obtain a second output result;

[0187] Obtaining a predicted loss value according to the second output result and the classification result of the first target data;

[0188] According to the predicted loss value, the hyperparameters of the first preset model are adjusted to obtain a first classifier.

[0189] Optionally, obtaining a predicted loss value according to the second output result and the classification result of the first target data includes:

[0190] A prediction loss value is obtained by using a cross entropy loss function between the second output result and the classification result of the first target data. The cross entropy loss function formula is:

[0191]

[0192] Indicates loss, w i represents the weight assigned to each category i, y i is the true category label, is the predicted probability of class i.

[0193] Optionally, the training process of the first classifier further includes:

[0194] Inputting the second output result into the activation mapping module of the first preset model to obtain an output gradient;

[0195] A feature heat map is obtained according to the output gradient and the first output result.

[0196] Optionally, the second classifier training process includes:

[0197] Acquire second training set data, where the second training set data includes classification results of building images with building location information;

[0198] Preprocessing the second training set data to obtain second target data;

[0199] Inputting the second target data into the first processing layer of the second preset model and performing random sampling to obtain a third output result;

[0200] Inputting the third output result into the second processing layer of the second preset model, performing random feature selection and a preset number of node splitting, to obtain a fourth output result;

[0201] According to the fourth output result and the classification result of the second target data, the hyperparameters of the second preset model are adjusted to obtain a second classifier.

[0202] Optionally, preprocessing the second training set data to obtain second target data includes:

[0203] Extracting elevation data and location information from the second training set data;

[0204] Obtaining second intermediate data based on the elevation data and the position information;

[0205] performing interpolation processing on the second intermediate data to obtain second target data;

[0206] Wherein, the geographic information in the second intermediate data is obtained according to the elevation data, and the normalized vegetation index and earthquake impact data in the second intermediate data are obtained according to the position information.

[0207] Optionally, the first classification result and the second classification result are input into a fusion model to perform building damage assessment processing to obtain a building damage assessment classification result, including:

[0208] According to the first classification result and the second classification result, a dynamic weight is obtained;

[0209] According to the dynamic weight, a building damage degree assessment classification result is obtained.

[0210] It should be noted that the device is a device corresponding to the above method, and all implementation methods in the above method embodiments are applicable to the embodiments of the device and can achieve the same technical effects.

[0211] An embodiment of the present invention further provides a computing device comprising: one or more processors; and a storage device configured to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the building damage assessment method of the present invention. All implementations in the aforementioned method embodiments are applicable to the embodiments of this computing device and can achieve the same technical effects.

[0212] Embodiments of the present invention further provide a computer-readable storage medium storing a program that, when executed by a processor, implements the building damage assessment method of the present invention. All implementations described in the aforementioned method embodiments are applicable to the embodiments of this computer-readable storage medium and achieve the same technical effects.

[0213] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0214] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0215] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0216] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0217] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0218] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the various embodiments of the method of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard disk, ROM, RAM, a magnetic disk, or an optical disk.

[0219] In addition, it should be noted that, in the apparatus and method of the present invention, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. Moreover, the steps of performing the above-mentioned series of processing can naturally be performed in chronological order according to the order of description, but it is not necessary to perform them in chronological order, and some steps can be performed in parallel or independently of each other. For those of ordinary skill in the art, it will be understood that all or any steps or components of the method and apparatus of the present invention can be implemented in any computing device (including processors, storage media, etc.) or a network of computing devices in hardware, firmware, software or a combination thereof, which can be achieved by those of ordinary skill in the art using their basic programming skills after reading the description of the present invention.

[0220] Therefore, the purpose of the present invention can also be achieved by running a program or a group of programs on any computing device. The computing device can be a well-known general-purpose device. Therefore, the purpose of the present invention can also be achieved simply by providing a program product containing program code for implementing the method or device. That is to say, such a program product also constitutes the present invention, and the storage medium storing such a program product also constitutes the present invention. Obviously, the storage medium can be any well-known storage medium or any storage medium developed in the future. It should also be pointed out that in the device and method of the present invention, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. In addition, the steps of performing the above-mentioned series of processing can naturally be performed in chronological order according to the order of description, but do not necessarily need to be performed in chronological order. Certain steps can be performed in parallel or independently of each other.

[0221] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for assessing the extent of building damage, characterized in that: include: Acquiring visual data reflecting the extent of damage to the building in a first scene, the visual data including an on-site image of the building; Acquiring prior knowledge data related to the extent of building damage in a second scenario, the prior knowledge data including geographic location data, including latitude and longitude coordinates; Inputting the visual data into a first classifier to classify the degree of building damage to obtain a first classification result, wherein the first classifier is trained by a first preset model; Inputting the prior knowledge data into a second classifier to perform building damage degree classification processing to obtain a second classification result, wherein the second classifier is trained by a second preset model; Inputting the first classification result and the second classification result into a fusion model to perform building damage assessment processing to obtain a building damage assessment classification result; The training process of the first classifier includes: Acquire first training set data, where the first training set includes visual images of buildings and classification results of their corresponding damage levels; Preprocessing the first training set data to obtain first target data; Inputting the first target data into multiple convolution processing layers of the first preset model to obtain a first output result; Inputting the first output result into the fully connected layer of the first preset model to obtain a second output result; Obtaining a predicted loss value according to the second output result and the classification result of the first target data; According to the predicted loss value, adjusting the hyperparameters of the first preset model to obtain a first classifier; Wherein, obtaining a predicted loss value according to the second output result and the classification result of the first target data includes: A prediction loss value is obtained by using a cross entropy loss function between the second output result and the classification result of the first target data. The cross entropy loss function is: Indicates loss, represents the weight assigned to each category 𝑖, is the true category label, is the predicted probability of class 𝑖; The first classification result and the second classification result are input into the fusion model to perform building damage assessment processing, and the building damage assessment classification result is obtained, including: According to the first classification result and the second classification result, a dynamic weight is obtained; According to the dynamic weight, a building damage degree assessment classification result is obtained.

2. The building damage assessment method according to claim 1, characterized in that: The training process of the first classifier further includes: Inputting the second output result into the activation mapping module of the first preset model to obtain an output gradient; A feature heat map is obtained according to the output gradient and the first output result.

3. The building damage assessment method according to claim 1, wherein: The second classifier training process includes: Acquire second training set data, wherein the second training set data includes building image knowledge data having building location information and corresponding damage degree classification results; Preprocessing the second training set data to obtain second target data; Inputting the second target data into the first processing layer of the second preset model and performing random sampling to obtain a third output result; Inputting the third output result into the second processing layer of the second preset model, performing random feature selection and a preset number of node splitting, to obtain a fourth output result; According to the fourth output result and the classification result of the second target data, the hyperparameters of the second preset model are adjusted to obtain a second classifier.

4. The building damage assessment method according to claim 3, characterized in that: Preprocessing the second training set data to obtain second target data includes: Extracting elevation data and location information from the second training set data; Obtaining second intermediate data based on the elevation data and the position information; performing interpolation processing on the second intermediate data to obtain second target data; Wherein, the geographic information in the second intermediate data is obtained according to the elevation data, and the normalized vegetation index and earthquake impact data in the second intermediate data are obtained according to the position information.

5. A building damage assessment device, characterized in that: include: an acquisition module, configured to acquire visual data reflecting the extent of damage to the building in a first scenario and prior knowledge data related to the extent of damage to the building in a second scenario, wherein the visual data includes an on-site image of the building, and the prior knowledge data includes geographic location data, provided with latitude and longitude coordinates; A processing module, configured to input the visual data into a first classifier to classify the degree of building damage and obtain a first classification result, wherein the first classifier is trained by a first preset model; Inputting the prior knowledge data into a second classifier to perform building damage degree classification processing to obtain a second classification result, wherein the second classifier is trained by a second preset model; Inputting the first classification result and the second classification result into a fusion model to perform building damage assessment processing to obtain a building damage assessment classification result; The training process of the first classifier includes: Acquire first training set data, where the first training set includes visual images of buildings and classification results of their corresponding damage levels; Preprocessing the first training set data to obtain first target data; Inputting the first target data into multiple convolution processing layers of the first preset model to obtain a first output result; Inputting the first output result into the fully connected layer of the first preset model to obtain a second output result; Obtaining a predicted loss value according to the second output result and the classification result of the first target data; According to the predicted loss value, adjusting the hyperparameters of the first preset model to obtain a first classifier; Wherein, obtaining a predicted loss value according to the second output result and the classification result of the first target data includes: A prediction loss value is obtained by using a cross entropy loss function between the second output result and the classification result of the first target data. The cross entropy loss function is: Indicates loss, represents the weight assigned to each category 𝑖, is the true category label, is the predicted probability of class 𝑖; The first classification result and the second classification result are input into the fusion model to perform building damage assessment processing, and the building damage assessment classification result is obtained, including: According to the first classification result and the second classification result, a dynamic weight is obtained; According to the dynamic weight, a building damage degree assessment classification result is obtained.

6. A computing device, characterized in that include: one or more processors; A storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program, which implements the method according to any one of claims 1 to 4 when executed by a processor.

Images