Building damage degree assessment method, device and equipment

Abstract

The invention provides a building damage degree assessment method, device and equipment. The building damage degree assessment method comprises the steps of obtaining visual data reflecting the building damage degree in a first scene; obtaining priori knowledge data related to the building damage degree in the second scene; the visual data are input into a first classifier for building damage degree classification processing, a first classification result is obtained, and the first classifier is obtained through training of a first preset model; the knowledge data is input into a second classifier for building damage degree classification processing, a second classification result is obtained, and the second classifier is obtained through training of a second preset model; and inputting the first classification result and the second classification result into a fusion model to carry out building damage degree evaluation processing to obtain a building damage degree evaluation classification result. According to the invention, the accuracy of building damage degree classification can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image information evaluation and processing, and particularly to a method, device and equipment for evaluating the damage degree of buildings. Background Art

[0002] Earthquakes cause significant damage to buildings and infrastructure, resulting in huge property losses and casualties. Building collapses are the main cause of death and injury during earthquakes, especially in areas with poor building standards or ineffective building code enforcement. In addition, damage to infrastructure makes rescue operations more complex and increases the risk of secondary disasters such as fires and environmental pollution. Therefore, evaluating the degree of building damage is crucial for post-disaster loss assessment.

[0003] Building damage assessment methods usually detect structural damage based on macro-level images, and are less effective in identifying subtle damage features, resulting in poor assessment results. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method, device and equipment for evaluating the damage degree of buildings, which can improve the accuracy of evaluating the damage degree of buildings.

[0005] To solve the above technical problem, the technical solution of the present invention is as follows:

[0006] A method for evaluating the damage degree of buildings, comprising:

[0007] Obtaining visual data reflecting the damage degree of a building in a first scenario;

[0008] Obtaining prior knowledge data related to the damage degree of the building in a second scenario;

[0009] Inputting the visual data into a first classifier for classifying the damage degree of the building to obtain a first classification result, where the first classifier is trained by a first preset model;

[0010] Inputting the knowledge data into a second classifier for classifying the damage degree of the building to obtain a second classification result, where the second classifier is trained by a second preset model;

[0011] Inputting the first classification result and the second classification result into a fusion model for evaluating the damage degree of the building to obtain an evaluation classification result of the damage degree of the building.

[0012] An embodiment of the present invention further provides a device for evaluating the damage degree of buildings, comprising:

[0013] An acquisition module, configured to acquire visual data reflecting the damage degree of a building in a first scenario and prior knowledge data related to the damage degree of the building in a second scenario;

[0014] A processing module, configured to input the visual data into a first classifier for classifying the damage degree of the building to obtain a first classification result, where the first classifier is trained by a first preset model; input the knowledge data into a second classifier for classifying the damage degree of the building to obtain a second classification result, where the second classifier is trained by a second preset model; and input the first classification result and the second classification result into a fusion model for evaluating the damage degree of the building to obtain an evaluation classification result of the damage degree of the building.

[0015] An embodiment of the present invention further provides a computing device, including: one or more processors; a storage device, configured to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method for evaluating the damage degree of a building according to the present invention.

[0016] An embodiment of the present invention further provides a computer-readable storage medium, where a program is stored in the computer-readable storage medium, and when the program is executed by a processor, the method for evaluating the damage degree of a building according to the present invention is implemented.

[0017] The above technical solution of the present invention has at least the following technical effects:

[0018] The above method for evaluating the damage degree of a building according to the present invention includes: acquiring visual data reflecting the damage degree of a building in a first scenario; acquiring prior knowledge data related to the damage degree of the building in a second scenario; inputting the visual data into a first classifier for classifying the damage degree of the building to obtain a first classification result, where the first classifier is trained by a first preset model; inputting the knowledge data into a second classifier for classifying the damage degree of the building to obtain a second classification result, where the second classifier is trained by a second preset model; and inputting the first classification result and the second classification result into a fusion model for evaluating the damage degree of the building to obtain an evaluation classification result of the damage degree of the building. Thereby, the accuracy of evaluating the damage degree of a building is improved. Description of the Drawings

[0019] Figure 1 is a schematic flowchart of the method for evaluating the damage degree of a building according to the present invention;

[0020] Figure 2 is a schematic diagram of the module architecture of the first classifier of the method for evaluating the damage degree of a building according to the present invention;

[0021] Figure 3Schematic diagram of the training accuracy curve of the first classifier of the building damage degree evaluation method of the present invention;

[0022] Figure 4 Schematic diagram of the training loss curve of the second classifier of the building damage degree evaluation method of the present invention;

[0023] Figure 5 Schematic diagram of the evaluation index of the performance of the building damage degree evaluation model and the non-fusion model of the present invention;

[0024] Figure 6 Schematic diagram of the building damage degree evaluation device of the present invention. Detailed implementation manners

[0025] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present invention can be more thoroughly understood and the scope of the present invention can be completely conveyed to those skilled in the art.

[0026] As Figure 1 shown, an embodiment of the present invention provides a building damage degree evaluation method, including:

[0027] Step S1, obtaining visual data reflecting the building damage degree in the first scenario; here, the first scenario may be the ground scenario of the post-disaster building; the visual data may be image data such as ground photos of the post-disaster building. It should be noted that such visual data does not include geographical location information;

[0028] Step S2, obtaining prior knowledge data related to the building damage degree in the second scenario; here, the second scenario may be the ground scenario of the post-disaster building. The prior knowledge data includes ground photos of the post-disaster building and geographical location data stored in the Exchangeable Image File Format (EXIF) metadata, providing latitude and longitude coordinates. These geographical location data are crucial for understanding the spatial distribution of the damage;

[0029] Step S3, inputting the visual data into the first classifier for building damage degree classification processing to obtain a first classification result, where the first classifier is trained through a first preset model;

[0030] Step S4, inputting the knowledge data into the second classifier for building damage degree classification processing to obtain a second classification result, where the second classifier is trained through a second preset model;

[0031] Step S5: Input the first classification result and the second classification result into a fusion model for building damage degree assessment processing to obtain a building damage degree assessment classification result.

[0032] In this embodiment, as Figure 1 shown, in the building damage degree assessment method,

[0033] First, obtain building image data, where the image data includes visual data reflecting the building damage degree in the first scenario and prior knowledge data related to the building damage degree in the second scenario;

[0034] Then, perform preprocessing such as size adjustment and image pixel value normalization on the visual data in the obtained building image data, and perform preprocessing such as prior knowledge variable lookup and calculation on the knowledge data in the building image data to obtain visual data and knowledge data that are convenient for processing and use;

[0035] Again, input the preprocessed visual data into a first classifier for processing to obtain a first classification result. The first classifier is trained through a first preset model, and the first preset model is a deep learning model based on images. Through learning and training on the visual data of a certain number of building images whose damage degree classifications have been determined, the first classifier is formed;

[0036] Again, input the preprocessed knowledge data into a second classifier for processing to obtain a second classification result. The second classifier is trained through a second preset model, and the second preset model is a machine learning model based on geographic information system (GIS). Through learning and training on the knowledge data of a certain number of building images whose damage degree classifications have been determined, the second classifier is formed;

[0037] Finally, input the first classification result and the second classification result into a fusion model for processing to obtain a building damage degree classification result. The fusion model provides a dynamic confidence weighting mechanism to integrate the predictions of the first classifier and the second classifier and generate a comprehensive and accurate classification.

[0038] The ground images used in the solution of the present invention include visual data and prior knowledge data of geographical location data, enabling the inclusion of more extensive and comprehensive information related to the severity of building damage, thereby facilitating large-scale and rapid assessment of disaster damage. By using a multimodal fusion method, the classification method integrates visual and non-visual factors, improves the accurate assessment of building damage degree, and enhances its applicability in a wider range of scenarios and emergency response phases.

[0039] In an optional embodiment of the present invention, in step S3, the training process of the first classifier includes:

[0040] Step S31: Obtain the first training set data, where the first training set contains the classification results of building images;

[0041] Step S32: Preprocess the first training set data to obtain the first target data;

[0042] Step S33: Input the first target data into multiple convolutional processing layers of the first preset model to obtain a first output result;

[0043] Step S34: Input the first output result into the fully connected layer of the first preset model to obtain a second output result;

[0044] Step S35: Obtain a prediction loss value according to the second output result and the classification result of the first target data;

[0045] Step S36: Adjust the hyperparameters of the first preset model according to the prediction loss value to obtain a first classifier.

[0046] In this embodiment, the first training set data is a certain amount of visual data of building images whose damage degrees have been determined, including the classification results of damage degrees. The classification criteria for the severity of building damage are shown in the following table:

[0047] Table 1 Criteria for the Severity of Building Damage

[0048] Degree of damage Description Total damage (TD) The building collapses or suffers irreparable damage and needs to be completely rebuilt. Severe damage (SD) The load-bearing structure is severely damaged or partially collapsed, requiring extensive repairs or partial demolition. Minor damage (MD) Minor cracks appear in the load-bearing structure and can be restored through minor repairs. No damage (ND) No visible structural damage, no repairs or reconstruction are required.

[0049] Then, preprocess the first training set data. Since there is an issue of unbalanced data distribution in the first training set data, it is necessary to supplement the original post-disaster building image dataset through network search. To further enhance the diversity and generalization ability of the visual image dataset, the crisis benchmark dataset can also be used in combination. This dataset aggregates widely used post-disaster image collections and contains building damage images from different disaster events and time periods. The addition of this dataset introduces more extensive visual information and significantly improves the diversity and overall representativeness of the dataset. Preprocess the visual data in the obtained building image dataset, such as resizing and normalizing the image pixel values. Adjust the first training set data to a preset size to obtain the first intermediate data. Normalize the first intermediate data to obtain the first target data. Specifically, when preprocessing the first training set data, first uniformly resize the visual images in the dataset to a fixed size, preferably 224×224 pixels, to obtain the first intermediate data. Then normalize the image pixel values by subtracting the mean of the dataset from each pixel value and mapping the pixel values to the range of 0-1 or -1 to 1 to accelerate the convergence during data usage. Again, divide the dataset into a training set, a validation set, and a test set in the ratio of 8:1:1, and use the training set as the first target data.

[0050] Next, input the first target data into the convolutional processing layer of the first preset model to obtain the first output result. Preferably, the first preset model is a convolutional neural network model including 16 weight layers, specifically including 13 convolutional layers and 3 fully connected layers. These layers gradually process the input image, and the extracted visual features become more complex as the layer depth increases. The first preset model uses small 3×3 filters in all convolutional layers, enabling it to efficiently capture details in the image while maintaining computational feasibility. The first preset model contains multiple convolutional layers. These convolutional layers perform convolution operations by sliding the convolution kernel over the image to extract image features. During the sliding process, the convolution kernel performs mathematical operations with each small region of the image to generate a new feature map. Each convolutional layer learns different levels of features, from simple features such as edges and lines at the bottom layer to more complex and abstract features at the high layer, such as the local shape and texture of objects. The first preset model uses a 3×3-sized convolution kernel, and the depth of the model is increased by stacking multiple convolutional layers, enabling it to learn richer and more advanced features. After the convolutional layer, a pooling layer usually follows. The role of the pooling layer is to downsample the feature map extracted by the convolutional layer, reduce the data volume, lower the computational complexity of the model, and also prevent overfitting to a certain extent. The first preset model mainly uses the maximum pooling method, that is, taking the maximum value within the pooling window as the pooling result.

[0051] Next, input the first output result into the fully connected layer of the first preset model to obtain a second output result; after being processed by multiple convolutional layers, the obtained feature map is flattened into a one-dimensional vector and then input into the fully connected layer. The neurons in the fully connected layer are connected to all neurons in the previous layer. Its function is to comprehensively analyze and judge the extracted features, map the features to different categories. The first preset model usually contains 3 fully connected layers. The number of neurons in the first two fully connected layers is relatively large, generally 4096, and the number of neurons in the last fully connected layer is equal to the number of categories in the classification task. In this embodiment, the number of categories is set to 4. The output of the last fully connected layer is processed by the Softmax function to convert the output value into a probability distribution belonging to each category, and the category with the highest probability is the prediction result of the model for the input image.

[0052] In the training process of the first preset model in this embodiment, an optimizer is used, and the learning rate is set to 0.0001 because it can adjust the learning rate during the training process to optimize the model; the batch size is set to 8, and 20 epochs are trained;

[0053] To further improve the model performance and reduce overfitting, this method also applies data augmentation techniques such as random cropping, horizontal flipping, and rotation, which artificially increase the diversity of the training images. Finally, the selected optimizer is used to update the hyperparameters of the model. The hyperparameters include the convolutional kernel size, learning rate, batch size, etc. The hyperparameters are updated along the opposite direction of the gradient to gradually reduce the loss value. After multiple trainings, a first classifier is obtained.

[0054] In an optional embodiment of the present invention, in step S35, according to the second output result and the classification result of the first target data, a prediction loss value is obtained, including:

[0055] Step S351, obtain the prediction loss value through the cross-entropy loss function of the second output result and the classification result of the first target data. The formula of the cross-entropy loss function is as follows:

[0056]

[0057] represents the loss, w i represents the weight assigned to each category i, y i is the true category label, is the predicted probability of category i. Here, if the image belongs to category i, it is 1, otherwise it is 0;

[0058] In this embodiment, due to the imbalance problem in the dataset (i.e., there are fewer samples in some building damage categories), a weighted categorical cross-entropy loss function is applied in this embodiment, so that the first preset model can better identify minority-class images, thereby improving the overall classification accuracy of all damage levels.

[0059] In an optional embodiment of the present invention, in step S3, the training process of the first classifier further includes:

[0060] Step S37, inputting the second output result into the activation mapping module of the first preset model to obtain an output gradient;

[0061] Step S38, obtaining a feature heat map according to the output gradient and the first output result.

[0062] In this embodiment, in order to improve the interpretability of the prediction of the first preset model, the training process of the first classifier further includes: inputting the second output result into the activation mapping module of the first preset model to obtain an output gradient; and obtaining a feature heat map according to the output gradient and the first output result.

[0063] The activation mapping module of the first preset model highlights the regions in the image that the model considers to be most relevant to the classification decision. The activation mapping module of the first preset model can help intuitively understand why the model classifies the image into a specific damage category, for example, showing the collapsed area in a severely damaged building. The activation mapping module of the first preset model calculates the output gradient for a specific category (e.g., "severely damaged") and compares it with the activation value of the last convolutional layer, thereby visualizing the category-specific decision process in detail. By calculating the importance of the feature maps, the activation mapping module of the first preset model assigns weights to these maps and generates a feature heat map, highlighting the image regions most relevant to the model prediction.

[0064] This process, as shown below, provides interpretability insights into the regions that contribute the most.

[0065]

[0066] Here, represents the heat map for class c. A k represents the activation (detected features) from the last convolutional layer for feature map k. is the weight of feature map k, which comes from the gradient of the class score of class c with respect to activation A k The ReLU function (ReLU(x) = max(0, x)) ensures that only positive influences are considered when generating the heat map.

[0067] It is calculated using the following formula

[0068]

[0069] Here, is the activation gradient of the output score of class c with respect to the position (i, j) in the feature map k, and Z is the total number of pixels in the feature map, which is used for normalization.

[0070] Essentially, the first preset model highlights the parts of the image that have a significant impact on the decision-making of the deep learning model. By tracing the gradient to the last convolutional layer, it identifies the regions crucial for classification. For example, if a building image is classified as "severely damaged", the first preset model may highlight the collapsed walls or rubble, indicating the importance of these regions in the model's decision-making.

[0071] In an optional embodiment of the present invention, in step S4, the training process of the second classifier includes:

[0072] Step S41, obtaining second training set data, where the second training set data includes the classification results of building images with building location information;

[0073] Step S42, preprocessing the second training set data to obtain second target data;

[0074] Step S43, inputting the second target data into the first processing layer of the second preset model for random sampling to obtain a third output result;

[0075] Step S44, inputting the third output result into the second processing layer of the second preset model for random feature selection and node splitting a preset number of times to obtain a fourth output result;

[0076] Step S45, adjusting the hyperparameters of the second preset model according to the fourth output result and the classification result of the second target data to obtain a second classifier.

[0077] In this embodiment, the process of training the second preset model includes: First, obtain the second training set data, which is a certain amount of building image knowledge data with the classified damage degrees already determined, including the classification results of the damage degrees; Then, perform preprocessing such as prior knowledge variable search and calculation on the knowledge data in the building image data to obtain the second target data; Through the first processing layer of the second preset model, randomly sample with replacement from the second target data to construct multiple different bootstrap sample sets, and obtain the third output result. The size of each bootstrap sample set is the same as the size of the original training set. However, due to sampling with replacement, each bootstrap sample set may contain some duplicate samples, and at the same time, some samples may not be sampled; Again, input the third output result into the second processing layer of the second preset model for random feature selection and node splitting for a preset number of times to obtain the fourth output result; When splitting each node, randomly select a feature subset, and then select the optimal feature from this feature subset for splitting. In this embodiment, the size of the selected feature subset is the square root of the total number of features, which can increase the diversity between decision trees and reduce the correlation between trees; For each node, according to the selected optimal feature and the corresponding splitting criterion (such as information gain, Gini index, etc.), divide the samples on the node into two child nodes, so that the impurity (such as Gini impurity, information entropy, etc.) of the divided child nodes is reduced as much as possible; Repeat this process until a preset stop condition is reached, such as the number of samples on the node is less than a certain threshold, the depth of the tree reaches a preset value, the decrease in impurity is less than a certain threshold, etc.; To prevent the decision tree from overfitting, pruning operations also need to be performed on the generated decision tree, and two methods, pre-pruning and post-pruning, can be used. Pre-pruning is to stop the growth of the tree in advance according to some conditions during the generation of the decision tree; Post-pruning is to start from the leaf nodes after the decision tree is generated and judge whether some subtrees need to be pruned from bottom to top; Repeat the above steps of generating the decision tree to construct multiple decision trees to form a random forest. Each decision tree is trained based on different bootstrap sample sets and feature subsets, so there are certain differences between them; For classification tasks, the voting method is usually used, that is, each decision tree classifies and predicts the samples, and finally counts the prediction results of all decision trees, and selects the category with the most votes as the prediction result of the random forest; The hyperparameters of the second preset model include the number of decision trees (n_estimators), the maximum depth of each tree (max_depth), and the number of features considered during each split (max_features). These hyperparameters are fine-tuned through grid search and cross-validation techniques, and finally a robust and accurate second classifier is obtained.

[0078] In an alternative embodiment of the present invention, in step S42, preprocessing the second training set data to obtain the second target data includes:

[0079] Step S421: Extract elevation data and location information from the second training set data;

[0080] Step S422: Obtain second intermediate data based on the elevation data and location information;

[0081] Step S423: Perform interpolation processing on the second intermediate data to obtain second target data;

[0082] Among them, according to the elevation data, obtain the geographic information in the second intermediate data, and according to the location information, obtain the normalized difference vegetation index and seismic impact data in the second intermediate data.

[0083] In this embodiment, preprocess the second training set data. By using the elevation data (DEM) in the second training set data, calculate the altitude (i1), slope (i2), azimuth (i3), and terrain undulation (i4) of the corresponding area to enhance the geographical factors, so as to better represent the geographical background of the disaster area. In addition, the normalized difference vegetation index (NDVI) (i5) of the relevant area is also added. NDVI provides valuable insights into the health and coverage of vegetation, and this information can reflect the post-disaster vegetation recovery and its potential impact on the landscape and building stability. Further, according to the earthquake situation information and image location information, calculate the distance from each image location to the epicenter (i6), the distance to the central fault zone (i7), and the epicenter intensity (i8) corresponding to each image. These three parameters are used as important indicators to measure the impact of the earthquake on buildings. These eight parameters are used as the prior knowledge data set of machine learning, as shown in the following table:

[0084] Table 2 Prior knowledge variables of building image data

[0085] Variable Parameter Type <![CDATA[i1]]> Elevation Continuous variable <![CDATA[i2]]> Slope Continuous variable <![CDATA[i3]]> Aspect Continuous variable <![CDATA[i4]]> Geomorphic undulation Continuous variable <![CDATA[i5]]> NDVI Continuous variable <![CDATA[i6]]> Epicentral distance Continuous variable <![CDATA[i7]]> Fault zone distance Continuous variable <![CDATA[i8]]> Seismic intensity Discrete variable

[0086] The structured prior knowledge data set extracted from the geographical location of the image will be used for the construction and training of the building damage severity assessment model.

[0087] Then, perform interpolation processing on the second intermediate data by using the synthetic minority over-sampling technique; the synthetic minority over-sampling technique is a method for balancing class labels in machine learning. It generates synthetic samples by interpolating between minority class instances, thereby obtaining a more balanced training data set. It improves the representativeness of minority class samples instead of simply replicating existing instances, thereby enhancing the robustness of the training process.

[0088] Specifically, for each minority class instance x, the synthetic minority over-sampling technique selects one of its k nearest neighbors and generates a new synthetic sample x through linear interpolation new, the formula is as follows:

[0089] x new = x + λ × (x neighbor - x)

[0090] where λ is a random number in the range [0, 1], x represents the feature vector of the original instance from the minority class, and x neighbor represents the feature vector of one of its selected nearest neighbors. The newly generated synthetic sample x new lies on the line segment in the feature space connecting and x neighbor . This process enriches the dataset by effectively diversifying the representation of the minority class, enabling the model to learn a more balanced representation of all classes.

[0091] In an alternative embodiment of the present invention, in step S5, the first classification result and the second classification result are input into a fusion model for building damage degree assessment processing to obtain a building damage degree assessment classification result, including:

[0092] Step S51, obtaining dynamic weights according to the first classification result and the second classification result;

[0093] Step S52, obtaining a building damage degree assessment classification result according to the dynamic weights.

[0094] In this embodiment, the probability outputs of the first classification result and the second classification result are fused to improve the accuracy of building damage severity classification; this fusion is performed by a confidence-based dynamic weighting method, aiming to integrate the evaluation results of the two models to draw a more comprehensive conclusion; specifically, the confidence of each model is defined as the highest probability value in its predicted probability distribution, representing the classification choice that the model is most confident in. The dynamic weights of the models are calculated as follows:

[0095]

[0096] where confidence rf represents the highest probability in the output distribution of the second classifier, and confidence dl represents the highest probability in the output of the first classifier. These equations ensure that the sum of the weights w rf and w dl is 1, thus balancing the contributions of the two models.

[0097] The final combined prediction probability combined_prob is obtained by weighted averaging the probability distributions predicted by the two models, and the expression is as follows:

[0098] combined_prob = w rf ·Prf +w dl ·P dl

[0099] Among them, P rf and P dl Denote the probability vectors predicted by the second classifier and the first classifier, respectively. The final building damage category is determined by selecting the category with the maximum value in the combined probability vector combined_prob. By exploiting the relative strengths of the two models, this adaptive fusion method dynamically optimizes their contributions, thereby improving the robustness and accuracy of classification.

[0100] The specific implementation process of the above method of the present invention is described below:

[0101] 1. Obtain photos of damaged buildings on the ground

[0102] The main data sources include images taken during post-disaster field surveys conducted after an earthquake in a certain area. A total of 10,171 images were obtained, recording the conditions of damaged buildings and infrastructure after the earthquake, providing a detailed visual record of damage and ruins. Among them, 6,806 images contain geolocation data stored in EXIF ​​metadata, providing latitude and longitude coordinates. This geolocation data is crucial to understand the spatial distribution of damage. It can be observed that most of the photos are from the epicenter areas of magnitude VII and VIII. The remaining 3,365 images, although lacking geolocation data, still provide valuable visual evidence for damage assessment. The number of photos of buildings with mild damage according to the criteria for building damage severity is relatively high, while the number of images showing more severe damage is relatively small. Secondly, building damage severity assessment based on deep learning faces the problem of data imbalance, which is also clearly reflected in the building damage photo dataset. To address this problem, the original post-disaster building image dataset was supplemented by web search. After screening, 668 images of intact buildings and 671 images of completely damaged buildings were obtained, which are called the "Web-sourced building damage dataset". These additional images introduced necessary diversity to the dataset and helped balance the label distribution.

[0103] In order to further enhance the diversity and generalization ability of the dataset, a crisis benchmark dataset is also combined. This dataset summarizes a widely used collection of post-disaster images, including images of building damage from different disaster events and time periods. The addition of this dataset introduces a wider range of visual information, significantly improving the diversity and overall representativeness of the dataset.

[0104] Geographic Information System (GIS) data also plays a crucial role in post-disaster scenario damage assessment, providing key insights into spatial factors for effective damage identification. When selecting prior knowledge data extracted from seismic images, three key factors were focused on: (1) geographical factors related to buildings, (2) environmental conditions, and (3) destructive earthquake factors related to the earthquake. Based on the availability of relevant information, Digital Elevation Model (DEM) data ， , Normalized Difference Vegetation Index (NDVI) data, and the locations of earthquake epicenters, central fault zones, and epicenter intensity distributions were finally selected as the main data sources for prior knowledge extraction.

[0105] II. Dataset Reconstruction

[0106] According to the definition of earthquake damage severity, images from the building damage dataset from web sources and the crisis benchmark dataset were re-annotated to ensure that all images met the damage level requirements in the research criteria. After completing the annotation, images from these three sources were integrated to form a more comprehensive and multi-source image dataset. Therefore, this dataset not only effectively demonstrates the building damage caused by the earthquake, but also contains building damage images from different regions and events, thus enhancing the diversity of the dataset.

[0107] In addition, this multi-source image dataset helps to address the problem of unbalanced label distribution; the original seismic image dataset was severely skewed, with most images classified as slightly damaged, while images showing complete destruction and no damage were relatively scarce. By introducing new data sources, the constructed dataset achieved a balanced distribution of the four damage levels, providing a more favorable basis for subsequent modeling work. Finally, the dataset was divided into a training set, a validation set, and a test set at a ratio of 8:1:1, as shown in the following table:

[0108] Table 3 Distribution of sources and number of images in the training set, validation set, and test set

[0109]

[0110] The current data partition was maintained, and only geotagged images in the training set, validation set, and test set were selected to construct a structured prior knowledge dataset for machine learning. By using DEM data, the elevation (i1), slope (i2), azimuth (i3), and terrain undulation (i4) of the corresponding area were calculated to enhance geographical factors and better represent the geographical background of the disaster area. In addition, the normalized difference vegetation index NDVI (i5) of the relevant area was added. The normalized difference vegetation index provides valuable insights into vegetation health and coverage, and this information can reflect the post-disaster vegetation recovery and its potential impact on landscape and building stability. Further, based on the earthquake information, the distance from the location of each image to the epicenter (i6), the distance to the central fault zone (i7), and the epicenter intensity corresponding to each image (i8) were calculated. These three parameters are important indicators for measuring the impact of the earthquake on buildings. Generally speaking, eight parameters were obtained to form a prior knowledge dataset for machine learning.

[0111] Two different types of datasets were constructed above: one is a multi-source image dataset created by integrating multiple post-disaster building images, and the other is a structured prior knowledge dataset extracted from the geographical locations of the images. These two datasets will be used for the construction of a building damage severity assessment model.

[0112] III. Deep learning method for evaluating the severity of building damage

[0113] Using the building damage photo dataset, the web-source building damage dataset, and the crisis benchmark dataset, the severity of building damage was classified by a pre-trained first classifier. In addition, an activation mapping module was adopted to visualize the prediction results of the model, thus enhancing the interpretability of the classification results. This method helps to deeply understand the decision-making process of the model, especially in identifying key damage features in the images.

[0114] The first classifier includes 13 convolutional layers and 3 fully connected layers. Small 3x3 filters are used in all convolutional layers, enabling it to efficiently capture details in the images while maintaining computational feasibility.

[0115] As Figure 2 shown, in order to adapt the model to the task of the present invention (four damage severity categories), the last fully connected layer was adjusted to output these four categories instead of the original 1,000 categories.

[0116] During the training process, an optimizer was used with a learning rate set to 0.0001 because it can adjust the learning rate during training to optimize the model. The batch size was set to 8 and 20 epochs were trained. Due to the imbalance problem in the dataset (i.e., there are fewer samples in some building damage categories), a weighted categorical cross-entropy loss function was applied, which gives a higher penalty to misclassifications of the smaller categories, as shown in the following formula:

[0117]

[0118] Here, represents the loss, and w i represents the weight assigned to each class i. (Greater weights are given to less representative classes), and y i is the true class label (1 if the image belongs to class i and 0 otherwise), is the predicted probability of class i.

[0119] This loss function helps the model better identify minority class images, thereby improving the overall classification accuracy for all damage levels. To further improve the model performance and reduce overfitting, data augmentation techniques such as random cropping, horizontal flipping, and rotation were applied, which artificially increase the diversity of the training images.

[0120] It should be noted that during the model validation process, not all 1,428 images in the dataset were used. Instead, the validation was specifically performed on 702 images that contain geographical location information. This decision was made to ensure the comparability of the results of the deep learning model with those of the machine learning-based method, which also used geotagged images for evaluation.

[0121] To improve the interpretability of the model predictions, gradient-weighted class activation mapping was used. This method highlights the regions in the image that the model considers most relevant to the classification decision. The activation mapping module can help intuitively understand why the model classifies an image into a specific damage category, such as showing the collapsed area in a severely damaged building.

[0122] The activation mapping analyzes the output gradients of the model for a specific category (e.g., "severely damaged") and compares them with the activation values of the last convolutional layer, thus visualizing the category-specific decision process in detail. By calculating the importance of the feature maps, the activation mapping assigns weights to these maps and generates a heatmap that highlights the image regions most relevant to the model prediction. This process is shown in the following formula, providing interpretability insights into the regions that contribute the most.

[0123]

[0124] Here The heatmap $A_k$ representing the activation map of class $c$ represents the activation (detected features) from the last convolutional layer for feature map $k$. $w_{ij}^k$ is the weight of feature map $k$, which is derived from the gradient of the class score of class $c$ with respect to activation $A_k$. The ReLU function ($ReLU(x)=\max(0, x)$) ensures that only positive influences are considered when generating the heatmap.

[0125] It is calculated using the following formula

[0126]

[0127] Here, $\frac{\partial s_c}{\partial a_{ij}^k}$ is the activation gradient of the output score of class $c$ with respect to the position $(i, j)$ in feature map $k$, and $Z$ is the total number of pixels in the feature map, which is used for normalization.

[0128] By tracing the gradient back to the last convolutional layer, it identifies the regions crucial for classification. For example, if a building image is classified as "severely damaged", the activation map module may highlight the collapsed walls or rubble, indicating the importance of these regions in the model's decision-making. This technique improves the interpretability of the model by visually demonstrating the features related to the severity of damage.

[0129] IV. Machine Learning Methods for Assessing the Severity of Building Damage

[0130] In the context of assessing the severity of building damage using a structured prior knowledge dataset based on image geographical location information, the lack of geographical location information in the images limits the enhancement of the distribution of class and functional labels by integrating data from other sources. To address this issue, the Synthetic Minority Over-sampling Technique (SMOTE) is adopted, which is very effective in balancing the class label distribution. Subsequently, a random forest model is used to effectively estimate the severity of building damage.

[0131] The Synthetic Minority Over-sampling Technique (SMOTE) generates synthetic samples by interpolating between existing minority class instances, thus obtaining a more balanced training dataset. The advantage of this technique is that it improves the representativeness of minority class samples rather than simply replicating existing instances, thereby enhancing the robustness of the training process. Mathematically, for each minority class instance $x$, the Synthetic Minority Over-sampling Technique (SMOTE) selects one of its $k$ nearest neighbors and generates a new synthetic sample $x'$ through linear interpolation new .

[0132] $x'$ new $=x + \lambda\times(x_{nn}$ neighbor $- x)$

[0133] Here, $\lambda$ is a random number in the range $[0, 1]$, $x$ represents the feature vector of the original instance from the minority class, and $x_{nn}$neighbor The feature vector representing one of its selected nearest neighbors. The newly generated synthetic sample x new is located on the line segment in the feature space connecting x and x neighbor . This process enriches the dataset by effectively diversifying the representations of the minority class, enabling the model to learn more balanced representations of all classes.

[0134] After addressing the class imbalance problem, the random forest algorithm is used for damage severity classification. Random forest is an ensemble learning method that improves traditional decision trees by introducing bootstrap sampling and random feature selection, reducing variance. It constructs multiple decision trees from subsets of the training data and averages their prediction results to determine the final class. Random forest is well-suited for handling high-dimensional data, resistant to overfitting, and capable of effectively capturing complex feature interactions, making it more robust than a single decision tree.

[0135] The random forest model of the present invention's solution is trained using a training dataset containing 5,475 samples, and the hyperparameters are optimized to ensure optimal performance. The key parameters include the number of decision trees (n_estimators), the maximum depth of each tree (max_depth), and the number of features considered at each split (max_features). These parameters are fine-tuned through grid search and cross-validation techniques, ultimately resulting in a robust and accurate model.

[0136] Subsequently, using a test dataset of 702 samples, the ability of the model to accurately predict damage severity under different disaster characteristics was verified.

[0137] V. Fusing Machine Learning and Deep Learning at the Decision Level for Damage Severity Classification

[0138] Considering the dimensional differences between deep learning and machine learning methods, decision-level fusion was selected to effectively combine their outputs. Specifically, the probability outputs of the deep learning model (VGG16) and the machine learning model (random forest) were fused to improve the accuracy of building damage severity classification. This fusion was performed through a confidence-based dynamic weighting method, aiming to synthesize the evaluation results of the two models to draw a more comprehensive conclusion.

[0139] Specifically, the confidence of each model is defined as the highest probability value in its predicted probability distribution, representing the classification choice that the model is most confident in. The dynamic weight of the model is calculated as follows:

[0140]

[0141] Here, confidence rfrepresents the highest probability in the output distribution of the random forest model, while confidence dl represents the highest probability in the output of the VGG16 model. These equations ensure that the sum of the weights w rf and w dl is 1, thus balancing the contributions of the two models.

[0142] The final combined prediction probability combined_prob is obtained by weighted averaging the probability distributions predicted by the two models, and the expression is as follows:

[0143] combined_prob = w rf ·P rf + w dl ·P dl

[0144] Here, P rf and P dl represent the probability vectors predicted by the random forest and VGG16 models respectively. The final building damage category is determined by selecting the category where the maximum value in the combined probability vector combined_prob lies. By leveraging the relative advantages of the two models, this adaptive fusion method dynamically optimizes their contributions, thereby improving the robustness and accuracy of classification.

[0145] VI. Evaluation Metrics

[0146] To evaluate the performance of deep learning, machine learning, and decision-level fusion models in building damage severity assessment, four key metrics are adopted: accuracy, precision, recall, and F1-score. These metrics provide valuable insights into the classification efficacy of each model at various damage levels. Accuracy measures the overall proportion of correctly classified instances, while precision evaluates the reliability of the model's predictions for the positive class. Recall assesses the model's ability to identify actual positive classes among all real cases, and the F1-score balances precision and recall, providing a comprehensive perspective on the model's performance. The mathematical representations of these metrics are as follows:

[0147]

[0148] where TP and TN represent true positives and true negatives respectively, while FP and FN represent false positives and false negatives respectively.

[0149] The proposed framework is comprehensively evaluated by analyzing the performance of the base model and the decision-level fusion model. The performance of each model is evaluated through key metrics such as accuracy, precision, recall, and F1-score, so as to clearly compare their advantages and disadvantages in the four damage level categories. This analysis helps to deeply explore the classification effects of each model.

[0150] Deep learning classification results: As Figure 3 shown, the accuracy of the model during the training process is stable and continuously increasing, reaching approximately 90% on the training set and stabilizing at 85% on the validation set. This indicates that the model has a good learning effect and strong generalization ability, without obvious overfitting phenomenon. In addition, as Figure 4 shown, it shows that both the training loss and the validation loss show a continuous downward trend, which further indicates that the model can effectively learn building damage features during the training process.

[0151] When evaluated on a test set containing 702 geotagged images, the model shows a high classification accuracy for all damage categories, and the results are summarized in Table 4. In particular, the classification accuracies for the severe damage and complete destruction categories are relatively high, with the accuracy of complete destruction reaching 98.72% and the accuracy of severe damage being 83.62%. These results indicate that the model has a strong ability to identify key damage levels. However, there is still a certain degree of confusion between minor damage and severe damage, which reflects that there are still certain challenges in differentiating features between adjacent damage levels.

[0152] Table 4 Deep learning performance

[0153] Category Number of samples TP TN FP FN Accuracy (%) Precision (%) Recall (%) F1 score (%) ND 60 44 624 18 16 95.16 70.97 73.33 72.13 MD 355 301 267 80 54 80.91 79.00 84.79 81.79 SD 259 189 398 45 70 83.62 80.77 72.97 76.67 TD 28 22 671 3 6 98.72 88.00 78.57 83.02

[0154] Precision, Recall, and F1-score further demonstrate the balanced performance of the model across various categories. For example, the Precision for the completely damaged category is the highest (88.00%), indicating that the model can effectively reduce False Positives in key classification tasks. Although the Recall for the severely damaged category is relatively low (72.97%), its Precision is high (80.77%), showing the model's robust ability to identify truly severely damaged cases. On the other hand, the F1-score for the slightly damaged category is 81.79%, and the Recall is relatively high (84.79%), indicating that the model can effectively detect subtle damage features. However, the classification performance for the undamaged category is slightly lacking, with a Precision of 70.97% and an F1-score of 72.13%. This may be due to the model occasionally misclassifying background features or minor irregularities with low image quality as damage. To further improve the model's ability to distinguish all damage levels, the dataset can be optimized by collecting more high-quality undamaged images.

[0155] The heatmap visually shows the regions that the deep learning model focuses on when classifying the severity of damage. In the undamaged category, the model mainly focuses on the unaffected structural areas, such as intact walls and roofs, but occasionally also pays attention to irrelevant areas, such as background vegetation. In the slightly damaged category, the model can successfully highlight local cracks and surface defects, indicating its ability to identify subtle damage features. However, in some cases, the heatmap shows scattered attention to undamaged areas, which may indicate certain limitations in feature discrimination. In the severely damaged category, the model accurately identifies major structural damages, including large cracks and partial collapses, and focuses its attention on the key failure areas. But in individual cases, the attention may extend to the surrounding debris or peripheral areas, thus affecting the classification accuracy. For the completely damaged category, the model shows strong and concentrated attention to the completely structurally failed areas (such as collapsed walls and roofs), and can effectively capture the degree of damage.

[0156] These results confirm that the model can localize features for different damage severities. However, the attention overlap between the slightly damaged and severely damaged categories indicates that further optimization is still needed in feature extraction to improve classification accuracy, especially in the presence of ambiguous situations. Overall, the Grad-CAM visualization results not only verify the practical value of the model in identifying and localizing damages in post-earthquake scenes but also reveal potential improvement directions.

[0157] The second classifier was optimized through grid search (n_estimators = 163, max_depth = 16, and max_features = 3) and evaluated on a test set containing location information, achieving an accuracy of 62.11% in the end. Although the performance of this machine learning model is inferior to that of deep learning models, it should be noted that the random forest only relies on non-visual prior knowledge and cannot integrate additional visual information during the training process. This limitation inherently reduces its classification accuracy, but the model still performs well in the classification tasks of slightly damaged categories.

[0158] As shown in Table 5, the model achieved a precision of 63.12%, a recall of 81.97%, and an F1-score of 71.32% on slightly damaged categories. These results indicate that the model is highly efficient in identifying low-severity damages, probably benefiting from the relatively high proportion of slightly damaged samples in the training set, enabling the model to effectively capture the characteristics of this category.

[0159] In addition, the model showed robustness in the identification of non-damaged buildings, with a classification accuracy of 92.45%. However, its precision was only 62.07%, and the recall rate was as low as 30.00%, indicating that there is still much room for improvement. The low recall rate means that many non-damaged instances were misclassified as slightly damaged, which may be due to the fact that the random forest model relies on manually designed features and cannot learn more complex visual representations like deep learning models. This limitation suggests that when conducting damage assessment based on non-visual data, further optimization of feature engineering is still needed to improve the classification accuracy and robustness.

[0160] Table 5 Machine Learning Performance

[0161]

[0162]

[0163] Table 6 summarizes the classification performance of the multi-model decision-level fusion method, which combines the output results of VGG16 and the random forest classifier. The fusion model achieved the highest overall accuracy of 77.92% on the test dataset, indicating that combining visual information with non-visual information for damage assessment is effective.

[0164] Table 6 Fusion Model Performance

[0165] Category Number of samples TP TN FP FN Accuracy (%) Precision (%) Recall (%) F1 score (%) ND 60 48 621 21 12 95.30 69.57 80.00 74.42 MD 355 327 232 115 28 79.63 73.98 92.11 82.06 SD 259 151 429 14 108 82.62 91.52 58.30 71.23 TD 28 21 669 5 7 98.29 80.77 75.00 77.78

[0166] Among the three models evaluated, the fusion model demonstrated the highest overall accuracy of 77.92%, slightly exceeding the deep learning model (76.92%) and significantly outperforming the machine learning model (62.11%). This result emphasizes the feasibility of integrating visual and non-visual information in the decision-making stage.

[0167] As Figure 5 shown, the performance of the two base classifiers varied significantly across different injury categories. The deep learning model excelled in identifying high-severity injuries, with all evaluation metrics exceeding 70%. In contrast, the machine learning model performed poorly in classifying high-severity injuries, tending to predict samples as mild injuries. This performance difference is closely related to the characteristics of the data: the vision-based deep learning model is better at identifying injury severity because visual cues directly reflect the degree of injury, which closely aligns with the actual injury assessment scenario that typically relies on visual inspection. On the other hand, the non-visual machine learning approach is limited by its inability to integrate visual cues. Additionally, during training, the machine learning model could not enhance the data through additional non-visual prior knowledge sources, leading to potential data imbalance issues. Despite these challenges, the non-visual machine learning approach still has considerable potential for improvement and can provide complementary insights into the spatial distribution of injuries, with a faster inference speed, which makes it valuable in specific applications.

[0168] Although the fusion model had the highest overall accuracy, its performance in the severe and complete injury categories lagged behind the deep learning model. Specifically, the recall rate and F1 score of the fusion model in the severe injury category were 58.30% and 71.23% respectively, while the recall rate of the deep learning model was 72.97% and the F1 score was 76.67%. In the complete injury category, the recall rate of the fusion model was 75.00% and the F1 score was 77.78%, lower than the recall rate of 78.57% and F1 score of 83.02% of the deep learning model. These differences indicate that the noise possibly introduced by the machine learning component in the fusion model weakened its performance in high-severity categories. The deep learning model benefits from direct visual injury analysis, while the machine learning model relies on non-visual information, which may introduce potential inaccuracies, especially when classifying severe or complete injuries. These noises seem to be one of the main reasons for the poorer performance of the fusion model compared to the deep learning model in these categories.

[0169] Nonetheless, the fusion model performs excellently in identifying minor damages, with a recall rate of 92.11% and an F1 score of 82.06%, comparable to the recall rate of 84.79% and F1 score of 81.79% of the deep learning model. Additionally, the fusion model outperforms the machine learning model in all categories, demonstrating an effective synergy between the two base classifiers. The fusion model also exceeds the machine learning model in terms of precision, recall, and F1 score for undamaged buildings, with a recall rate of 80.00% and an F1 score of 74.42%, while the machine learning model has a recall rate of 30.00% and an F1 score of 40.45%.

[0170] These results highlight the advantages of each model in different damage categories. The deep learning model performs excellently in classifying high-severity damages due to its ability to utilize visual cues, while the fusion model achieves a balanced performance in all categories by combining visual and non-visual information. Despite the current limitations of data imbalance and the inability to integrate visual features, the machine learning model still has the potential for further development in specific scenarios that require rapid inference or spatial distribution analysis.

[0171] In conclusion, although the fusion model provides reliable performance, especially in identifying minor and moderate damages, there is still significant room for improvement in classifying high-severity damages. Future research should focus on increasing the diversity and quantity of high-severity damage samples, addressing data imbalance, especially in non-visual information, and enhancing the feature set of the fusion model to improve its overall classification performance.

[0172] The present invention uniquely combines structured disaster-related prior knowledge with post-disaster imagery, leveraging the advantages of two types of base classifiers. The vision-based deep learning method acts as a visual interpreter, assessing building damage in post-disaster scenarios by identifying structural damages. Such models are particularly suitable for refined damage assessment and can robustly extract complex image features with high precision using computer vision techniques. Additionally, the present invention further combines heatmaps to enhance the interpretability of the model, providing strong support for decision-making in disaster response. The knowledge-based non-visual structured data is used to predict the damage severity of unknown buildings, especially suitable for rapid assessment. When visual data is difficult to obtain or unclear, the machine learning model can integrate prior knowledge and geospatial data to clarify the damage level using spatial information, providing support for accurate decision-making.

[0173] The decision-level fusion model combines the advantages of deep learning in image analysis and the context information understanding ability of machine learning, achieving high accuracy in large-scale assessments and providing richer background information in complex scenarios. This hybrid model provides a flexible and adaptable solution for post-disaster building damage assessment.

[0174] As shown in Table 7, the multi-modal framework proposed by the present invention can adapt to various disaster scenarios. In the case where visual information is available, deep learning methods can be preferentially used for refined and comprehensive damage assessment. These models use advanced computer vision technology to accurately identify structural damage and are particularly suitable for detecting high-severity damage in post-disaster images. However, when visual data is missing or of low quality (such as blurred images or scene occlusion), machine learning methods that rely on non-visual structured data and geospatial information can still provide reliable predictions. This ensures that the severity of damage can be quickly evaluated even under data-limited conditions.

[0175] Table 7 Prior knowledge variables used in the present invention

[0176]

[0177] The fusion of visual and non-visual data enables the framework to provide rapid screening and in-depth damage assessment in a complementary manner. This is particularly important for disaster management because efficient emergency response and detailed damage analysis are crucial for making effective decisions. The flexibility of the framework enables it to adapt to different levels of information availability, whether in large-scale disaster assessments where high-resolution images are scarce or in local scenarios where valuable insights can be provided by relying solely on spatial data.

[0178] The multi-modal framework established by the present invention can not only perform refined damage assessment but also achieve rapid screening, enabling it to adapt to different scenarios, including cases where visual data is incomplete or unavailable. This multi-modal method expands the data sources, improves the flexibility of damage assessment, and enhances its applicability in real disaster situations.

[0179] As Figure 6 shown, an embodiment of the present invention also provides a building damage degree assessment device 60, including:

[0180] An acquisition module 61, configured to acquire visual data reflecting the building damage degree in the first scenario and prior knowledge data related to the building damage degree in the second scenario;

[0181] A processing module 62, configured to input the visual data into a first classifier for building damage degree classification processing to obtain a first classification result, where the first classifier is trained by a first preset model; input the knowledge data into a second classifier for building damage degree classification processing to obtain a second classification result, where the second classifier is trained by a second preset model; and input the first classification result and the second classification result into a fusion model for building damage degree assessment processing to obtain a building damage degree assessment classification result.

[0182] Optionally, the training process of the first classifier includes:

[0183] Obtain the first training set data, where the first training set contains the classification results of building images;

[0184] Preprocess the first training set data to obtain the first target data;

[0185] Input the first target data into multiple convolutional processing layers of the first preset model to obtain a first output result;

[0186] Input the first output result into the fully connected layer of the first preset model to obtain a second output result;

[0187] Obtain a prediction loss value according to the second output result and the classification result of the first target data;

[0188] Adjust the hyperparameters of the first preset model according to the prediction loss value to obtain a first classifier.

[0189] Optionally, obtaining a prediction loss value according to the second output result and the classification result of the first target data includes:

[0190] Obtain a prediction loss value through the cross-entropy loss function of the second output result and the classification result of the first target data. The formula of the cross-entropy loss function is:

[0191]

[0192] represents the loss, w i represents the weight assigned to each category i, y i is the true category label, is the predicted probability of category i.

[0193] Optionally, the training process of the first classifier further includes:

[0194] Input the second output result into the activation mapping module of the first preset model to obtain an output gradient;

[0195] Obtain a feature heat map according to the output gradient and the first output result.

[0196] Optionally, the training process of the second classifier includes:

[0197] Obtain the second training set data, where the second training set data contains the classification results of building images with building location information;

[0198] Preprocess the second training set data to obtain the second target data;

[0199] Input the second target data into the first processing layer of the second preset model for random sampling to obtain a third output result;

[0200] Input the third output result into the second processing layer of the second preset model for random feature selection and node splitting for a preset number of times to obtain a fourth output result;

[0201] Adjust the hyperparameters of the second preset model according to the fourth output result and the classification result of the second target data to obtain a second classifier.

[0202] Optionally, preprocess the second training set data to obtain second target data, including:

[0203] Extract elevation data and location information from the second training set data;

[0204] Obtain second intermediate data according to the elevation data and location information;

[0205] Perform interpolation processing on the second intermediate data to obtain second target data;

[0206] Among them, obtain the geographic information in the second intermediate data according to the elevation data, and obtain the normalized vegetation index and seismic impact data in the second intermediate data according to the location information.

[0207] Optionally, input the first classification result and the second classification result into a fusion model for building damage degree assessment processing to obtain a building damage degree assessment classification result, including:

[0208] Obtain a dynamic weight according to the first classification result and the second classification result;

[0209] Obtain a building damage degree assessment classification result according to the dynamic weight.

[0210] It should be noted that this device corresponds to the above method, and all implementation manners in the above method embodiments are applicable to the embodiments of this device and can also achieve the same technical effects.

[0211] An embodiment of the present invention further provides a computing device, including: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enabling the one or more processors to implement the building damage degree assessment method of the present invention. All implementation manners in the above method embodiments are applicable to the embodiments of this computing device and can also achieve the same technical effects.

[0212] An embodiment of the present invention further provides a computer-readable storage medium, in which a program is stored, and when the program is executed by a processor, the method for evaluating the damage degree of a building according to the present invention is implemented. All implementation manners in the above method embodiments are applicable to the embodiments of this computer-readable storage medium and can achieve the same technical effects.

[0213] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0214] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be repeated here.

[0215] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings, direct couplings, or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of devices or units can be in an electrical, mechanical, or other forms.

[0216] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0217] In addition, the functional units in each embodiment of the present invention can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0218] When the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0219] In addition, it should be noted that in the devices and methods of the present invention, obviously, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of the present invention. And, the steps of performing the above series of processes can naturally be executed in chronological order according to the described order, but it is not necessary to be executed in chronological order. Some steps can be executed in parallel or independently of each other. For those of ordinary skill in the art, it can be understood that all or any steps or components of the methods and devices of the present invention can be implemented in any computing device (including processors, storage media, etc.) or a network of computing devices in the form of hardware, firmware, software, or a combination thereof, which can be achieved by those of ordinary skill in the art using their basic programming skills after reading the description of the present invention.

[0220] Therefore, the object of the present invention can also be achieved by running a program or a set of programs on any computing device. The computing device can be a well-known general device. Therefore, the object of the present invention can also be achieved only by providing a program product containing program codes for implementing the method or device. That is to say, such a program product also constitutes the present invention, and the storage medium storing such a program product also constitutes the present invention. Obviously, the storage medium can be any well-known storage medium or any storage medium developed in the future. It should also be noted that in the devices and methods of the present invention, obviously, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of the present invention. And, the steps of performing the above series of processes can naturally be executed in chronological order according to the described order, but it is not necessary to be executed in chronological order. Some steps can be executed in parallel or independently of each other.

[0221] The above is the preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A method for evaluating the damage degree of a building, characterized in that, Including: Obtain visual data reflecting the damage degree of a building in a first scenario; Obtain prior knowledge data related to the damage degree of the building in a second scenario; Input the visual data into a first classifier for classifying the damage degree of the building to obtain a first classification result, where the first classifier is trained through a first preset model; Input the knowledge data into a second classifier for classifying the damage degree of the building to obtain a second classification result, where the second classifier is trained through a second preset model; Input the first classification result and the second classification result into a fusion model for evaluating the damage degree of the building to obtain an evaluation classification result of the damage degree of the building.

2. The method for evaluating the damage degree of a building according to claim 1, wherein, The training process of the first classifier includes: Obtain first training set data, where the first training set contains classification results of building images; Preprocess the first training set data to obtain first target data; Input the first target data into multiple convolutional processing layers of the first preset model to obtain a first output result; Input the first output result into the fully connected layer of the first preset model to obtain a second output result; Obtain a prediction loss value according to the second output result and the classification result of the first target data; Adjust hyperparameters of the first preset model according to the prediction loss value to obtain the first classifier.

3. The method for evaluating the damage degree of a building according to claim 2, wherein Obtain a prediction loss value according to the second output result and the classification result of the first target data, including: Obtain a prediction loss value through a cross-entropy loss function of the second output result and the classification result of the first target data, where the cross-entropy loss function is: Denotes loss, w i Denotes the weight assigned to each class i, y i Is the true class label, Is the predicted probability for class i.

4. The method for evaluating the degree of damage to a building according to claim 3, characterized in that, The training process of the first classifier further includes: Input the second output result into an activation mapping module of the first preset model to obtain an output gradient; Obtain a feature heat map according to the output gradient and the first output result.

5. The method for evaluating the damage degree of a building according to claim 1, characterized in that The training process of the second classifier includes: Obtain second training set data, where the second training set data contains classification results of building images with building location information; Preprocess the second training set data to obtain second target data; Input the second target data into a first processing layer of the second preset model for random sampling to obtain a third output result; Input the third output result into a second processing layer of the second preset model for random feature selection and node splitting for a preset number of times to obtain a fourth output result; Adjust hyperparameters of the second preset model according to the fourth output result and the classification result of the second target data to obtain the second classifier.

6. The method for evaluating the degree of damage to a building according to claim 5, characterized in that, Preprocess the second training set data to obtain second target data, including: Extract elevation data and location information from the second training set data; Obtain second intermediate data according to the elevation data and the location information; Perform interpolation processing on the second intermediate data to obtain second target data; Wherein, according to the elevation data, obtain geographical information in the second intermediate data, and according to the location information, obtain the normalized vegetation index and seismic impact data in the second intermediate data.

7. The method for evaluating the damage degree of a building according to claim 1, wherein Input the first classification result and the second classification result into a fusion model for building damage degree assessment processing to obtain a building damage degree assessment classification result, including: Obtain a dynamic weight according to the first classification result and the second classification result; Obtain a building damage degree assessment classification result according to the dynamic weight.

8. An apparatus for evaluating the damage degree of a building, characterized in that, Including: An acquisition module, configured to acquire visual data reflecting the building damage degree in a first scenario and prior knowledge data related to the building damage degree in a second scenario; A processing module, configured to input the visual data into a first classifier for building damage degree classification processing to obtain a first classification result, where the first classifier is trained by a first preset model; Input the knowledge data into a second classifier for building damage degree classification processing to obtain a second classification result, where the second classifier is trained by a second preset model; Input the first classification result and the second classification result into a fusion model for building damage degree assessment processing to obtain a building damage degree assessment classification result.

9. A computing device, characterized in that, Including: One or more processors; A storage device, configured to store one or more programs, and when the one or more programs are executed by the one or more processors, enable the one or more processors to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A program is stored in the computer-readable storage medium, and when the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images