A multi-level disaster detection method based on hierarchical learning

Abstract

The application discloses a multi-grade disaster detection method based on hierarchical learning, which comprises the following steps: performing hierarchical analysis on the multi-grade building damage detection problem from the perspective of task decomposition, and constructing a disaster detection model based on multi-task hierarchy; or performing hierarchical analysis on the multi-grade building damage detection problem by using a graph structure, transferring representation information according to the hierarchical relationship between different nodes, and constructing a disaster detection model based on graph coding hierarchy; and detecting the double-time-phase feature maps before and after the disaster by using the disaster detection model based on multi-task hierarchy or the disaster detection model based on graph coding hierarchy, to obtain a building damage detection result. The application models the experience of the multi-grade building damage detection problem into a network from the perspective of hierarchical learning, and improves the building damage detection precision.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image disaster detection technology, and more specifically to a multi-level disaster detection method based on hierarchical learning. Background Technology

[0002] In multi-level disaster detection, participants in the 2019 xView competition released a basic method along with a multi-level disaster detection dataset. This method decomposes the problem into two steps: the first step is to train the target localization, and the second step is to classify the damage level of the target. The localization network uses semantic segmentation, specifically U-Net, an encoding / decoding structure frequently used in semantic segmentation that effectively extracts features from building targets. The classification network uses the ResNet50 residual network, which considers no damage as a form of damage and directly outputs the damage classification, resulting in less than ideal performance. A paper published at the ICLR workshop in 2020 transformed the two-step prediction problem into a one-step prediction by fusing localization and classification into a single network, directly outputting the damage classification result. A paper published in the same year proposed a bilateral feature fusion network to enhance the interaction and correlation of features between preceding and following time images, and used the CutMix method for data augmentation to learn hard samples. In 2021, BDANet proposed a two-stage convolutional neural network. The first stage uses U-Net to extract the location of buildings, and the parameters share the weights of the second-stage dual-branch multi-scale classification network. It also proposed a bilateral attention mechanism to connect information from pre-disaster and post-disaster images, which achieved good results.

[0003] Current research on disaster detection largely focuses on change detection, specifically identifying changed areas in multi-temporal images and pinpointing damaged regions by comparing pre- and post-disaster images. This approach only considers pixel-level "whether a disaster has occurred" questions and lacks specific algorithms designed for multi-level disaster assessment. While a small number of algorithms do address multi-level issues, they merely extend the classification portion of change detection from binary to multi-class classification, failing to consider the prior information inherent in disaster assessment, thus resulting in less than ideal performance.

[0004] Therefore, how to provide a multi-level disaster detection method based on hierarchical learning that takes into account the prior information contained in disaster assessment and improves detection accuracy is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the present invention provides a multi-level disaster detection method based on hierarchical learning, which incorporates empirical modeling of the multi-level building damage detection problem into the network from the perspective of hierarchical learning, thereby improving the accuracy of building damage detection.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A multi-level disaster detection method based on hierarchical learning includes:

[0008] From the perspective of task decomposition, a hierarchical analysis of the multi-level building damage detection problem is conducted to construct a disaster detection model based on multi-task hierarchical analysis; or a graph structure is used to conduct a hierarchical analysis of the multi-level building damage detection problem, and the representation information is transmitted according to the hierarchical relationship between different nodes to construct a disaster detection model based on graph coding hierarchical analysis.

[0009] The disaster detection model based on multi-task hierarchical structure or the disaster detection model based on graph coding hierarchical structure is used to detect the dual-temporal feature maps before and after the disaster, and the building damage detection results are obtained.

[0010] Furthermore, the disaster detection model based on multi-task hierarchy is an explicit multi-stage disaster detection model, which divides the multi-level building damage detection task into three tasks, each task corresponding to a prediction network.

[0011] Among them, the prediction network corresponding to Task 1 detects whether it is a building, and its mathematical expression is: P(T=1)=φ1(I);

[0012] The prediction network corresponding to Task 2 detects whether the building is damaged. Its mathematical expression is: P(F=1)=P(F=1|T=1)×P(T=1)=φ2(T);

[0013] The prediction network corresponding to Task 3 detects the degree of damage to the building. Its mathematical expression is: P(V)=P(V|F=1,T=1)×P(F=1|T=1)×P(T=1)=φ3(F);

[0014] Wherein, P(T=1) means "the current pixel is a building", P(F=1|T=1) means "the current pixel is a building and is damaged", and P(V=1|F=1,T=1) means "the current pixel is a building and is damaged, with the degree of damage being a certain level"; φ1, φ2, and φ3 are the network parameters corresponding to the three tasks in the three stages.

[0015] Furthermore, the disaster detection model based on multi-task hierarchical structure is an implicit joint disaster detection model. The implicit joint disaster detection model divides the multi-level building damage detection task into three tasks, each task corresponding to a prediction network. The three prediction networks corresponding to the three tasks share the underlying features at a shallower level, and each learns its own unique parameters at a deeper level.

[0016] The loss functions of the prediction networks for the three tasks are as follows:

[0017]

[0018]

[0019]

[0020] Where N represents the total number of pixels in the image, t i y i z i These represent the labels for three independent tasks, θ1, θ2, and θ3 represent the parameters of the three prediction networks, and x i Feature map representing the difference between two temporal images; f(x) i θ1) represents the result of the network prediction;

[0021] The losses from the three tasks are weighted and summed to obtain the final loss of the implicit joint disaster detection model.

[0022] loss=α1loss1+α2loss2+α3loss3

[0023] Here, α1, α2, and α3 represent the weights of the three losses, respectively.

[0024] Furthermore, the weights of the three losses can be set according to the difficulty of the tasks, or according to the importance of the tasks, or by using attention to allow the implicit joint disaster detection model to learn a set of weights.

[0025] Furthermore, in the graph structure constructed for the multi-level detection of building damage, each edge represents a valid transition relationship branch, and data is transmitted between nodes through the representation transformation along the edge; nodes that have a transition relationship with the current node are neighbor nodes; lower-level nodes are used as parent nodes, and higher-level nodes are used as child nodes.

[0026] Furthermore, the graph structure of the disaster detection model based on graph coding hierarchy is a directed graph, and the message propagation direction is top-down; for a top-down directed graph, it can be expressed mathematically as:

[0027]

[0028] The above formula means that for each node k, considering its neighboring nodes {parent(k)}, node k itself is merged with its neighboring nodes, explicitly passing low-level information to high-level information to help high-level task learning; where σ represents the linear mapping of nodes to the latent space. This represents the initial representation of node k. {parent(k)} represents the initial representation of the lower-level neighbor nodes, and {parent(k)} represents the set of all lower-level neighbor nodes of node k.

[0029] Furthermore, the graph structure of the disaster detection model based on graph coding hierarchy is a directed graph, and the message propagation direction is bottom-up; for a bottom-up directed graph, the mathematical expression is:

[0030]

[0031] The above formula means that for each node k, its neighboring nodes {child(k)} are considered, and node k itself is merged with its neighboring nodes, explicitly passing high-level information to low-level nodes to help low-level task learning; where, {child(k)} is the initial representation of the higher-level neighbor nodes, and {child(k)} is the set of all higher-level neighbor nodes of node k.

[0032] Furthermore, the graph structure of the disaster detection model based on graph coding hierarchy is an undirected graph. In this undirected graph, the edges are bidirectionally connected, and the message propagation direction is bidirectional. The undirected graph is a combination of a top-down directed graph and a bottom-up directed graph, and its mathematical expression is:

[0033]

[0034] Furthermore, the graph structure of the disaster detection model based on graph coding hierarchy is an unweighted graph, in which the connecting edges between nodes have the same weight, indicating that the message transmission probability between nodes is the same.

[0035] Furthermore, the graph structure of the disaster detection model based on graph coding hierarchy is a weighted graph. In this weighted graph, the weight of each connecting edge is set according to the prior message transition probability. The mathematical formula for calculating the prior message transition probability is:

[0036]

[0037] f p (e j,k ) = 1

[0038] Among them, f c (e j,k f represents the prior message transition probability used in the top-down message propagation edge. p (e j,k ) represents the prior message transition probability used by the edge in the bottom-up message propagation, N. k N represents the number of pixels at node k. j Represents the number of pixels in node j; e j,kThis represents the connection from node j to node k.

[0039] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a multi-level disaster detection method based on hierarchical learning, with the following beneficial effects:

[0040] 1. This invention proposes a general disaster detection architecture that uses hierarchical learning to achieve multi-level disaster detection (MLDD) in order to realize fine-grained damage level classification.

[0041] 2. This invention constructs an end-to-end hierarchical learning architecture that is compatible with multiple MLDD methods. By establishing task associations from coarse to fine, two new methods are proposed: a multi-task hierarchy and a graph coding hierarchy.

[0042] 3. The method proposed in this invention achieves state-of-the-art performance on the large-scale building damage assessment dataset xBD and demonstrates advantages in challenging tasks involving fine-grained levels of building damage. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0044] Figure 1 A flowchart of the multi-level disaster detection method based on hierarchical learning provided by the present invention;

[0045] Figure 2 This is a schematic diagram of the structure of the explicit multi-stage disaster detection model provided by the present invention;

[0046] Figure 3 This is a schematic diagram of the implicit joint disaster detection model provided by the present invention;

[0047] Figure 4 A schematic diagram illustrating the hierarchical relationship of multi-level building damage detection provided by this invention;

[0048] Figure 5 This is a schematic diagram of the structure of the disaster detection model based on graph coding hierarchy provided by the present invention;

[0049] Figure 6 The graph structures corresponding to the three message passing methods provided by this invention;

[0050] Figure 7 This is a schematic diagram of the structure of the unweighted diagram and the weighted diagram provided by the present invention;

[0051] Figure 8 This is a visual comparison chart of disaster detection results between the hierarchical learning-based model and other detection models provided by this invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] like Figure 1 As shown, this embodiment of the invention discloses a multi-level disaster detection method based on hierarchical learning, including:

[0054] From the perspective of task decomposition, a hierarchical analysis of the multi-level building damage detection problem is conducted to construct a disaster detection model based on multi-task hierarchical analysis; or a graph structure is used to conduct a hierarchical analysis of the multi-level building damage detection problem, and the representation information is transmitted according to the hierarchical relationship between different nodes to construct a disaster detection model based on graph coding hierarchical analysis.

[0055] The disaster detection model based on multi-task hierarchical approach or graph coding hierarchical approach is used to detect the dual-temporal feature maps before and after the disaster, and the building damage detection results are obtained.

[0056] In one specific embodiment, the present invention decomposes the complex problem of multi-level building damage detection into: (1) input the original image and output "whether it is a building"; (2) based on "is a building", output "whether the building is damaged"; (3) based on "the building is damaged", output "the degree of damage to the building", which is expressed by the following mathematical formula:

[0057] P(T)=φ1(I)

[0058] P(F)=P(F|T=1)×P(T)=T2(T)

[0059] P(V)=p(V|F=1,T=1)×P(F|T=1)×P(T)=φ3(F)

[0060] Wherein, P(T) is the predicted probability of "whether it is a building", P(F) is the predicted probability of "whether the building is damaged", P(V) is the predicted probability of "the degree of damage to the building", and φ1, φ2, and φ3 are the network parameters corresponding to the three tasks in the three stages.

[0061] As can be seen from the above mathematical formulas, the method of this invention utilizes the multiplication of conditional probabilities for separate learning. For the multiplication of conditional probabilities, this invention proposes two model methods: an explicit multi-stage model and an implicit joint model.

[0062] Specifically, if the disaster detection model based on multi-task hierarchical structure is an explicit multi-stage disaster detection model, such as Figure 2 As shown, the explicit multi-stage disaster detection model divides the multi-level building damage detection task into three tasks, each corresponding to a prediction network;

[0063] Among them, the prediction network corresponding to Task 1 detects whether it is a building, and its mathematical expression is: P(T=1)=φ1(I);

[0064] The prediction network corresponding to Task 2 detects whether the building is damaged. Its mathematical expression is: P(F=1)=P(F=1|T=1)×P(T=1)=φ2(T);

[0065] The prediction network corresponding to Task 3 detects the degree of damage to the building. Its mathematical expression is: P(V)=P(V|F=1,T=1)×P(F=1|T=1)×P(T=1)=φ3(F);

[0066] Wherein, P(T=1) means "the current pixel is a building", P(F=1|T=1) means "the current pixel is a building and is damaged", and P(V=1|F=1,T=1) means "the current pixel is a building and is damaged, with the degree of damage being a certain level"; φ1(I), φ2(T), and φ3(F) represent the network prediction results of the above three tasks, respectively.

[0067] Task 1, "Is it a building?", is a binary classification task, and the prediction network focuses solely on predicting whether a building is damaged. Task 2, "Is the building damaged?", is also a binary classification task; the prediction network receives only "is a building" data and then predicts whether it is damaged. Task 3, "Degree of damage to the building?", differs from the previous tasks; it is a three-class classification problem ("slightly damaged", "severely damaged", "completely destroyed"). The prediction network receives only "damaged building" data and then predicts the degree of damage.

[0068] This invention trains three prediction networks for three tasks, with each prediction network focusing only on its own small part of the task. This can effectively reduce the bias caused by the coupling of multiple tasks. By reducing the bias brought by each part, the learning objective of the model is simplified, allowing the model to better summarize the key points of the task.

[0069] More importantly, multi-stage disaster models require separate prediction networks for each task, making it impossible to utilize the entire sample space and resulting in data waste. For example, in the problem of "whether a building is damaged," only samples that are "buildings" can be extracted, and then their "damage" can be predicted. In this case, the task can be described as "assuming this pixel is a building, what is the probability that it is damaged?" This is why the model cannot be trained using all samples, because the unavailable information is whether the "non-building" pixels are "damaged." Directly using 0 as the label would significantly mislead the model's learning.

[0070] Based on the mathematical expressions of the aforementioned multi-stage disaster detection model, P(T=1), meaning "is a building," has a clear label; P(F=1|T=1), meaning "the current pixel is a building and is damaged," has a clear label; and P(V=1|F=1,T=1), meaning "the current pixel is a building and is damaged, with a damage level of a certain degree," also has a clear label. Furthermore, all samples can be used for the three transformed tasks. Therefore, by learning the three transformed tasks, the previously defined three tasks can be implicitly learned.

[0071] Therefore, this embodiment proposes to adopt an implicit joint disaster detection model instead of a multi-task hierarchical disaster detection model, such as... Figure 3 As shown, the implicit joint disaster detection model divides the multi-level building damage detection task into three tasks, each corresponding to a prediction network. The three prediction networks corresponding to the three tasks share the underlying features at shallower levels, and learn their own unique parameters at deeper levels.

[0072] Since each prediction network is supervised by label information, each branch will generate a loss for the current task. The loss functions for the prediction networks corresponding to the three tasks are as follows:

[0073]

[0074]

[0075]

[0076] Where N represents the total number of pixels in the image, t i y i z i These represent the labels for three independent tasks, θ1, θ2, and θ3 represent the parameters of the three prediction networks, and x i Feature map representing the difference between two temporal images; f(x) i θ1) represents the result of the network prediction;

[0077] The losses from the three tasks are weighted and summed to obtain the final loss of the implicit joint disaster detection model:

[0078] loss=α1loss1+α2loss2+α3loss3

[0079] Here, α1, α2, and α3 represent the weights of the three losses, respectively. The three prediction networks, sharing underlying features, are simultaneously optimized according to the target, making full use of the entire sample space.

[0080] Specifically, the weights of the three losses are set according to the difficulty between tasks, or according to the importance of tasks, or by using attention to allow the implicit joint disaster detection model to learn a set of weights.

[0081] It can be seen that the implicit joint disaster detection model has optimized the sample selection space compared with the explicit multi-stage disaster detection model, making use of all samples. Furthermore, multiple tasks can improve each other by sharing underlying features, and the number of parameters and computations has been reduced. Only a small number of parameters are independent.

[0082] In other embodiments, the present invention further optimizes the model based on the association between each task, uses a graph structure to transmit hierarchical information between nodes, encodes hierarchical information from the perspective of information transmission, and improves the efficiency of the model.

[0083] In classifying multi-level disaster severity from multi-temporal images, different neurons are typically used to predict each class separately. However, in the task of multi-level disaster detection for buildings, nodes have logical relationships and belong to different hierarchical levels. Therefore, this invention designs a disaster detection model based on graph coding hierarchy, tailored to the characteristics of the task. Its specific structure is as follows: Figure 5 As shown, the core idea is to transmit representational information according to the hierarchical relationship between nodes, enabling predictions at different levels to learn from each other and increasing information flow between levels. For example, in predicting the "damage level of a building," the node representation of "whether the building is damaged" greatly helps in determining the "damage level." For instance, if the building is undamaged, the probability of the damage level should approach 0. This hierarchical relationship can be modeled using a graph, where each prediction target corresponds to a node in the graph structure, and the correlation between targets can be represented by edges connecting them in the graph structure. Based on this, the multi-level disaster detection problem of buildings can be solved by... Figure 4 express.

[0084] In graph structures, there are three basic elements to explore: (1) nodes. In this problem, based on human experience, the five-node classification problem is extended to a seven-node classification problem, namely "not a building", "is a building", "undamaged", "damaged", "slightly damaged", "severely damaged", and "completely damaged"; (2) node connection relationships. For message passing between levels, it is explored from the perspectives of directed graphs and undirected graphs. Among them, directed graphs are divided into top-down connections and bottom-up connections, while undirected graphs do not consider specific directions and can also be regarded as a combination of top-down and bottom-up connections; (3) edge weights, i.e., the state transition matrix between nodes. The information flow probability between different levels and different nodes needs to be studied, and two methods, average message passing and prior message passing, are proposed and verified by experiments.

[0085] Graph Convolutional Neural Networks (GCNs) are a type of convolutional neural network, but they can perform message passing on graphs and leverage the structural information of the graph to enhance node representations. This invention utilizes the basic ideas of graph convolutional neural networks and proposes a graph coding network model for hierarchical tasks. In the hierarchical graph, each edge represents a valid transition relationship branch, and data can be passed between nodes through representation transformations along the edges. Nodes with transition relationships to the current node are called neighboring nodes. For ease of identification, lower-level nodes are called parent nodes (e.g., the lowest level in this task is "whether it is a building" prediction), and higher-level nodes are called child nodes (the highest level in this task is "damage level" prediction).

[0086] Therefore, for node k, its potential neighbor relationships, which involve message passing, consist of its own transformations, its parent neighbor node, and its child neighbor nodes, forming N(k) = {n}. k ,child(k),parent(k)}. Then the hidden state of node k after graph encoding can be expressed mathematically as:

[0087]

[0088]

[0089]

[0090] in, This represents the transition of nodes. Denotes a real number field of dimension dim. Represents the real number field of N×dim. Represents the N-dimensional real number field. Let represent the real number field, dim represent the dimension of the feature vector, and N represent the number of neighboring nodes. j This represents the embedded representation of node j. σ represents the bias when node j transmits a message to node k, and v represents the nonlinear activation. k Represents the embedded representation of node k. This represents the bias when node g sends a message to node k, where N(k) represents all neighboring nodes of node k. k,j This represents the message that node j sends to node k, u k,j This represents the weight of the message exchange between nodes j and k. d(j,k) represents the hierarchical connection from node j to node k, including top-down, bottom-up, and self-circular edges. f represents the hierarchical transition probability d(k,j) (e kj ), where the self-circulating edge uses a k,k =1, use f for top-down edges c (e j,k ), use f for bottom-up edges p (e j,k );f d(k,j) e represents the hierarchical transition probability. kj f represents the connection from node k to node j. c e represents the transition probability in the top-down direction. j,k f represents the connection from node j to node k. p This represents the bottom-up transition probability. The feature matrix F = {a} of the edges of the entire graph. 0,0 ,a 0,1 ,…,a C-1,C-1} represents the weighted adjacency matrix of the hierarchical graph. Finally, node k outputs the hidden state h. k This represents a representation that incorporates the hierarchical graph structure, which serves as the final node feature used for prediction.

[0091] The above calculation process is a summary formula description, but there are still some important parameters that need to be confirmed. For example, when f c (e j,k )=0,f p (e j,k When f = 1, it degenerates into a bottom-up directed graph. c (e j,k )=1,f p (e j,k When f = 0, it degenerates into a top-down directed graph; the f between different levels d(k,j) (e kj )=t k,j These can vary, representing the probability of message transfer between levels. Below, we will analyze this from the perspectives of message transfer direction and message transfer probability.

[0092] Specifically, in multi-level disaster detection tasks, it is believed that information from different levels can mutually reinforce each other. Similar to a feature pyramid, fusing feature maps at different scales is beneficial for the model to learn representations.

[0093] Therefore, regarding the direction of message transmission, such as Figure 6 As shown, this invention proposes hierarchical graphs in two forms: directed and undirected. The directed graph further includes top-down and bottom-up propagation methods. For top-down propagation, for each node k, its neighboring nodes {parent(k)} are considered, and the node itself is merged with its neighbors, explicitly passing information from lower levels to higher levels to aid in task learning at higher levels. This can be expressed mathematically as follows:

[0094] For a directed graph where message passing proceeds from top to bottom, the mathematical expression is as follows:

[0095]

[0096] Where σ represents a linear mapping from a node to the latent space. This represents the initial representation of node k. {parent(k)} represents the initial representation of the lower-level neighbor nodes, and {parent(k)} represents the set of all lower-level neighbor nodes of node k.

[0097] For a directed graph where message passing follows a bottom-up propagation pattern, the mathematical expression is as follows:

[0098]

[0099] in, {child(k)} is the initial representation of the higher-level neighbor nodes, and {child(k)} is the set of all higher-level neighbor nodes of node k.

[0100] Undirected graphs can be viewed as a special type of directed graph, where all edges are bidirectionally connected. For multi-level disaster detection tasks, high-level and low-level features can be fused and mutually promoted to improve classification performance at each level, achieving hierarchical fine-grained classification. In bidirectional message propagation, the neighboring nodes of node k are {parent(k), child(k)}, meaning that "whether it is a building" and "the degree of damage" are fused to improve the classification accuracy of "whether the building is damaged." Undirected graphs can be seen as a combination of top-down and bottom-up directed graphs, considering information flow in both directions, which can be expressed mathematically as:

[0101]

[0102] In other embodiments, the weight of the connecting edges between nodes is also an important factor in the hierarchical structure. For example... Figure 7 As shown, graph structures can be further divided into weighted graphs and unweighted graphs. In a weighted graph, each edge has a different weight, while in an unweighted graph, each edge has the same weight, or a weight of 1. In other words, an unweighted graph does not distinguish between the weights of nodes compared to a weighted graph. In a hierarchical graph, different weights represent different message transition probabilities. For example, for the "building damaged" node, the transition probabilities to the "slightly damaged," "severely damaged," and "completely destroyed" nodes are different. Therefore, this invention proposes two message transition probabilities: the average message probability and the prior message probability.

[0103] Specifically, average message probability propagation corresponds to an unweighted graph, meaning the model does not specifically consider the probability transition between each level; as long as there is a connection between nodes, the message will be propagated.

[0104] Prior message probabilistic propagation is a type of weighted graph where each edge has a different propagation probability. It can be argued that different propagation probabilities between levels aid model learning. For example, in a severe disaster (such as an 8-magnitude earthquake), most buildings are "completely destroyed." Therefore, for a "destroyed" node, there is a high probability of transitioning to a "completely destroyed" node. Thus, the weight between these two edges needs to be increased in the graph structure to improve model learning. Based on this assumption, prior probabilities in the data are obtained according to the feature distribution and used as the weights of connecting edges. It's important to note that in detection tasks, this prior transition probability only exists from parent node to child node (top-down). However, each child node corresponds to a parent node; that is, when a child node occurs, a unique parent node can be found. Therefore, the transition probability from child node to parent node (bottom-up) only has a weight of 1. The mathematical formula for the prior transition probability is shown below.

[0105]

[0106] f p (e j,k ) = 1

[0107] Among them, f c (e j,k f represents the prior message transition probability used in the top-down message propagation edge. p (e j,k ) represents the prior message transition probability used by the edge in the bottom-up message propagation, N. k N represents the number of pixels at node k. j Represents the number of pixels in node j; e j,k This represents the connection from node j to node k.

[0108] The following specific experiments further verify the effectiveness of the method of the present invention.

[0109] 1. Data Acquisition

[0110] xBD is the largest building damage assessment dataset to date, containing 850,736 labeled building instances within 45,362 square kilometers of imagery. The dataset's annotations were guided by experts worldwide specializing in various disaster types to assess building damage in pre- and post-disaster satellite imagery. With a spatial resolution of 0.8m, the dataset provides a wealth of ground detail for different levels of damage assessment. xBD captures damage to buildings of varying types and densities caused by different disasters worldwide, including five types: earthquakes, floods, volcanic eruptions, wildfires, and hurricanes.

[0111] Furthermore, the xBD dataset goes beyond simply classifying buildings into binary categories of undamaged / damaged; it refines the classification of damage levels, allowing for a more precise assessment of the extent of damage. Damage levels are categorized into four types: undamaged, slightly damaged, severely damaged, and completely destroyed.

[0112] This dataset covers a wide geographical area, considers many types of disasters, and has a detailed classification of the degree of damage. Compared with datasets that only predict binary disaster situations and only have a single disaster type, it places higher demands on the algorithm.

[0113] 2. Evaluation Indicators

[0114] The evaluation metrics adopted in this invention are consistent with those used in the official xView2 competition, facilitating comparison with other models. Specifically, this invention calculates an F1 score (loc) for building location, which primarily examines the effectiveness of building location. Simultaneously, regarding damage, four damage levels—no damage (dmg1), minor damage (dmg2), severe damage (dmg3), and complete damage (dmg4)—each yield four F1 scores, representing the classification effectiveness for different levels. These four levels are combined to obtain the damage F1 score (dmg). The final score considers both building location accuracy and building damage accuracy; the specific score calculation formula is as follows.

[0115] score = 0.3 × loc + 0.7 × dmg

[0116] 3. Comparative Experiment Based on Multi-Task Hierarchy

[0117] This experiment primarily aims to verify the effectiveness of explicit multi-stage disaster detection models and implicit joint disaster detection models within a multi-task hierarchical disaster detection model. To verify the generalization ability of the strategy, experiments were conducted on four different skeleton networks to eliminate the influence of errors from single structures. Furthermore, comparisons were made with benchmark models that do not employ multi-task hierarchical architecture to validate the effectiveness of the proposed algorithm. The experimental results are shown in Table 1.

[0118] Table 1

[0119]

[0120]

[0121] The experimental results show that using multi-task hierarchical methods improves the scores of the baseline model compared to the one without. Furthermore, the implicit joint detection model demonstrates superior performance compared to the explicit multi-stage disaster detection model. Particular attention is paid to the scores for disaster levels 2 and 3, where the implicit joint disaster detection model exhibits very strong detection performance. While the explicit multi-stage disaster detection model's overall performance is not as good as the implicit joint model, it still significantly outperforms the baseline model without this strategy in terms of overall disaster severity. This further demonstrates the effectiveness of the two multi-level disaster detection models based on multi-task hierarchical methods proposed in this invention.

[0122] 4. Message Passing Direction Ablation Experiment Based on Graph Coding Hierarchy

[0123] This experiment primarily aims to verify the impact of different message passing directions on disaster detection performance in a graph-coded hierarchical disaster detection model. Graph coding structures can be divided into directed graphs and undirected graphs. The message aggregation direction of directed graphs can be further divided into top-down and bottom-up types. The baseline model is a disaster detection model without graph coding hierarchy. Similarly, to verify the generalization of the strategy, experiments were conducted on four different skeleton networks to eliminate the influence of errors from a single structure. The experimental results are shown in Table 2.

[0124] Table 2

[0125]

[0126]

[0127] The experimental results show that using the graph coding hierarchical strategy improves the scores of the baseline model without hierarchical structure. Regarding message passing direction, top-down message passing performs better than bottom-up message passing, demonstrating that low-level classification features (e.g., whether the data is damaged) help improve the accuracy of high-level classifications (e.g., damage level). High-level features also have some effect on low-level classification, but it is not very significant. Among the four models, bidirectional message passing shows the best performance, with at least a 2-point improvement. This indicates that the current classification node interacts with its parent and child neighbors, conveying hierarchical information and mutually improving classification accuracy.

[0128] 5. Message Passing Probabilistic Ablation Experiment Based on Graph Coding Hierarchy

[0129] This experiment primarily aims to verify the impact of different message passing probabilities on disaster detection performance in a graph-coded hierarchical disaster detection model. The baseline model is one without a graph-coded hierarchical strategy. In this model, average message passing does not consider the transition probabilities between each level, and the interactions between each node follow the same probability distribution. In contrast, prior message passing uses the transition probabilities between levels in the statistical data distribution to guide information exchange between nodes. Similarly, to verify the generalization ability of the strategy, experiments were conducted on four different models, and the results are shown in Table 3.

[0130] Table 3

[0131]

[0132]

[0133] The experimental results above show that using graph coding hierarchy outperforms the baseline model without a hierarchy strategy, and that message passing between levels based on prior probabilities is more effective than message passing based on average probabilities. This demonstrates that considering transition probabilities is effective for multi-level disaster detection tasks. Furthermore, the results of the four models show that graph coding hierarchy based on prior probabilities offers an improvement of more than one percentage point compared to the baseline model.

[0134] 6. Comparison of Four Hierarchical Disaster Detection Models

[0135] This experiment mainly compares four disaster detection methods, and the experimental results are shown in Table 4.

[0136] Table 4

[0137]

[0138]

[0139] Experimental results show that the pre-trained hierarchical disaster detection method outperforms the one-step disaster detection method, but both are inferior to the newly proposed multi-task hierarchical disaster detection model and the graph coding hierarchical disaster detection model. In terms of performance, the graph coding hierarchical disaster detection model achieves the best results and is significantly faster to train than the pre-trained hierarchical disaster detection method, eliminating the need for repeated training. While the multi-task hierarchical disaster detection model is slightly less effective than the graph coding hierarchical model, its advantage lies in its adaptability to new levels, allowing for the addition of new levels. However, the graph coding method requires a pre-defined hierarchical graph, necessitating retraining if new levels are added. In summary, the two hierarchical disaster detection methods proposed in this invention improve upon existing methods and offer advantages for different scenarios.

[0140] 7. Disaster detection visualization based on hierarchical learning

[0141] This invention compares the performance of the four disaster detection models in Table 4 on the xBD dataset and visualizes the final results. Black represents non-building areas, green represents undamaged buildings, yellow represents slightly damaged buildings, orange represents severely damaged buildings, and red represents completely destroyed buildings. The visualization is as follows: Figure 8 As shown.

[0142] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0143] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-level disaster detection method based on hierarchical learning, characterized in that, include: From the perspective of task decomposition, a hierarchical analysis of the multi-level building damage detection problem is carried out, and a disaster detection model based on multi-task hierarchy is constructed. Alternatively, a graph structure approach can be used to perform hierarchical analysis of the multi-level building damage detection problem, transmitting representational information according to the hierarchical relationship between different nodes, and constructing a disaster detection model based on graph coding hierarchy; The disaster detection model based on multi-task hierarchical structure or the disaster detection model based on graph coding hierarchical structure is used to detect the dual-temporal feature maps before and after the disaster to obtain the building damage detection results; The disaster detection model based on multi-task hierarchy is an explicit multi-stage disaster detection model. The explicit multi-stage disaster detection model divides the multi-level building damage detection task into three tasks, and each task corresponds to a prediction network. In Task 1, the prediction network detects whether an object is a building; its mathematical expression is: ; The prediction network corresponding to Task 2 detects whether a building is damaged. Its mathematical expression is: ; The prediction network corresponding to Task 3 detects the degree of damage to the building, and its mathematical expression is: ; in, This indicates that "the current pixel is a building". This indicates that "the current pixel is a building, and it is damaged". This indicates that "the current pixel is a building, and it is damaged, with a damage level of a certain degree"; , , These are the network parameters for the three stages and three tasks respectively; In the graph structure constructed for the multi-level detection of building damage, each edge represents a valid transition relationship branch, and data is transmitted between nodes through the representation transformation along the edge; nodes that have a transition relationship with the current node are neighbor nodes; lower-level nodes are designated as parent nodes, and higher-level nodes are designated as child nodes.

2. A multi-level disaster detection method based on hierarchical learning, characterized in that, include: From the perspective of task decomposition, a hierarchical analysis of the multi-level building damage detection problem is carried out, and a disaster detection model based on multi-task hierarchy is constructed. Alternatively, a graph structure approach can be used to perform hierarchical analysis of the multi-level building damage detection problem, transmitting representational information according to the hierarchical relationship between different nodes, and constructing a disaster detection model based on graph coding hierarchy; The disaster detection model based on multi-task hierarchical structure or the disaster detection model based on graph coding hierarchical structure is used to detect the dual-temporal feature maps before and after the disaster to obtain the building damage detection results; The disaster detection model based on multi-task hierarchical structure is an implicit joint disaster detection model. The implicit joint disaster detection model divides the multi-level building damage detection task into three tasks. Each task corresponds to a prediction network. The three prediction networks corresponding to the three tasks share the underlying features at a shallower level, and each learns its own unique parameters at a deeper level. The loss functions of the prediction networks for the three tasks are as follows: ； ； ； in, This represents the total number of pixels in the image. , , These represent labels for three independent tasks. , , These represent the parameters of the three prediction networks. Feature maps representing the differences between two temporal images; This represents the result of the network prediction; The losses from the three tasks are weighted and summed to obtain the final loss of the implicit joint disaster detection model. ； in, , , These represent the weights of the three losses; In the graph structure constructed for the multi-level detection of building damage, each edge represents a valid transition relationship branch, and data is transmitted between nodes through the representation transformation along the edge; nodes that have a transition relationship with the current node are neighbor nodes; lower-level nodes are designated as parent nodes, and higher-level nodes are designated as child nodes.

3. The multi-level disaster detection method based on hierarchical learning according to claim 2, characterized in that, The weights of the three losses are set according to the difficulty between tasks, or according to the importance of tasks, or the implicit joint disaster detection model learns a set of weights using attention.

4. A multi-level disaster detection method based on hierarchical learning according to claim 1 or 2, characterized in that, The disaster detection model based on graph coding hierarchy has a directed graph structure, and the message propagation direction is top-down. A top-down directed graph can be expressed mathematically as follows: ； The above formula means: for each node Consider its neighboring nodes This involves merging node k itself with its neighboring nodes, explicitly passing low-level information to higher-level nodes to aid in higher-level task learning; among these... This represents a linear mapping from a node to the latent space. Represents a node The initial characterization, This represents the initial representation of lower-level neighbor nodes. Represents a node The set of all lower-level neighbor nodes.

5. A multi-level disaster detection method based on hierarchical learning according to claim 1 or 2, characterized in that, The disaster detection model based on graph coding hierarchy has a directed graph structure, and the message propagation direction is bottom-up. The bottom-up directed graph can be expressed mathematically as follows: ； The above formula means: for each node Consider its neighboring nodes It also merges node k itself with its neighboring nodes, explicitly passing high-level information to low-level nodes to aid in low-level task learning; among which... This serves as the initial representation of the higher-level neighbor nodes. For nodes The set of all high-level neighbor nodes.

6. A multi-level disaster detection method based on hierarchical learning according to claim 1 or 2, characterized in that, The graph structure of the disaster detection model based on graph coding hierarchy is an undirected graph. In this undirected graph, the edges are bidirectional, and the message propagation direction is bidirectional. The undirected graph is a combination of a top-down directed graph and a bottom-up directed graph, and its mathematical expression is: 。 7. A multi-level disaster detection method based on hierarchical learning according to claim 1 or 2, characterized in that, The graph structure of the disaster detection model based on graph coding hierarchy is an unweighted graph. In the unweighted graph, the connecting edges between nodes have the same weight, indicating that the message transmission probability between nodes is the same.

8. A multi-level disaster detection method based on hierarchical learning according to claim 1 or 2, characterized in that, The graph structure of the disaster detection model based on graph coding hierarchy is a weighted graph. In this weighted graph, the weight of each connecting edge is set according to the prior message transition probability. The mathematical formula for calculating the prior message transition probability is as follows: ； ； in, This represents the prior message transition probability used in the top-down message propagation edge. This represents the prior message transition probability used by the edges that propagate messages from bottom to top. Represents a node The number of pixels, Represents a node The number of pixels; This represents the connection from node j to node k.

Images