A method and system for identifying potential landslides based on graph neural networks
By constructing a multi-level graph attention network model and combining feature fusion of large and small sample datasets, the problems of insufficient model generalization ability and small sample size in landslide identification are solved, and effective landslide identification in different regions is achieved.
Patent Information
- Application Number
- CN202411520923.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-29
- Publication Date
- 2025-10-28
- Estimated Expiration
- 2044-10-29
AI Technical Summary
Existing technologies for landslide identification suffer from insufficient model generalization ability and small sample size, making it difficult to effectively identify the diversity of complex geological conditions and the poor reliability of identification due to the insignificant characteristics of potential landslides. In particular, when there are large differences in landslide hazards between different regions, the model performance is greatly reduced.
A feature fusion model based on a multi-level graph attention network is constructed. The node layer provides reliable landslide sample features from dataset 1, the sub-network layer learns the similarity of landslide features between regions, and multimodal features are fused at the global layer. The model is pre-trained using a merged dataset of large sample dataset 1 and small sample dataset 2 to generate auxiliary set node features, thereby improving the model's classification performance and generalization ability.
It effectively extracts features in a small dataset with only 2 samples, improving the model's classification performance and generalization ability. It also demonstrates good recognition performance across different regions, solving the problems of wide-area generalization and small sample size in landslide identification.
Smart Images
Figure CN119418112B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of pattern recognition technology, and in particular to a potential landslide identification method and a potential landslide identification system based on graph neural networks. Background Technology
[0002] Approximately 70% of my country's territory is mountainous, with complex geological conditions and frequent geological disasters. Landslides, in particular, can cause massive losses of life and economy, leading to disasters such as traffic disruptions, river and canal blockages, farmland destruction, factory and mine collapses, village burials, and the deaths of people and livestock. Utilizing technologies such as satellite remote sensing and artificial intelligence for wide-area landslide detection and early warning is of great significance for disaster reduction and prevention. Furthermore, landslide early warning can provide important reference information for urban planning, avoiding large-scale construction activities in areas with a high probability of landslides and reducing potential risks.
[0003] Potential landslides that have begun geological activity but have not yet occurred typically have inconspicuous surface features, usually lacking boundary height differences and overall sliding surface morphology characteristics. Furthermore, their surface subsidence rate is usually on the order of millimeters per year. In addition, the diversity of topography and landslide causes leads to poor identification reliability and generalization performance.
[0004] The task of detecting potential landslides over a wide area faces the following main technical challenges:
[0005] 1) The problem of model generalization caused by landslides and diverse topography
[0006] The paradigm for using deep learning methods in landslide identification involves training a model on a local landslide hazard dataset and then validating and testing it on other landslide hazard datasets within the same region. However, the diversity of topography and geological structures leads to significant differences in landslide hazards across different regions, resulting in poor generalization ability of the identification model. This means that samples trained in one region exhibit drastically reduced performance when transferred to other regions. Given the extremely limited number of high-quality samples, the generalization problem cannot be solved by merging landslide samples from multiple regions for larger-scale model training, nor can domain-adaptive transfer learning improve generalization ability. Furthermore, the uneven distribution of landslide samples across different regions further exacerbates the model's generalization capability.
[0007] 2) Small sample size problem
[0008] Due to the indistinct surface morphology and insignificant surface subsidence of potential landslides, coupled with the complexity of preprocessing tasks such as InSAR data calibration, sample creation is difficult, and sample delineation and annotation are extremely time-consuming and labor-intensive. Consequently, high-quality sample data is scarce, making this a typical small-sample task. To extract highly abstract target semantic features, the model's learning ability typically needs to be strong enough, thus requiring a high quantity, diversity, and quality of training samples. Therefore, it is necessary to carefully design potential landslide identification models and model generalization methods, utilizing multi-regional landslide samples collaboratively to extract the essential features of potential landslides, thereby reducing the requirement for a large number of landslide samples. Summary of the Invention
[0009] To address the aforementioned problems, this invention provides a potential landslide identification method and system based on graph neural networks. By constructing a feature fusion model based on a multi-level graph attention network, a large number of reliable features from landslide samples in dataset 1 are provided at the node layer for the landslide samples in dataset 2 to be predicted. At the sub-network layer, the model learns the similarity between the landslide features of the two regions, enhancing reliable landslide features and suppressing background features. At the global layer, the multimodal features from the sub-network layer are fused and the classification result is output. Thus, only a small number of samples from dataset 2 are needed to effectively utilize the features of dataset 1, achieving effective extraction of features from dataset 2 and improving the classification performance and generalization ability of the model.
[0010] To achieve the above objectives, the present invention provides a potential landslide identification method based on graph neural networks, comprising:
[0011] A multi-level feature fusion graph attention network model is constructed, including a node layer, a sub-network layer, and a global layer. The graph attention network model adopts a dual-branch encoder-decoder structure.
[0012] The dual-branch encoder-decoder structure model was pre-trained until convergence using a merged dataset consisting of a large sample dataset 1 and a small sample dataset 2, thus forming a pre-trained model.
[0013] Based on the auxiliary set node features obtained by inputting the dataset 1 and the dataset 2 into the pre-trained model, the graph attention network model is trained and validated until the model converges and training is completed.
[0014] The image that needs to be identified as a potential landslide is input into the trained graph attention network model to obtain the prediction result of the potential landslide.
[0015] The pre-trained model generates auxiliary set node features based on dataset 1 and dataset 2, and the encoder in the node layer of the graph attention network model generates sample node features based on dataset 1 and dataset 2.
[0016] The sub-network layer constructs the InSAR graph attention sub-network and the DEM graph attention sub-network using the auxiliary set node features and the sample node features;
[0017] The global layer concatenates the output features of the InSAR map attention subnet and the DEM map attention subnet, and then uses the decoder to classify the samples.
[0018] In the above technical solution, preferably, the process of pre-training the dual-branch encoder-decoder structure model to convergence using the merged dataset of large-sample dataset 1 and small-sample dataset 2 to form a pre-trained model includes:
[0019] Select a subset of samples from dataset 2 as a small sample dataset for training;
[0020] The large sample dataset 1 and the small sample dataset 2 are merged to obtain the merged dataset;
[0021] The merged dataset is used to pre-train the dual-branch encoder-decoder structure model. The model is trained until it converges using a binary classification cross-entropy loss function and is used as the pre-trained model.
[0022] In this model, the two branch encoders do not share weights. During the pre-training process, the two encoders extract the InSAR node features and DEM node features of the merged dataset, respectively.
[0023] In the above technical solution, preferably, the specific process by which the pre-trained model generates auxiliary set node features based on dataset 1 and dataset 2 includes:
[0024] For the InSAR node features and DEM node features of dataset 1 and dataset 2 extracted by the encoder, the K-means clustering algorithm is used to cluster them into four types respectively.
[0025] In the above technical solution, preferably, the sub-network layer constructs the InSAR map attention sub-network and the DEM map attention sub-network through the auxiliary set node features and the sample node features, and the specific process includes:
[0026] The similarity between the auxiliary set node features and the sample node features is calculated.
[0027] Difficult samples are selected from the auxiliary set node features. Specifically, the same number of samples that are least similar to the sample node features and most similar to the sample node features are selected from the landslide type samples and the background type samples in the auxiliary set.
[0028] Both the InSAR map attention subnet and the DEM map attention subnet of the subnet layer include three multilayer perceptrons (MLPs), which are used to update the input node features, source node attention, and target node attention, respectively.
[0029] Each node of the input subnet updates its features through the first multilayer perceptron, and calculates the attention weights between the source node and the target node through the second and third multilayer perceptrons;
[0030] The attention weights are used to weight the features of each node updated by the first multilayer perceptron to obtain the comprehensive features of each node itself and its neighboring nodes.
[0031] In the above technical solution, preferably, after the global layer concatenates the output features of the InSAR image attention subnet and the DEM image attention subnet, the sample classification is performed by the decoder. The specific process includes:
[0032] After concatenating the output features of the InSAR map attention subnet and the DEM map attention subnet at the global layer, the input is fed into a decoder consisting of batch normalization, ReLU activation function and multilayer perceptron classification head;
[0033] The decoder outputs the class probability of the sample, and the sample classification is determined based on the class probability.
[0034] This invention also proposes a potential landslide identification system based on graph neural networks, applying the potential landslide identification method based on graph neural networks disclosed in any of the above technical solutions, including:
[0035] The model building module is used to build a multi-level feature fusion graph attention network model including a node layer, a sub-network layer and a global layer, wherein the graph attention network model adopts a dual-branch encoder-decoder structure;
[0036] The model pre-training module is used to pre-train the dual-branch encoder-decoder structure model until convergence using the merged dataset of large sample dataset 1 and small sample dataset 2, forming a pre-trained model.
[0037] The model training module trains and validates the graph attention network model based on the auxiliary set node features obtained by inputting dataset 1 and dataset 2 into the pre-trained model, until the model converges and training is completed.
[0038] The landslide prediction module is used to input the image that needs to be identified as a potential landslide into the trained graph attention network model to obtain the prediction result of the potential landslide.
[0039] The pre-trained model generates auxiliary set node features based on dataset 1 and dataset 2, and the encoder in the node layer of the graph attention network model generates sample node features based on dataset 1 and dataset 2.
[0040] The sub-network layer constructs the InSAR graph attention sub-network and the DEM graph attention sub-network using the auxiliary set node features and the sample node features;
[0041] The global layer concatenates the output features of the InSAR map attention subnet and the DEM map attention subnet, and then uses the decoder to classify the samples.
[0042] In the above technical solution, preferably, the model pre-training module is specifically used for:
[0043] Select a subset of samples from dataset 2 as a small sample dataset for training;
[0044] The large sample dataset 1 and the small sample dataset 2 are merged to obtain the merged dataset;
[0045] The graph attention network model is pre-trained using the merged dataset, and trained until the model converges using a binary classification cross-entropy loss function.
[0046] In this model, the two branch encoders do not share weights. During the pre-training process, the two encoders extract the InSAR node features and DEM node features of the merged dataset, respectively.
[0047] In the above technical solution, preferably, the specific process by which the pre-trained model generates auxiliary set node features based on dataset 1 and dataset 2 includes:
[0048] For the InSAR node features and DEM node features of dataset 1 and dataset 2 extracted by the encoder, the K-means clustering algorithm is used to cluster them into four types respectively.
[0049] In the above technical solution, preferably, the sub-network layer constructs the InSAR map attention sub-network and the DEM map attention sub-network through the auxiliary set node features and the sample node features, and the specific process includes:
[0050] The similarity between the auxiliary set node features and the sample node features is calculated.
[0051] Difficult samples are selected from the auxiliary set node features. Specifically, the same number of samples that are least similar to the sample node features and most similar to the sample node features are selected from the landslide type samples and the background type samples in the auxiliary set.
[0052] Both the InSAR map attention subnet and the DEM map attention subnet of the subnet layer include three multilayer perceptrons (MLPs), which are used to update the input node features, source node attention, and target node attention, respectively.
[0053] Each node of the input subnet updates its features through the first multilayer perceptron, and calculates the attention weights between the source node and the target node through the second and third multilayer perceptrons;
[0054] The attention weights are used to weight the features of each node updated by the first multilayer perceptron to obtain the comprehensive features of each node itself and its neighboring nodes.
[0055] In the above technical solution, preferably, after the global layer concatenates the output features of the InSAR image attention subnet and the DEM image attention subnet, the sample classification is performed by the decoder. The specific process includes:
[0056] After concatenating the output features of the InSAR map attention subnet and the DEM map attention subnet at the global layer, the input is fed into a decoder consisting of batch normalization, ReLU activation function and multilayer perceptron classification head;
[0057] The decoder outputs the class probability of the sample, and the sample classification is determined based on the class probability.
[0058] Compared with the prior art, the beneficial effects of the present invention are as follows: by constructing a feature fusion model based on a multi-level graph attention network, a large number of reliable features of landslide samples from dataset 1 are provided for the landslide samples of dataset 2 to be predicted at the node layer. In the sub-network layer, the model learns the similarity between the landslide features of the two regions, enhances the reliable landslide features, and suppresses the background features. In the global layer, the multimodal features of the sub-network layer are fused and the classification results are output. Thus, only a small number of dataset 2 samples are needed to effectively utilize the features of dataset 1, realize the effective extraction of features of dataset 2, and improve the classification performance and generalization ability of the model. Attached Figure Description
[0059] Figure 1 This is a schematic diagram of the overall structure of a multi-level graph attention network model disclosed in one embodiment of the present invention;
[0060] Figure 2 This is a schematic diagram of the structure of a pre-trained model disclosed in one embodiment of the present invention;
[0061] Figure 3 This is a schematic diagram of the subnet layer node selection strategy for dataset 2 disclosed in one embodiment of the present invention;
[0062] Figure 4This is a schematic diagram of the subnet layer node selection strategy for dataset 1 disclosed in one embodiment of the present invention;
[0063] Figure 5 This is a schematic diagram of a sub-network layer graph attention network structure disclosed in one embodiment of the present invention. Detailed Implementation
[0064] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0065] The present invention will now be described in further detail with reference to the accompanying drawings:
[0066] like Figure 1 As shown, a potential landslide identification method based on a graph neural network according to the present invention includes:
[0067] A multi-level graph attention network model with feature fusion, including node layer, sub-network layer and global layer, is constructed. The graph attention network model adopts a dual-branch encoder-decoder structure.
[0068] The dual-branch encoder-decoder structure model was pre-trained until convergence using a merged dataset consisting of a large sample dataset 1 and a small sample dataset 2, thus forming a pre-trained model.
[0069] Based on the auxiliary set node features obtained by inputting dataset 1 and dataset 2 into the pre-trained model, the graph attention network model is trained and validated until the model converges and training is completed.
[0070] The images that need to be identified as potential landslides are input into the trained graph attention network model to obtain the prediction results of potential landslides.
[0071] Among them, the pre-trained model generates auxiliary set node features based on dataset 1 and dataset 2, and the encoder in the node layer of the graph attention network model generates sample node features based on dataset 1 and dataset 2.
[0072] The sub-network layer constructs the InSAR graph attention sub-network and the DEM graph attention sub-network using auxiliary set node features and sample node features;
[0073] The global layer concatenates the output features of the InSAR map attention subnet and the DEM map attention subnet, and then uses a decoder to classify the samples.
[0074] In this implementation, a feature fusion model based on a multi-level graph attention network is constructed. At the node layer, a large number of reliable features of landslide samples from dataset 1 are provided for the landslide samples in dataset 2 to be predicted. At the sub-network layer, the model learns the similarity between the landslide features of the two regions, enhances the reliable landslide features, and suppresses the background features. At the global layer, the multimodal features of the sub-network layer are fused and the classification results are output. Thus, only a small number of dataset 2 samples are needed to effectively utilize the features of dataset 1, realizing the effective extraction of features of dataset 2 and improving the classification performance and generalization ability of the model.
[0075] Specifically, to overcome the challenge of wide-area generalization caused by the diversity of landslides and topography, under the constraint of small sample size, InSAR (Interferometric Synthetic Aperture Radar) data and DEM (Digital Elevation Model) data are used. Based on semantic similarity metric learning and multi-level feature fusion of multiple regional landslide samples, a graph attention network model is used to perform a binary classification task on the input image to determine whether there is a potential landslide at the regional scale, which meets the practical need for wide-area generalization with good recognition performance in different regions with significant differences.
[0076] The multi-level feature fusion graph attention network model includes a node layer, a sub-network layer, and a global layer. At the node layer, features are extracted from the input samples, and clustering is used to determine typical classes of potential landslides. Then, a semantic similarity metric is used to learn and select the sample that contributes the most to the current sample from the typical landslide classes as an auxiliary sample. At the sub-network layer, the semantic similarity between the target sample and the auxiliary samples is learned through the graph attention network, and an adjacency matrix is constructed. The features of the auxiliary samples, determined by annotation, are compared with the features of the target sample, thereby achieving a more comprehensive and reliable extraction of the semantic features of the target sample. At the global layer, the semantic features of the target sample are further abstracted, and its category is predicted.
[0077] This method enables the model to outperform models trained by merging multiple regions with only a small number of new region samples, extracts more comprehensive and essential landslide semantic features, exhibits better classification performance in new regions, and significantly improves the model's generalization ability.
[0078] The embodiments of this invention primarily study two datasets of potential landslide hazards in the target area, which differ significantly in terms of topography, landslide causes, and sample size. In the smaller dataset 2, landslides are mostly caused by significant deformation of the topsoil under rainwater erosion, resulting in small, fragmented, and varied landslide hazards that are difficult to identify. In contrast, landslides in dataset 1 are mostly caused by weak surfaces formed by fractured rock bases, which gradually slide under the influence of gravity. The landslide morphology is more fixed and the distribution is more uniform. Compared to dataset 2, dataset 1 has more significant features, is easier to identify, and has a larger sample size.
[0079] To improve the generalization performance of the model on the difficult-to-identify and small-sample dataset 2, features from dataset 1 are used to assist the model in identifying dataset 2. By merging a large number of samples from dataset 1 and a small number of samples from dataset 2 for pre-training, the pre-trained model can achieve high classification performance on the dataset 1 test set and extract reliable target and background features from dataset 1, while also learning some features from dataset 2.
[0080] The pre-trained model's role in landslide identification for Dataset 2 is twofold. First, the parameters of the pre-trained model from the merged datasets are used as the initialization parameters for the Dataset 2 model, giving it prior knowledge of landslide hazard characteristics. Second, the pre-trained model generates auxiliary feature nodes at the node layer. At the sub-network layer, a multi-level graph attention network compares the similarity of target and background features between Dataset 2 and Dataset 1. This allows the model to learn more comprehensive target semantic information, enhancing similar high-level semantic features while suppressing background information. Therefore, with the help of reliable features from Dataset 1, the designed multi-level graph attention network model learns the correlation between features in the two regions, achieving high classification performance for potential landslide hazards in this area with only a small number of samples from Dataset 2.
[0081] In the above implementation, preferably, a merged dataset consisting of a large sample dataset 1 and a small sample dataset 2 is used to pre-train the dual-branch encoder-decoder structure model until convergence, forming a pre-trained model. The specific process includes:
[0082] Select a subset of samples from dataset 2 as a small sample dataset for training;
[0083] The large sample dataset 1 and the small sample dataset 2 are merged to obtain the merged dataset;
[0084] The graph attention network model was pre-trained by merging datasets and trained until convergence using a binary classification cross-entropy loss function.
[0085] In this model, the two branch encoders do not share weights. During the pre-training process, the two encoders extract the InSAR node features and DEM node features of the merged dataset, respectively.
[0086] like Figure 2 As shown, specifically, the encoders Encoder1 and Encoder2 consist of dilated convolutional ResNet101 layers, ASPP, and max pooling layers. The decoder consists of a BN layer, a ReLU activation function, a Dropout layer, and a fully connected layer classification head. Encoders Encoder1 and Encoder2 extract features from InSAR and DEM data, respectively. The extracted features are concatenated along the channel dimension and then input into the decoder to obtain the final probabilities of each category. The pre-trained model is trained on a combined dataset of Dataset 1 and Small Sample Dataset 2. Auxiliary set samples are input into the pre-trained model encoder to obtain node features of the auxiliary set at the node layer, and then the node features of the auxiliary set are clustered.
[0087] In the above embodiments, preferably, the specific process by which the pre-trained model generates auxiliary set node features based on dataset 1 and dataset 2 includes:
[0088] For the InSAR node features and DEM node features of datasets 1 and 2 extracted by the encoder, the K-means clustering algorithm was used to cluster them into four types respectively.
[0089] Specifically, since the sub-network layer features are selected from the pre-trained node layer auxiliary set, in order to avoid the random selection method from resulting in nodes with similar semantic features, leading to less semantic information and limited auxiliary role in the graph attention network, this invention uses the K-means clustering algorithm to cluster the node layer features. A certain number of nodes are selected from each type of node obtained by clustering to construct the sub-network layer, ensuring that the selected landslide and background node features are diverse and can provide sufficient semantic information for the input samples.
[0090] After testing, it was found that for nodes in dataset 2 and dataset 1, the K-means algorithm with parameter K=4, i.e., four classes, has a better clustering effect on features.
[0091] For dataset 1, the clustering of InSAR features is strongly correlated with landslide and background features. Specifically, of the four resulting clusters, two are entirely landslide features and two are entirely background features, demonstrating a clear distinction between landslide and background features. However, the four clusters obtained from DEM feature clustering each include both landslide and background features, leading to a mixture of landslide and background features in the clusters resulting from the concatenation of InSAR and DEM features. This proves that DEM data does not contribute to the extraction of landslide features in dataset 1. Therefore, the auxiliary set for dataset 1 uses only InSAR features as node-level features and employs InSAR feature clustering.
[0092] For dataset 2, t samples are selected from the existing m-sample dataset M as the small sample dataset T to be trained, p samples are selected as the validation set, and n samples are selected as the test set. The mpn images that do not contain the validation and test sets are defined as the cluster set, where m >> t and p > n > t. After merging dataset 1 and dataset 2T, the model is pre-trained. Then, using the InSAR and DEM features of the cluster set obtained from the pre-trained model, the K-means clustering algorithm is used to cluster them. Although each cluster type contains landslide and background features, the number of landslide and background features differs significantly, indicating that the model has learned some features of dataset 2. The pre-trained auxiliary nodes of dataset 2 can provide auxiliary semantic information for the small sample model of dataset 2.
[0093] Furthermore, experiments verified that fusing DEM data did not improve the classification of landslide hazards in Dataset 1, and even led to a decrease in classification performance. However, for landslide hazards in Dataset 2, adding DEM data significantly improved classification performance. This is because the InSAR data features of landslide hazards in Dataset 1 are obvious, and high classification accuracy can be achieved using InSAR data alone. However, for landslide hazards in Dataset 2, the landslide features are altered due to the terrain, making it difficult to effectively extract semantic features of landslides using only InSAR data. Therefore, terrain features provided by DEM data are necessary. Thus, the auxiliary set for Dataset 2 includes InSAR feature nodes and DEM feature nodes, and clustering is performed using a concatenation method of InSAR and DEM features. Based on the sub-network layer node selection strategy, a similarity metric is calculated between the input sample features and the clustered auxiliary set nodes to select the difficult samples that contribute the most to the model's learning of the input sample features.
[0094] In the above implementation, preferably, the sub-network layer constructs the InSAR map attention sub-network and the DEM map attention sub-network by using auxiliary set hard node features and sample node features. The specific process includes:
[0095] Calculate the similarity between the auxiliary set node features and the sample node features;
[0096] Difficult samples are selected from the auxiliary set node features. Specifically, the same number of samples that are least similar to the sample node features and most similar to the sample node features are selected from the landslide class samples and the background class samples in the auxiliary set.
[0097] Both the InSAR map attention subnet and the DEM map attention subnet of the subnet layer include three multilayer perceptrons (MLPs), which are used to update the input node features, source node attention, and target node attention, respectively.
[0098] Each node of the input subnet updates its features through the first multilayer perceptron, and calculates the attention weights between the source node and the target node through the second and third multilayer perceptrons;
[0099] The attention weights are used to weight the features of each node updated by the first multilayer perceptron, resulting in a comprehensive feature that aggregates the features of each node itself and its neighboring nodes.
[0100] Specifically, sub-network layer nodes will select difficult samples from the auxiliary set of samples. Difficult samples are defined as the least similar samples of the same class and the most similar samples of different classes that are least similar to the input features. Since the auxiliary set of samples has label information and is considered a relatively reliable pre-trained feature, learning the correlation between the input sample and the difficult samples through the graph attention network can provide the input sample with the most semantic information.
[0101] For landslide-type input samples, the least dissimilar auxiliary set of landslide-type samples and the most similar auxiliary set of background-type samples should be selected. Conversely, for background-type input samples, the least dissimilar auxiliary set of background-type samples and the most similar auxiliary set of landslide-type samples should be selected. However, since the input sample category is unknown during training, it is difficult to determine which category of auxiliary set samples to calculate similarity with to select difficult samples. Therefore, for each input sample, the same number of least dissimilar and most similar samples are selected from both the landslide and background-type auxiliary sets, ensuring that difficult samples are included in the selected nodes regardless of the input sample's category. Furthermore, this sub-network layer node selection strategy is used during training, validation, and testing, maintaining consistency in the model's forward propagation process.
[0102] like Figure 3 As shown, for the node features of the auxiliary set of dataset 2, since the K-means algorithm is applied to the node features of the auxiliary set of dataset 2 using InSAR and DEM features concatenated, the InSAR and DEM features of the input sample are also concatenated. The similarity between the concatenated input features and the features concatenated according to the target in the auxiliary set of dataset 2 is calculated. Based on the above selection method for difficult samples, samples are selected from each cluster type, where the number of node features selected for each type is a hyperparameter.
[0103] like Figure 4As shown, for the node features of the auxiliary set of dataset 1, since the K-means algorithm is only applied to the InSAR features of the nodes of the auxiliary set of dataset 1, the similarity between the InSAR features of the input sample and the InSAR node features of the auxiliary set of dataset 1 is calculated. Based on the above selection method for difficult samples, samples are selected from each cluster type and input into the sub-network layer.
[0104] like Figure 5 As shown, the graph attention network in each subnet contains three multilayer perceptrons (MLPs), which are used to update the input node features, source node attention, and target node attention, respectively.
[0105] The connections between nodes form a complete graph, meaning that every two distinct vertices are connected by an edge, and an attention weight is calculated between any two vertices. Each node in the input subnet first updates its features through the first multilayer perceptron Θ1:
[0106] x ′ i =Θ1(x i )
[0107] The attention weights between the source node and the target node are then calculated using the second and third multilayer perceptrons Θ2 and Θ3:
[0108]
[0109] Where N(i) represents the set of neighboring nodes of node i, and φ represents the LeakyReLU function:
[0110] LeakyReLU(x)=max(0,x)+0.01min(0,x)
[0111] Finally, the calculated attention weights are used to weight the features of each node updated by Θ1, resulting in a summary of the features of each node itself and its neighboring nodes:
[0112]
[0113] In the above implementation, preferably, after the global layer concatenates the output features of the InSAR map attention subnet and the DEM map attention subnet, the sample is classified by a decoder. The specific process includes:
[0114] After concatenating the output features of the InSAR map attention subnet and the DEM map attention subnet at the global layer, the input is a decoder consisting of batch normalization, ReLU activation function and multilayer perceptron classification head;
[0115] The decoder outputs the class probability of the sample, and the sample classification is determined based on the class probability.
[0116] Specifically, the InSAR and DEM features of the training samples are updated through two subnets, then concatenated at the global layer and input into a decoder consisting of batch normalization, ReLU activation function, and a multilayer perceptron classification head to obtain the class probability of the samples. Both the pre-trained model and the multilayer graph attention network model use the binary cross-entropy loss function.
[0117] L CE = -ylog p + (1-y)log(1-p)
[0118] Where y represents the true label of the sample, with the background label being 0 and the landslide hazard label being 1, and p represents the probability that the sample is classified as a landslide.
[0119] According to the latent landslide identification method based on graph neural networks disclosed in the above embodiments, the training process of the graph attention network model is as follows:
[0120] Step 1:
[0121] The model was pre-trained using the combined datasets of Dataset 1 (training set) and Dataset 2 (cluster set) until convergence.
[0122] Step 2:
[0123] Input all samples from the training set of dataset 1 and the cluster set of dataset 2 into the pre-trained encoders Encoder1 and Encoder2 respectively to obtain the auxiliary set InSAR node feature and DEM node feature models.
[0124] Step 3:
[0125] Select a landslide sample from the small sample dataset 2, and input its InSAR and DEM data into the encoders Encoder1 and Encoder2 of the designed multi-level graph attention network model to obtain the input InSAR and input DEM features.
[0126] Step 4:
[0127] By applying the K-means clustering algorithm to the node features of the auxiliary sets of dataset 2 and dataset 1 respectively, the features of dataset 2 and dataset 1 can be divided into four categories.
[0128] Step 5:
[0129] Calculate the similarity between the input InSAR and input DEM features and the features of the four types of auxiliary set nodes, and select difficult InSAR and DEM auxiliary nodes from the auxiliary set nodes of dataset 2 and dataset 1 to input into the subnet layer.
[0130] Step 6:
[0131] At the sub-network layer, the InSAR and DEM node features selected from the node layer and the InSAR and DEM features of the input sample are used to construct two graph attention sub-networks, InSAR and DEM, respectively. The similarity measure between the landslide hazard features and background features in the input target and auxiliary target is learned, and the features of the input sample are updated.
[0132] Step 7:
[0133] After concatenating the output features of each subnetwork at the global layer, the results are input into the decoder to obtain the classification results. The loss function is then calculated and backpropagated to update the parameters of the multi-level graph attention network model.
[0134] Step 8:
[0135] Validate the performance of the multi-level graph attention network model on the validation set of dataset 2.
[0136] Step 9:
[0137] Repeat steps 3 to 8 until the model converges, and save the model parameters and convergence curve data.
[0138] Specifically, a dual-branch pre-trained model is trained by merging dataset 1 and small sample dataset 2, generating auxiliary set node features for the node layers of dataset 1 and dataset 2. K-means clustering is used, and difficult nodes are selected from the auxiliary set features obtained by clustering based on their similarity to the input sample features. This ensures that the selected auxiliary nodes have the richest semantic information for the sample to be predicted and are then input into the sub-network layer.
[0139] Among them, the InSAR and DEM graph attention subnetworks learn the similarity of features between the target to be predicted and each target in the auxiliary set from the InSAR and DEM modalities of the landslide sample, respectively, suppressing unreliable information and strengthening reliable information. Finally, the learned multimodal features are fused to enable the model to learn the comprehensive and essential semantic features of the landslide in the new area.
[0140] The present invention also proposes a potential landslide identification system based on graph neural networks, which applies the potential landslide identification method based on graph neural networks disclosed in any of the above embodiments, including:
[0141] The model building module is used to build a multi-level feature fusion graph attention network model that includes a node layer, a sub-network layer and a global layer. The graph attention network model adopts a dual-branch encoder-decoder structure.
[0142] The model pre-training module is used to pre-train the dual-branch encoder-decoder structure model until convergence using the merged dataset of large sample dataset 1 and small sample dataset 2, forming a pre-trained model.
[0143] The model training module trains and validates the graph attention network model based on the auxiliary set node features obtained by inputting dataset 1 and dataset 2 into the pre-trained model, until the model converges and training is complete.
[0144] The landslide prediction module is used to input images that need to be identified as potential landslides into a trained graph attention network model to obtain prediction results of potential landslides.
[0145] Among them, the pre-trained model generates auxiliary set node features based on dataset 1 and dataset 2, and the encoder in the node layer of the graph attention network model generates sample node features based on dataset 1 and dataset 2.
[0146] The sub-network layer constructs the InSAR graph attention sub-network and the DEM graph attention sub-network using auxiliary set node features and sample node features;
[0147] The global layer concatenates the output features of the InSAR map attention subnet and the DEM map attention subnet, and then uses a decoder to classify the samples.
[0148] In the above embodiments, preferably, the model pre-training module is specifically used for:
[0149] Select a subset of samples from dataset 2 as a small sample dataset for training;
[0150] The large sample dataset 1 and the small sample dataset 2 are merged to obtain the merged dataset;
[0151] The graph attention network model was pre-trained by merging datasets and trained until convergence using a binary classification cross-entropy loss function.
[0152] In this model, the two branch encoders do not share weights. During the pre-training process, the two encoders extract the InSAR node features and DEM node features of the merged dataset, respectively.
[0153] In the above embodiments, preferably, the specific process by which the pre-trained model generates auxiliary set node features based on dataset 1 and dataset 2 includes:
[0154] For the InSAR node features and DEM node features of datasets 1 and 2 extracted by the encoder, the K-means clustering algorithm is used to cluster them into four types respectively.
[0155] In the above implementation, preferably, the sub-network layer constructs the InSAR map attention sub-network and the DEM map attention sub-network by using auxiliary set hard node features and sample node features. The specific process includes:
[0156] Calculate the similarity between the auxiliary set node features and the sample node features;
[0157] Difficult samples are selected from the auxiliary set node features. Specifically, the same number of samples that are least similar to the sample node features and most similar to the sample node features are selected from the landslide class samples and the background class samples in the auxiliary set.
[0158] Both the InSAR map attention subnet and the DEM map attention subnet of the subnet layer include three multilayer perceptrons (MLPs), which are used to update the input node features, source node attention, and target node attention, respectively.
[0159] Each node of the input subnet updates its features through the first multilayer perceptron, and calculates the attention weights between the source node and the target node through the second and third multilayer perceptrons;
[0160] The attention weights are used to weight the features of each node updated by the first multilayer perceptron, resulting in a comprehensive feature that aggregates the features of each node itself and its neighboring nodes.
[0161] In the above implementation, preferably, after the global layer concatenates the output features of the InSAR map attention subnet and the DEM map attention subnet, the sample is classified by a decoder. The specific process includes:
[0162] After concatenating the output features of the InSAR map attention subnet and the DEM map attention subnet at the global layer, the input is a decoder consisting of batch normalization, ReLU activation function and multilayer perceptron classification head;
[0163] The decoder outputs the class probability of the sample, and the sample classification is determined based on the class probability.
[0164] The potential landslide identification method and system based on graph neural networks disclosed in the above embodiments are described in the following examples to illustrate the identification process and effect of the above method and system.
[0165] The dataset 1 used in this invention contains 962 images, of which 481 are positive samples of landslide hazards and 481 are negative background samples. It is divided into a training set, a validation set, and a test set in a 6:2:2 ratio, containing 576, 190, and 196 images respectively. The number of positive and negative samples in the training set, validation set, and test set are equal. The training set is used as an auxiliary set for dataset 1 to obtain auxiliary set node features.
[0166] The dataset 2 used in this invention contains 446 images, of which 223 are positive samples representing landslide hazards and 223 are negative background samples. These are divided into a clustering set, a validation set, and a test set in a 6:2:2 ratio, containing 268, 92, and 86 images respectively. Sixty images are randomly selected from the clustering set as a small training set, and another 20 images are selected based on the clustering results as auxiliary samples for dataset 2. The number of positive and negative samples in the clustering set, small training set, auxiliary set, validation set, and test set are all equal.
[0167] To enhance the model's ability to distinguish between positive and negative samples, this patent selects slope units with similar texture, color, and terrain variation characteristics to positive samples as negative samples. The negative sample selection strategy is as follows.
[0168]
[0169]
[0170] The trained model is evaluated using three metrics: average accuracy, landslide accuracy, and background accuracy. The formulas for these evaluation metrics are as follows:
[0171]
[0172] TP, TN, FP, and FN represent the number of accurately predicted landslide samples, the number of accurately predicted non-landslide samples, the number of landslide samples mistakenly identified as non-landslide samples, and the number of non-landslide samples mistakenly identified as landslide samples, respectively.
[0173] Using the above evaluation index formulas, the accuracy rates of the pre-trained model and the graph attention network model of this invention were calculated, as shown in the table below:
[0174]
[0175] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for identifying potential landslides based on graph neural networks, characterized in that, include: A multi-level feature fusion graph attention network model is constructed, including a node layer, a sub-network layer, and a global layer. The graph attention network model adopts a dual-branch encoder-decoder structure. The dual-branch encoder-decoder structure model was pre-trained until convergence using a merged dataset consisting of a large sample dataset 1 and a small sample dataset 2, thus forming a pre-trained model. Based on the auxiliary set node features obtained by inputting the dataset 1 and the dataset 2 into the pre-trained model, the graph attention network model is trained and validated until the model converges and training is completed. The images that need to be identified in the small sample dataset 2 are input into the trained graph attention network model to obtain the prediction results of potential landslides. The pre-trained model generates auxiliary set node features based on dataset 1 and dataset 2, and the encoder in the node layer of the graph attention network model generates sample node features based on dataset 1 and dataset 2. The sub-network layer constructs the InSAR map attention sub-network and the DEM map attention sub-network using the auxiliary set node features and the sample node features. The specific process includes: The similarity between the auxiliary set node features and the sample node features is calculated. Difficult samples are selected from the auxiliary set node features. Specifically, the same number of samples that are least similar to the sample node features and most similar to the sample node features are selected from the landslide type samples and the background type samples in the auxiliary set. Both the InSAR map attention subnet and the DEM map attention subnet of the subnet layer include three multilayer perceptrons (MLPs), which are used to update the input node features, source node attention, and target node attention, respectively. Each node of the input subnet updates its features through the first multilayer perceptron, and calculates the attention weights between the source node and the target node through the second and third multilayer perceptrons; The attention weights are used to weight the features of each node updated by the first multilayer perceptron to obtain the comprehensive features of each node itself and its neighboring nodes. The global layer concatenates the output features of the InSAR map attention subnet and the DEM map attention subnet, and then uses the decoder to classify the samples.
2. The potential landslide identification method based on graph neural networks according to claim 1, characterized in that, The combined dataset of large-sample dataset 1 and small-sample dataset 2 is used to pre-train the dual-branch encoder-decoder structure model until convergence, forming a pre-trained model. The specific process includes: Select a subset of samples from dataset 2 as a small sample dataset for training; The large sample dataset 1 and the small sample dataset 2 are merged to obtain the merged dataset; The merged dataset is used to pre-train the dual-branch encoder-decoder structure model. The model is trained until it converges using a binary classification cross-entropy loss function and is used as the pre-trained model. In this model, the two branch encoders do not share weights. During the pre-training process, the two encoders extract the InSAR node features and DEM node features of the merged dataset, respectively.
3. The potential landslide identification method based on graph neural networks according to claim 2, characterized in that, The specific process by which the pre-trained model generates auxiliary set node features based on dataset 1 and dataset 2 includes: For the InSAR node features and DEM node features of dataset 1 and dataset 2 extracted by the encoder, the K-means clustering algorithm is used to cluster them into four types respectively.
4. The potential landslide identification method based on graph neural networks according to claim 1, characterized in that, The global layer concatenates the output features of the InSAR image attention subnet and the DEM image attention subnet, and then performs sample classification through the decoder. The specific process includes: After concatenating the output features of the InSAR map attention subnet and the DEM map attention subnet at the global layer, the input is fed into a decoder consisting of batch normalization, ReLU activation function and multilayer perceptron classification head; The decoder outputs the class probability of the sample, and the sample classification is determined based on the class probability.
5. A potential landslide identification system based on graph neural networks, characterized in that, The method for identifying potential landslides based on graph neural networks as described in any one of claims 1 to 4 includes: The model building module is used to build a multi-level feature fusion graph attention network model including a node layer, a sub-network layer and a global layer, wherein the graph attention network model adopts a dual-branch encoder-decoder structure; The model pre-training module is used to pre-train the dual-branch encoder-decoder structure model until convergence using the merged dataset of large sample dataset 1 and small sample dataset 2, forming a pre-trained model. The model training module trains and validates the graph attention network model based on the auxiliary set node features obtained by inputting dataset 1 and dataset 2 into the pre-trained model, until the model converges and training is completed. The landslide prediction module is used to input the image that needs to be identified as a potential landslide into the trained graph attention network model to obtain the prediction result of the potential landslide. The pre-trained model generates auxiliary set node features based on dataset 1 and dataset 2, and the encoder in the node layer of the graph attention network model generates sample node features based on dataset 1 and dataset 2. The sub-network layer constructs the InSAR graph attention sub-network and the DEM graph attention sub-network using the auxiliary set node features and the sample node features; The global layer concatenates the output features of the InSAR map attention subnet and the DEM map attention subnet, and then uses the decoder to classify the samples.
6. The potential landslide identification system based on graph neural networks according to claim 5, characterized in that, The model pre-training module is specifically used for: Select a subset of samples from dataset 2 as a small sample dataset for training; The large sample dataset 1 and the small sample dataset 2 are merged to obtain the merged dataset; The merged dataset is used to pre-train the dual-branch encoder-decoder structure model. The model is trained until it converges using a binary classification cross-entropy loss function and is used as the pre-trained model. In this model, the two branch encoders do not share weights. During the pre-training process, the two encoders extract the InSAR node features and DEM node features of the merged dataset, respectively.
7. The potential landslide identification system based on graph neural networks according to claim 6, characterized in that, The specific process by which the pre-trained model generates auxiliary set node features based on dataset 1 and dataset 2 includes: For the InSAR node features and DEM node features of dataset 1 and dataset 2 extracted by the encoder, the K-means clustering algorithm is used to cluster them into four types respectively.
8. The potential landslide identification system based on graph neural networks according to claim 5, characterized in that, The sub-network layer constructs the InSAR map attention sub-network and the DEM map attention sub-network using the auxiliary set node features and the sample node features. The specific process includes: The similarity between the auxiliary set node features and the sample node features is calculated. Difficult samples are selected from the auxiliary set node features. Specifically, the same number of samples that are least similar to the sample node features and most similar to the sample node features are selected from the landslide type samples and the background type samples in the auxiliary set. Both the InSAR map attention subnet and the DEM map attention subnet of the subnet layer include three multilayer perceptrons (MLPs), which are used to update the input node features, source node attention, and target node attention, respectively. Each node of the input subnet updates its features through the first multilayer perceptron, and calculates the attention weights between the source node and the target node through the second and third multilayer perceptrons; The attention weights are used to weight the features of each node updated by the first multilayer perceptron to obtain the comprehensive features of each node itself and its neighboring nodes.
9. The potential landslide identification system based on graph neural networks according to claim 5, characterized in that, The global layer concatenates the output features of the InSAR image attention subnet and the DEM image attention subnet, and then performs sample classification through the decoder. The specific process includes: After concatenating the output features of the InSAR map attention subnet and the DEM map attention subnet at the global layer, the input is fed into a decoder consisting of batch normalization, ReLU activation function and multilayer perceptron classification head; The decoder outputs the class probability of the sample, and the sample classification is determined based on the class probability.