Potential landslide identification method and system based on graph neural network

Abstract

The invention discloses a potential landslide identification method and system based on a graph neural network, and the method comprises the steps: constructing a multi-level feature fusion graph attention network model which comprises a node layer, a subnet layer and a global layer, and employs a dual-branch encoder-decoder structure; the method comprises the following steps of: pre-training a dual-branch encoder-decoder structure model until convergence by adopting a combined data set of a data set 1 of a large sample and a data set 2 of a small sample to form a pre-training model; based on auxiliary set node features obtained by inputting the data set 1 and the data set 2 into the pre-training model, training and verifying the graph attention network model until model convergence training is completed; and inputting the pictures needing to be identified in the data set 2 into the trained picture attention network model to obtain a prediction result of the potential landslide. Through the technical scheme of the invention, effective extraction of the target features in the small data set 2 is realized, and the classification performance and generalization ability of the model are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of pattern recognition, and in particular to a potential landslide recognition method based on a graph neural network and a potential landslide recognition system based on a graph neural network. Background Art

[0002] About 70% of my country's territory is mountainous, with complex geological conditions and frequent geological disasters. Among them, landslides can cause traffic interruptions, river damming, farmland destruction, factory and mine destruction, village burial, people and livestock being crushed to death and other disasters, causing huge personal and economic losses. Using satellite remote sensing and artificial intelligence and other technologies to detect and warn of landslides in a wide area is of great significance for disaster reduction and prevention. In addition, landslide warnings can also provide important reference information for urban planning, avoid large-scale construction activities in areas with high landslide probabilities, and reduce potential risks.

[0003] The surface features of potential landslides that have begun geological activities but have not yet occurred are not significant. They usually do not have the surface morphological characteristics of boundary height difference and overall sliding. At the same time, the surface settlement rate is usually in the millimeter level per year. Coupled with the diversity of topography and landslide causes, the recognition reliability and generalization performance are poor.

[0004] The wide-area generalized potential landslide detection task mainly has the following technical difficulties:

[0005] 1) The problem of wide-area generalization of models caused by landslides and landform diversity

[0006] The paradigm of using deep learning methods in the field of landslide identification is to train the model on a landslide hazard dataset in a certain area, and verify and test it on other landslide hazards in the same area. Due to the diversity of topography and geological structure, the landslide hazards in different regions vary greatly, which makes the generalization ability of the identification model low, that is, the performance of samples trained in a certain area is greatly reduced when they are migrated to other areas. Under the constraint of extremely limited number of high-quality samples, the model generalization problem cannot be solved by merging landslide samples from multiple regions for larger-scale model training, nor can the generalization ability be improved by transfer learning in a domain adaptive way. In addition, the current situation of uneven number of landslide samples in different regions further deteriorates the generalization ability of the model.

[0007] 2) Small sample problem

[0008] Due to the unclear surface morphological features of potential landslides and the insignificant surface settlement, coupled with the complexity of pre-processing work such as InSAR data calibration, sample preparation is difficult, and sample circle marking is extremely time-consuming and laborious, resulting in a scarcity of high-quality sample data, which is a typical small sample task. In order to extract highly abstract target semantic features, the learning ability of the model is usually strong enough, which places high requirements on the number, diversity and quality of training samples. Therefore, it is necessary to carefully design the potential landslide identification model and model generalization method, and use multi-regional landslide samples to collaboratively extract the essential characteristics of potential landslides, thereby reducing the number of landslide samples required. Summary of the invention

[0009] In view of the above problems, the present invention provides a potential landslide identification method and system based on graph neural network. By constructing a feature fusion model based on a multi-level graph attention network, a large number of reliable data set 1 landslide sample features are provided for the landslide samples of data set 2 to be predicted at the node layer. The subnetwork layer model learns the similarity between the landslide features of the two regions, enhances the reliable landslide features, and suppresses the background features. The subnetwork layer multimodal features are fused at the global layer and the classification results are output. In this way, only a small number of data set 2 samples are needed to effectively utilize the features of data set 1, effectively extract the features of data set 2, and improve the classification performance and generalization ability of the model.

[0010] To achieve the above object, the present invention provides a potential landslide identification method based on graph neural network, comprising:

[0011] Constructing a multi-level feature fusion graph attention network model including a node layer, a subnetwork layer and a global layer, wherein the graph attention network model adopts a dual-branch encoder-decoder structure;

[0012] Using a combined data set of a large sample data set 1 and a small sample data set 2, a dual-branch encoder-decoder structure model is pre-trained until convergence to form a pre-trained model;

[0013] Based on the auxiliary set node features obtained by inputting the data set 1 and the data set 2 into the pre-trained model, the graph attention network model is trained and verified until the model converges and the training is completed;

[0014] Inputting the image that needs to be identified as a potential landslide into the trained graph attention network model to obtain a prediction result of the potential landslide;

[0015] Wherein, the pre-trained model generates auxiliary set node features according to the data set 1 and the data set 2, and the encoder in the node layer of the graph attention network model generates sample node features according to the data set 1 and the data set 2;

[0016] The subnetwork layer constructs an InSAR image attention subnetwork and a DEM image attention subnetwork through the auxiliary set node features and the sample node features;

[0017] The global layer concatenates the output features of the InSAR image attention subnet and the DEM image attention subnet, and then performs sample classification through the decoder.

[0018] In the above technical solution, preferably, the combined data set of the large sample data set 1 and the small sample data set 2 is used to pre-train the dual-branch encoder-decoder structure model until convergence to form a pre-training model, and the specific process includes:

[0019] Select some samples from data set 2 as the small sample data set to be trained;

[0020] Merge the large sample data set 1 and the small sample data set 2 to obtain a merged data set;

[0021] The dual-branch encoder-decoder structure model is pre-trained using the combined data set, and the binary cross entropy loss function is used to train the model until the model converges as a pre-training model;

[0022] The two branch encoders of the pre-training model do not share weights. During the pre-training process, the two encoders respectively extract the InSAR node features and the DEM node features of the merged data set.

[0023] In the above technical solution, preferably, the specific process of the pre-trained model generating auxiliary set node features according to the data set 1 and the data set 2 includes:

[0024] The InSAR node features and DEM node features of the dataset 1 and the dataset 2 extracted by the encoder are clustered into four types using a K-means clustering algorithm.

[0025] In the above technical solution, preferably, the subnetwork layer constructs the InSAR image attention subnetwork and the DEM image attention subnetwork through the auxiliary set node features and the sample node features, and the specific process includes:

[0026] Calculating similarity between the auxiliary set node features and the sample node features;

[0027] Selecting difficult samples from the auxiliary set node features, specifically, selecting the same number of the most dissimilar samples and the most similar samples to the sample node features from the landslide samples and background samples in the auxiliary set;

[0028] The InSAR image attention subnet and the DEM image attention subnet of the subnet layer both include three multi-layer perceptrons (MLPs) for updating input node features, source node attention, and target node attention, respectively;

[0029] Each node of the input subnet updates its features through the first multi-layer perceptron, and the attention weights between the source node and the target node are calculated through the second and third multi-layer perceptrons;

[0030] The attention weights are used to weight the features of each node updated by the first multi-layer perceptron to obtain the comprehensive features of each node itself and its neighboring nodes.

[0031] In the above technical solution, preferably, after the global layer splices the output features of the InSAR image attention subnet and the DEM image attention subnet, the decoder is used to perform sample classification, and the specific process includes:

[0032] After splicing the output features of the InSAR image attention subnet and the DEM image attention subnet at the global layer, the features are input into a decoder composed of batch normalization, ReLU activation function and multi-layer perceptron classification head;

[0033] The decoder outputs the category probability of the sample and determines the sample classification according to the category probability.

[0034] The present invention further proposes a potential landslide identification system based on a graph neural network, which uses the potential landslide identification method based on a graph neural network disclosed in any one of the above technical solutions, including:

[0035] A model building module, used to build a graph attention network model including a node layer, a subnetwork layer and a global layer with multi-level feature fusion, wherein the graph attention network model adopts a dual-branch encoder-decoder structure;

[0036] A model pre-training module is used to use a combined data set of a large sample data set 1 and a small sample data set 2 to pre-train a dual-branch encoder-decoder structure model until convergence to form a pre-trained model;

[0037] A model training module, which trains and verifies the graph attention network model based on the auxiliary set node features obtained by inputting the data set 1 and the data set 2 into the pre-trained model until the model convergence training is completed;

[0038] A landslide prediction module, used for inputting a picture that needs to be identified as a potential landslide into the trained graph attention network model to obtain a prediction result of a potential landslide;

[0039] Wherein, the pre-trained model generates auxiliary set node features according to the data set 1 and the data set 2, and the encoder in the node layer of the graph attention network model generates sample node features according to the data set 1 and the data set 2;

[0040] The subnetwork layer constructs an InSAR image attention subnetwork and a DEM image attention subnetwork through the auxiliary set node features and the sample node features;

[0041] The global layer concatenates the output features of the InSAR image attention subnet and the DEM image attention subnet, and then performs sample classification through the decoder.

[0042] In the above technical solution, preferably, the model pre-training module is specifically used for:

[0043] Select some samples from data set 2 as the small sample data set to be trained;

[0044] Merge the large sample data set 1 and the small sample data set 2 to obtain a merged data set;

[0045] The graph attention network model is pre-trained using the merged data set, and the binary cross entropy loss function is used to train the model until it converges as a pre-trained model;

[0046] The two branch encoders of the pre-training model do not share weights. During the pre-training process, the two encoders respectively extract the InSAR node features and the DEM node features of the merged data set.

[0047] In the above technical solution, preferably, the specific process of the pre-trained model generating auxiliary set node features according to the data set 1 and the data set 2 includes:

[0048] The InSAR node features and DEM node features of the dataset 1 and the dataset 2 extracted by the encoder are clustered into four types using a K-means clustering algorithm.

[0049] In the above technical solution, preferably, the subnetwork layer constructs the InSAR image attention subnetwork and the DEM image attention subnetwork through the auxiliary set node features and the sample node features, and the specific process includes:

[0050] Calculating similarity between the auxiliary set node features and the sample node features;

[0051] Selecting difficult samples from the auxiliary set node features, specifically, selecting the same number of the most dissimilar samples and the most similar samples to the sample node features from the landslide samples and background samples in the auxiliary set;

[0052] The InSAR image attention subnet and the DEM image attention subnet of the subnet layer both include three multi-layer perceptrons (MLPs) for updating input node features, source node attention, and target node attention, respectively;

[0053] Each node of the input subnet updates its features through the first multi-layer perceptron, and the attention weights between the source node and the target node are calculated through the second and third multi-layer perceptrons;

[0054] The attention weights are used to weight the features of each node updated by the first multi-layer perceptron to obtain the comprehensive features of each node itself and its neighboring nodes.

[0055] In the above technical solution, preferably, after the global layer splices the output features of the InSAR image attention subnet and the DEM image attention subnet, the decoder is used to perform sample classification, and the specific process includes:

[0056] After splicing the output features of the InSAR image attention subnet and the DEM image attention subnet at the global layer, the features are input into a decoder composed of batch normalization, ReLU activation function and multi-layer perceptron classification head;

[0057] The decoder outputs the category probability of the sample and determines the sample classification according to the category probability.

[0058] Compared with the prior art, the beneficial effects of the present invention are as follows: by constructing a feature fusion model based on a multi-level graph attention network, a large number of reliable data set 1 landslide sample features are provided for the landslide samples of data set 2 to be predicted at the node layer, the subnetwork layer model learns the similarity between the landslide features of the two regions, enhances the reliable landslide features, suppresses the background features, and fuses the subnetwork layer multimodal features at the global layer and outputs the classification results, so that only a small number of data set 2 samples are needed to effectively utilize the features of data set 1, thereby realizing the effective extraction of the features of data set 2 and improving the classification performance and generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A schematic diagram of the overall structure of a multi-level graph attention network model disclosed in an embodiment of the present invention;

[0060] Figure 2 A schematic diagram of the structure of a pre-training model disclosed in an embodiment of the present invention;

[0061] Figure 3 A schematic diagram of a data set 2 subnet layer node selection strategy disclosed in an embodiment of the present invention;

[0062] Figure 4A schematic diagram of a data set 1 subnet layer node selection strategy disclosed in an embodiment of the present invention;

[0063] Figure 5 A schematic diagram of a subnetwork layer graph attention network structure disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0065] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0066] like Figure 1 As shown, a potential landslide identification method based on graph neural network provided by the present invention includes:

[0067] Construct a graph attention network model with multi-level feature fusion including node layer, subnetwork layer and global layer, where the graph attention network model adopts a dual-branch encoder-decoder structure;

[0068] Using a combined data set of a large sample data set 1 and a small sample data set 2, a dual-branch encoder-decoder structure model is pre-trained until convergence to form a pre-trained model;

[0069] Based on the auxiliary set node features obtained by inputting dataset 1 and dataset 2 into the pre-trained model, the graph attention network model is trained and verified until the model converges and the training is completed;

[0070] Input the image that needs to be identified as a potential landslide into the trained graph attention network model to obtain the prediction result of the potential landslide;

[0071] Among them, the pre-trained model generates auxiliary set node features according to dataset 1 and dataset 2, and the encoder in the node layer of the graph attention network model generates sample node features according to dataset 1 and dataset 2;

[0072] The subnetwork layer constructs the InSAR image attention subnetwork and the DEM image attention subnetwork through the auxiliary set node features and sample node features;

[0073] The global layer concatenates the output features of the InSAR image attention subnet and the DEM image attention subnet, and then performs sample classification through the decoder.

[0074] In this implementation, by constructing a feature fusion model based on a multi-level graph attention network, a large number of reliable data set 1 landslide sample features are provided for the landslide samples of data set 2 to be predicted at the node layer, the subnetwork layer model learns the similarity between the landslide features of the two regions, enhances the reliable landslide features, suppresses the background features, and fuses the subnetwork layer multimodal features at the global layer and outputs the classification results, so that only a small number of data set 2 samples are needed to effectively utilize the features of data set 1, thereby achieving effective extraction of data set 2 features and improving the classification performance and generalization ability of the model.

[0075] Specifically, in order to overcome the problem of wide-area generalization brought about by the diversity of landslides and topography, under the constraint of small samples, InSAR (Interferometric Synthetic Aperture Radar) data and DEM (Digital Elevation Model) data are used. A graph attention network model with multi-level feature fusion based on semantic similarity metric learning and collaborative landslide samples from multiple regions is used to perform a binary classification task on whether there is a potential landslide in the input image at a regional scale, which meets the actual needs of wide-area generalization with good recognition performance in different regions with significant differences.

[0076] Among them, the graph attention network model with multi-level feature fusion includes node layer, subnetwork layer and global layer. At the node layer, the input samples are feature extracted, and the typical class of potential landslides is determined by clustering. Then, the samples that contribute the most to the current sample are selected from the typical class of landslides as auxiliary samples through semantic similarity metric learning. At the subnetwork layer, the semantic similarity of the target sample and the auxiliary sample is learned through the graph attention network, and then the adjacency matrix is ​​constructed. The features of the auxiliary samples determined by annotation are compared with the features of the target sample, so as to achieve a more comprehensive and reliable extraction of the semantic features of the target sample. At the global layer, the semantic features of the target sample are further abstracted and its category is predicted.

[0077] Through this method, the model only needs a small number of new area samples to make the model performance surpass the model performance of merging multiple areas for training, extract more comprehensive and essential landslide semantic features, have better classification performance in new areas, and achieve a significant improvement in the model generalization ability.

[0078] The main research target of the embodiment of the present invention is two potential landslide hazard data sets with significant differences in topography, landslide causes, sample quantity, etc. The landslide causes of the small sample data set 2 are mostly caused by the large deformation of the surface loose soil under the erosion of rainwater, and most of them are small and broken, with various shapes and difficult to identify landslide hazards, while the landslide causes of data set 1 are mostly caused by the formation of weak surfaces at the bottom of the broken rock, which gradually slide under the influence of gravity. The landslide morphology is relatively fixed and the distribution is relatively uniform. Compared with data set 2, the characteristics are more significant, easier to identify, and there are more sample data.

[0079] In order to improve the generalization performance of the model on dataset 2, which is difficult to identify and has a small sample size, the sample features of dataset 1 are used to assist the recognition model of dataset 2. By combining a large number of dataset 1 samples and a small sample dataset of dataset 2 for pre-training, the pre-trained model can achieve higher classification performance in the dataset 1 test set, extract reliable dataset 1 target and background features, and the model can also learn certain dataset 2 features.

[0080] The role of the pre-trained model in landslide identification in dataset 2 can be divided into two aspects. First, the parameters of the pre-trained model of the merged dataset are used as the initialization parameters of the dataset 2 model, so that the dataset 2 model has certain prior knowledge of landslide hazard characteristics. Secondly, the pre-trained model is used to generate auxiliary set feature nodes at the node layer. At the subnetwork layer, the similarity of the target and background features of dataset 2 and dataset 1 is compared through a multi-level graph attention network. The model can learn more comprehensive target semantic information, enhance similar high-level semantic features and suppress background information. Therefore, with the help of reliable features of dataset 1, the designed multi-level graph attention network model is used to learn the correlation between the features of the two regions, and only a small number of samples from dataset 2 are needed to achieve a high classification performance of potential landslide hazards in the region.

[0081] In the above implementation, preferably, a combined data set of a large sample data set 1 and a small sample data set 2 is used to pre-train the dual-branch encoder-decoder structure model until convergence to form a pre-training model, and the specific process includes:

[0082] Select some samples from data set 2 as the small sample data set to be trained;

[0083] Merge the large sample data set 1 and the small sample data set 2 to obtain a merged data set;

[0084] The merged dataset is used to pre-train the graph attention network model, and the binary cross entropy loss function is used to train the model until it converges as the pre-training model;

[0085] Among them, the two-branch encoders of the pre-trained model do not share weights. During the pre-training process, the two encoders extract the InSAR node features and DEM node features of the merged dataset respectively.

[0086] like Figure 2 As shown in the figure, specifically, encoders Encoder1 and Encoder2 are composed of dilated convolution ResNet101, ASPP and maximum pooling layers. The decoder is composed of BN layer, ReLU activation function, Dropout layer and fully connected layer classification head. Encoders Encoder1 and Encoder2 extract InSAR data and DEM data features respectively. After the extracted features are spliced ​​in the channel dimension, they are input into the decoder to obtain the final probability of each category. The pre-trained model is trained on the combined dataset of dataset 1 and small sample dataset 2, and the auxiliary set samples are input into the pre-trained model encoder to obtain the node features of the auxiliary set of the node layer, and then the auxiliary set node features are clustered.

[0087] In the above implementation, preferably, the specific process of the pre-trained model generating auxiliary set node features according to data set 1 and data set 2 includes:

[0088] The K-means clustering algorithm is used to cluster the InSAR node features and DEM node features of dataset 1 and dataset 2 extracted by the encoder into four types.

[0089] Specifically, since the sub-network layer features are selected from the pre-trained node layer auxiliary set nodes, in order to avoid the random selection method that may cause the selected nodes to have similar semantic features, resulting in less semantic information and limited auxiliary role in the graph attention network, the present invention uses the K-means clustering algorithm to cluster the node layer features, and selects a certain number of nodes from each type of nodes obtained by clustering to construct the sub-network layer, ensuring that the selected landslide and background node features are diverse and can provide sufficient semantic information for the input samples.

[0090] After testing, for the nodes of dataset 2 and dataset 1, the K-means algorithm parameter K=4, that is, when divided into four categories, the clustering effect of the features is better.

[0091] For dataset 1, the clustering of InSAR features is strongly correlated with landslides and backgrounds, that is, of the four types, two types are all landslide features, and two types are all background features. The landslide and background features are clearly distinguished, while the four types obtained by clustering DEM features each include landslide features and background features, and lead to the mixing of landslide and background features in each clustering type of InSAR and DEM features. This proves that DEM data does not contribute to the extraction of landslide features in dataset 1. Therefore, the auxiliary set of dataset 1 only uses InSAR features as node layer features and uses the InSAR feature clustering method.

[0092] For dataset 2, t samples are selected from the existing m sample datasets M as the small sample dataset T to be trained, p samples are selected as the validation set, n samples are selected as the test set, and mpn images that do not contain the validation set and the test set are defined as cluster sets, where m>>t and p>n>t. After merging dataset 1 with dataset 2T, the model is pre-trained, and then the cluster set InSAR and DEM features obtained by the pre-trained model are clustered by the K-means clustering algorithm. Although each cluster type contains landslide and background features, the number of landslides and backgrounds is quite different, that is, the model has learned certain features of dataset 2, and the pre-trained dataset 2 auxiliary nodes can provide auxiliary semantic information for the small sample model of dataset 2.

[0093] In addition, experiments have verified that the fusion of DEM data has no effect on the classification of landslide hazards in dataset 1, and even leads to a decrease in classification performance. However, for the landslide hazards in dataset 2, adding DEM data can significantly improve the classification performance. This is because the InSAR data features of the landslide hazards in dataset 1 are obvious, and only using InSAR data can achieve a high classification accuracy. For the landslide hazards in dataset 2, due to the changes in landslide features caused by its topography, it is difficult to effectively extract the semantic features of the landslide only by relying on InSAR data, and it is necessary to supplement it with the terrain features provided by DEM data. Therefore, the auxiliary set of dataset 2 contains InSAR feature nodes and DEM feature nodes, and uses the method of splicing and clustering InSAR and DEM features. According to the subnetwork layer node selection strategy, the input sample features and the auxiliary set nodes after clustering are calculated by similarity measurement, and the difficult samples that have the greatest contribution to the input sample features of the model learning are selected.

[0094] In the above implementation, preferably, the subnetwork layer constructs the InSAR image attention subnetwork and the DEM image attention subnetwork through the auxiliary set difficult node features and sample node features, and the specific process includes:

[0095] Calculate the similarity between the auxiliary set node features and the sample node features;

[0096] Selecting difficult samples from the auxiliary set node features, specifically, selecting the same number of the most dissimilar samples and the most similar samples to the sample node features from the landslide samples and background samples in the auxiliary set;

[0097] The InSAR image attention subnet and the DEM image attention subnet of the subnet layer both include three multi-layer perceptrons (MLPs), which are used to update input node features, source node attention, and target node attention, respectively;

[0098] Each node of the input subnet updates its features through the first multi-layer perceptron, and the attention weights between the source node and the target node are calculated through the second and third multi-layer perceptrons;

[0099] The attention weights are used to weight the features of each node updated by the first multi-layer perceptron to obtain the comprehensive features of each node itself and its neighboring nodes.

[0100] Specifically, the subnetwork layer nodes will select difficult samples from the auxiliary set samples, which are defined as the same type of samples that are the least similar to the input features and the most similar different type of samples. Since the auxiliary set samples have label information and are considered to be more reliable pre-trained features, learning the correlation between input samples and difficult samples through the graph attention network can provide the most semantic information for the input samples.

[0101] For landslide input samples, the most dissimilar auxiliary set landslide samples and the most similar auxiliary set background samples should be selected. For background input samples, the most dissimilar auxiliary set background samples and the most similar auxiliary set landslide samples should be selected. However, since the input sample category is unknown during training, it is difficult to determine which category of auxiliary set samples to calculate the similarity with in order to select difficult samples. Therefore, for each input sample, the same number of the most dissimilar samples and the most similar samples are selected from the auxiliary set landslide and background class nodes, so that no matter which category the input sample belongs to, it can be guaranteed that the selected nodes contain difficult samples. In addition, the subnetwork layer node selection strategy is used during training, validation, and testing to maintain the consistency of the model forward propagation process.

[0102] like Figure 3 As shown in the figure, for the node features of the auxiliary set of dataset 2, since the K-means algorithm is applied to the node features of the auxiliary set of dataset 2 spliced ​​using InSAR and DEM features, the InSAR and DEM features of the input samples are also spliced, and the similarity between the spliced ​​input features and the features spliced ​​according to the target in the auxiliary set of dataset 2 is calculated. According to the above-mentioned difficult sample selection method, samples are selected from each clustering type, where the number of node features selected for each type is a hyperparameter.

[0103] like Figure 4As shown in the figure, for the node features of the auxiliary set of dataset 1, since the K-means algorithm is only applied to the InSAR features of the nodes of the auxiliary set of dataset 1, only the similarity between the InSAR features of the input samples and the InSAR node features of the auxiliary set of dataset 1 is calculated. According to the above-mentioned method of selecting difficult samples, samples are selected from each clustering type and input into the subnetwork layer.

[0104] like Figure 5 As shown in the figure, the graph attention network in each subnetwork contains three multilayer perceptrons (MLPs), which are used to update the input node features, source node attention, and target node attention respectively.

[0105] The connection relationship between each node is a complete graph, that is, there is an edge connecting two different vertices, and an attention weight will be calculated between the two vertices. Each node of the input subnet first updates the features through the first multilayer perceptron Θ1:

[0106] x ′ i =θ1(x i )

[0107] Then the attention weights between the source node and the target node are calculated by the second and third multilayer perceptrons Θ2 and Θ3:

[0108]

[0109] Where N(i) represents the set of neighbor nodes of node i, and φ represents the LeakyReLU function:

[0110] LeakyReLU(x)=max(0,x)+0.01min(0,x)

[0111] Finally, the calculated attention weights are used to weight the features of each node updated by Θ1, and the features of each node and its neighboring nodes are obtained:

[0112]

[0113] In the above implementation, preferably, after the global layer splices the output features of the InSAR image attention subnet and the DEM image attention subnet, the sample classification is performed through the decoder. The specific process includes:

[0114] After the output features of the InSAR image attention subnet and the DEM image attention subnet are concatenated at the global layer, they are input into a decoder consisting of batch normalization, ReLU activation function and multi-layer perceptron classification head;

[0115] The decoder outputs the category probability of the sample and determines the sample classification based on the category probability.

[0116] Specifically, the InSAR features and DEM features of the training samples are updated through two subnets, concatenated by channels in the global layer, and input into a decoder consisting of batch normalization, ReLU activation function, and multi-layer perceptron classification head to obtain the class probability of the samples. The loss function used by the pre-trained model and the multi-level graph attention network model is the binary cross entropy loss function:

[0117] L CE =-ylog p+(1-y)log(1-p)

[0118] Among them, y represents the true label of the sample, where the background label is 0 and the landslide hazard label is 1, and p represents the probability that the sample is classified as a landslide category.

[0119] According to the potential landslide identification method based on graph neural network disclosed in the above embodiment, the training process of the graph attention network model is as follows:

[0120] Step 1:

[0121] The model is pre-trained using the combined dataset of dataset 1 training set and dataset 2 clustering set until convergence.

[0122] Step 2:

[0123] All samples of the training set of dataset 1 and the clustering set of dataset 2 are input into the pre-trained Encoder1 and Encoder2 encoders respectively to obtain the auxiliary set InSAR node feature and DEM node feature models.

[0124] Step 3:

[0125] A landslide sample from the small sample data set of Dataset 2 is selected, and its InSAR and DEM data are respectively input into the Encoder1 and Encoder2 encoders in the designed multi-level graph attention network model to obtain the input InSAR and input DEM features.

[0126] Step 4:

[0127] Using the K-mean clustering algorithm on the node features of the auxiliary set of dataset 2 and the auxiliary set of dataset 1, the features of dataset 2 and dataset 1 can be divided into four categories respectively.

[0128] Step 5:

[0129] The similarity between the input InSAR and input DEM features and the features of the four types of auxiliary set nodes is calculated, and the difficult InSAR and DEM auxiliary nodes are selected from the auxiliary set nodes of dataset 2 and dataset 1 and input into the subnetwork layer.

[0130] Step 6:

[0131] At the subnetwork layer, the auxiliary set InSAR and DEM node features selected from the node layer and the InSAR and DEM features of the input samples will construct two graph attention subnetworks, InSAR and DEM, to learn the similarity measures between the landslide hazard features and background features in the input targets and auxiliary targets, respectively, and update the input sample features.

[0132] Step 7:

[0133] After concatenating the output features of each subnetwork at the global layer, the features are input into the decoder to obtain the classification result, and the loss function is calculated and back-propagated to update the parameters of the multi-level graph attention network model.

[0134] Step 8:

[0135] Verify the performance of the multi-level graph attention network model on the validation set of Dataset 2.

[0136] Step 9:

[0137] Repeat steps 3 to 8 until the model converges, and save data such as model parameters and convergence curves.

[0138] Among them, the dual-branch pre-training model is trained by merging dataset 1 and small sample dataset 2 to generate node features of auxiliary sets at the node layer of dataset 1 and dataset 2. K-means algorithm is used for clustering, and difficult nodes are selected from various auxiliary set features obtained by clustering according to their similarity with the input sample features, so that the selected auxiliary nodes have the richest semantic information for the samples to be predicted and are input into the subnetwork layer.

[0139] Among them, the two graph attention subnetworks, InSAR and DEM, learn the similarity of features between the target to be predicted and the targets in the auxiliary set from the two modalities of InSAR and DEM of the landslide samples, suppress unreliable information and enhance reliable information. Finally, the learned multimodal features are integrated to enable the model to learn the comprehensive and essential semantic features of landslides in the new area.

[0140] The present invention further proposes a potential landslide identification system based on a graph neural network, which uses the potential landslide identification method based on a graph neural network disclosed in any one of the above embodiments, including:

[0141] A model building module is used to build a graph attention network model with multi-level feature fusion including node layer, subnetwork layer and global layer, wherein the graph attention network model adopts a dual-branch encoder-decoder structure;

[0142] A model pre-training module is used to use a combined data set of a large sample data set 1 and a small sample data set 2 to pre-train a dual-branch encoder-decoder structure model until convergence to form a pre-trained model;

[0143] The model training module trains and verifies the graph attention network model based on the auxiliary set node features obtained by inputting dataset 1 and dataset 2 into the pre-trained model until the model converges and the training is completed;

[0144] The landslide prediction module is used to input the images that need to be identified as potential landslides into the trained graph attention network model to obtain the prediction results of potential landslides;

[0145] Among them, the pre-trained model generates auxiliary set node features according to dataset 1 and dataset 2, and the encoder in the node layer of the graph attention network model generates sample node features according to dataset 1 and dataset 2;

[0146] The subnetwork layer constructs the InSAR image attention subnetwork and the DEM image attention subnetwork through the auxiliary set node features and sample node features;

[0147] The global layer concatenates the output features of the InSAR image attention subnet and the DEM image attention subnet, and then performs sample classification through the decoder.

[0148] In the above implementation, preferably, the model pre-training module is specifically used for:

[0149] Select some samples from data set 2 as the small sample data set to be trained;

[0150] Merge the large sample data set 1 and the small sample data set 2 to obtain a merged data set;

[0151] The merged dataset is used to pre-train the graph attention network model, and the binary cross entropy loss function is used to train the model until it converges as the pre-training model;

[0152] Among them, the two-branch encoders of the pre-trained model do not share weights. During the pre-training process, the two encoders extract the InSAR node features and DEM node features of the merged dataset respectively.

[0153] In the above implementation, preferably, the specific process of the pre-trained model generating auxiliary set node features according to data set 1 and data set 2 includes:

[0154] The K-means clustering algorithm is used to cluster the InSAR node features and DEM node features of dataset 1 and dataset 2 extracted by the encoder into four types respectively.

[0155] In the above implementation, preferably, the subnetwork layer constructs the InSAR image attention subnetwork and the DEM image attention subnetwork through the auxiliary set difficult node features and sample node features, and the specific process includes:

[0156] Calculate the similarity between the auxiliary set node features and the sample node features;

[0157] Selecting difficult samples from the auxiliary set node features, specifically, selecting the same number of the most dissimilar samples and the most similar samples to the sample node features from the landslide samples and background samples in the auxiliary set;

[0158] The InSAR image attention subnet and the DEM image attention subnet of the subnet layer both include three multi-layer perceptrons (MLPs), which are used to update input node features, source node attention, and target node attention, respectively;

[0159] Each node of the input subnet updates its features through the first multi-layer perceptron, and the attention weights between the source node and the target node are calculated through the second and third multi-layer perceptrons;

[0160] The attention weights are used to weight the features of each node updated by the first multi-layer perceptron to obtain the comprehensive features of each node itself and its neighboring nodes.

[0161] In the above implementation, preferably, after the global layer splices the output features of the InSAR image attention subnet and the DEM image attention subnet, the sample classification is performed through the decoder. The specific process includes:

[0162] After the output features of the InSAR image attention subnet and the DEM image attention subnet are concatenated at the global layer, they are input into a decoder consisting of batch normalization, ReLU activation function and multi-layer perceptron classification head;

[0163] The decoder outputs the category probability of the sample and determines the sample classification based on the category probability.

[0164] According to the potential landslide identification method and system based on graph neural network disclosed in the above-mentioned embodiment, during the implementation process, the identification process and effect of the above-mentioned method and system are explained with the following examples.

[0165] The data set 1 used in the present invention has a total of 962 pictures, of which 481 are positive samples of landslide hazards and 481 are background negative samples. According to the ratio of 6:2:2, it is divided into a training set, a validation set and a test set, which contain 576, 190 and 196 pictures respectively. The number of positive samples and negative samples in the training set, validation set and test set is equal, and the training set is used as an auxiliary set of data set 1 to obtain the node features of the auxiliary set.

[0166] The data set 2 used in the present invention has a total of 446 pictures, of which 223 are positive samples of landslide hazards and 223 are background negative samples. According to the ratio of 6:2:2, it is divided into a cluster set, a validation set and a test set, which contain 268, 92 and 86 pictures respectively. 60 pictures are randomly selected from the cluster set as a small sample training set, and then 20 pictures are selected as samples of the auxiliary set of data set 2 according to the clustering results of the cluster set. The number of positive and negative samples in the cluster set, small sample training set, auxiliary set, validation set and test set is equal.

[0167] In order to enhance the model's ability to distinguish positive and negative samples, this patent selects slope units with similar texture, color, and terrain change characteristics as positive samples as negative samples. The negative sample selection strategy is as follows.

[0168]

[0169]

[0170] The trained model is evaluated using three indicators: average accuracy, landslide accuracy, and background accuracy. The evaluation indicator formula is as follows:

[0171]

[0172] TP, TN, FP and FN are the number of accurately predicted landslide samples, the number of accurately predicted non-landslide samples, the number of landslide samples mistaken for non-landslides and the number of non-landslide samples mistaken for landslides, respectively.

[0173] Through the above evaluation index formula, the accuracy of the pre-trained model and the graph attention network model of the present invention are calculated, as shown in the following table:

[0174]

[0175] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A potential landslide identification method based on graph neural network, characterized in that: include: Constructing a multi-level feature fusion graph attention network model including a node layer, a subnetwork layer and a global layer, wherein the graph attention network model adopts a dual-branch encoder-decoder structure; Using a combined data set of a large sample data set 1 and a small sample data set 2, a dual-branch encoder-decoder structure model is pre-trained until convergence to form a pre-trained model; Based on the auxiliary set node features obtained by inputting the data set 1 and the data set 2 into the pre-trained model, the graph attention network model is trained and verified until the model converges and the training is completed; Input the pictures to be identified in the small sample data set 2 into the trained graph attention network model to obtain the prediction results of potential landslides; Wherein, the pre-trained model generates auxiliary set node features according to the data set 1 and the data set 2, and the encoder in the node layer of the graph attention network model generates sample node features according to the data set 1 and the data set 2; The subnetwork layer constructs an InSAR image attention subnetwork and a DEM image attention subnetwork through the auxiliary set node features and the sample node features; The global layer concatenates the output features of the InSAR image attention subnet and the DEM image attention subnet, and then performs sample classification through the decoder.

2. The potential landslide identification method based on graph neural network according to claim 1 is characterized in that: The combined data set of the large sample data set 1 and the small sample data set 2 is used to pre-train the dual-branch encoder-decoder structure model until convergence to form a pre-training model. The specific process includes: Select some samples from data set 2 as the small sample data set to be trained; Merge the large sample data set 1 and the small sample data set 2 to obtain a merged data set; The dual-branch encoder-decoder structure model is pre-trained using the combined data set, and the binary cross entropy loss function is used to train the model until the model converges as a pre-training model; The two branch encoders of the pre-training model do not share weights. During the pre-training process, the two encoders respectively extract the InSAR node features and the DEM node features of the merged data set.

3. The potential landslide identification method based on graph neural network according to claim 1 is characterized in that: The specific process of the pre-trained model generating auxiliary set node features according to the data set 1 and the data set 2 includes: The InSAR node features and DEM node features of the dataset 1 and the dataset 2 extracted by the encoder are clustered into four types using a K-means clustering algorithm.

4. The potential landslide identification method based on graph neural network according to claim 1 is characterized in that: The subnetwork layer constructs the InSAR image attention subnetwork and the DEM image attention subnetwork through the auxiliary set node features and the sample node features, and the specific process includes: Calculating similarity between the auxiliary set node features and the sample node features; Selecting difficult samples from the auxiliary set node features, specifically, selecting the same number of the most dissimilar samples and the most similar samples to the sample node features from the landslide samples and background samples in the auxiliary set; The InSAR image attention subnet and the DEM image attention subnet of the subnet layer both include three multi-layer perceptrons MLP, which are used to update input node features, source node attention, and target node attention, respectively; Each node of the input subnet updates its features through the first multi-layer perceptron, and the attention weights between the source node and the target node are calculated through the second and third multi-layer perceptrons; The attention weights are used to weight the features of each node updated by the first multi-layer perceptron to obtain the comprehensive features of each node itself and its neighboring nodes.

5. The potential landslide identification method based on graph neural network according to claim 1 is characterized in that: After the global layer splices the output features of the InSAR image attention subnet and the DEM image attention subnet, the decoder is used to perform sample classification. The specific process includes: After splicing the output features of the InSAR image attention subnet and the DEM image attention subnet at the global layer, the features are input into a decoder composed of batch normalization, ReLU activation function and multi-layer perceptron classification head; The decoder outputs the category probability of the sample and determines the sample classification according to the category probability.

6. A potential landslide identification system based on graph neural network, characterized in that: The method for identifying potential landslides based on a graph neural network as claimed in any one of claims 1 to 5 comprises: A model building module, used to build a graph attention network model including a node layer, a subnetwork layer and a global layer with multi-level feature fusion, wherein the graph attention network model adopts a dual-branch encoder-decoder structure; A model pre-training module is used to use a combined data set of a large sample data set 1 and a small sample data set 2 to pre-train a dual-branch encoder-decoder structure model until convergence to form a pre-trained model; A model training module, which trains and verifies the graph attention network model based on the auxiliary set node features obtained by inputting the data set 1 and the data set 2 into the pre-trained model until the model convergence training is completed; A landslide prediction module, used for inputting a picture that needs to be identified as a potential landslide into the trained graph attention network model to obtain a prediction result of a potential landslide; Wherein, the pre-trained model generates auxiliary set node features according to the data set 1 and the data set 2, and the encoder in the node layer of the graph attention network model generates sample node features according to the data set 1 and the data set 2; The subnetwork layer constructs an InSAR image attention subnetwork and a DEM image attention subnetwork through the auxiliary set node features and the sample node features; The global layer concatenates the output features of the InSAR image attention subnet and the DEM image attention subnet, and then performs sample classification through the decoder.

7. The potential landslide identification system based on graph neural network according to claim 6 is characterized in that: The model pre-training module is specifically used for: Select some samples from data set 2 as the small sample data set to be trained; Merge the large sample data set 1 and the small sample data set 2 to obtain a merged data set; The dual-branch encoder-decoder structure model is pre-trained using the combined data set, and the binary cross entropy loss function is used to train the model until the model converges as a pre-training model; The two branch encoders of the pre-training model do not share weights. During the pre-training process, the two encoders respectively extract the InSAR node features and the DEM node features of the merged data set.

8. The potential landslide identification system based on graph neural network according to claim 6 is characterized in that: The specific process of the pre-trained model generating auxiliary set node features according to the data set 1 and the data set 2 includes: The InSAR node features and DEM node features of the dataset 1 and the dataset 2 extracted by the encoder are clustered into four types using a K-means clustering algorithm.

9. The potential landslide identification system based on graph neural network according to claim 6 is characterized in that: The subnetwork layer constructs the InSAR image attention subnetwork and the DEM image attention subnetwork through the auxiliary set node features and the sample node features, and the specific process includes: Calculating similarity between the auxiliary set node features and the sample node features; Selecting difficult samples from the auxiliary set node features, specifically, selecting the same number of the most dissimilar samples and the most similar samples to the sample node features from the landslide samples and background samples in the auxiliary set; The InSAR image attention subnet and the DEM image attention subnet of the subnet layer both include three multi-layer perceptrons MLP, which are used to update input node features, source node attention, and target node attention, respectively; Each node of the input subnet updates its features through the first multi-layer perceptron, and the attention weights between the source node and the target node are calculated through the second and third multi-layer perceptrons; The attention weights are used to weight the features of each node updated by the first multi-layer perceptron to obtain the comprehensive features of each node itself and its neighboring nodes.

10. The potential landslide identification system based on graph neural network according to claim 6 is characterized in that: After the global layer splices the output features of the InSAR image attention subnet and the DEM image attention subnet, the decoder is used to perform sample classification. The specific process includes: After splicing the output features of the InSAR image attention subnet and the DEM image attention subnet at the global layer, the features are input into a decoder composed of batch normalization, ReLU activation function and multi-layer perceptron classification head; The decoder outputs the category probability of the sample and determines the sample classification according to the category probability.

Images