A method for improving landslide hazard assessment in data-limited areas using data migration
By constructing a deep learning convolutional neural network model and using data migration technology to screen out similar samples from areas with sufficient data, the problem of landslide risk assessment in areas with limited data is solved, and the accuracy and reliability of landslide risk assessment is improved.
Patent Information
- Application Number
- CN202210403151.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-15
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2042-04-15
AI Technical Summary
In areas with limited data, landslide risk assessment is difficult to model and evaluate, and the lack of sufficient data makes the model unreliable.
Deep learning convolutional neural network model is constructed by extracting historical landslide data from areas with sufficient data, and using the distance function evaluation algorithm of similar coefficients to screen samples similar to the finite data sets, and data migration is carried out to improve the landslide risk assessment model.
It effectively solves the problem that reliable landslide risk assessment model cannot be established in areas with limited data, improves classification accuracy and area under the ROC curve, and provides more reliable landslide risk assessment results.
Smart Images

Figure CN114742419B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method for improving landslide hazard assessment in a limited area through data migration, and belongs to the technical field of geological disaster prevention and control. Background Art
[0002] Landslide is a geological disaster that seriously endangers the production and life of local residents. my country is a landslide-prone area, which causes huge economic losses and casualties every year. It is very important to conduct hazard assessment to reasonably avoid and reduce the risk of landslide disasters. However, due to the widespread distribution of landslide disasters but the relatively local impact range, and the lack of monitoring station data for disasters such as earthquakes and typhoons, it is extremely challenging to carry out landslide hazard assessment modeling in data-limited areas such as the Qinghai-Tibet Plateau or the southwestern mountainous areas. Summary of the invention
[0003] In order to overcome the shortcomings of the prior art, the present invention provides a method for improving landslide hazard assessment in data-limited areas by data migration, so as to improve the assessment of landslide disaster hazard in data-limited areas and solve the problem that landslide hazard cannot be modeled and assessed in areas with scarce or very limited data.
[0004] The technical solution of the present invention:
[0005] The present invention provides a method for data migration to improve landslide hazard assessment in a limited area of data, which comprises the following steps:
[0006] Step 1: Define area A as having sufficient and complete historical landslide hazard data and area B as having only very limited landslide hazard data;
[0007] Step 2: Extract factors affecting landslides from historical landslide disaster data of area A to construct a landslide hazard model based on deep learning convolutional neural network. The landslide hazard model includes two parts: feature extraction module and classification module. The feature extraction module is a convolutional neural network structure, including convolution layer, pooling layer and full connection layer. The convolution layer is used for feature extraction, by extracting specific and effective image features that can reflect the association between original data and target tasks; the pooling layer is used to achieve downsampling to achieve the hierarchical nature of convolutional neural network feature extraction, so that the field of view of feature extraction features gradually expands with the depth of the network; the full connection layer is used to achieve full connection between convolutional neural network units, and finally proposes high-dimensional influencing factor features suitable for probability estimation of landslide hazard classification targets;
[0008] SaidThe classification module selects the Softmax classifier to make probability estimates of historical landslide classifications through high-dimensional influencing factor characteristic variables, and then determines the final parameter combination of the deep learning convolutional neural network model based on the indicator evaluation method as the deep learning landslide hazard assessment model for the complete historical area A;
[0009] Step 3: Establish a distance function evaluation algorithm based on similarity coefficient, and its calculation formula is as follows:
[0010]
[0011] Where X is the value of the sample in area A on the characteristic variable of the factors affecting landslides, and Y is the value of the sample in area B on the characteristic variable of the factors affecting landslides. is the similarity distance between samples in area A and area B in terms of the factors affecting landslides, X A1 is the characteristic vector of the first landslide disaster point in area A under different factors affecting landslide; X An Y is the characteristic vector of the Nth landslide disaster point in area A under different factors affecting landslide; B1 Y is the characteristic vector of the first landslide disaster point in area B under different factors affecting landslide; Bn is the characteristic vector of the Nth landslide hazard point in region B under different factors affecting landslide;
[0012] For each historical landslide data sample in region A, the distance between the sample and the limited data set of region B is evaluated by the distance function evaluation algorithm based on the similarity coefficient, and a certain number of samples in region A that are closer to the sample characteristics of region B are sorted and selected, and samples that are greatly different from the limited data samples of region B are removed, while samples that are closer to the limited samples of region B are retained as potential auxiliary samples C;
[0013] Step 4: randomly divide the limited data of region B into a training set of region B and a test set of region B, use the limited training set samples of region B to fine-tune the parameters of the deep learning landslide hazard assessment model of the complete historical region A, and apply it to the test set of region B to calculate its index evaluation method;
[0014] Step 5: Introduce the potential auxiliary samples C screened in step 3 into the process of step 4 one by one, to improve and enrich the sample size for parameter optimization adjustment, and evaluate the contribution and influence of the potential auxiliary samples C on the evaluation performance of the test set of region B each time. The loss function is calculated one by one by the cross-validation method using the confusion matrix classification accuracy. The cross-validation method first introduces all potential auxiliary samples C into the training set of region B to calculate the classification accuracy E of all auxiliary samples introduced. m, and then remove the potential auxiliary samples C from the training set of region B one by one, and calculate the classification accuracy E of the samples removed one by one i , compared with E i and E m , if the classification accuracy is reduced, the removed sample is a sample with beneficial contribution and needs to be retained, otherwise it is a useless sample and is removed from the potential auxiliary sample C, and finally the beneficial historical sample D of area B is obtained;
[0015] Step six, the beneficial historical samples D of the area B perform parameter optimization and adjustment on the deep learning landslide hazard assessment model of the complete historical area A established in step two, and obtain an improved deep learning data migration landslide hazard assessment model for area B.
[0016] Furthermore, in step 2, the indicator evaluation method includes a classification accuracy evaluation method and an area under the ROC curve evaluation method.
[0017] Furthermore, in step 2, the factors affecting landslides include topography, geological lithology, vegetation coverage, soil type, precipitation, and distance from roads, rivers, and fault zones.
[0018] Furthermore, in step 2, the classification accuracy is the proportion of training landslide and non-landslide samples in the complete historical area A correctly classified by the model to all training samples under different model parameter combinations.
[0019] Furthermore, in step 5, the loss function is calculated by using the confusion matrix classification accuracy, including the following steps: according to whether the actual landslide disaster occurs and whether the model predicts a landslide disaster, a landslide disaster is set to 1 and a non-landslide disaster is set to 0, and a 2×2 confusion matrix can be obtained. The confusion matrix can be divided into four cases according to whether the actual and predicted landslide disasters occur, namely: f a is the actual non-landslide disaster sample, and the predicted one is also a non-landslide disaster sample; f b is the actual non-landslide disaster sample, and is predicted to be a landslide disaster sample; f c is the actual landslide disaster sample, and the predicted one is the non-landslide disaster sample; f d is the actual landslide disaster sample, and is predicted to be a landslide disaster sample. Based on this, the classification accuracy O can be calculated. a The formula is as follows:
[0020] .
[0021] Beneficial Effects
[0022] The present invention discloses a method for improving landslide hazard assessment in data-limited areas through data migration. The method uses a data migration assessment algorithm to select historical samples from areas with complete and rich data to be beneficial to landslide hazard assessment modeling in data-limited areas. The method can effectively solve the problem that reliable models cannot be established in areas with very limited data due to insufficient samples. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1 A schematic flow chart of a method for improving landslide hazard assessment in a limited area by data migration according to the present invention;
[0024] Figure 2 It is a schematic diagram of the process structure of the feature extraction module;
[0025] Figure 3 To obtain a useful historical sample D process diagram;
[0026] Figure 4 Representative areas for complete historical landslide hazard samples A: southeastern mountainous and hilly Fujian Province;
[0027] Figure 5 Representative area B for limited landslide hazard samples: Hengduan Mountain area in the southwest;
[0028] Figure 6 ROC curve for landslide hazard modeling using the method of the present invention and directly using limited data;
[0029] Figure 7 The landslide hazard distribution map is obtained directly based on limited data modeling;
[0030] Figure 8 The landslide hazard distribution map is obtained by modeling in data-limited areas using the method of the present invention. DETAILED DESCRIPTION
[0031] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.
[0032] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0033] In some implementations of this embodiment, the present invention proposes a method for improving landslide hazard assessment in a limited area by data migration, which comprises the following steps:
[0034] Step 1: Define area A as having sufficient and complete historical landslide hazard data and area B as having only very limited landslide hazard data;
[0035] Step 2: Based on the complete historical landslide disaster data of area A, multiple factors affecting landslides, such as landform, geological lithology, vegetation cover, soil type, precipitation, distance from roads, rivers and fault zones, are used as factors affecting landslides to construct a landslide hazard model based on deep learning convolutional neural network, which is the source model. The schematic diagram of the source model is shown in Figure 1 , mainly including feature extraction module and classification module. The feature extraction module is a convolutional neural network structure, including convolution layer, pooling layer and fully connected layer. The convolution layer is used for feature extraction, by extracting specific and effective image features that can reflect the association between the original data and the target task; the pooling layer is used to achieve downsampling to realize the hierarchical nature of feature extraction of the convolutional neural network, so that the field of view of feature extraction gradually expands with the depth of the network. At the same time, the pooling layer can also effectively reduce the total number of parameters and model complexity, and improve the computational efficiency; the fully connected layer is used to achieve full connection between convolutional neural network units, and finally proposes high-dimensional influencing factor features suitable for probability estimation of landslide hazard classification targets. For each influencing factor, its high-dimensional influencing factor features can be extracted through deep learning convolutional neural network and used as a predictive variable for landslide hazard assessment classification. Figure 2 This is a schematic diagram of the feature extraction process using terrain variables as an example. Other influencing factor features can also be extracted according to this process to output high-dimensional influencing factor feature variables as prediction feature variables.
[0036] On this basis, the Softmax classifier is selected to make probability estimates for historical landslide classifications through high-dimensional influencing factor feature variables, and then the final parameter combination of the deep learning convolutional neural network model is determined based on the classification accuracy, area under the ROC curve and other indicators as the deep learning landslide hazard assessment model for the complete historical area A; the classification accuracy is the proportion of training landslide and non-landslide samples in the complete historical area A that are correctly classified by the model to all training samples under different model parameter combinations. For example: if there are 360 correctly classified landslide samples and non-landslide samples, and all training samples are 400, the classification accuracy is 90%.
[0037] Step 3: Establish a distance function evaluation algorithm based on similarity coefficient, and its calculation formula is as follows:
[0038]
[0039] Where X is the value of the sample in region A on the characteristic variables of influencing factors such as landform, geological lithology, vegetation cover, and soil type; Y is the value of the sample in region B on the characteristic variables of influencing factors such as landform, geological lithology, vegetation cover, and soil type. is the similarity distance between samples in region A and region B in terms of various influencing factors, X A1 is the characteristic vector of the first landslide disaster point in area A under different influencing factors; X An is the characteristic vector of the Nth landslide disaster point in area A under different influencing factors; Y B1 is the characteristic vector of the first landslide disaster point in area B under different influencing factors; Y Bn is the characteristic vector of the Nth landslide hazard point in area B under different influencing factors.
[0040] For each historical landslide data sample in region A, its distance from the limited data set of region B is evaluated by a distance function evaluation algorithm based on the similarity coefficient, and a certain number of samples in region A that are closer to the sample characteristics of region B are sorted and selected. Samples that are significantly different from the limited data samples of region B are removed, while samples that are closer to the limited samples of region B are retained as potential auxiliary samples C.
[0041] Step 4: Randomly divide the limited data of region B into a training set of region B and a test set of region B. Using the limited training set samples of region B, fine-tune the parameters of the deep learning landslide hazard assessment model for the complete history region A established in step 2, and apply it to the test set of region B to calculate evaluation indicators such as classification accuracy and the area under the ROC curve.
[0042] Step 5. Introduce the potential auxiliary samples C screened in step 3 into process step 4 one by one to improve and enrich the sample size for parameter optimization adjustment, and evaluate the contribution and impact of the sample on the evaluation performance of the test set in region B each time. The loss function is calculated using the confusion matrix classification accuracy.
[0043] According to whether the actual disaster is a landslide and whether the model predicts a landslide, set the landslide disaster to 1 and the non-landslide disaster to 0, a 2x2 confusion matrix can be obtained. The confusion matrix can be divided into four cases according to whether the actual and predicted landslide disasters occur, namely: a is the actual non-landslide disaster sample, and the predicted one is also a non-landslide disaster sample; f b is the actual non-landslide disaster sample, and is predicted to be a landslide disaster sample; f c is the actual landslide disaster sample, and the predicted one is the non-landslide disaster sample; f d is the actual landslide disaster sample, and is predicted to be a landslide disaster sample. Based on this, the confusion matrix classification accuracy O can be calculated. a The formula is as follows: .
[0044] Table 1: Confusion matrix table and formula
[0045]
[0046] The classification accuracy is calculated one by one through the cross-validation method, that is, first all auxiliary samples C are introduced into the calculation to obtain the confusion matrix classification accuracy E m , and then remove the auxiliary samples C case by case, and calculate the confusion matrix classification accuracy E each time i , compared with E i and E m , if the classification accuracy is reduced, it is a sample with beneficial contribution and is retained, otherwise it is a useless sample and is removed from C, and finally a beneficial historical sample D is obtained, such as Figure 3 shown.
[0047] Step six, randomly divide the limited data of area B into a training set and use comprehensive beneficial historical samples D as the parameters for optimizing and adjusting the model established in step two, so as to obtain an improved deep learning data migration landslide hazard assessment model for area B with limited data, that is, to obtain the target model. Example
[0048] Fujian Province, a typical southeastern mountainous and hilly area with sufficient and complete historical landslide hazard data, was selected as Region A. Figure 4 ), the southwestern Hengduan Mountain area, where historical landslide hazard data are very limited, is used as region B ( Figure 5 ). Region A covers an area of about 124,000 square kilometers in Fujian Province, located in the southeastern mountainous and hilly areas, and is severely affected by landslide disasters. There are 5,290 landslide disaster samples recorded in the historical landslide disaster database. Region B covers an area of about 53,400 square kilometers in the southwestern Hengduan Mountain area, located at the junction of Sichuan and Tibet. The natural environment conditions such as topography, hydrology, and meteorology are very complex. It is a landslide-prone area in my country, and the terrain of the Hengduan Mountain area is very undulating. Figure 5 It can also be seen that its highest altitude is above 7000m, which has created a good geomorphic environment for the occurrence of landslide disasters. However, it is precisely because of its complex landforms and high altitude that it is very difficult to conduct landslide disaster surveys and obtain data in the area. There are only 493 landslide disaster samples recorded in the historical landslide disaster database. If the limited historical landslide disaster samples in the Hengduan Mountain area are directly used to construct a landslide hazard assessment model, the reliability of the modeling results is difficult to guarantee. To this end, a method for improving landslide hazard assessment in data-limited areas based on deep learning data migration is proposed and applied to landslide hazard modeling in the Hengduan Mountain area to solve the problem that limited data in the area cannot establish a reliable model.
[0049] The proposed method of data migration to improve landslide hazard assessment in limited data areas was applied to the landslide hazard modeling in the Hengduan Mountain area (Scheme 1) and compared with the results of the landslide hazard modeling in the Hengduan Mountain area that directly used limited landslide hazard samples (Scheme 2). The results showed that the confusion matrix accuracy of Scheme 1 was about 93.7%, which was about 11.4 higher than that of Scheme 2 (confusion matrix accuracy 82.3%). From the evaluation results of the ROC curve, Scheme 1 (AUROC = 0.962) was also significantly better than Scheme 2 (AUROC = 0.858) ( Figure 6 The landslide hazard map of the Hengduan Mountain area obtained by the final simulation of Scheme 2 (modeling directly based on limited data) and Scheme 1 (modeling using the method of the present invention) is as follows: Figure 7 and Figure 8 From the distribution relationship between the landslide hazard map of the Hengduan Mountain area and the limited historical landslide samples, it can be seen that the landslide hazard map of the Hengduan Mountain area obtained by using the method for improving the landslide hazard assessment in a limited area based on deep learning data migration proposed in the present invention is more reliable and more consistent with the distribution of historical landslide samples, which can provide a more effective scientific basis for formulating strategies to reduce landslide disaster risks.
[0050] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the embodiments of the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A method for landslide hazard assessment in a limited area using data migration to improve data, comprising the following steps: Step 1: Define area A as having sufficient and complete historical landslide hazard data and area B as having only very limited landslide hazard data; Step 2: Extract factors affecting landslides from historical landslide disaster data of area A to construct a landslide hazard model based on deep learning convolutional neural network. The landslide hazard model includes two parts: feature extraction module and classification module. The feature extraction module is a convolutional neural network structure, including convolution layer, pooling layer and full connection layer. The convolution layer is used for feature extraction, by extracting specific and effective image features that can reflect the association between original data and target tasks; the pooling layer is used to achieve downsampling to achieve the hierarchical nature of convolutional neural network feature extraction, so that the field of view of feature extraction features gradually expands with the depth of the network; the full connection layer is used to achieve full connection between convolutional neural network units, and finally proposes high-dimensional influencing factor features suitable for probability estimation of landslide hazard classification targets; Said The classification module selects the Softmax classifier to make probability estimates of historical landslide classifications through high-dimensional influencing factor characteristic variables, and then determines the final parameter combination of the deep learning convolutional neural network model based on the indicator evaluation method as the deep learning landslide hazard assessment model for the complete historical area A; Step 3: Establish a distance function evaluation algorithm based on similarity coefficient, and its calculation formula is as follows: Where X is the value of the sample in area A on the characteristic variable of the factors affecting landslides, and Y is the value of the sample in area B on the characteristic variable of the factors affecting landslides. is the similarity distance between samples in area A and area B in terms of the factors affecting landslides, X A1 is the characteristic vector of the first landslide disaster point in area A under different factors affecting landslide; X An Y is the characteristic vector of the Nth landslide disaster point in area A under different factors affecting landslide; B1 Y is the characteristic vector of the first landslide disaster point in area B under different factors affecting landslide; Bn is the characteristic vector of the Nth landslide hazard point in region B under different factors affecting landslide; For each historical landslide data sample in region A, the distance between the sample and the limited data set of region B is evaluated by the distance function evaluation algorithm based on the similarity coefficient, and a certain number of samples in region A that are closer to the sample characteristics of region B are sorted and selected, and samples that are greatly different from the limited data samples of region B are removed, while samples that are closer to the limited samples of region B are retained as potential auxiliary samples C; Step 4: randomly divide the limited data of region B into a training set of region B and a test set of region B, use the limited training set samples of region B to fine-tune the parameters of the deep learning landslide hazard assessment model of the complete historical region A, and apply it to the test set of region B to calculate its index evaluation method; Step 5: Introduce the potential auxiliary samples C screened in step 3 into the process of step 4 one by one, to improve and enrich the sample size for parameter optimization adjustment, and evaluate the contribution and influence of the potential auxiliary samples C on the evaluation performance of the test set of region B each time. The loss function is calculated one by one by the cross-validation method using the confusion matrix classification accuracy. The cross-validation method first introduces all potential auxiliary samples C into the training set of region B to calculate the classification accuracy E of all auxiliary samples introduced. m , and then remove the potential auxiliary samples C from the training set of region B one by one, and calculate the classification accuracy E of the samples removed one by one i , compared with E i and E m , if the classification accuracy is reduced, the removed sample is a sample with beneficial contribution and needs to be retained, otherwise it is a useless sample and is removed from the potential auxiliary sample C, and finally the beneficial historical sample D of area B is obtained; Step six, the beneficial historical samples D of the area B perform parameter optimization and adjustment on the deep learning landslide hazard assessment model of the complete historical area A established in step two, and obtain an improved deep learning data migration landslide hazard assessment model for area B.
2. The method according to claim 1, characterized in that In step 2, the indicator evaluation method includes a classification accuracy evaluation method and an area under the ROC curve evaluation method.
3. The method according to claim 1, characterized in that In step 2, the factors affecting landslides include topography, geological lithology, vegetation coverage, soil type, precipitation, and distance from roads, rivers, and fault zones.
4. The method according to claim 1, characterized in that: In step 2, the classification accuracy is the proportion of training landslide and non-landslide samples in the complete historical area A that are correctly classified by the model to all training samples under different model parameter combinations.
5. The method according to claim 1, characterized in that In step 5, the loss function is calculated by using the confusion matrix classification accuracy, including the following steps: according to whether the actual landslide disaster occurs and whether the model predicts a landslide disaster, a 2x2 confusion matrix is obtained by setting the landslide disaster as 1 and the non-landslide disaster as 0. The confusion matrix can be divided into four cases according to whether the landslide disaster actually occurs and is predicted, namely: a is the actual non-landslide disaster sample, and the predicted one is also a non-landslide disaster sample; f b is the actual non-landslide disaster sample, and is predicted to be a landslide disaster sample; f c is the actual landslide disaster sample, and the predicted one is the non-landslide disaster sample; f d is the actual landslide disaster sample, and is predicted to be a landslide disaster sample. Based on this, the classification accuracy O can be calculated. a The formula is as follows: .
