Semi-supervised landslide early warning method fusing concept drift detection and mark migration

Through the semi-supervised landslide early warning method that integrates conceptual drift detection and marking migration, the problems of complexity of rock and soil medium and high data labeling cost in existing landslide early warning technology are solved, and high-precision and real-time landslide early warning effect is achieved.

CN118940140AActive Publication Date: 2024-11-12CHINA UNIV OF MINING & TECH
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411003894.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-25
Publication Date
2024-11-12
Estimated Expiration
2044-07-25

AI Technical Summary

Technical Problem

The existing landslide early warning technology in the non-uniformity and discontinuity of rock and soil media makes it difficult to apply mechanical analysis models, and machine learning methods require a large amount of labeled data support. The existing landslide data set has no labels or sparse labels, resulting in early warning accuracy and Insufficient timeliness.

Method used

Using a semi-supervised landslide early warning method that integrates conceptual drift detection and marker migration, features are derived and clustered by obtaining source domain historical data and target domain current data, using KNN model and ADWIN conceptual drift detection algorithm, combining sliding windows and Manhattan distance Perform data migration and tag updates, and build an initial warning model to improve warning accuracy.

Benefits of technology

It realizes high classification accuracy of single landslide warning and real-time update of the model, effectively improves the warning accuracy, solves the problem of label updates in the semi-supervised model, and improves the timeliness of landslide warning.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118940140A_ABST
    Figure CN118940140A_ABST
Patent Text Reader

Abstract

The invention discloses a semi-supervised landslide early warning method fusing concept drift detection and mark migration. The method comprises the following steps: acquiring a source domain historical data set and target domain current landslide data; carrying out feature derivation on the data of the two domains, clustering the data of the source domain, carrying out feature derivation on clusters generated by clustering, and finally storing the clusters and the features thereof and recording the clusters and the features thereof as an initial early warning model; taking a cluster centroid as a training set to train K KNN models; eMD filtering is carried out on a target domain data stream, an ADWIN algorithm is used for filtering data to detect a concept drift point, and the concept drift point is a to-be-migrated data point; speed reciprocal and tangent angle characteristics closely related to the landslide curve are used as historical label migration basis; and for the classified data, judging the consistency of the migration tag and the classification tag to update the model. According to the invention, the features and labels of the historical landslide data are utilized, the historical labels are migrated to the current data, and classified early warning is carried out by using semi-supervised learning, so that the accuracy of landslide early warning is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of geological disaster prevention, and in particular to a semi-supervised landslide early warning method integrating concept drift detection and label migration. Background Art

[0002] Landslide refers to the phenomenon that the rock and soil on the slope slides downward along a certain sliding surface under the action of gravity. Landslides often cause casualties, property losses and infrastructure damage, causing serious impacts on human society and the natural environment. At present, the relevant work on landslide warning can be roughly divided into regional meteorological warning and single landslide warning. Regional meteorological warning mainly considers rainfall factors, and uses historical rainfall data and landslide occurrences in the corresponding area to establish a critical rainfall threshold model. However, the warning accuracy and timeliness of this model still have room for improvement. For single landslide warning, a variety of landslide warning models have been developed, such as deformation trend models, mechanical analysis models, threshold models and comprehensive risk models. However, due to the heterogeneity and discontinuity of rock and soil media, as well as the strong discreteness and randomness of rock and soil parameters, the above physical models based on mechanical analysis are difficult to apply in actual warning projects. In recent years, with the development of machine learning and deep learning, the use of K-means and ResNet models to construct a landslide displacement warning classification model has obtained good classification results, indicating that machine learning methods are expected to be applied in the field of landslide warning. However, this approach requires sufficient labeled data to support it. Most existing landslide datasets are unlabeled or sparsely labeled, and labeling datasets is time-consuming, labor-intensive, and costly. Summary of the invention

[0003] In order to overcome the defects in the prior art, the present invention provides a semi-supervised landslide early warning method integrating concept drift detection and label migration.

[0004] The technical solution adopted by the present invention is: a semi-supervised landslide early warning method integrating concept drift detection and label migration, comprising:

[0005] Step S1. Obtain historical data sets in the source domain and current landslide data in the target domain;

[0006] Step S2. Deriving features from the data in the source domain and the target domain, clustering the historical data in the source domain and deriving features from the clusters generated by the clustering, and saving the clusters and their features as the initial warning model;

[0007] Step S3. Take the centroid of the cluster generated by clustering as the training set, train K KNN models, maintain a weight for each KNN model, and use the classifier with the largest weight to classify the data;

[0008] Step S4. Filter the current landslide data stream in the target domain using EMD decomposition, and use the ADWIN concept drift detection algorithm to detect the concept drift points of the filtered data, and define the points within a small range near the drift points as near drift points;

[0009] Step S5. For the drift point of the curve, the sliding window is used to calculate the velocity mean and the tangent angle mean in the window, and the labeled landslide dataset in the source domain is migrated to the current landslide data by comparing the window features between the source domain and the target domain;

[0010] Step S6. For the classified data, determine the consistency between the migration label and the classification label. If they are consistent, add the data to the nearest cluster and update the cluster features. If they are inconsistent, use a heuristic method to create a new cluster.

[0011] As a further improvement of the present invention, in step S2, the feature derivation of the source domain and target domain data, clustering of the source domain historical data and feature derivation of the clusters generated by the clustering, and saving the clusters and their features as an initial warning model specifically include:

[0012] Step S21. Derived the corresponding velocity, velocity inverse, curvature, tangent angle and daily average velocity as its features through the landslide displacement values ​​and corresponding time of the source domain data and the target domain data, clustered the data and features in each type of data, and subdivided the data to obtain clusters;

[0013] Step S22. Calculate the amount of data in the cluster, linear sum LS, square sum SS, weight W, time τ, label CL, radius R and centroid C mc , and describe its characteristics. The set of all characteristics is called micro-cluster MC, and the set of all micro-clusters is recorded as the early warning initial model.

[0014] As a further improvement of the present invention, in step S3, the centroids of the clusters generated by clustering are taken as training sets to train K KNN models, a weight is maintained for each KNN model, and the classifier with the largest weight is used to classify the data, specifically including:

[0015] Step S31. Each classifier is i The weight matrix calculation of the time step is shown in formula (1):

[0016]

[0017] m is the number of labeled points before this point, and the function k-NN(x i ) assigns a class label to instance x i ,y i is the actual class label.

[0018] Step S32: Select the classifier with the largest weight as the best classifier to predict the class label of the test instance.

[0019] As a further improvement of the present invention, in step S4, the current data stream of the target domain is filtered using EMD decomposition, and the concept drift point of the filtered data is detected using the ADWIN concept drift detection algorithm, and points within a small range near the drift point are defined as near drift points, specifically including:

[0020] Step S41. Record the landslide displacement sequence as x(t)={x1,x2,...,x n}, x n It is the landslide displacement value corresponding to the nth moment. After the landslide displacement sequence is decomposed by EMD, the intrinsic modal component IMF is obtained. The formula is as follows:

[0021] IMF i (t)=EMD(x(t)) (2)

[0022] Step S42: To reduce the impact of data volatility on feature calculation, the decomposed components are reconstructed into trend term q(t) and noise term r(t), as follows:

[0023] x(t)=q(t)+r(t) (3)

[0024] Step S43. For each IMF obtained by decomposition i (t){i=1,2,...,n}, calculate its correlation coefficient with the original sequence, and finally obtain a set of correlation coefficient values ​​Cset={c1,c2,...,c i}, c i The formula is as follows:

[0025] c i =correlate(x(t),IMF i (t)) (4)

[0027] Among them, correlate means calculating the correlation coefficient between two time series, c i Decomposition component IMF i The correlation coefficient between (t) and the original sequence x(t);

[0028] Step S44. Correlation coefficient c in Cset i Keep a threshold α, if c i If it is greater than the threshold, the corresponding IMF i (t) is added to the reconstructed trend term q(t), and the remaining components are added to the noise term r(t);

[0029] Step S45. After removing the noise term, ADWIN is used to calculate the concept drift points in the trend term q(t) to obtain a set of concept drift point values ​​Pset = {p1, p2, ..., p n}, p i is the i-th drift point, and the formula is as follows:

[0030] Pset=ADWIN(q(t)) (5)

[0031] Step S46. For each detected drift point p i , define the β points before and after it as its near drift point Gp i , the formula is as follows:

[0032] Gp i ∈(p i -β,p i +β) (6)

[0033] As a further improvement of the present invention, in step S5, the sliding window is used to calculate the velocity mean and the tangent angle mean in the window for the curve drift point, and the source domain labeled landslide data set is migrated to the current landslide data by comparing the window features between the source domain and the target domain, specifically including:

[0034] Step S51. For each source domain landslide displacement data, a sliding window with a length of δ (δ>1) is maintained for it, and the reciprocal mean velocity Mspeeda is calculated in each sliding window, and the formula is as follows:

[0035] Mspeeda=1 / (δ*(Δdeform / Δtime)) (7)

[0036] And calculate the average tangent angle Mangle, the formula is as follows:

[0037] Mangle=1 / δ*atan(Δdeform / Δtime) (8)

[0038] And the label corresponding to the last point of the sliding window is counted as the label of the segment;

[0039] Step S52: Aggregate the calculation results of all source domain landslide data segments to obtain the migration model model Q , the formula is as follows:

[0040] model Q =[Mspeeda i ,Mangle i ,label i ] i∈(1,2,3...) (9)

[0041] Step S53: The features calculated from the source domain data and the target domain data are calculated using the Manhattan distance calculation formula to obtain their corresponding distances. The formula is as follows:

[0042] dist min =|Mspeeda i -Mspeeda r |+|Mangle i -Mangle r | (10)

[0043] Then, assign the label corresponding to the nearest migration model to the current data point x i , the formula is as follows:

[0044] Label(x i )←model Q (dist min )(11)

[0045] As a further improvement of the present invention, in step S52, in order to maintain the real-time nature of target domain data collection, at a new data point x i When it arrives, concatenate the previous δ-1 data points with it, and then calculate its corresponding Mspeeda r and Mangle r .

[0046] As a further improvement of the present invention, in step S6, the consistency between the migration label and the classification label is determined for the classified data. If they are consistent, the data is added to the nearest cluster and the cluster features are updated. If they are inconsistent, a heuristic method is used to create a new cluster, which specifically includes:

[0047] Step S61. At the subsequent data point x i When it arrives, the classification model is first called to classify it. Then if the label value y of the point i If it is not empty, then judge y i Are the labels of all micro-clusters MC equal? ​​If they are equal, the weight of the micro-cluster is increased by one and the corresponding time τ is updated; otherwise, the weight of the micro-cluster is reduced by one;

[0048] Step S62. Calculate x i Dist to all microclusters in the early warning model and return the nearest distance Dist min and the corresponding micro-cluster MC min , if Dist min >R mc Or instance x i Labeling and micro-cluster MC min The labels of are not empty and they are not equal, then a new micro-cluster is constructed using a heuristic method, the formula is as follows:

[0049]

[0050] Otherwise, the micro-cluster is updated with the following formula:

[0051] MC(LS=LS+x,SS=SS+x,N=N+1,T=curT) (13)

[0052] Step S63. For the number of microclusters in the entire early warning model, a threshold maxMC is maintained. If the model size is equal to the threshold, the two microclusters with the closest distance among the microclusters with labels and without labels are first found and returned to obtain [MC1, MC2], and then the two microclusters are merged;

[0053] If there are no unlabeled microclusters, find the two closest microclusters from the microclusters with the largest number of microclusters of a certain type and merge them. The formula is as follows:

[0054]

[0055] The beneficial effects of the present invention are:

[0056] (1) The present invention provides early warning for single landslides. Based on the fact that the information contained in historical landslide data is closely related to current landslide prediction, the semi-supervised data stream classification technology is applied to the field of landslide early warning. The concept drift can be passively dealt with by updating the model, and the classification accuracy is relatively high.

[0057] (2) In order to solve the problem that the semi-supervised model needs to use certain labels to update the model, the present invention proposes a migration algorithm that integrates concept drift detection. The concept drift detection algorithm is used to capture the "inflection point" in the landslide curve and the labels of the points most similar to the "inflection point" in the historical landslide data set are migrated using the inverse of the velocity and the tangent angle. This solves the label problem required by the semi-supervised model and effectively improves the warning accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0059] Figure 1 It is a flow chart of the semi-supervised landslide early warning method integrating concept drift detection and label migration of the present invention;

[0060] Figure 2 is a target domain original landslide displacement curve diagram in an embodiment of the present invention;

[0061] Figure 3 It is a landslide displacement filtering curve diagram after EMD decomposition and reconstruction in an embodiment of the present invention;

[0062] Figure 4is a drift point detection graph processed by the ADWIN concept drift algorithm in an embodiment of the present invention;

[0063] Figure 5 It is a data migration effect diagram in the real-time example of the present invention;

[0064] Figure 6 It is a diagram of the overall early warning effect in the embodiment of the present invention. DETAILED DESCRIPTION

[0065] like Figure 1 As shown, a semi-supervised landslide early warning method integrating concept drift detection and label migration of the present invention comprises the following steps:

[0066] Step S1. Obtain historical data sets in the source domain and current landslide data in the target domain. The main ways to obtain historical data and current data include geological surveys, satellite remote sensing technology, and monitoring equipment.

[0067] Step S2: derive features from the source domain and target domain data, cluster the source domain historical data and derive features from the clusters generated by clustering, and save the clusters and their features as the initial warning model, specifically including:

[0068] Step S21. Derived the corresponding velocity, velocity inverse, curvature, tangent angle and daily average velocity as its features through the landslide displacement values ​​and corresponding time of the source domain data and the target domain data, clustered the data and features in each type of data, and subdivided the data to obtain clusters;

[0069] Step S22. Calculate the amount of data N, linear sum LS, square sum SS, weight W, time τ, label CL, radius R and centroid C in the cluster mc , and describe its characteristics. The set of all characteristics is called micro-cluster MC, and the set of all micro-clusters is recorded as the early warning initial model.

[0070] Step S3. Take the centroid of the cluster generated by clustering as the training set, train K KNN models, K is an odd number such as 1, 3, 5, 7, etc. Keep a weight for each KNN model, and use the classifier with the largest weight to classify the data, specifically including:

[0071] Step S31. Each classifier is i The weight matrix calculation of the time step is shown in formula (1):

[0072]

[0073] m is the number of labeled points before this point, and the function k-NN(x i ) assigns a class label to instance x i ,y i is the actual class label.

[0074] Step S32: Select the classifier with the largest weight as the best classifier to predict the class label of the test instance.

[0075] Step S4. The current landslide data stream in the target domain is filtered using EMD (Empirical Mode Decomposition) decomposition, and the concept drift detection algorithm is used to detect the concept drift points of the filtered data, and the points within a small range near the drift points are defined as near drift points, specifically including:

[0076] Step S41. Record the landslide displacement sequence as x(t)={x1,x2,...,x n}, x n It is the landslide displacement value corresponding to the nth moment. After the landslide displacement sequence is decomposed by EMD, the intrinsic modal component IMF is obtained. The formula is as follows:

[0077] IMF i (t)=EMD(x(t)) (2)

[0078] Step S42: To reduce the impact of data volatility on feature calculation, the decomposed components are reconstructed into trend term q(t) and noise term r(t), as follows:

[0079] x(t)=q(t)+r(t) (3)

[0080] Step S43. For each IMF obtained by decomposition i (t){i=1,2,...,n}, calculate its correlation coefficient with the original sequence, and finally obtain a set of correlation coefficient values ​​Cset={c1,c2,...,c i}, c i The formula is as follows:

[0081] c i =correlate(x(t),IMF i (t)) (4)

[0082] Among them, correlate means calculating the correlation coefficient between two time series, c i Decomposition component IMF i The correlation coefficient between (t) and the original sequence x(t);

[0083] Step S44. Correlation coefficient c in Cset i Keep a threshold α, if c i If it is greater than the threshold, the corresponding IMF i(t) is added to the reconstructed trend term q(t), and the remaining components are added to the noise term r(t);

[0084] Step S45. After removing the noise term, ADWIN is used to calculate the concept drift points in the trend term q(t) to obtain a set of concept drift point values ​​Pset = {p1, p2, ..., p n}, p i is the i-th drift point, and the formula is as follows:

[0085] Pset=ADWIN(q(t)) (5)

[0086] Step S46. For each detected drift point p i , define the β points before and after it as its near drift point Gp i , the formula is as follows:

[0087] Gp i ∈(p i -β,p i +β) (6)

[0088] Step S5. For the drift point of the curve, use the sliding window to calculate the velocity mean and tangent angle mean in the window, and migrate the labeled landslide dataset in the source domain to the current landslide data by comparing the window features between the source domain and the target domain, specifically including:

[0089] Step S51. For each source domain landslide displacement data, a sliding window with a length of δ (δ>1) is maintained for it, and the reciprocal mean velocity Mspeeda is calculated in each sliding window, and the formula is as follows:

[0090] Mspeeda=1 / (δ*(Δdeform / Δtime)) (7)

[0091] And calculate the average tangent angle Mangle, the formula is as follows:

[0092] Mangle=1 / δ*atan(Δdeform / Δtime) (8)

[0093] And the label corresponding to the last point of the sliding window is counted as the label of the segment;

[0094] Step S52: Aggregate the calculation results of all source domain landslide data segments to obtain the migration model model Q , the formula is as follows:

[0095] model Q =[Mspeeda i ,Mangle i ,label i] i∈(1,2,3...) (9)

[0096] In order to maintain the real-time nature of target domain data collection, at a new data point x i When it arrives, concatenate the previous δ-1 data points with it, and then calculate its corresponding Mspeeda r and Mangle r .

[0097] Step S53: The features calculated from the source domain data and the target domain data are calculated using the Manhattan distance calculation formula to obtain their corresponding distances. The formula is as follows:

[0098] dist min =|Mspeeda i -Mspeeda r |+|Mangle i -Mangle r | (10)

[0099] Then, assign the label corresponding to the nearest migration model to the current data point x i , the formula is as follows:

[0100] Label(x i )←model Q (dist min )(11)

[0101] Step S6. For the classified data, determine the consistency between the migration label and the classification label. If they are consistent, add the data to the nearest cluster and update the cluster features. If they are inconsistent, use a heuristic method to create a new cluster. Specifically include:

[0102] Step S61. At the subsequent data point x i When it arrives, the classification model is first called to classify it. Then if the label value y of the point i If it is not empty, then judge y i Are the labels of all micro-clusters MC equal? ​​If they are equal, the weight of the micro-cluster is increased by one and the corresponding time τ is updated; otherwise, the weight of the micro-cluster is reduced by one;

[0103] Step S62. Calculate x i Dist to all microclusters in the early warning model and return the nearest distance Dist min and the corresponding micro-cluster MC min , if Dist min >R mc Or instance x i Labeling and micro-cluster MC minThe labels of are not empty and they are not equal, then a new micro-cluster is constructed using a heuristic method, the formula is as follows:

[0104]

[0105] Otherwise, the micro-cluster is updated with the following formula:

[0106] MC(LS=LS+x,SS=SS+x,N=N+1,T=curT); (13)

[0107] Step S63. For the number of microclusters in the entire early warning model, a threshold maxMC is maintained. If the model size is equal to the threshold, the two microclusters with the closest distance among the microclusters with labels and without labels are first found and returned to obtain [MC1, MC2], and then the two microclusters are merged;

[0108] If there are no unlabeled microclusters, find the two closest microclusters from the microclusters with the largest number of microclusters of a certain type and merge them. The formula is as follows:

[0109]

[0110] [Example]

[0111] The specific process and data migration effect of the present invention are described below with specific embodiments.

[0112] The original data is the radar monitoring landslide displacement data of a certain area actually obtained by a company in Suzhou. Each data has about 2,000 points, and the sampling interval is about six minutes. Figure 2 There are seven groups of data in total. In this embodiment, the first six groups are used as the historical data set of landslide in the source domain, and the seventh group is used as the current data in the target domain. The subsequent effects are all displayed with the seventh group of data.

[0113] The original data is processed by the EMD algorithm to obtain the various IMF components. The correlation coefficients between the various IMF components and the original data are calculated, and the items with correlation coefficients greater than 0.1 are added to the reconstructed sequence, such as Figure 3 As shown. Figure 3 It can be seen that the reconstructed sequence is smoother than the original sequence while retaining the trend information of the original data.

[0114] A complete landslide displacement monitoring sequence will contain data of various distributions, each of which corresponds to a warning level. The switching between various data distributions responds to the concept drift phenomenon. The present invention uses the ADWIN algorithm to perform active concept drift detection on the reconstructed sequence generated by the above filtering to capture the drift points in the landslide data stream. The drift point monitoring results are as follows: Figure 4It can be seen that the algorithm can detect the drift point in the landslide displacement data and the detected drift point position has a certain randomness. The ADWIN window is set to 30, and the near drift point is 20 points before and after the drift point (the parameter is adjustable).

[0115] Label migration was performed on all subsequent landslide data streams. The migration results are shown in the figure below. Figure 5 As shown. Figure 4 It can be seen that the landslide displacement curve is different from the ordinary concept drift curve type, and the concept drift is almost every moment. Therefore, there are a large number of drift points and near drift points in each landslide data stream. Changing the size of the parameter β can change the number of near drift points. The larger the value of β, the more near drift points there are, and the more labels are provided to the semi-supervised early warning model. A large number of experiments were conducted on the sliding window δ of the migration algorithm, and it was finally fixed to 5, that is, when a new data point arrives, the previous 4 points are spliced ​​to obtain a landslide data segment, and then the feature mean of the segment data is calculated and input into the migration model to obtain its label.

[0116] Constructing an initial landslide warning model using an existing labeled landslide dataset

[0117] Since the data itself only has the accumulated displacement of the landslide and its corresponding time, the characteristics are relatively simple and cannot fully reflect the stage of the landslide. Therefore, the present invention calculates the six characteristics corresponding to each landslide displacement data point, namely, velocity, acceleration, inverse velocity, curvature, tangent angle, and daily rate, to better reflect the state of the landslide. All characteristic data are normalized by inverse trigonometric functions. The present invention focuses on the correlation between the six characteristics and the warning level label value, which is discussed in detail in the following table.

[0118] Table 1 Correlation analysis between data characteristics and warning label values

[0119]

[0120] Table 1 calculates the correlation coefficients between the seven sets of landslide data label values ​​and the remaining features, and then obtains the corresponding mean. Table 1 shows that the correlation between the landslide displacement value and acceleration features and the warning level label is low, so speed, inverse speed, curvature, tangent angle, and daily average speed are finally selected as semi-supervised learning features to build the initial landslide warning model.

[0121] Model parameter setting: The detection number parameter α is set to 5, and drift detection is performed once for every 5 new data points, which means that the label value provided by the semi-supervised early warning model is limited to less than 20%. The near drift point parameter β is set to 20. If the new data point is within 20 points of the nearest drift point, a label value is given to the point. The migration sliding window δ is set to 5, and each new point is spliced ​​with its previous 4 points. The parameter c is set to 500. After the cumulative number of new points is greater than 500 points, the accumulated data segments are filtered using EMD decomposition. m is set to 50, where m is the marked point within 50 points before the current point. k=25, and each class is divided into 25 clusters using a hierarchical clustering algorithm. K is set to 4, and 4 KNN classifiers (1, 3, 5, 7) are used for weighted classification in the classification stage.

[0122] Warning results: Figure 6 As shown, it can be seen that the early warning system did not give a blue warning, the orange warning came a little later, and the red warning came in advance to a certain extent. The overall early warning accuracy rate was 94.77%, which met the actual engineering needs.

[0123] Model comparison: The Saito model of the prior art is compared with the velocity inverse method and compared with the real label of the present invention. First, the abbreviations in Table 2 are, ACR (overall accuracy), ACR-YOR (accuracy of yellow-orange-red warning levels), AR (false alarm rate), MAR (missed alarm rate), SNV (sensitivity), SPF (specificity). The accuracy rate ACR is defined as the ratio of the number of correct model warnings to the number of all warning results. The false alarm rate (False Alarm Rate, FAR) refers to the probability that the early warning system issues an erroneous landslide warning, which measures the extent to which the early warning system generates false alarms when no real landslide event occurs. The lower the false alarm rate, the fewer times the system issues false alarms, indicating that the system has higher accuracy. It is defined here as the ratio of the number of points where the early warning results are different from the label value to all warning points. The missed alarm rate (Missed Alarm Rate, MAR) refers to the proportion of landslide events that the early warning system warns incorrectly, which measures the extent to which the early warning system fails to issue an alarm when a real landslide event occurs. The lower the false alarm rate, the better the system can identify and predict landslide events. It is defined here as the ratio of the number of points where the warning result is no warning but the real label has a warning level to all warning points. Sensitivity SNV refers to the ability of the warning system to correctly predict and issue landslide alarms. It measures the ability of the warning system to identify actual landslide events. It is defined here as the ratio of the number of correct alarms to the sum of the number of correct alarms and the number of false alarms. Specificity SPF refers to the ability of the warning system to correctly exclude non-landslide events. It measures the ability of the system to correctly not issue an alarm. It is defined here as the ratio of the number of correct no alarms to the sum of the number of correct no alarms and the number of false alarms.

[0124] Table 2 Comparison of model accuracy

[0125]

[0126] Comparison conclusion: By comparison, it can be obtained that the present invention has a significant improvement in overall warning and high-risk warning accuracy compared to the traditional Saito model and velocity inverse method, and the false alarm rate has also been reduced to a certain extent. However, there is a certain false alarm rate, that is, when the landslide curve changes slowly, the present invention may give a lower warning level. Because there is a certain false alarm rate, the sensitivity of the present invention is lower than that of the traditional method, but the results of observing the real labels show that the present invention is superior to the traditional method in all indicators when using the real label to guide the semi-supervised model.

[0127] The embodiments of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited thereto. Various changes that can be made within the knowledge scope of technicians in the relevant technical field without departing from the spirit of the present invention are all within the protection scope of the claims of the present invention.

Claims

1. A semi-supervised landslide early warning method integrating concept drift detection and label transfer, It is characterized in that include: Step S1. Obtain historical data sets in the source domain and current landslide data in the target domain; Step S2. Deriving features from the data in the source domain and the target domain, clustering the historical data in the source domain and deriving features from the clusters generated by the clustering, and saving the clusters and their features as the initial warning model; Step S3. Take the centroid of the cluster generated by clustering as the training set, train K KNN models, maintain a weight for each KNN model, and use the classifier with the largest weight to classify the data; Step S4. Filter the current landslide data stream in the target domain using EMD decomposition, and use the ADWIN concept drift detection algorithm to detect the concept drift points of the filtered data, and define the points within a small range near the drift points as near drift points; Step S5. For the drift points of the target domain data, the sliding window is used to calculate the velocity mean and the tangent angle mean in the window, and the labeled landslide dataset in the source domain is migrated to the current landslide data by comparing the window features between the source domain and the target domain; Step S6. For the classified data, determine the consistency between the migration label and the classification label. If they are consistent, add the data to the nearest cluster and update the cluster features. If they are inconsistent, use a heuristic method to create a new cluster.

2. According to claim 1, the semi-supervised landslide early warning method integrating concept drift detection and label migration, It is characterized in that In step S2, the feature derivation is performed on the source domain and target domain data, the source domain historical data is clustered, and the feature derivation is performed on the clusters generated by the clustering, and the clusters and their features are saved as the initial warning model, which specifically includes: Step S21. Derived the corresponding velocity, velocity inverse, curvature, tangent angle and daily average velocity as its features through the landslide displacement values ​​and corresponding time of the source domain data and the target domain data, clustered the data and features in each type of data, and subdivided the data to obtain clusters; Step S22. Calculate the amount of data N, linear sum LS, square sum SS, weight W, time τ, label CL, radius R and centroid C in the cluster mc , and describe its characteristics. The set of all characteristics is called micro-cluster MC, and the set of all micro-clusters is recorded as the initial warning model.

3. According to claim 1, the semi-supervised landslide early warning method integrating concept drift detection and label migration, It is characterized in that In step S3, the centroids of the clusters generated by clustering are taken as training sets to train K KNN models, a weight is maintained for each KNN model, and the classifier with the largest weight is used to classify the data, specifically including: Step S31. Each classifier is i The weight matrix calculation of the time step is shown in formula (1): m is the number of labeled points before this point, and the function k-NN(x i ) assigns a class label to instance x i ,y i is the true class label; Step S32: Select the classifier with the largest weight as the best classifier to predict the class label of the test instance.

4. The semi-supervised landslide early warning method integrating concept drift detection and label migration according to claim 1, It is characterized in that In step S4, the current data stream of the target domain is filtered using EMD decomposition, and the concept drift point of the filtered data is detected using the ADWIN concept drift detection algorithm, and points within a small range near the drift point are defined as near drift points, specifically including: Step S41. Record the landslide displacement sequence as x(t)={x 1 ,x 2 ,...,x n }, x n It is the landslide displacement value corresponding to the nth moment. After the landslide displacement sequence is decomposed by EMD, the intrinsic modal component IMF is obtained. The formula is as follows: IMF i (t)=EMD(x(t)); (2) Step S42: To reduce the impact of data volatility on feature calculation, the decomposed components are reconstructed into trend term q(t) and noise term r(t), as follows: x(t)=q(t)+r(t); (3) Step S43. For each IMF obtained by decomposition i (t){i=1,2,...,n}, calculate its correlation coefficient with the original sequence, and finally obtain a set of correlation coefficient values ​​Cset={c 1 ,c 2 ,...,c i },c i The formula is as follows: c i =correlate(x(t),IMF i (t)) (4) Among them, correlate means calculating the correlation coefficient between two time series, c i Decomposition component IMF i The correlation coefficient between (t) and the original sequence x(t); Step S44. Correlation coefficient c in Cset i Keep a threshold α, if c i If it is greater than the threshold, the corresponding IMF i (t) is added to the reconstructed trend term q(t), and the remaining components are added to the noise term r(t); Step S45. After removing the noise term, ADWIN is used to calculate the concept drift points in the trend term q(t) to obtain a set of concept drift point values ​​Pset={p 1 ,p 2 ,...,p n }, p i is the i-th drift point, and the formula is as follows: Pset=ADWIN(q(t)); (5) Step S46. For each detected drift point p i , define the β points before and after it as its near drift point Gp i , the formula is as follows: Gp i ∈(p i -β,p i +β)。 (6)。 5. The semi-supervised landslide early warning method integrating concept drift detection and label migration according to claim 1, It is characterized in that In step S5, the sliding window is used to calculate the velocity mean and tangent angle mean in the window for the curve drift point, and the source domain labeled landslide dataset is migrated to the current landslide data by comparing the window features between the source domain and the target domain, specifically including: Step S51. For each source domain landslide displacement data, a sliding window with a length of δ (δ>1) is maintained for it, and the reciprocal mean velocity Mspeeda is calculated in each sliding window, and the formula is as follows: Mspeeda=1 / (δ*(Δdeform / Δtime)) (7) And the formula for calculating the average tangent angle Mangle is as follows: Mangle=1 / δ*atan(Δdeform / Δtime) (8) And the label corresponding to the last point of the sliding window is counted as the label of the segment; Step S52: Aggregate the calculation results of all source domain landslide data segments to obtain the migration model model Q , the formula is as follows: model Q =[Mspeeda i ,Missing i ,label i ] i∈(1,2,3...) (9) Step S53: The features calculated from the source domain data and the target domain data are calculated using the Manhattan distance calculation formula to obtain their corresponding distances. The formula is as follows: dist min =|Mspeeda i -Mspeeda r |+|Mangle i -Mangle r | (10) Then, assign the label corresponding to the nearest migration model to the current data point x i , the formula is as follows: Label(x i )←model Q (dist min )。(11)。 6. The semi-supervised landslide early warning method integrating concept drift detection and label migration according to claim 5, It is characterized in that In step S52, in order to maintain the real-time nature of target domain data collection, at a new data point x i When it arrives, concatenate the previous δ-1 data points with it, and then calculate its corresponding Mspeed r a and Mangle r .

7. The semi-supervised landslide early warning method integrating concept drift detection and label migration according to claim 1, It is characterized in that In step S6, the consistency between the migration label and the classification label is determined for the classified data. If they are consistent, the data is added to the nearest cluster and the cluster features are updated. If they are inconsistent, a new cluster is created using a heuristic method, which specifically includes: Step S61. At the subsequent data point x i When it arrives, the classification model is first called to classify it. Then if the label value y of the point i If it is not empty, then judge y i Are the labels of all micro-clusters MC equal? ​​If they are equal, the weight of the micro-cluster is increased by one and the corresponding time τ is updated; otherwise, the weight of the micro-cluster is reduced by one; Step S62. Calculate x i Dist to all microclusters in the early warning model and return the nearest distance Dist min and the corresponding micro-cluster MC min , if Dist min >R mc Or instance x i Labeling and micro-cluster MC min The labels of are not empty and they are not equal, then a new micro-cluster is constructed using a heuristic method, the formula is as follows: MC new =(LS=x,SS=x 2 ,N=1,W=1,τ=curT CL=x.label,R=R MC ) (12) Otherwise, the micro-cluster is updated with the following formula: MC(LS=LS+x,SS=SS+x,N=N+1,T=curT); (13) Step S63. For the number of micro-clusters in the entire early warning model, a threshold maxMC is maintained. If the model size is equal to the threshold, the two micro-clusters with the nearest distance among the micro-clusters with labels and without labels are first found and returned to obtain [MC 1 ,MC 2 ], then the two microclusters are merged; If there are no unlabeled microclusters, find the two closest microclusters from the microclusters with the largest number of microclusters of a certain type and merge them. The formula is as follows:

Citation Information

Patent Citations

  • Intelligent remote sensing identification method for large landslide in mountainous area based on semi-supervised deep learning

    CN115761513A

  • Increment-based suspected fire point data identification method and device

    CN117197679A

  • Concept drift data stream ensemble classification method

    CN117892243A

  • Method for Real-Time Enhancement of a Predictive Algorithm by a Novel Measurement of Concept Drift Using Algorithmically-Generated Features

    US20170103340A1

  • Real drift detector on partial labeled data in data streams

    US20220036201A1