Landslide GNSS monitoring data noise filtering effect evaluation method

Abstract

The invention discloses a landslide GNSS monitoring data noise filtering effect evaluation method in the technical field of landslide monitoring and early warning, and the method comprises the following steps: carrying out the analysis of the original data and noise-filtered data in the X and Y directions based on GNSS monitoring, and obtaining outlier data and an outlier rate; based on the plane resultant displacement data monitored by the GNSS and the noise-filtered data, performing analysis to obtain reverse direction displacement data and a reverse direction displacement rate; performing prediction based on plane resultant displacement data monitored by the GNSS to obtain an average absolute error, an average absolute percentage error, a root-mean-square error and a mean-square error between a predicted value and a measured value, namely, sub-indexes; and taking the sub-index, the outlier rate and the reverse direction displacement rate as evaluation indexes for evaluating the data noise filtering effect, and comparing the applicability and noise reduction efficiency of various data noise filtering methods. Based on the effect evaluation method, the problem that various data noise filtering methods cannot be preferentially selected by geological disaster monitoring and early warning workers in a unified standard system is solved.

Description

Technical Field

[0001] The invention belongs to the technical field of landslide monitoring and early warning, and in particular is a method for evaluating the effect of noise filtering of landslide GNSS monitoring data. Background Art

[0002] The Global Navigation Satellite System (GNSS), also known as the Global Satellite Navigation System, is an airborne radio navigation and positioning system that can provide users with all-weather 3D coordinates, speed and time information at any location on the Earth's surface or near-Earth space. In recent years, with the rapid development of GNSS technology, this technology has been widely used in the field of landslide monitoring with high accuracy and stability requirements due to its advantages such as all-round, real-time and dynamic monitoring. However, since the GNSS measurement principle is to determine the position of the ground point by receiving the signal transmitted by the satellite, there are errors in the three aspects of the transmitter, the propagation end and the receiver, such as the clock error of the satellite clock, the satellite orbit error, the satellite ephemeris error, the ionosphere refraction error, the tropospheric refraction error, the terminal clock error, the antenna relative center error and the multipath effect error. In addition, in the practice of landslide GNSS monitoring, the first step is to build a base station and an observation station, and then measure the baseline length between the base station and the observation station, so as to achieve the purpose of monitoring the surface deformation. The complex observation conditions during landslide monitoring, such as improper site selection, poor communication, unstable power supply, human damage and monitoring terminal failure, will cause errors in the observation values ​​and reduce the accuracy of deformation monitoring.

[0003] Traditional methods for eliminating GNSS monitoring gross errors include Laida's rule, Grubbs' criterion, Dixon's criterion and data jump method. At present, the methods that are widely used in GNSS data noise filtering include moving window averaging, frequency domain transformation and Kalman filtering. The noise filtering effect of the moving window averaging method is related to the window size and moving period. When the window size and moving period are consistent with the change of the observation data set, the effect is better. The spectrum change method is currently commonly used for wavelet analysis denoising. The wavelet denoising effect is affected by the wavelet basis function and the number of decomposition layers, which makes the optimal estimate of the real monitoring data obtained by wavelet analysis not stable enough. Kalman filtering is an algorithm that uses the linear system state equation to optimally estimate the system state through the system input and output observation data.

[0004] In recent years, it has been widely used in military and civilian fields including navigation, positioning, mapping, control, artificial intelligence, surface deformation, etc. The limitation of Kalman filtering is that it assumes that the error is Gaussian distributed and can only be used in linear systems. In order to overcome the above limitations, domestic and foreign scholars have improved it into extended Kalman filtering, robust-adaptive Kalman filtering, inertial Kalman filtering, etc. In summary, domestic and foreign scholars have carried out data noise filtering research in various fields, but there is currently no noise filtering effect evaluation method for the physical significance and early warning needs of landslide GNSS monitoring, which makes it impossible for various data noise filtering methods to be selected in a unified standard system for geological disaster monitoring and early warning workers. Summary of the invention

[0005] The present invention aims to provide an evaluation method for the noise filtering effect of landslide GNSS monitoring data. Based on the effect evaluation method, the problem that various data noise filtering methods cannot be selected in a unified standard system for selection by geological disaster monitoring and early warning workers is avoided.

[0006] In order to achieve the above object, the technical solution of the present invention is as follows: A method for evaluating the effect of noise filtering of landslide GNSS monitoring data, comprising the following steps:

[0007] Step 1: Based on the original data and noise-filtered data of the X-direction and Y-direction monitoring time series of the GNSS monitoring, the outlier data and outlier rate are analyzed;

[0008] Step 2: Based on the plane combined displacement data and noise-filtered data of the GNSS monitoring time series, considering the physical significance of landslide deformation, the reverse displacement data and reverse displacement rate are analyzed;

[0009] Step 3: Based on the plane displacement data and noise-filtered data of the GNSS monitoring time series, machine learning regression analysis is performed to predict the mean absolute error, mean absolute percentage error, root mean square error and mean square error between the predicted value and the measured value, and these four indicators are used as sub-indicators;

[0010] Step 4: Use the sub-index, outlier rate and reverse displacement rate as evaluation indicators to evaluate the data denoising effect, compare the applicability and noise reduction efficiency of various data denoising methods, and evaluate the effect of landslide GNSS monitoring data denoising.

[0011] Furthermore, the original data refers to the data acquired by the GNSS monitoring equipment, and the denoised data refers to the original data after denoising by the moving window method, the original data after denoising by the frequency domain transformation method, and the original data after denoising by the Kalman filter.

[0012] Furthermore, outliers are defined as abnormal values ​​in the X or Y displacement data, and the quartile method is used to detect data anomalies. When a data point is outside the range of 1.5 times the distance between the lower quartile and the upper quartile, it is considered an outlier:

[0013] Outlier lower limit: Q1-1.5×IQR

[0014] Outlier upper limit: Q3+1.5×IQR

[0015] In the formula, Q1 is the first quartile, Q3 is the third quartile; IQR = Q3-Q1.

[0016] Furthermore, the outlier rate is the ratio of outlier data to the total data volume, and the formula is as follows:

[0017]

[0018] Where: P O ——outlier rate, N p ——Number of outliers in the monitoring data, N d ——Total amount of monitoring data.

[0019] Furthermore, the reverse displacement data refers to the displacement data when the angle between the reverse displacement data and the free surface direction of the slope is greater than 150°.

[0020] Furthermore, the reverse displacement rate is the ratio of the reverse displacement data to the total data volume, and the formula is as follows:

[0021]

[0022] Where: R r ——Reverse displacement rate, N R ——Number of reverse displacement data in the monitoring data, N d ——Total amount of monitoring data.

[0023] Beneficial effects: The present invention proposes a noise filtering effect evaluation index for the physical significance and early warning needs of landslide GNSS monitoring, evaluates the effects of various data noise filtering methods, and enables various data noise filtering methods to be selected in a unified standard system for geological disaster monitoring and early warning workers. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 The flowchart of the method for evaluating the effect of noise filtering of landslide GNSS monitoring data in an embodiment of the present invention is shown.

[0025] Figure 2 Schematic diagram of common errors in landslide GNSS monitoring according to the method for evaluating the effect of noise filtering of landslide GNSS monitoring data in an embodiment of the present invention.

[0026] Figure 3 This is a distribution map of GNSS monitoring points of a landslide in a certain place according to the method for evaluating the effect of noise filtering of GNSS monitoring data of a landslide in an embodiment of the present invention.

[0027] Figure 4 This is a graph of the cumulative plane displacement-time curve of a landslide in a certain place according to the method for evaluating the effect of noise filtering of landslide GNSS monitoring data in an embodiment of the present invention.

[0028] Figure 5 Schematic diagram of the basic principle of the moving window method of the method for evaluating the effect of noise filtering of landslide GNSS monitoring data in an embodiment of the present invention.

[0029] Figure 6 Schematic diagram of the basic principle of the frequency domain transformation method of the method for evaluating the effect of noise filtering of landslide GNSS monitoring data in an embodiment of the present invention.

[0030] Figure 7 The figure is a Kalman filter calculation principle diagram of the method for evaluating the effect of noise filtering of landslide GNSS monitoring data in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The following is further described in detail through specific implementation methods:

[0032] The embodiment is basically as follows Figure 1-Figure 5 As shown: A method for evaluating the effect of noise filtering of landslide GNSS monitoring data, comprising the following steps:

[0033] Step 1: Based on the original data and noise-filtered data of the X-direction and Y-direction monitoring time series of the GNSS monitoring, the outlier data and outlier rate are analyzed.

[0034] Among them, through the analysis of the original total data of the GNSS monitoring time series in the X and Y directions, it can be found that there is a type of outlier data. Because the creep deformation of the landslide has a short-term persistence, it can be judged that this type of data is error data ( Figure 2 ). Further, an outlier is defined as an abnormal value in the X or Y displacement data, and the algorithm for data anomaly detection using the quartile method in statistics is used, that is, when a data point is outside the range of 1.5 times the distance between the lower quartile and the upper quartile, it is considered an outlier:

[0035] Outlier lower limit: Q1-1.5×IQR

[0036] Outlier upper limit: Q3+1.5×IQR

[0037] In the formula, Q1 is the first quartile, Q3 is the third quartile; IQR = Q3-Q1.

[0038] The "outlier rate" is the ratio of outlier data to the total data volume. The formula is as follows:

[0039]

[0040] Where: P O ——outlier rate, N p ——Number of outliers in the monitoring data, N d ——Total amount of monitoring data.

[0041] Step 2: Based on the plane combined displacement data and noise-filtered data of the GNSS monitoring time series, considering the physical significance of landslide deformation, the reverse displacement data and reverse displacement rate are analyzed.

[0042] Among them, through the original total data of the GNSS monitoring time series, it can be found that there are many data with the combined displacement direction toward the deformation inside the slope body. From the physical meaning of landslide surface deformation monitoring, it can be judged that this type of deformation is error data. That is, the combined displacement direction of the landslide deformation should mostly be consistent with the direction of the free surface of the slope. For example: the free surface direction of the Xinpu landslide is 350°. Through the synthesis of the GNSS combined displacement vector, it can be judged that its main sliding direction is also 350°. When the combined displacement direction of the GNSS measured value is 180°, it can be judged that the combined displacement direction is toward the inside of the slope, and this type of data can be judged as error data ( Figure 2 ). The reverse displacement is defined as the displacement when the angle with the free surface of the slope is greater than 150°. The reverse displacement rate is the ratio of the reverse displacement data to the total data volume, and its calculation formula is as follows:

[0043]

[0044] Where: R r ——Reverse displacement rate, N R ——Number of reverse displacement data in the monitoring data, N d ——Total amount of monitoring data.

[0045] Step 3: Based on the plane displacement data and noise-filtered data of the GNSS monitoring time series, machine learning regression analysis is performed to predict the mean absolute error, mean absolute percentage error, root mean square error and mean square error between the predicted value and the measured value, and these four indicators are used as sub-indicators;

[0046] Among them, the regression analysis machine learning algorithm based on the displacement time series curve uses the predicted value instead of the measured value to filter the data noise. Its advantages are that it is suitable for real-time big data analysis, has good compatibility and is easy to promote and apply. At present, the commonly used evaluation indicators of machine learning regression algorithms include mean absolute error (MAE), mean absolute percentage error (MAPE), root mean square error (RMSE) and mean square error (MSE). The above indicators can be used to evaluate the difference between the predicted value and the true value.

[0047] Step 4: Use sub-indicators, outlier rate and reverse displacement rate as evaluation indicators for data noise filtering to evaluate the effect of landslide GNSS monitoring data noise filtering.

[0048] This embodiment is based on the GNSS monitoring data of a landslide in a certain place, and adopts the effect evaluation method of GNSS monitoring data noise filtering of landslide in this embodiment to evaluate the effects of three data noise filtering methods: moving window method, frequency domain transformation method and Kalman filtering.

[0049] GNSS monitoring data of a landslide in a certain place:

[0050] 41 GNSS data from GNSS automatic monitoring from July 3, 2014 to December 31, 2022 were selected ( Figure 3 ), a total of about 2,372,904 data, the data collection frequency is 15 seconds / data, the data push frequency is 60 minutes / data when the deformation is not large and 6 minutes / data when there is abnormal deformation. In order to keep consistent with the warning judgment frequency of the monitoring and early warning platform, this embodiment uses the monitoring data of the actual push frequency to evaluate the data noise filtering effect. According to the characteristics of the landslide GNSS cumulative displacement-time curve, the monitoring data is divided into two types: steady type and step type ( Figure 4 ), perform data noise filtering on the monitoring time series data, and conduct a comparative analysis on the noise filtering effects.

[0051] The principles of the three data noise filtering methods are as follows:

[0052] The basic principle of the moving window method is to shift the data according to the window size based on the time series, and calculate the eigenvalues ​​in a certain window size in sequence ( Figure 5), and then use the eigenvalue to fit the original data sequence. According to the different methods of calculating eigenvalues ​​within the window range, the three most commonly used methods are moving average method, moving median method and moving polynomial method. Among them, the moving average method replaces the measured data with the arithmetic mean of all data in the window, and the moving median method replaces the measured data with the median of all data in the window. The main parameter affecting the noise filtering effect of the moving average method and the moving median method is the window size. The moving polynomial method is to perform polynomial fitting on the data in the sliding window, and replace the measured data with the fitted data. The main parameters affecting the noise filtering effect in this method are the window size and the number of polynomial fittings.

[0053] The basic principle of the frequency domain transformation method is to regard the measured data as a time domain signal, convert the time domain signal into a frequency domain signal by Fourier transform, and then filter (cut off) the high-order spectrum of the original signal and convert the cut-off frequency domain signal back to the time domain signal curve to achieve the filtering effect. The data noise filtering effect is controlled by the cut-off frequency of the filter ( Figure 6 ).

[0054] The conventional method of time-frequency conversion is shown in the following formula:

[0055]

[0056] Where, w is the frequency; b(t) is the transformation basis function; ψ(t) is the basic wavelet function; a is the wavelet scale; τ is the wavelet translation.

[0057] The time domain function f(t) of the original signal can be transformed into the frequency domain function F(w). This study selected the wavelet transformation method in the frequency domain transformation method for data noise filtering. The core of the wavelet transformation method lies in the selection of the wavelet basis function b(t). If the basis function b(t) is selected as a trigonometric function of infinite length, its transformation is a conventional Fourier transform; if the basis function b(t) is selected as a finite length and attenuated wavelet basis, the transformation method changes to a wavelet transform. Wavelet transform is different from Fourier transform in that the variable is only frequency w. Wavelet transform also has scale a (controls the expansion and contraction of the wavelet function) and translation (controls the translation of the wavelet function). The wavelet transform method is the most widely used and widely used in the frequency domain transformation method.

[0058] Kalman filtering is an algorithm that uses the linear system state equation to optimally estimate the system state through the system input and output observation data. Since the observation data includes the influence of noise and interference in the system, the optimal estimate can also be used for data noise filtering. The core of the Kalman filtering method is not to completely trust the actual measurement value, but to adjust the Kalman gain parameter and perform weighted calculations in combination with the estimated value to obtain the current estimate ( Figure 7 ).

[0059] The effect evaluation of the noise reduction of GNSS monitoring data of a landslide based on the above three data denoising methods is shown in Table 1:

[0060] Table 1 Evaluation index table of GNSS data noise filtering effect

[0061]

[0062] Combining the six indicators, it can be seen that the moving median method has the lowest mean absolute error (MAE) among all methods, which is 0.351, 0.567 and 0.746 when the window size is 5, 10 and 20 respectively; the mean absolute percentage error (MAPE) is the lowest among all methods, which is 0.074%, 0.169% and 0.260% when the window size is 5, 10 and 20 respectively; the mean square error (MSE) is second only to the wavelet denoising among all methods, which is 0.12 when the window size is 5, 10 and 20 respectively. 3, 0.576 and 0.706; the root mean square error (RMSE) is the lowest among all the methods. When the window size is more than 10, the root mean square error is second only to wavelet denoising. When the window size is 5, 10 and 20, the root mean square error is 7.351, 25.295 and 26.452 respectively; the reduction rate of outlier rate is 0.00%, 0.00% and 0.00% respectively, which shows that the effect in outlier analysis is not ideal; the reduction rate of reverse displacement rate is the best among all the methods, which are 17.80%, 18.59% and 19.04% respectively.

[0063] In summary, the moving window method has the best noise filtering effect in stable and step deformation curves. The wavelet noise filtering method may be more suitable for noise filtering of sudden deformation curves, and it is also necessary to collect sudden deformation curves for analysis and determination. Kalman filtering is more suitable for noise filtering of stable curves.

[0064] The above is only an embodiment of the present invention. The common sense such as the known specific structure and characteristics in the scheme is not described in detail here. The ordinary technicians in the relevant field know all the common technical knowledge in the technical field of the invention before the application date or priority date, can obtain all the existing technologies in the field, and have the ability to apply the conventional experimental means before that date. The ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the relevant field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the protection scope of the present invention, which will not affect the effect of the implementation of the present invention and the practicality of the patent. The protection scope required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A method for evaluating the noise filtering effect of landslide GNSS monitoring data, characterized in that: The steps include: Step 1: Based on the original data and noise-filtered data of the X-direction and Y-direction monitoring time series of the GNSS monitoring, the outlier data and outlier rate are analyzed; Step 2: Based on the plane combined displacement data and noise-filtered data of the GNSS monitoring time series, considering the physical significance of landslide deformation, the reverse displacement data and reverse displacement rate are analyzed; Step 3: Based on the plane displacement data and noise-filtered data of the GNSS monitoring time series, machine learning regression analysis is performed to predict the mean absolute error, mean absolute percentage error, root mean square error and mean square error between the predicted value and the measured value, and these four indicators are used as sub-indicators; Step 4: Use sub-indicators, outlier rate and reverse displacement rate as evaluation indicators to evaluate the data noise filtering effect, compare the applicability and noise reduction efficiency of various data noise filtering methods, and evaluate the effect of landslide GNSS monitoring data noise filtering.

2. The method for evaluating the effect of noise filtering of landslide GNSS monitoring data according to claim 1 is characterized in that: The original data refers to the data obtained by the GNSS monitoring equipment, and the denoised data refers to the original data after denoising by the moving window method, the data after denoising by the frequency domain transformation method, and the data after denoising by the Kalman filter.

3. The method for evaluating the effect of noise filtering of landslide GNSS monitoring data according to claim 1 is characterized in that: Outliers are defined as abnormal values ​​in the X or Y displacement data. The quartile method is used to detect data anomalies. When a data point is outside the range of 1.5 times the distance between the lower quartile and the upper quartile, it is considered an outlier: Outlier lower limit: Q1-1.5×IQR Outlier upper limit: Q3+1.5×IQR In the formula, Q1 is the first quartile, Q3 is the third quartile; IQR = Q3-Q1.

4. The method for evaluating the effect of noise filtering of landslide GNSS monitoring data according to claim 3 is characterized in that: The outlier rate is the ratio of outlier data to the total data volume, and the formula is as follows: Where: P O ——outlier rate, N p ——Number of outliers in the monitoring data, N d ——Total amount of monitoring data.

5. The method for evaluating the effect of noise filtering of landslide GNSS monitoring data according to claim 1, characterized in that: The reverse displacement data refers to the displacement data when the angle with the free surface of the slope is greater than 150°.

6. The method for evaluating the effect of noise filtering of landslide GNSS monitoring data according to claim 5 is characterized by: The reverse displacement rate is the ratio of the reverse displacement data to the total data volume, and the formula is as follows: Where: R r ——Reverse displacement rate, N R ——Number of reverse displacement data in the monitoring data, N d ——Total amount of monitoring data.

Images