Fast processing method for gross errors in slope monitoring data

Abstract

The present invention discloses a method for quickly processing gross errors in slope monitoring data, comprising the following steps: 1) obtaining time-series slope monitoring data, with a total of h observed data, and grouping the observed data according to the observation time interval to obtain multiple groups of observed data; the observed data includes the observation time and its corresponding observed value; 2) performing within-group segmentation on each group of observed data in step 1), and then calculating the credibility of the observed values within the segment; 3) calculating the average credibility and the standard deviation of credibility of the observed values of each group, and identifying and replacing the gross errors by using the average credibility and the standard deviation of credibility; 4) repeating steps 1) to 3) for the replaced data to optimize the observed data until there are no gross errors. The method for quickly processing gross errors in slope monitoring data provided by the present invention can more scientifically guide the elimination of gross errors in slope monitoring data, so as to more accurately perform slope monitoring, control the safety state of the reservoir bank slope, and ensure project safety.

Description

Technical Field

[0001] The present invention relates to the technical field of safety monitoring of water conservancy projects, and particularly to a method for quickly processing gross errors in slope monitoring data. Background Art

[0002] Water energy resources are mainly concentrated in areas with complex geological environments and special natural climates. Sudden geological disasters such as collapses and landslides are relatively common in these areas. The impoundment of high dam reservoirs and the periodic cyclic rise and fall of reservoir water levels pose potential risks of further inducing slope instability. The instability and failure of slopes of high dam reservoirs have a wider influence range and more diverse harmful forms than those of slopes under other conditions, including not only the direct harm caused by slope instability but also the indirect harm within the influence range of the landslide.

[0003] Safety monitoring is an important measure to control the safety state of reservoir bank slopes and ensure project safety. The perception of slope safety monitoring instruments is the basis, and the timely, efficient, and accurate analysis and processing of data are the key points. The monitoring of reservoir bank slopes has the characteristics of scattered distribution of monitoring objects, inconvenient transportation, long monitoring periods, uncertain monitoring frequencies, and many influencing factors of monitoring effect quantities, resulting in many difficulties in the processing of gross errors in monitoring data. Filtering analysis of monitoring data is an effective means of preprocessing monitoring data. However, due to the differences in the monitoring periods of reservoir bank slopes, for example, some have a very short observation interval due to intensified observations, while some have a very long observation interval due to poor observation traffic and observation conditions, conventional filtering analysis methods cannot effectively identify gross errors, resulting in the problem of "masking" of gross errors. In addition, in conventional filtering analysis, credibility indicators are mainly used to identify errors. When the credibility is small, it is determined as a gross error, but there is no clear indicator for how "small" it should be.

[0004] Therefore, in view of the above deficiencies, the applicant considers providing a method for quickly identifying gross errors in slope monitoring data, which can not only effectively avoid the problem of masking of gross errors in long-time series monitoring data but also give the criteria for gross error determination, realizing the quick identification and processing of gross errors in slope monitoring data. Summary of the Invention

[0005] To overcome the deficiencies of the above technologies, the purpose of the present invention is to provide a method for quickly processing gross errors in slope monitoring data, which solves the problems that the monitoring frequencies of reservoir bank slope monitoring data vary greatly due to intensified observations or poor traffic and observation conditions, and at the same time, the long-term observation of reservoir bank slopes leads to a large difference in the change values of the observed effect quantities between two consecutive observations in long-time series observation data, the problem of "masking" of gross errors in conventional filtering methods, and the lack of clear judgment criteria in conventional filtering methods.

[0006] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0007] A method for quickly processing gross errors in slope monitoring data, comprising the following steps:

[0008] 1) Obtain a total of h observed data of time series slope monitoring data, and group the observed data according to the observation time interval to obtain multiple groups of observed data; the observed data includes the observation time and its corresponding observed value; set L 1 、L 2 、...、L i 、...、L h to represent the observed values corresponding to each observation time;

[0009] 2) Perform within-group segmentation on each group of observed data in step 1), and then calculate the credibility of the observed values within the segment;

[0010] 3) Based on the credibility, calculate the average credibility and the standard deviation of credibility of the observed values within each group, and identify and replace the gross errors within the group using the average credibility and the standard deviation of credibility;

[0011] 4) Repeat steps 1) to 3) for the replaced data for optimization until there are no gross errors.

[0012] Preferably, in step 1), the grouping method includes: dividing into g groups according to the observation frequency in different periods, with the difference between adjacent two observations within each group not exceeding 1.5 times, and renumbering the observed values within the group starting from 1, denoted as {L 1 ,L 2 ,...,L i ,...,L n}.

[0013] Preferably, in step 2), the within-group segmentation method includes: taking the observed value for which the credibility is to be calculated as the center, and taking a total of m data before and / or after the observed value for which the credibility is to be calculated within the group to form a data segment, that is, each data segment contains m + 1 data; where m ≥ 4 and m < n, and n is the number of observed values within the group.

[0014] Preferably, in step 2), the credibility is calculated by the following formula:

[0015]

[0016] where is the credibility of the observed value; ξ i represents the number of L i in the neighborhood {L||L - L i | ≤ λ, λ > 0} that contains L j (j ≠ i), and L j is the same as L iOther observed values in the same data segment, j≠i, and λ is the neighborhood radius.

[0017] Preferably, the neighborhood radius λ is calculated by the following formula:

[0018]

[0019] Preferably, in step 2), the selection of the data segment is specified as follows:

[0020] (a) When i = 1, take the 1st and 2nd observed values as the data segment where L i is located and use it as the search area for ξ i ;

[0021] (b) When , take the 1st to 2i - 1 observed values as the search area for ξ i ;

[0022] (c) When , take the to observed values as the search area for ξ i ;

[0023] (d) When , take the 2i - n to n observed values as the search area for ξ i ;

[0024] (e) When i = n, take the n - 1st and nth observed values as the search area for ξ i ;

[0025] where m / 2 is taken as an integer and m≥4.

[0026] Preferably, in step 3), the average credibility is calculated by the following formula:

[0027]

[0028] where is the average credibility, and is the credibility of the observed value in step 2).

[0029] Preferably, in step 3), the standard deviation of credibility is calculated by the following formula:

[0030]

[0031] where is the standard deviation of credibility, and is the credibility of the observed value in step 2).

[0032] Preferably, the method for identifying and replacing gross errors in step 3) includes:

[0033] Find the credibility The smallest observed value L i , if its credibility And the average credibility And the standard deviation of credibility Satisfy the following formula:

[0034]

[0035] Then for the credibility The smallest observed value L i Temporarily eliminate it, and use E(A) to replace the credibility The smallest observed value L i , if the standard deviation s' of the differences in the data group after replacement and the standard deviation s of the differences within the data group before replacement satisfy: s′ < s, then the credibility The smallest observed value L i Is determined as a gross error, and E(A) is used to replace the credibility The smallest observed value L i ;

[0036] The E(A) is calculated by the following formula:

[0037]

[0038] Preferably, the standard deviation s of the differences is calculated by the following formula:

[0039]

[0040] Where n is the number of observed values in the data group.

[0041] Preferably, step 4) includes: re-filtering the observed data after replacement, repeating steps 1) to 3) for iteration, and realizing the optimization of the standard deviation s of the differences until the standard deviation s of the differences converges to a state without gross errors.

[0042] A rapid processing system for gross errors in slope monitoring data, used to implement the above rapid processing method for gross errors in slope monitoring data, includes:

[0043] A data storage module, used to store time-series slope monitoring data;

[0044] A data processing module, used to cyclically process the time-series slope monitoring data to eliminate gross errors;

[0045] The data processing module includes:

[0046] A grouping unit, used to group the observed data of the time-series slope monitoring data;

[0047] Segmentation unit, used for segmenting the grouped observation data within each group;

[0048] Calculation unit, used for calculating credibility, average credibility, and standard deviation of credibility;

[0049] Analysis unit, used for identifying gross errors;

[0050] Replacement unit, used for replacing gross errors.

[0051] A computer device, including a memory and a processor, the memory is used for storing at least one program, and the processor is used for loading the at least one program to execute the above-mentioned rapid processing method for gross errors in slope monitoring data.

[0052] Compared with the prior art, the beneficial effects of the present invention are:

[0053] The method of the present invention is based on a grouping method based on the observation time interval. Observation data with similar observation time intervals are divided into one group, and gross errors are identified within the group, which can effectively avoid the problem of gross error masking caused by large changes in the observed values before and after due to too long an observation time interval.

[0054] This method proposes criteria for gross error identification, elimination, and replacement, that is, using the mean of credibility and the standard deviation of credibility to identify gross errors, avoiding the deficiency that although the credibility is small enough, there is no clear standard for how small it should be, and it can more scientifically guide the elimination of gross errors in slope monitoring data, thereby more accurately conducting slope monitoring, controlling the safety state of the reservoir bank slope, and ensuring project safety. Description of the Drawings

[0055] Figure 1 It is a schematic flow chart of a rapid processing method for gross errors in slope monitoring data of the present invention;

[0056] Figure 2 It is a comparison chart before and after gross error processing for the first data group of a slope surface displacement monitoring point using the method of the present invention;

[0057] Figure 3 It is a comparison chart before and after gross error processing for the second data group of a slope surface displacement monitoring point using the method of the present invention;

[0058] Figure 4 It is a comparison chart before and after gross error processing for the third data group of a slope surface displacement monitoring point using the method of the present invention;

[0059] Figure 5 It is a comparison chart before and after gross error processing for the fourth data group of a slope surface displacement monitoring point using the method of the present invention;

[0060] Figure 6It is a comparison chart before and after gross error processing of all data of a slope surface displacement monitoring point using the method of the present invention. Detailed implementation manners

[0061] To better explain the present invention, the main content of the present invention will be further clarified below in conjunction with specific embodiments, but the content of the present invention is not limited to the following embodiments.

[0062] As Figure 1 shown, a method for rapid gross error processing of slope monitoring data of the present invention includes the following steps:

[0063] 1) Grouping of observed data

[0064] Taking the principle that the observation time intervals are similar, the time series slope monitoring data {L 1 , L 2 ,..., L i ,..., L h} with a total of h observed data are divided into g data groups according to the observation frequencies in different periods. The number of data in each data group can be different, but the observation frequencies are similar. Specifically, the difference between adjacent two observations in each data group is not greater than 1.5 times. The observed data includes the observation time and its corresponding observed value. L 1 , L 2 ,..., L i ,..., L h represent the observed values corresponding to each observation time.

[0065] 2) Segmenting the observed data within a group and calculating the credibility of the observed values within a segment

[0066] Assume that after grouping, there are n observed data in a certain group. The n observed values of the observation sequence are divided into several data segments, and each data segment contains m + 1 (m < n) observed values. Then, the credibility of each observed value is calculated for each data segment respectively When segmenting, the observed value to be calculated for credibility can be used as the center, and a total of m data before and / or after the observed value to be calculated for credibility within the group are taken to form a data segment.

[0067] The credibility of the observed value is calculated according to the following formula:

[0068]

[0069] Among them, is the credibility of the observed value; ξ i represents the number of L i in the neighborhood {L||L - L i | ≤ λ, λ > 0} that contains L j , and L jFor L i For other observations in the same data segment, j≠i, λ is the neighborhood radius.

[0070] The neighborhood radius λ is calculated by the following formula:

[0071]

[0072] The selection of data segments is specified as follows (where m / 2 is an integer, m≥4):

[0073] (a) When i = 1, take the first and second observation values ​​as L i The data segment where it is located and as ξ i Search area;

[0074] (b) When When , take the 1st to 2i-1th observation values ​​as ξ i Search area;

[0075] (c) When When to observations as ξ i Search area;

[0076] (d) When When , take the 2i-nth to nth observation values ​​as ξ i Search area;

[0077] (e) When i = n, take the n-1th and nth observation values ​​as ξ i Search area.

[0078] 3) Calculate the mean and standard deviation of the credibility of each observation data group, and use the mean and standard deviation of the credibility to identify and replace gross errors.

[0079] Reliability average and the reliability standard deviation The calculation formulas are as follows:

[0080]

[0081] Use the credibility mean and credibility standard deviation to identify and replace gross errors. The identification and replacement methods are as follows:

[0082] Find the observation value L with the minimum credibility i ,like and When , the observation value L with the minimum credibility i Temporarily remove and use E(A) to replace the observation value L with the lowest credibility i, if the standard deviation of the difference s' of the data group after replacement < s, then the observation value L with the lowest credibility i is determined as a gross error, and E(A) is used to replace the gross error observation value L i . Among them, the calculation formula of E(A) is:

[0083]

[0084] The standard deviation of the difference s is calculated by the following formula:

[0085]

[0086] Among them, n is the number of observation values in the data group. The calculation formula of the standard deviation of the difference s' of the data group after replacement is the same as that of s, only the observation values in the data group are replaced.

[0087] 4) Circularly filter the original data until it converges to a state without gross errors.

[0088] On the basis of step 3), re-filter the original data, repeat steps 1) to 3), and cycle in this way. Through the iteration of the unascertained filtering process, the optimization of s is realized, and finally s converges to a state without gross errors.

[0089] The following further illustrates the solution of the present invention through specific embodiments.

[0090] There are observation data of a slope surface displacement monitoring point for 13 years (from 2005 to 2017). The minimum time interval of the measuring point data is 2 days, and the maximum is 114 days. Considering the data observation frequency, about 3 years is selected as a data group, and a total of 4 groups are divided. Each group has about 48 data. Select m = 6, and calculate the credibility of each data in each group respectively. Taking the first group of data as an example, calculate the standard deviation of the difference s of the data in the group = 3.38. The credibility of each observation value is shown in Table 1.

[0091] As can be seen from Table 1, after the credibility of the data in the first data group is calculated for the first time, the average credibility of the data group is 0.92, the standard deviation is 0.16, and the credibility of the measurement value with the serial number 35 is the lowest, which is 0.33, less than 0.92 - 3 * 0.16 = 0.44, and Therefore, the measurement value with the serial number 35 is temporarily excluded, and E(A) = 9.55 is used for replacement. The standard deviation of the difference s' of the first data group after replacement = 3.16 is less than 3.38. Therefore, the measurement value with the serial number 35 is a gross error. Repeat the above operations until the standard deviation of the difference s converges to a state without gross errors. Remove the gross errors from each data group in turn, and the results are shown in Table 2 and Figures 2 - 6 .

[0092] Table 1: Calculated values of the credibility of the first group of the surface displacement monitoring point for the first time (m = 6)

[0093] Serial number Time Measured value Confidence level Serial number Time Measured value Confidence level Serial number Time Measured value Confidence level 1 2005 / 5 / 18 0.00 1.00 17 2005 / 8 / 25 7.97 1.00 33 2006 / 4 / 4 12.07 0.83 2 2005 / 5 / 22 -0.78 1.00 18 2005 / 9 / 5 2.99 0.83 34 2006 / 4 / 21 10.06 1.00 3 2005 / 5 / 24 1.31 1.00 19 2005 / 9 / 15 6.46 1.00 35 2006 / 5 / 17 5.16 0.33 4 2005 / 5 / 26 -2.15 1.00 20 2005 / 9 / 24 12.15 0.50 36 2006 / 5 / 29 12.56 0.83 5 2005 / 5 / 30 0.47 0.83 21 2005 / 10 / 9 4.69 0.83 37 2006 / 6 / 7 13.04 0.83 6 2005 / 6 / 3 0.00 0.83 22 2005 / 10 / 18 3.44 0.83 38 2006 / 6 / 20 14.11 0.83 7 2005 / 6 / 12 0.71 0.83 23 2005 / 10 / 26 5.86 1.00 39 2006 / 7 / 3 16.45 1.00 8 2005 / 6 / 19 7.59 0.33 24 2005 / 11 / 4 4.70 1.00 40 2006 / 8 / 30 18.28 1.00 9 2005 / 6 / 26 -1.21 0.67 25 2005 / 11 / 15 8.19 1.00 41 2006 / 11 / 1 17.41 1.00 10 2005 / 7 / 3 0.83 1.00 26 2005 / 12 / 6 8.94 1.00 42 2007 / 1 / 14 20.94 1.00 11 2005 / 7 / 11 2.02 1.00 27 2005 / 12 / 17 10.14 1.00 43 2007 / 3 / 27 23.74 1.00 12 2005 / 7 / 16 6.09 0.83 28 2006 / 1 / 3 8.74 1.00 44 2007 / 4 / 15 21.56 1.00 13 2005 / 7 / 24 1.74 1.00 29 2006 / 2 / 14 9.48 1.00 45 2007 / 6 / 23 26.54 1.00 14 2005 / 7 / 31 3.00 1.00 30 2006 / 2 / 25 9.86 1.00 46 2007 / 8 / 31 28.21 1.00 15 2005 / 8 / 8 4.05 1.00 31 2006 / 3 / 4 12.39 1.00 47 2007 / 9 / 7 26.92 1.00 16 2005 / 8 / 16 2.49 1.00 32 2006 / 3 / 19 10.11 1.00 48 2007 / 9 / 15 26.84 1.00

[0094] Table 2: Results of eliminating gross errors by grouping and segmentation

[0095]

[0096] From Table 2 and Figures 2 - 6 it can be seen that after eliminating and replacing gross errors by the method of the present invention, the slope monitoring data is more reasonable and scientific, and slope monitoring can be accurately carried out to control the safety state of the reservoir bank slope and ensure project safety.

Claims

1. A method for rapid processing of gross errors in slope monitoring data, characterized by: It includes the following steps: 1) Obtain a total of h observation data of time series slope monitoring data, and group the observation data according to the observation time interval to obtain multiple observation data groups; the observation data includes the observation time and its corresponding observation value; set L1, L2, ..., L i ,...,L h Represents the observation value corresponding to each observation time; 2) Segment each group of observation data in step 1) within the group, and then calculate the credibility of the observed values within the segment; 3) Based on the credibility, calculate the average credibility and the standard deviation of credibility of the observed values within each group, and identify and replace the gross errors within the group by using the average credibility and the standard deviation of credibility; The method for identifying and replacing the gross errors within the group includes: Find the observation value L with the minimum credibility i If its credibility The average reliability and the reliability standard deviation When the following equation is satisfied: and Then for the observation value L with the minimum credibility i Temporarily remove and use E(A) to replace the observation value L with the lowest credibility i If the standard deviation s' of the difference between the data group after replacement and the standard deviation s of the difference within the data group before replacement satisfy: s'<s, then the observation value L with the minimum credibility is i It is judged as a gross error, and E(A) replaces the observation value L with the minimum credibility. i ; The E(A) is calculated by the following formula: 4) Repeat steps 1) to 3) for the replaced observation data for optimization until there are no gross errors.

2. The method for rapid processing of gross errors in slope monitoring data according to claim 1 is characterized by: In step 1), the grouping method includes: dividing into g groups according to the observation frequency in different periods, the interval between two adjacent observations in each group differs by no more than 1.5 times, and renumbering the observation values ​​in the group from 1, recorded as {L1, L2, ..., L i , ..., L n }.

3. The method for rapid processing of gross errors in slope monitoring data according to claim 1 is characterized by: In step 2), the method for segmenting within the group includes: taking the observed value for which the credibility is to be calculated as the center, and taking a total of m data before and / or after the observed value for which the credibility is to be calculated within the group to form a data segment, that is, each data segment contains m + 1 data; where m ≥ 4, m < n, and n is the number of observed values within the group.

4. The method for rapid processing of gross errors in slope monitoring data according to claim 3 is characterized by: In step 2), the credibility is calculated by the following formula: in, is the credibility of the observed value; i Indicates L i Neighborhood {L||LL i |≤λ,λ>0} contains L j The number of L j For L i For other observations in the same data segment, j≠i, λ is the neighborhood radius.

5. The method for rapid processing of gross errors in slope monitoring data according to claim 4 is characterized by: The neighborhood radius λ is calculated by the following formula:

6. The method for rapid processing of gross errors in slope monitoring data according to claim 4 is characterized by: In step 2), the selection of the data segment is specified as follows: (a) When i = 1, take the first and second observation values ​​as L i The data segment where it is located and serves as ξ i Search area; (b) When When , take the 1st to 2i-1th observation values ​​as ξ i Search area; (c) When When to The observed value is ξ i Search area; (d) When When , take the 2i-nth to nth observation values ​​as ξ i Search area; (e) When i = n, take the n-1th and nth observation values ​​as ξ i Search area; Where m / 2 is an integer.

7. The method for rapid processing of gross errors in slope monitoring data according to claim 1 is characterized by: In step 3), the average credibility is calculated by the following formula: in, is the mean reliability value, is the credibility of the observation in step 2).

8. The method for rapid processing of gross errors in slope monitoring data according to claim 1 is characterized by: In step 3), the standard deviation of credibility is calculated by the following formula: in, is the reliability standard deviation, is the mean reliability value, is the credibility of the observation in step 2).

9. The method for rapid processing of gross errors in slope monitoring data according to claim 1 is characterized by: The standard deviation s of the difference is calculated by the following formula: Where n is the number of observed values within the data group.

10. The method for rapid processing of gross errors in slope monitoring data according to any one of claims 1 to 9, characterized in that: Step 4) includes: re-filtering the replaced observation data, repeating steps 1) to 3) for iteration, and realizing the optimization of the standard deviation s of the difference until the standard deviation s of the difference converges to a state without gross errors.

11. A rapid processing system for gross errors in slope monitoring data, characterized by: For implementing the method for rapid processing of gross errors in slope monitoring data according to any one of claims 1 to 10, it includes: A data storage module for storing time series slope monitoring data; A data processing module for circularly processing the time series slope monitoring data to eliminate gross errors; The data processing module includes: A grouping unit for grouping the observation data of the time series slope monitoring data; A segmenting unit for segmenting the grouped observation data within the group; A calculating unit for calculating the credibility, the average credibility, and the standard deviation of credibility; An analyzing unit for identifying gross errors; A replacing unit for replacing gross errors.

12. A computer device, characterized in that: It includes a memory and a processor, the memory is used for storing at least one program, and the processor is used for loading the at least one program to execute the method for rapid processing of gross errors in slope monitoring data according to any one of claims 1 to 10.

Images