Slope monitoring data gross error rapid processing method

Abstract

The invention discloses a side slope monitoring data gross error rapid processing method, which comprises the following steps: 1) obtaining time sequence side slope monitoring data and h observation data, and grouping the observation data according to observation time intervals to obtain a plurality of observation data groups; the observation data comprises observation time and observation values corresponding to the observation time; 2) carrying out intra-group segmentation on each observation data group in the step 1), and then carrying out credibility calculation on observation values in the segments; 3) calculating a credibility average value and a credibility variance of each group of observation values, and identifying and replacing gross errors by using the credibility average value and the credibility variance; and 4) repeating the steps 1)-3) on the replaced data to perform observation data optimization until no gross error exists. According to the slope monitoring data gross error rapid processing method provided by the invention, the removal of the gross error of the slope monitoring data can be more scientifically guided, so that slope monitoring is more accurately carried out, the safety state of a bank slope is mastered, and the engineering safety is ensured.

Description

Technical Field

[0001] The invention relates to the technical field of water conservancy project safety monitoring, and in particular to a method for quickly processing gross errors in slope monitoring data. Background Art

[0002] Hydropower resources are mainly concentrated in areas with complex geological environments and special natural climates. These areas are prone to sudden geological disasters such as collapse and landslides. The storage of water in high dams and large reservoirs and the periodic fluctuation of reservoir water levels pose potential risks such as further inducing slope instability. The instability and damage of slopes in high dams and large reservoirs has a wider impact range than the instability and damage of slopes under other conditions, and the forms of damage are more diverse, including not only the direct damage caused by slope instability, but also the indirect damage to the landslide area.

[0003] Safety monitoring is an important measure to control the safety status of reservoir bank slopes and ensure the safety of projects. The perception of slope safety monitoring instruments is the foundation, and timely, efficient and accurate analysis and processing of data is the key. Reservoir bank slope monitoring has the characteristics of scattered distribution of monitoring objects, inconvenient transportation, long monitoring cycle, uncertain monitoring frequency, and many factors affecting the monitoring effect, which leads to many difficulties in the processing of gross errors in monitoring data. Filtering analysis of monitoring data is an effective means of preprocessing monitoring data, but due to the differences in the monitoring cycle of reservoir bank slopes, such as: some have a short observation interval due to intensive observation; some have a long observation interval due to poor observation traffic and observation conditions, which leads to the inability of conventional filtering analysis methods to effectively identify gross errors, and there is a problem of gross error "masking". In addition, in conventional filtering analysis, the credibility index is mainly used to identify errors. When the credibility is small, it is judged as a gross error, but there is no clear indicator of how "small" it is.

[0004] Therefore, in response to the above-mentioned deficiencies, the applicant has considered providing a method for rapidly identifying gross errors in slope monitoring data, which can not only effectively avoid the problem of gross error masking in long-term series monitoring data, but also provide a standard for gross error determination, thereby achieving rapid identification and processing of gross errors in slope monitoring data. Summary of the invention

[0005] In order to overcome the shortcomings of the above-mentioned technology, the purpose of the present invention is to provide a method for quickly processing gross errors in slope monitoring data, so as to solve the problem that the observation frequency of reservoir bank slope monitoring data is quite different due to encrypted observation or poor traffic observation conditions. At the same time, the observation of reservoir bank slope requires long-term observation, resulting in a large difference in the change value of the effect quantity of the two observations before and after in the long-term series observation data. Conventional filtering methods have the problem of gross error "masking" and the lack of clear judgment criteria in conventional filtering methods.

[0006] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0007] A method for quickly processing gross errors in slope monitoring data includes the following steps:

[0008] 1) Obtain a total of h observed data of time series slope monitoring data, and group the observed data according to the observation time interval to obtain multiple groups of observed data; the observed data includes the observation time and its corresponding observed value; set L1, L2,..., L i 、...、L h to represent the observed values corresponding to each observation time;

[0009] 2) Perform within-group segmentation on each group of observed data in step 1), and then calculate the credibility of the observed values within the segment;

[0010] 3) Based on the credibility, calculate the average credibility and credibility variance of the observed values within each group, and use the average credibility and credibility variance to identify and replace the gross errors within the group;

[0011] 4) Repeat steps 1) to 3) for the replaced data for optimization until there are no gross errors.

[0012] Preferably, in step 1), the grouping method includes: dividing into g groups according to the observation frequency in different periods, the difference between adjacent two observations within each group is not greater than 1.5 times, and re-number the observed values within the group starting from 1, denoted as {L1, L2,..., L i ,..., L n}.

[0013] Preferably, in step 2), the within-group segmentation method includes: taking the observed value to be calculated for credibility as the center, and taking a total of m data before and / or after the observed value to be calculated for credibility within the group to form a data segment, that is, each data segment contains m + 1 data; where m≥4, m < n, and n is the number of observed values within the group.

[0014] Preferably, in step 2), the credibility is calculated by the following formula:

[0015]

[0016] where φ(x) is the credibility of the observed value; ξ i represents the number of L i in the neighborhood {L||L - L i |≤λ, λ>0} that contains L j (j≠i), L j is other observed value in the same data segment as L i , j≠i, and λ is the neighborhood radius.

[0017] Preferably, the neighborhood radius λ is calculated by the following formula:

[0018]

[0019] Preferably, in step 2), the selection of data segments is specified as follows:

[0020] (a) When i = 1, take the first and second observation values ​​as L i The data segment where it is located and serves as ξ i Search area;

[0021] (b) When When , take the 1st to 2i-1th observation values ​​as ξ i Search area;

[0022] (c) When When observations as ξ i Search area;

[0023] (d) When When , take the 2i-nth to nth observation values ​​as ξ i Search area;

[0024] (e) When i = n, take the n-1th and nth observation values ​​as ξ i Search area;

[0025] Here, m / 2 is an integer, and m≥4.

[0026] Preferably, in step 3), the average credibility value is calculated by the following formula:

[0027]

[0028] in, is the mean reliability value, is the credibility of the observation in step 2).

[0029] Preferably, in step 3), the credibility variance is calculated by the following formula:

[0030]

[0031] in, is the reliability standard deviation, is the credibility of the observation in step 2).

[0032] Preferably, the method for identifying and replacing gross errors in step 3) includes:

[0033] Finding Credibility The smallest observed value L i If its credibility The average reliability and the credibility variance When the following equation is satisfied:

[0034]

[0035] Credibility The smallest observed value L i Temporarily remove and use E(A) instead of credibility The smallest observed value L i If the variance of the difference in the data group after replacement s' and the variance of the difference in the data group before replacement s satisfy: s'<s, then the credibility The smallest observed value L i It is judged as a gross error and E(A) is used instead of the credibility The smallest observed value L i ;

[0036] The E(A) is calculated by the following formula:

[0037]

[0038] Preferably, the difference variance s is calculated by the following formula:

[0039]

[0040] Here, n is the number of observations in the data set.

[0041] Preferably, step 4) includes: re-filtering the replaced observation data, repeating steps 1) to 3) for iteration to optimize the difference variance s until the difference variance s converges to a state without gross errors.

[0042] A system for rapidly processing gross errors in slope monitoring data, used to implement the aforementioned method for rapidly processing gross errors in slope monitoring data, comprising:

[0043] A data storage module, used for storing time series slope monitoring data;

[0044] The data processing module is used to perform cyclic processing on the time series slope monitoring data to eliminate gross errors;

[0045] The data processing module comprises:

[0046] A grouping unit, used to group the observation data of the time series slope monitoring data;

[0047] A segmentation unit is used to segment the grouped observation data within the group;

[0048] A calculation unit, used for calculating the credibility, the credibility mean and the credibility variance;

[0049] Analysis unit, used to identify gross errors;

[0050] Replacement unit, used to replace gross errors.

[0051] A computer device comprises a memory and a processor, wherein the memory is used to store at least one program, and the processor is used to load the at least one program to execute the above-mentioned method for rapid processing of gross errors in slope monitoring data.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] The method of the present invention is based on a grouping method based on observation time intervals. The observation data with similar observation time intervals are grouped together, and gross errors are identified within the group. This can effectively avoid the gross error masking problem caused by a large change in the previous and next observation values ​​due to a long observation time interval.

[0054] This method proposes the criteria for gross error identification, elimination and replacement, that is, using the mean and variance of the credibility to identify gross errors, avoiding the deficiency of using a credibility that is small enough but has no clear standard for how small it is. It can provide more scientific guidance for the elimination of gross errors in slope monitoring data, thereby conducting more accurate slope monitoring, controlling the safety status of the reservoir bank slope and ensuring project safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 A schematic diagram of a flow chart of a method for rapidly processing gross errors in slope monitoring data according to the present invention;

[0056] Figure 2 This is a comparison diagram before and after the gross error processing of the first data group of a displacement monitoring point on a slope surface by using the method of the present invention;

[0057] Figure 3 A comparison diagram before and after the gross error processing of the second data set of a displacement monitoring point on a slope surface by using the method of the present invention;

[0058] Figure 4 A comparison diagram before and after the gross error processing of the third data set of a displacement monitoring point on a slope surface using the method of the present invention;

[0059] Figure 5 This is a comparison diagram before and after the gross error processing of the fourth data group of a displacement monitoring point on a slope surface using the method of the present invention;

[0060] Figure 6 The figure is a comparison diagram before and after the gross error processing of all data of a displacement monitoring point on a slope surface using the method of the present invention. DETAILED DESCRIPTION

[0061] To better explain the present invention, the main content of the present invention will be further clarified below in conjunction with specific embodiments, but the content of the present invention is not limited to the following embodiments.

[0062] As Figure 1 shown, a method for quickly processing gross errors in slope monitoring data according to the present invention includes the following steps:

[0063] 1) Grouping of observation data

[0064] Based on the principle of similar observation time intervals, the time series slope monitoring data {L1, L2,..., L i ,..., L h} with a total of h observation data are divided into g data groups according to the observation frequency in different periods. The number of data in each data group can be different, but the observation frequencies are similar. Specifically, the difference between adjacent two observations within each data group is not greater than 1.5 times. The observation data includes the observation time and its corresponding observation value. L1, L2,..., L i ,..., L h represent the observation values corresponding to each observation time.

[0065] 2) Segmenting the observation data within the group and calculating the credibility of the observation values within the segment

[0066] Assume that after grouping, there are n observation data in a certain group. The n observation values of the observation sequence are divided into several data segments, each data segment containing m + 1 (m < n) observation values, and then the credibility φ(x) of each observation value is calculated for each data segment respectively. When segmenting, the observation value for which the credibility is to be calculated can be used as the center, and a total of m data before and / or after the observation value for which the credibility is to be calculated within the group are taken to form a data segment.

[0067] The calculation formula for the credibility φ(x) of the observation value is as follows:

[0068]

[0069] Where is the credibility of the observation value; ξ i represents the number of L i in the neighborhood {L||L - L i | ≤ λ, λ > 0} that contains L j , L j is other observation value in the same data segment as L i , j ≠ i, and λ is the neighborhood radius.

[0070] The neighborhood radius λ is calculated by the following formula:

[0071]

[0072] The selection of data segments is specified as follows (where m / 2 is an integer, m≥4):

[0073] (a) When i = 1, take the first and second observation values ​​as L i The data segment where it is located and serves as ξ i Search area;

[0074] (b) When When , take the 1st to 2i-1th observation values ​​as ξ i Search area;

[0075] (c) When When observations as ξ i Search area;

[0076] (d) When When , take the 2i-nth to nth observation values ​​as ξ i Search area;

[0077] (e) When i = n, take the n-1th and nth observation values ​​as ξ i Search area.

[0078] 3) Calculate the credibility mean and credibility variance of each observation data group, and use the credibility mean and credibility variance to identify and replace gross errors.

[0079] The mean value of the reliability φ(Li) and the credibility variance σ φ(Li) The calculation formulas are as follows:

[0080]

[0081] The credibility mean and credibility variance are used to identify and replace gross errors. The identification and replacement methods are as follows:

[0082] Find the observation value L with the minimum credibility i ,like and When , the observation value L with the minimum credibility i Temporarily remove and use E(A) to replace the observation value L with the lowest credibility i If the variance of the data set after replacement is s′<s, then the observation L with the smallest credibility is i It is judged as a gross error, and E(A) replaces the gross error observation value L i .

[0083] The calculation formula of E(A) is:

[0084]

[0085] The difference variance s is calculated by the following formula:

[0086]

[0087] Where n is the number of observations in the data set. The calculation formula of the difference variance s' of the data set after replacement is the same as that of the difference variance s, except that only the observations in the data set are replaced.

[0088] 4) Circularly filter the original data until it converges to a state without gross errors.

[0089] On the basis of step 3), the original data is refiltered, and steps 1) to 3) are repeated, and this cycle is repeated. The optimization of s is achieved through the iteration of the unascertained filtering process, and finally s converges to a state without gross errors.

[0090] The present invention is further described below through specific examples.

[0091] A slope surface displacement monitoring point has observation data for up to 13 years (2005-2017). The minimum time interval of the measurement point data is 2 days and the maximum is 114 days. Considering the frequency of data observation, about 3 years is selected as a data group, which is divided into 4 groups, with about 48 data in each group. m=6 is selected, and the credibility of each data in each group is calculated respectively. Taking the first group of data as an example, the variance of the data difference within the group is calculated to be s=3.38. The credibility of each observation value is shown in Table 1.

[0092] As shown in Table 1, after the first reliability calculation of the data in the first data group, the mean reliability of the data group is 0.92, the standard deviation is 0.16, and the reliability of the measured value with sequence number 35 is the smallest, which is 0.33, less than 0.92-3*0.16=0.44, and Therefore, the measured value of sequence number 35 is temporarily eliminated and replaced by E(A)=9.55. After the replacement, the difference variance s′= of the first data group is 3.16, which is less than 3.38. Therefore, the measured value of sequence number 35 is a gross error. Repeat the above operation until the difference variance s converges to a state without gross errors. The gross errors are eliminated for each data group in turn. The results are shown in Table 2 and Figures 2 to 6 shown.

[0093] Table 1: First group of first credibility calculation values ​​of surface displacement monitoring points (m=6)

[0094] Serial number time Measured value Credibility Serial number time Measured value Credibility Serial number time Measured value Credibility 1 2005 / 5 / 18 0.00 1.00 17 2005 / 8 / 25 7.97 1.00 33 2006 / 4 / 4 12.07 0.83 2 2005 / 5 / 22 -0.78 1.00 18 2005 / 9 / 5 2.99 0.83 34 2006 / 4 / 21 10.06 1.00 3 2005 / 5 / 24 1.31 1.00 19 2005 / 9 / 15 6.46 1.00 35 2006 / 5 / 17 5.16 0.33 4 2005 / 5 / 26 -2.15 1.00 20 2005 / 9 / 24 12.15 0.50 36 2006 / 5 / 29 12.56 0.83 5 2005 / 5 / 30 0.47 0.83 21 2005 / 10 / 9 4.69 0.83 37 2006 / 6 / 7 13.04 0.83 6 2005 / 6 / 3 0.00 0.83 22 2005 / 10 / 18 3.44 0.83 38 2006 / 6 / 20 14.11 0.83 7 2005 / 6 / 12 0.71 0.83 23 2005 / 10 / 26 5.86 1.00 39 2006 / 7 / 3 16.45 1.00 8 2005 / 6 / 19 7.59 0.33 24 2005 / 11 / 4 4.70 1.00 40 2006 / 8 / 30 18.28 1.00 9 2005 / 6 / 26 -1.21 0.67 25 2005 / 11 / 15 8.19 1.00 41 2006 / 11 / 1 17.41 1.00 10 2005 / 7 / 3 0.83 1.00 26 2005 / 12 / 6 8.94 1.00 42 2007 / 1 / 14 20.94 1.00 11 2005 / 7 / 11 2.02 1.00 27 2005 / 12 / 17 10.14 1.00 43 2007 / 3 / 27 23.74 1.00 12 2005 / 7 / 16 6.09 0.83 28 2006 / 1 / 3 8.74 1.00 44 2007 / 4 / 15 21.56 1.00 13 2005 / 7 / 24 1.74 1.00 29 2006 / 2 / 14 9.48 1.00 45 2007 / 6 / 23 26.54 1.00 14 2005 / 7 / 31 3.00 1.00 30 2006 / 2 / 25 9.86 1.00 46 2007 / 8 / 31 28.21 1.00 15 2005 / 8 / 8 4.05 1.00 31 2006 / 3 / 4 12.39 1.00 47 2007 / 9 / 7 26.92 1.00 16 2005 / 8 / 16 2.49 1.00 32 2006 / 3 / 19 10.11 1.00 48 2007 / 9 / 15 26.84 1.00

[0095] Table 2: Results of eliminating gross errors by group and segment

[0096]

[0097] From Table 2 and Figures 2 to 6It can be seen that after the gross errors are eliminated and replaced by the method of the present invention, the slope monitoring data is more reasonable and scientific, and the slope monitoring can be accurately carried out to control the safety status of the reservoir bank slope and ensure the safety of the project.

Claims

1. A method for rapid processing of gross errors in slope monitoring data, characterized by: Including the following steps: 1) Obtain a total of h observation data of time series slope monitoring data, and group the observation data according to the observation time interval to obtain multiple observation data groups; the observation data includes the observation time and its corresponding observation value; set L1, L2, ..., L i ,...,L h Represents the observation value corresponding to each observation time; 2) Segment each group of observation data in step 1), and then calculate the credibility of the observed values within the segment; 3) Based on the credibility, calculate the average credibility and credibility variance of the observed values within each group, and use the average credibility and credibility variance to identify and replace the gross errors within the group; 4) Repeat steps 1) to 3) for the replaced observation data for optimization until there are no gross errors.

2. The method for rapid processing of gross errors in slope monitoring data according to claim 1 is characterized by: In step 1), the grouping method includes: dividing into g groups according to the observation frequency in different periods, the interval between two adjacent observations in each group differs by no more than 1.5 times, and renumbering the observation values ​​in the group from 1, recorded as {L1, L2, ..., L i , ..., L n }.

3. The method for rapid processing of gross errors in slope monitoring data according to claim 1 is characterized by: In step 2), the method of segmenting within the group includes: taking the observed value for which the credibility is to be calculated as the center, and taking a total of m data before and / or after the observed value for which the credibility is to be calculated within the group to form a data segment, that is, each data segment contains m + 1 data; where m ≥ 4, m < n, and n is the number of observed values within the group.

4. The method for rapid processing of gross errors in slope monitoring data according to claim 3 is characterized by: In step 2), the credibility is calculated by the following formula: in, is the credibility of the observed value; i Indicates L i Neighborhood {L||LL i |≤λ,λ>0} contains L j The number of L j For L i For other observations in the same data segment, j≠i, λ is the neighborhood radius.

5. The method for rapid processing of gross errors in slope monitoring data according to claim 4 is characterized by: The neighborhood radius λ is calculated by the following formula:

6. The method for rapid processing of gross errors in slope monitoring data according to claim 4 is characterized by: In step 2), the selection of the data segment is specified as follows: (a) When i = 1, take the first and second observation values ​​as L i The data segment where it is located and serves as ξ i Search area; (b) When When , take the 1st to 2i-1th observation values ​​as ξ i Search area; (c) When When to observations as ξ i Search area; (d) When When , take the 2i-nth to nth observation values ​​as ξ i Search area; (e) When i = n, take the n-1th and nth observation values ​​as ξ i Search area; Where m / 2 is an integer.

7. The method for rapid processing of gross errors in slope monitoring data according to claim 1 is characterized by: In step 3), the average credibility is calculated by the following formula: in, is the mean reliability value, is the credibility of the observation in step 2).

8. The method for rapid processing of gross errors in slope monitoring data according to claim 1 is characterized by: In step 3), the credibility variance is calculated by the following formula: in, is the reliability standard deviation, is the mean reliability value, is the credibility of the observation in step 2).

9. The method for rapid processing of gross errors in slope monitoring data according to claim 1 is characterized by: The method of identifying and replacing gross errors in step 3) includes: Find the observation value L with the minimum credibility i If its credibility The average reliability and the credibility variance When the following equation is satisfied: and Then for the observation value L with the minimum credibility i Temporarily remove and use E(A) to replace the observation value L with the lowest credibility i , if the difference variance s' of the data group after replacement and the difference variance s within the data group before replacement satisfy: When s′<s, the observation value L with the minimum credibility is i It is judged as a gross error and E(A) is used instead of the credibility The smallest observed value L i ; The E(A) is calculated by the following formula:

10. The method for rapid processing of gross errors in slope monitoring data according to claim 9 is characterized in that: The difference variance s is calculated by the following formula: Where n is the number of observed values within the data group.

11. The method for rapid processing of gross errors in slope monitoring data according to any one of claims 1 to 10, characterized in that: Step 4) includes: re-filtering the replaced observation data, repeating steps 1) to 3) for iteration, and realizing the optimization of the difference variance s until the difference variance s converges to a state without gross errors.

12. A system for rapid processing of gross errors in slope monitoring data, characterized in that: Used to implement the method for rapid processing of gross errors in slope monitoring data according to any one of claims 1 to 11, including: A data storage module for storing time series slope monitoring data; A data processing module for circularly processing the time series slope monitoring data to eliminate gross errors; The data processing module includes: A grouping unit for grouping the observation data of the time series slope monitoring data; A segmenting unit for segmenting the grouped observation data within the group; A calculation unit for calculating credibility, average credibility, and credibility variance; An analysis unit for identifying gross errors; A replacement unit for replacing gross errors.

13. A computer device, characterized in that: Including a memory and a processor, the memory is used for storing at least one program, and the processor is used for loading the at least one program to execute the method for rapid processing of gross errors in slope monitoring data according to any one of claims 1 to 11.

Images