Dam monitoring data trend anomaly identification method and system and electronic equipment

Abstract

The invention relates to the technical field of dam safety monitoring, in particular to a dam monitoring data trend anomaly identification method and system and electronic equipment, and the method comprises the following steps: obtaining an observation data sequence of a dam based on different monitoring modes, and carrying out the preprocessing; performing timestamp alignment on the preprocessed observation data sequences to obtain a plurality of to-be-identified data sequences; respectively calculating the variable quantity of adjacent measurement times in each to-be-identified data sequence, and determining an accumulated trend deviation degree based on the instantaneous error of the corresponding variable quantity between the to-be-identified data sequences; and comparing a preset deviation degree threshold with the accumulated trend deviation degree to obtain a trend anomaly identification result. Instantaneous tiny deviation accumulation can be amplified, the problem that a traditional difference method is insufficient in sensitivity when data changes slightly for a long time is solved, and the method is particularly suitable for scenes with small environmental quantity changes or stable dam operation. Through mutual checking of multi-source data, trend abnormity can be found more comprehensively, and misjudgment is reduced.

Description

Technical Field

[0001] This invention relates to the field of dam safety monitoring technology, and more specifically, to a method, system, and electronic device for identifying anomalies in dam monitoring data trends. Background Technology

[0002] Currently, at key monitoring points of dams, it is common to use two or more monitoring methods simultaneously, such as combining manual and automated monitoring, or operating different types of automated monitoring equipment at the same time. However, different monitoring methods vary in terms of accuracy, frequency, and stability. While automated monitoring has the advantages of high frequency and low workload, monitoring equipment may malfunction or be affected by environmental interference, leading to data anomalies. Manual monitoring, on the other hand, may be subject to human error.

[0003] Currently, most dams in China have been in operation for more than 5 years. The monitoring data of dams that have entered a stable operating period are usually quite stable. When a certain monitoring data shows a slight abnormality in the monitoring trend, how to detect it in time and ensure the reliability of the monitoring data is an urgent problem to be solved in the field of dam safety monitoring.

[0004] For dam safety monitoring, existing technologies include the difference verification method, which verifies the accuracy of monitoring data by calculating the difference between two monitoring values. When the difference exceeds a limit, it is determined that a certain monitoring data may have a problem. However, this method is not sensitive enough to detect problems in the data in a timely manner when the measured data has small long-term changes, as it relies solely on difference analysis. In addition, Chinese invention patent CN114817373A provides an intelligent identification method and system for systematic error jumps in dam safety monitoring data. Specifically, it discloses that systematic error jumps in single-source monitoring data are identified by calculating the absolute value of the slope of each measurement and the absolute value of the difference between the arithmetic mean of the observation data within a set time period before and after. This method is more effective in identifying gross errors in single-source monitoring data. Chinese invention patent CN108319664A provides a method for identifying systematic error jumps in single-source monitoring data. A method and system for identifying gross errors in safety monitoring data are disclosed, specifically disclosing a method for constructing new statistics based on observation data and their corresponding measurement time, and identifying gross errors by calculating the arithmetic mean and standard deviation of the new data sequence. This method is suitable for identifying gross errors in single-source data. The two existing technologies mentioned above are mainly for identifying gross errors or systematic error jumps in single-source monitoring data. However, when faced with multi-source data, they cannot identify trend anomalies due to the lack of specificity in cross-verification of multi-source data. Chinese invention with publication number CN118094312A provides a method, device, equipment, and medium for identifying gross errors in dam deformation data, and specifically discloses the method for identifying gross errors in deformation data by constructing a correlation model between historical deformation data and temperature and water level. This method can effectively identify gross errors and relatively obvious trend anomalies, but it is difficult to capture subtle trend deviations when the monitoring data has small long-term changes or the monitoring time is short.

[0005] In summary, existing technologies do not consider dynamic cross-verification of multi-source data, making it difficult to identify anomalies in dam monitoring data trends, especially in scenarios where environmental changes are small or dam operation time is long and monitoring data is stable. Summary of the Invention

[0006] The purpose of this invention is to provide a method, system, and electronic device for identifying anomalies in dam monitoring data trends, in order to solve the technical problems pointed out in the background art.

[0007] This invention is achieved through the following technical solution: a method for identifying anomalies in dam monitoring data, comprising the following steps: The observation data sequences of the dam were obtained based on different monitoring methods and preprocessed. The preprocessed observation data sequences are timestamped to obtain multiple data sequences to be identified. The change in adjacent measurements in each of the data sequences to be identified is calculated, and the cumulative trend deviation is determined based on the instantaneous error of the corresponding change between the data sequences to be identified. The cumulative trend deviation is compared with the preset deviation threshold to obtain the trend anomaly identification result.

[0008] According to a preferred embodiment, the preprocessing includes: According to the preset algorithm, gross errors are identified and removed from each of the observed data sequences to obtain the processing result of removing gross errors.

[0009] According to a preferred embodiment, the preprocessed observation data sequences are timestamped and aligned, specifically including: The correlation between the observed data and environmental quantities is identified. If the correlation is significant, the processing results are linearly interpolated based on a pre-built correlation model. If the correlation is not significant, spline interpolation or sliding window average interpolation is used to interpolate the processing results. The interpolated processing result is then timestamped.

[0010] According to a preferred embodiment, the instantaneous error is the absolute value of the difference in change or the square root of the difference in change.

[0011] According to a preferred embodiment, when the data sequence to be identified contains Sequence and When performing a sequence, the instantaneous error is calculated using the following expression:

[0012]

[0013]

[0014] In the above formula, Indicates the first Instantaneous error of the number of measurements, express The change in adjacent measurements within a sequence, express No. 1 in the sequence The data from the test, express No. 1 in the sequence The data from the test, express The change in adjacent measurements within a sequence, express No. 1 in the sequence The data from the test, express No. 1 in the sequence Data from the number of measurements.

[0015] According to a preferred embodiment, the cumulative trend deviation is the sum or weighted sum of the instantaneous errors of the corresponding changes between the data sequences to be identified.

[0016] According to a preferred embodiment, the calculation expression for the cumulative trend deviation is:

[0017]

[0018]

[0019] In the above formula, This indicates the number of measurements included within the set time window. Indicates the first Continuous before the measurement The sum of the instantaneous errors of each measurement. Indicates the first Instantaneous error of the number of measurements, Indicates the first Continuous before the measurement The cumulative weighted sum of instantaneous errors from each measurement. Indicates the first Weighting coefficients for the number of tests, The closer , The larger.

[0020] According to a preferred embodiment, the preset process for the deviation threshold is as follows: Collect the cumulative instantaneous error value under normal operating conditions to construct a historical data sequence; When the length of the historical data sequence meets the threshold setting condition, the mean and standard deviation of the historical data sequence are calculated respectively, and the deviation threshold is determined based on the mean and standard deviation, as shown in the following expression:

[0021] In the above formula, This represents the deviation threshold when the length of the historical data sequence meets the threshold setting condition. This represents the mean of a historical data series. The standard deviation of a historical data series This is a dynamic adjustment coefficient; When the length of the historical data sequence does not meet the threshold setting condition, the measurement accuracy of the change is determined, and the maximum theoretical error limit is determined based on the measurement accuracy. The deviation threshold is then determined based on the maximum theoretical error limit, as expressed below:

[0022] In the above formula, This represents the deviation threshold when the length of the historical data sequence does not meet the threshold setting conditions. Indicates the safety factor. express Accuracy of sequence change measurement express Accuracy of measurement of changes in the sequence.

[0023] This invention also provides a dam monitoring data trend anomaly identification system, applied to the dam monitoring data trend anomaly identification method described above. The system includes: The data acquisition and processing module is used to acquire and preprocess the observation data sequences of the dam based on different monitoring methods; The data alignment module is used to align the timestamps of the preprocessed observation data sequences to obtain multiple data sequences to be identified. The cumulative trend deviation calculation module is used to calculate the change in adjacent measurements in each of the data sequences to be identified, and to determine the cumulative trend deviation based on the instantaneous error of the corresponding change between the data sequences to be identified. The trend anomaly identification module is used to compare the cumulative trend deviation with a preset deviation threshold to obtain the trend anomaly identification result.

[0024] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the dam monitoring data trend anomaly identification method as described above.

[0025] The technical solution of the method, system and electronic equipment for identifying trend anomalies in dam monitoring data provided by the present invention has at least the following advantages and beneficial effects: (1) By measuring the trend of monitoring data by cumulative trend deviation, the instantaneous small deviation can be amplified and effectively overcome the problem of insufficient sensitivity of the traditional difference method when the data changes slowly over a long period of time. It is especially suitable for scenarios where the environmental quantity changes little or the dam is operating stably; (2) Compared with the existing gross error identification of single source data, the present invention can more comprehensively discover trend anomalies by cross-checking multi-source data, thereby reducing misjudgments caused by single equipment failure or environmental interference. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the overall process of the method for identifying anomalies in dam monitoring data provided in Embodiment 1 of the present invention; Figure 2This is a structural block diagram of the dam monitoring data trend anomaly identification system provided in Embodiment 2 of the present invention; Detailed Implementation To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0027] Example 1 This invention provides a method for identifying anomalies in dam monitoring data trends. Figure 1 This is a schematic diagram of the overall process for identifying trend anomalies in the dam monitoring data. (See attached diagram) Figure 1 As shown, the method for identifying anomalies in dam monitoring data includes the following steps: Step S1: Data acquisition and processing; In this embodiment, this step aims to obtain the dam's observation data sequence based on different monitoring methods, and further preprocess it to resolve frequency differences between the data. The specific process is as follows; Step S101: Collect raw observation data sequences from at least two different monitoring methods, such as manual monitoring data sequences, automated monitoring data sequences, and monitoring data sequences from sensors of different brands. In this embodiment, the following examples use the collection of manual monitoring data sequences and automated monitoring data sequences, but no specific limitations are imposed.

[0028] Step S102: Preprocess the collected manual monitoring data sequence and automated monitoring data sequence, specifically including: according to the preset algorithm, identify gross errors in each of the observation data sequences and remove gross error data to obtain the processing result of removing gross error data.

[0029] In some implementations, the preset algorithm may be selected to classify gross errors into a functional model or into a stochastic model. The basic idea of ​​classifying gross errors into a functional model is to detect and locate gross errors before performing least squares adjustment, remove observations containing gross errors, and obtain a set of relatively cleaned observations. Then, the least squares adjustment is performed using this set of cleaned observations. The basic idea of ​​classifying gross errors into a stochastic model is to continuously change the weights or variances of the observations based on the results of successive iterative adjustment, so that the weights of the gross error observations tend to zero or infinity. This method can ensure that the estimated parameters are less affected by model errors, especially gross errors.

[0030] Step S2: Time alignment; In this embodiment, this step aims to address the issue of missing data obtained from different monitoring methods by interpolating the data to achieve time alignment, thereby resolving the difficulty of cross-verification caused by different monitoring frequencies. The specific process is as follows: Step S201: Timestamp alignment is performed on each of the preprocessed observation data sequences; In some implementations, the correlation between the observed data and environmental quantities is identified. If the correlation is significant, the processing results are linearly interpolated based on a pre-built correlation model. If the correlation is not significant, spline interpolation or sliding window average interpolation is used to interpolate the processing results.

[0031] Step S202: Timestamp alignment is performed on the interpolated processing result to obtain multiple data sequences to be identified.

[0032] Step S3: Calculate the cumulative trend deviation; In this embodiment, this step aims to calculate the cumulative trend deviation using the changes in adjacent measurements within the data sequence to be identified. The specific process is as follows: Step S301: Calculate the change in adjacent measurements in each of the data sequences to be identified; When the data sequence to be identified contains Sequence and When analyzing a sequence, the expression for calculating the change in adjacent measurements within the data sequence to be identified is as follows:

[0033]

[0034] In the above formula, express The change in adjacent measurements within a sequence, express No. 1 in the sequence The data from the test, express No. 1 in the sequence The data from the test, express The change in adjacent measurements within a sequence, express No. 1 in the sequence The data from the test, express No. 1 in the sequence Data from the number of measurements.

[0035] Step S302: Calculate the instantaneous error of the corresponding changes between each of the data sequences to be identified; In this embodiment, the instantaneous error is the absolute value of the difference in changes or the square root of the difference in changes, expressed as follows:

[0036] In the above formula, Indicates the first Instantaneous error of the number of measurements.

[0037] Step S303: Calculate the cumulative trend deviation based on the loss error; In this embodiment, the cumulative trend deviation is the sum or weighted sum of the instantaneous errors of the corresponding changes between the data sequences to be identified, as expressed below:

[0038]

[0039]

[0040] In the above formula, This indicates the number of measurements included within the set time window. Indicates the first Continuous before the measurement The sum of the instantaneous errors of each measurement. Indicates the first Instantaneous error of the number of measurements, Indicates the first Continuous before the measurement The cumulative weighted sum of instantaneous errors from each measurement. Indicates the first Weighting coefficients for the number of tests, The closer , The larger the value, the higher the weighting of recent data, reflecting a greater focus on recent trends.

[0041] Step S4: Identify trend anomalies; In this embodiment, this step aims to identify trend anomalies based on the cumulative trend deviation output in step S3. The specific process is as follows: Step S401: Preset deviation threshold; First, collect the cumulative instantaneous error values ​​under normal operating conditions to construct a historical data sequence; When the length of the historical data sequence meets the threshold setting condition, the mean and standard deviation of the historical data sequence are calculated respectively, and the deviation threshold is determined based on the mean and standard deviation, as shown in the following expression:

[0042] In the above formula, This represents the deviation threshold when the length of the historical data sequence meets the threshold setting condition. This represents the mean of a historical data series. The standard deviation of a historical data series This is the dynamic adjustment coefficient.

[0043] When the length of the historical data sequence does not meet the threshold setting condition, the measurement accuracy of the change is determined, and the maximum theoretical error limit is determined based on the measurement accuracy. The deviation threshold is then determined based on the maximum theoretical error limit, as expressed below:

[0044] In the above formula, This represents the deviation threshold when the length of the historical data sequence does not meet the threshold setting conditions. Indicates the safety factor. express Accuracy of sequence change measurement express Accuracy of measurement of changes in the sequence.

[0045] Step S402: Compare the cumulative trend deviation with the preset deviation threshold to obtain the trend anomaly identification result.

[0046] In some implementations, when the cumulative trend deviation exceeds a preset deviation threshold, it is determined that there is a problem with the data of a certain measurement; further, the reliability of subsequent monitoring data can be ensured by verifying whether the on-site instruments and equipment or manual observation methods are correct.

[0047] In summary, this embodiment measures the monitoring data trend by quantifying the trend deviation through cumulative trend deviation, which can amplify the instantaneous small deviations and effectively overcome the insufficient sensitivity of the traditional difference method when the data changes slowly over a long period of time. It is especially suitable for scenarios where environmental changes are small or the dam is operating stably. Compared with the existing gross error identification based on single-source data, this invention can more comprehensively detect trend anomalies through cross-verification of multi-source data, thereby reducing misjudgments caused by single equipment failure or environmental interference.

[0048] Example 2 This embodiment, based on the technical solution provided in Embodiment 1, provides a dam monitoring data trend anomaly identification system. This system applies the dam monitoring data trend anomaly identification method described in Embodiment 1. (See [link to previous document]). Figure 2 As shown, the system includes: The data acquisition and processing module is used to acquire and preprocess the observation data sequences of the dam based on different monitoring methods; The data alignment module is used to align the timestamps of the preprocessed observation data sequences to obtain multiple data sequences to be identified. The cumulative trend deviation calculation module is used to calculate the change in adjacent measurements in each of the data sequences to be identified, and to determine the cumulative trend deviation based on the instantaneous error of the corresponding change between the data sequences to be identified. The trend anomaly identification module is used to compare the cumulative trend deviation with a preset deviation threshold to obtain the trend anomaly identification result.

[0049] The functions of each module of the dam monitoring data trend anomaly identification system in this embodiment are the same as those in the embodiment of the dam monitoring data trend anomaly identification method, and the technical effects are the same, so they will not be repeated here.

[0050] Example 3 This embodiment is based on the technical solution provided in Embodiment 1, and provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the dam monitoring data trend anomaly identification method as described in Embodiment 1.

[0051] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for identifying anomalies in dam monitoring data, characterized in that, The method comprises the following steps: obtaining observation data sequences of a dam based on different monitoring methods and preprocessing; aligning time stamps of each of the preprocessed observation data sequences to obtain a plurality of to-be-identified data sequences; calculating variation amounts of adjacent measurement times in each of the to-be-identified data sequences respectively, and determining cumulative trend deviation degrees based on instantaneous errors of corresponding variation amounts between the to-be-identified data sequences; comparing the preset deviation degree threshold value with the cumulative trend deviation degrees to obtain a trend anomaly identification result.

2. The method of claim 1, wherein the dam monitoring data trend anomaly identification method is characterized by, The preprocessing comprises: performing gross error identification on each of the observation data sequences according to a preset algorithm and eliminating gross error data to obtain a processing result of the eliminated gross error data.

3. The method of claim 2, wherein the step of identifying the trend anomaly in the dam monitoring data comprises the steps of: aligning time stamps of each of the preprocessed observation data sequences, specifically comprising: ​ performing correlation identification on observation data and environmental quantities, if the correlation is significant, performing linear interpolation processing on the processing result based on a pre-constructed correlation model, if the correlation is not significant, performing interpolation processing on the processing result by using spline interpolation or sliding window average interpolation; aligning time stamps of the processing result after interpolation processing.

4. The method of claim 3, wherein the dam monitoring data trend anomaly identification method is characterized by, The instantaneous error is an absolute value of a variation amount difference or a square root of the variation amount difference.

5. The method of claim 4, wherein the method further comprises: When the data sequence to be identified comprises a sequence and a sequence, the calculation expression of the instantaneous error is: In the above formulae, represents the first instantaneous error of the measurement, represents the change between adjacent measurements in the sequence, represents the data of the first measurement in the sequence, represents the data of the first measurement in the sequence, represents the change between adjacent measurements in the sequence, represents the data of the first measurement in the sequence, represents the data of the first measurement in the sequence.

6. The method of claim 5, wherein the method further comprises: The cumulative trend deviation degree is a cumulative sum or a weighted sum of instantaneous errors of corresponding variation amounts between the to-be-identified data sequences.

7. The method of claim 6, wherein the method further comprises: The calculation expression of the cumulative trend deviation degree is: In the above formulae, represents the number of measurements included in the set time window, represents the instantaneous error of the measurement before the measurement, and represents the instantaneous error of the measurement, represents the instantaneous error of the measurement before the measurement, and represents the weight coefficient of the measurement, the closer to , the greater.

8. The method of claim 1 to 7, wherein The preset process of the deviation degree threshold value is as follows: collecting instantaneous error cumulative values under normal working conditions to construct a historical data sequence; when the length of the historical data sequence meets a threshold setting condition, calculating the mean and the standard deviation of the historical data sequence respectively, determining the deviation degree threshold value based on the mean and the standard deviation, and the expression is as follows: In the above formula, denotes a deviation degree threshold value when the length of the history data sequence satisfies a threshold setting condition, denotes a mean value of the history data sequence, denotes a standard deviation of the history data sequence, is a dynamic adjustment coefficient; when the length of the historical data sequence does not meet the threshold setting condition, determining the measurement accuracy of the variation amount, determining the maximum theoretical error limit value based on the measurement accuracy, and determining the deviation degree threshold value based on the maximum theoretical error limit value, and the expression is as follows: In the above formula, denotes a deviation degree threshold value when the length of the history data sequence does not satisfy a threshold setting condition, denotes a safety factor, denotes a change amount measurement accuracy of the sequence, denotes a change amount measurement accuracy of the sequence.

9. A dam monitoring data trend anomaly identification system, characterized by, The application is applied to the dam monitoring data trend anomaly identification method and system according to any one of claims 1 to 8, and the system comprises: a data acquisition and processing module configured to obtain observation data sequences of a dam based on different monitoring methods and preprocess the observation data sequences; a data alignment module configured to align time stamps of each of the preprocessed observation data sequences to obtain a plurality of to-be-identified data sequences; a cumulative trend deviation degree calculation module configured to calculate variation amounts of adjacent measurement times in each of the to-be-identified data sequences respectively, and determine cumulative trend deviation degrees based on instantaneous errors of corresponding variation amounts between the to-be-identified data sequences; a trend anomaly identification module configured to compare a preset deviation degree threshold value with the cumulative trend deviation degrees to obtain a trend anomaly identification result.

10. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the dam monitoring data trend anomaly identification method according to any one of claims 1 to 8 when executing the computer program.

Images