Method and device for processing monitoring data of safety of hydropower station

Abstract

The present disclosure provides a hydropower station safety monitoring data processing method and device, and relates to the technical field of data preprocessing. The method comprises the following steps: obtaining target monitoring data of a target index for monitoring the safety of a hydropower station; if there are missing points in the target monitoring data, filling the missing points with values based on a weighted average filling rule corresponding to a cycle type; if there are burrs, determining the width of a burr removal sliding window in multiple dimensions based on the data length of the filled target monitoring data and the number of valid data in the sliding window, or the data length, the fluctuation type and the number of valid data in the sliding window; calculating the median of the valid data of each data point corresponding to the burr removal sliding window; performing primary burr removal based on the median of each burr removal sliding window; and performing secondary burr removal based on the statistical characteristics after the primary burr removal. Based on the technical scheme, burrs can be effectively removed while ensuring the effective trend of the data.

Description

Technical Field

[0001] This disclosure relates to the field of data preprocessing technology, and in particular to a method and apparatus for processing monitoring data for hydropower station safety. Background Technology

[0002] Safety monitoring of hydropower stations is a crucial link in ensuring the stable operation of core facilities such as dams and generating units, and the accuracy of its monitoring data is directly related to the operational safety of hydropower stations.

[0003] Hydropower stations are typically located in complex environments. Factors such as strong water flow, geological structure changes, extreme weather, or electromagnetic interference can adversely affect monitoring data. For example, due to harsh sensor installation environments, aging transmission lines, and extreme weather (heavy rain, lightning strikes), monitoring data often shows gaps for a single day or several consecutive days. Water flow, unit vibration, and electromagnetic interference can cause sudden spikes in monitoring data.

[0004] Currently, to address the aforementioned adverse effects, methods for filling missing points typically employ linear interpolation or direct mean filling. However, the filled values ​​deviate significantly from the actual data. Traditional methods for handling spikes, such as a 5-day window, use fixed-window filtering. These methods either excessively smooth the effective trend or fail to completely eliminate high-frequency spikes, affecting the accurate identification of safety hazards. Therefore, current methods for processing hydropower station safety monitoring data result in unreliable processed results that cannot accurately reflect the true operating status of the hydropower station.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] To overcome the problems existing in related technologies, this disclosure provides a method and apparatus for processing monitoring data for hydropower station safety, which can effectively remove glitch while ensuring the effective trend of the data.

[0007] According to a first aspect of the present disclosure, a method for processing monitoring data for hydropower station safety is provided. The method includes: acquiring target monitoring data for target indicators of hydropower station safety; identifying whether there are missing points in the target monitoring data; if there are missing points in the target monitoring data, filling the missing points with values ​​based on a weighted average filling rule corresponding to the period type, wherein the period type indicator is a periodic monitoring indicator or a non-periodic monitoring indicator; identifying whether there are spikes in the filled target monitoring data; if there are spikes, adaptively determining the width of the spike removal sliding window in multiple dimensions based on the data length of the filled target monitoring data and the number of valid data in the sliding window, or the data length, fluctuation type, and the number of valid data in the sliding window, wherein the fluctuation type indicator is a slow-changing indicator or a fluctuation indicator; calculating the median of the valid data in the spike removal sliding window corresponding to each data point, performing initial spike removal based on the median of each spike removal sliding window, and performing secondary spike removal based on the statistical characteristics after the initial spike removal.

[0008] According to a second aspect of the present disclosure, a monitoring data processing device for hydropower station safety is provided. The monitoring data processing device includes: a data acquisition module, a missing point identification module, a missing point filling module, a burr identification module, a burr removal sliding window construction module, and a burr removal module. The data acquisition module is used to acquire target monitoring data for target indicators of hydropower station safety. The missing point identification module is used to identify whether missing points exist in the target monitoring data. The missing point filling module is used to fill in values ​​at the missing locations based on a weighted average filling rule corresponding to the period type if missing points exist in the target monitoring data. The period type indicator is either a periodic monitoring indicator or a non-periodic monitoring indicator. The system comprises: a burr identification module for identifying whether burrs exist in the filled target monitoring data; a burr removal sliding window construction module for adaptively determining the width of the burr removal sliding window based on the data length of the filled target monitoring data and the number of valid data within the sliding window, or the data length, fluctuation type, and the number of valid data within the sliding window, if burrs exist; and a burr removal module for calculating the median of valid data within the burr removal sliding window for each data point, performing initial burr removal based on the median of the burr removal sliding window, and performing secondary burr removal based on the statistical characteristics after the initial burr removal.

[0009] According to a third aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the monitoring data processing method for hydropower station safety as described in the first aspect.

[0010] According to a fourth aspect of the present disclosure, a computer device apparatus is provided, comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein the computer-readable instructions, when executed by the processor, implement the monitoring data processing method for hydropower station safety as described in the first aspect.

[0011] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0012] In this embodiment, target monitoring data for the safety indicators of a hydropower station are first acquired. Then, it is identified whether there are missing points in the target monitoring data. If missing points exist, values ​​are filled into the missing locations based on the weighted average filling rule corresponding to the period type. That is, for periodic and non-periodic indicators, different weighted average filling rules can be adaptively selected for various monitoring indicators according to the period type to fill missing points, which can improve the filling accuracy. After filling, it is identified whether there are spikes in the filled target monitoring data. If spikes exist, the width of the spike removal sliding window is adaptively determined from multiple dimensions based on the data length of the filled target monitoring data and the number of valid data in the sliding window, or the data length, fluctuation type, and the number of valid data in the sliding window. By adaptively constructing the width of the spike removal sliding window from multiple dimensions, the determined spike removal sliding window can better match the characteristics of the actual monitoring data, thereby avoiding the bottleneck problem of spike removal accuracy caused by inappropriate selection of the sliding window width. Initial spurious removal based on the median of the spurious removal sliding window better reflects the short-term trend characteristics of the monitoring data. Subsequent secondary spurious removal based on statistical characteristics further refines the short-term distribution of the data. This two-round spurious removal process, employing different methods, allows for the precise removal of sudden spurs while preserving key trend changes such as abnormal dam settlement. Verification with real monitoring data shows that the spur residual rate is below 3%. This multi-indicator differentiated processing approach, based on the periodicity and fluctuation type of the indicators, is adaptable to various core monitoring indicators of hydropower stations, such as dam displacement, seepage flow, seepage pressure, and opening / closing degree. It eliminates the need for developing separate algorithms for individual indicators, reducing system deployment costs and providing precise data support for hydropower station safety assessment, fault early warning, and operation and maintenance decisions.

[0013] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0014] The accompanying drawings, which are incorporated in and form part of this disclosure, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0015] Figure 1 This is a schematic diagram of a monitoring data processing system architecture for hydropower station safety provided in an embodiment of this disclosure.

[0016] Figure 2 This is a flowchart illustrating a method for processing monitoring data for the safety of a hydropower station, as provided in an embodiment of this disclosure.

[0017] Figure 3 This is a schematic diagram of the raw data process of a deep-hole strain gauge monitoring data of a power station, provided as an embodiment of this disclosure.

[0018] Figure 4 This is a schematic diagram of raw data and missing location markers provided in an embodiment of this disclosure.

[0019] Figure 5 This is a schematic diagram illustrating the position of a demonstration year marker in a raw data time series provided in an embodiment of this disclosure.

[0020] Figure 6 This is a schematic diagram of a seasonal coefficient distribution provided in an embodiment of the present disclosure.

[0021] Figure 7 This is a schematic diagram illustrating the seasonal correction filling effect provided in an embodiment of this disclosure.

[0022] Figure 8 This is one of the schematic diagrams of the original data process line provided in the embodiments of this disclosure.

[0023] Figure 9 This is one of the process flow diagrams provided in the embodiments of this disclosure after deburring.

[0024] Figure 10 This is the second schematic diagram of the original data process line provided in the embodiments of this disclosure.

[0025] Figure 11 This is the second schematic diagram of the process line after deburring provided in the embodiments of this disclosure.

[0026] Figure 12 This is the third schematic diagram of the original data process line provided in the embodiments of this disclosure.

[0027] Figure 13 This is the third schematic diagram of the process line after burr removal provided in the embodiments of this disclosure.

[0028] Figure 14 This is the fourth schematic diagram of the original data process line provided in the embodiments of this disclosure.

[0029] Figure 15 This is the fourth schematic diagram of the process line after burr removal provided in the embodiments of this disclosure.

[0030] Figure 16This is a hardware structure diagram of the computer equipment used in the monitoring data processing system for hydropower station safety provided in this embodiment of the disclosure.

[0031] Figure 17 This is a schematic diagram of the structure of a monitoring data processing device for the safety of a hydropower station, provided in an embodiment of this disclosure. Detailed Implementation

[0032] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0033] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0034] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0035] The embodiments of this disclosure will now be described in detail.

[0036] Figure 1 A schematic diagram of a monitoring data processing system architecture for hydropower station safety, applicable to embodiments of this disclosure, is shown. Figure 1As shown, the monitoring data processing system architecture may include: a data receiving unit 101, a core processing unit 102, and a result display unit 103. The data receiving unit 101 includes an Excel (spreadsheet) data import interface 1011, a database data access interface 1012, and a manual input interface 1013; the data receiving unit 101 is used to read the monitoring data to be processed. Specifically, the monitoring data processing system can directly read monitoring data from locally stored spreadsheets, or read data by configuring connections to other monitoring subsystems or connecting to a database, and obtain manually entered monitoring data through the manual input interface 1013. Multiple data entry methods enhance the flexibility of the device.

[0037] The core processing unit 102 includes: a data verification unit 1021, an indicator identification unit 1022, a missing point filling unit 1023, an outlier removal unit 1024, a sliding window generation unit 1025, a spike removal unit 1026, and a Kalman filtering and trend extraction unit 1027. The data verification unit 1021 performs a first preprocessing on the acquired data, which includes at least one of blank line filtering, non-data-type data filtering, and date-based deduplication. After successful data reading, the first preprocessing is performed on the read data to obtain a set of monitoring time series data arranged in ascending date order. For example, after reading the data, the monitoring data processing system automatically extracts the corresponding data columns, such as the observation date, the monitoring value, automatically filters blank lines and non-data-type data, parses the date format, sorts the data in ascending date order, removes duplicate dates, and retains the last valid value of the duplicate date. The indicator identification unit 1022 identifies the period type and fluctuation type of the indicator. The missing point filling unit 1023 identifies and fills in the values ​​of missing points. The outlier removal unit 1024 is used to pre-remove outliers from the data after filling missing points by setting a threshold and applying Layda's rule. The sliding window generation unit 1025 is used to generate a sliding window for removing spikes, including generating a spike removal sliding window and a statistical sliding window. The spike removal unit 1026 is used to remove spikes from the monitoring data based on the sliding window. The Kalman filtering and trend extraction unit 1027 is used to adaptively initialize the Kalman filter parameters on the data after spike removal and to extract trends based on the adaptively initialized Kalman filter parameters.

[0038] The results display unit 103 includes: a results verification unit 1031, a results visualization unit 1032, and a storage and download unit 1033. The results verification unit 1031 is used to verify the removal of glitch from the monitoring data, the results visualization unit 1032 is used to visually display the removal results to the user, and the storage and download unit 1033 is used to store and download the monitoring data after glitch removal and the trends extracted based on the glitch removal.

[0039] like Figure 2 As shown, Figure 2 The flowchart of a monitoring data processing method for hydropower station safety provided in this embodiment includes the following steps S201 to S206.

[0040] S201. The monitoring data processing system acquires target monitoring data for the target indicators of hydropower station safety.

[0041] For example, the target indicators are those used to monitor the safety of hydropower stations, which may include: dam displacement, seepage flow, seepage pressure, opening and closing degree, etc.

[0042] In this embodiment of the disclosure, monitoring data from a deep-hole strain gauge at a power station is used as an example for illustration. Figure 3 This is a schematic diagram of the raw data flow of deep-hole strain gauge monitoring data from a power plant, provided as an embodiment of this disclosure. Figure 3 The diagram shown illustrates a process flow composed of raw data from historical monitoring data between June 1, 2018, and December 1, 2023. Figure 3 As shown in the figure, the historical monitoring data contains missing monitoring time points and a large number of spikes.

[0043] S202. The monitoring data processing system identifies whether there are missing points in the target monitoring data.

[0044] Missing points include isolated missing points and consecutive missing segments.

[0045] For example, it can be determined whether there are missing monitoring data points at any given time based on the raw monitoring data. If so, it is determined that there are missing points.

[0046] First, a time index is constructed based on the data acquisition time series of the monitoring indicators. Specifically, the monitoring equipment provides an input configuration interface, typically configuring the data acquisition frequency, usually with default values, such as once every half day, once a day, once a week, or once a month. Users can modify these settings directly through the interface. The monitoring data processing system generates a continuous, equally spaced baseline time axis based on the acquisition frequency of the monitoring indicators and the start and end times of the records. Then, a full comparison is performed between the baseline time axis and the actual data sequence. If there is no data in the actual sequence corresponding to a certain time point on the baseline time axis, or if data exists but is null (e.g., NULL, NaN, or an invalid value), or if the value exceeds the sensor's range (e.g., -999 to 999), that time point is marked as a missing point. After marking the missing points, the monitoring data processing system sequentially scans the marked missing points on the baseline time axis. If the time interval between two adjacent missing points is equal to a single acquisition cycle, then these two missing points are considered consecutive missing points. If the duration of consecutive missing points exceeds a set threshold, the consecutive missing points are marked as missing segments. For example, if consecutive missing points last for more than 3 collection cycles, they are classified as missing segments; otherwise, they are considered isolated missing points. The monitoring data processing system records the start time, duration, and type of each missing segment, providing crucial information for subsequent numerical filling of differentiated missing points.

[0047] For example, Figure 4 A schematic diagram of original data and missing location markers provided for embodiments of this disclosure, such as... Figure 4 As shown in the figure, the straight line represents the original data (including spikes), and the cross indicates missing locations. It can be seen from the figure that the original data includes isolated missing points and consecutive missing segments.

[0048] S203. If there are missing points in the target monitoring data, the monitoring data processing system fills in the missing points with values ​​based on the weighted average filling rule corresponding to the period type.

[0049] Among them, the cycle type indicator is either a periodic monitoring indicator or a non-periodic monitoring indicator.

[0050] It is understandable that hydropower safety monitoring indicators typically include periodic and non-periodic monitoring indicators, and the weighted average filling rules for missing points for monitoring indicators of different periodic types are different.

[0051] It should be noted that, for the convenience of subsequent description, in the embodiments of this disclosure, when describing the processing steps after filling, the filling value of the missing point is also described as the monitoring value.

[0052] S204. The monitoring data processing system identifies whether there are burrs in the filled target monitoring data.

[0053] It should be noted that, in the embodiments of this disclosure, a glitch refers to an abnormal data point in which the measured value changes abruptly within a very short time, and the data trend before and after the change point immediately recovers.

[0054] For example, the monitoring data processing system determines whether the monitoring value at a monitoring time is a spike by comparing the difference in amplitude between the first monitoring time point and the adjacent monitoring time points before and after it, as well as the difference in amplitude between monitoring time points during the same period.

[0055] Specifically, if the monitoring value of the index at time t If both conditions 1 and 2 are met, then the monitoring value at time t is determined. For burrs.

[0056] Condition 1: ;

[0057] in, Indicates time The monitoring value, Let represent the standard deviation of all monitoring data within the four monitoring time points Q before time t, time t, and Q after time t, for a total of 2Q+1 monitoring time points. This represents the sensitivity coefficient, which can be set manually, typically between 2.5 and 3.5. For example, It can take the value 3.

[0058] Condition 2: The monitored value at time t deviates from the overall trend of other data within the local time window.

[0059] Specifically, The value deviates from the local trend baseline revealed by the local time window, and the degree of deviation is far beyond the normal fluctuation range within that local time window.

[0060] The width of the local time window is a preset width, which can be adjusted as needed. For example, it can be adjusted according to the monitoring length and monitoring indicators. This embodiment does not specifically limit this.

[0061] S205. If burrs exist, the monitoring data processing system adaptively determines the width of the burr removal sliding window based on the data length of the filled target monitoring data and the number of valid data in the sliding window, or the data length, fluctuation type, and the number of valid data in the sliding window.

[0062] Among them, the volatility type indicator is either a slow-change indicator or a volatility indicator.

[0063] Slow-changing indicators refer to physical quantities whose monitored data values ​​and changes over time are slow, stable, and continuous, without rapid oscillations or abrupt changes.

[0064] A volatility index refers to a physical quantity of monitored data that exhibits non-monotonic, reciprocal changes around its long-term trend or benchmark value in a time series.

[0065] Whether an indicator is a volatility indicator is determined by monitoring the frequency and rate of change of local extreme values ​​in the data. If the indicator reverses direction multiple times within a unit of time; for example, crossing above / below the average line, and the rate of change between adjacent extreme points exceeds the slow change threshold, such as the hourly change being more than three times its long-term daily average change, then the indicator is classified as a volatility indicator.

[0066] S206. The monitoring data processing system calculates the median of the effective data within the spur removal sliding window corresponding to each data point, performs initial spur removal based on the median of each spur removal sliding window, and performs secondary spur removal based on the statistical characteristics after the initial spur removal.

[0067] This disclosure provides a method for processing monitoring data on hydropower station safety. First, target monitoring data for the target indicators of hydropower station safety is acquired. Then, missing points are identified in the target monitoring data. If missing points exist, values ​​are filled into the missing locations based on a weighted average filling rule corresponding to the period type. In other words, for both periodic and non-periodic indicators, different weighted average filling rules can be adaptively selected for various monitoring indicators according to the period type to fill missing points, improving the filling accuracy. After filling, it is identified whether there are spikes in the filled target monitoring data. If spikes exist, the width of the spike removal sliding window is adaptively determined from multiple dimensions based on the data length of the filled target monitoring data and the number of valid data points within the sliding window, or the data length, fluctuation type, and the number of valid data points within the sliding window. By adaptively constructing the width of the spike removal sliding window from multiple dimensions, the determined spike removal sliding window can better match the characteristics of the actual monitoring data, thereby avoiding the bottleneck problem of spike removal accuracy caused by inappropriate selection of the sliding window width. Initial spurious removal based on the median of the spurious removal sliding window better reflects the short-term trend characteristics of the monitoring data. Subsequent secondary spurious removal based on statistical characteristics further refines the short-term distribution of the data. This two-round spurious removal process, employing different methods, allows for the precise removal of sudden spurs while preserving key trend changes such as abnormal dam settlement. Verification with real monitoring data shows that the spur residual rate is below 3%. This multi-indicator differentiated processing approach, based on the periodicity and fluctuation type of the indicators, is adaptable to various core monitoring indicators of hydropower stations, such as dam displacement, seepage flow, seepage pressure, and opening / closing degree. It eliminates the need for developing separate algorithms for individual indicators, reducing system deployment costs and providing precise data support for hydropower station safety assessment, fault early warning, and operation and maintenance decisions.

[0068] Optionally, in the hydropower station safety monitoring data processing method provided in the embodiments of this disclosure, the above-mentioned S203 may specifically include the following S203a or S203b.

[0069] S203a. If the target indicator is a non-periodic monitoring indicator, the monitoring data processing system fills in the values ​​of each missing point in the missing segment based on the mean of the effective data in the first time period to the left of the missing segment, the mean of the effective data in the second time period to the right of the missing segment, the length of the missing segment, and the position of the missing point in the missing segment.

[0070] The duration of both the first and second time periods is a first preset duration.

[0071] For example, the filling value of the kth missing point of the missing segment of the non-periodic monitoring indicator is determined based on formula (1).

[0072] ; Formula (1)

[0073] in, Indicates the first missing segment Fill value for each missing point Indicates the fill factor. , Indicates the length of the missing segment. This represents the mean of the valid data in the first time interval to the left of the missing segment. This represents the mean of the valid data in the second time period to the right of the missing segment.

[0074] For example, if the first preset duration is 30 days, then the mean of the 30 days of valid data to the left of the missing segment and the mean of the 30 days of valid data to the right of the missing segment are obtained.

[0075] It is understandable that, through formula (1), the filling value of non-periodic monitoring indicators can be adjusted according to the change of missing positions, which can fit the slow change trend.

[0076] S203b. If the target indicator is a periodic monitoring indicator, the monitoring data processing system fills in the missing points in the missing segment with values ​​based on the mean of the effective data of the first time period to the left of the missing segment, the mean of the effective data of the second time period to the right of the missing segment, the first seasonal coefficient corresponding to the first time period, and the second seasonal coefficient corresponding to the second time period.

[0077] For example, the kth missing point of the missing segment of the periodic monitoring indicator is filled based on formula (2).

[0078] ; Formula (2)

[0079] in, This represents the coefficient for the first season corresponding to the first time period. This represents the second seasonal coefficient corresponding to the second time period.

[0080] It should be noted that isolated missing points can also be filled with values ​​using the method described above, in which case the length of the missing segment will be... =1, formulas (1) and (2) are still applicable, which can ensure the integrity and consistency of the filling.

[0081] Figure 5 This is a schematic diagram illustrating the position of a demonstration year marker in a raw data time series provided in an embodiment of this disclosure, such as... Figure 5 As shown, the demonstration years are from July to September 2022. Figure 6 This is a schematic diagram of the seasonal coefficient distribution provided in an embodiment of the present disclosure. Figure 7 This is a schematic diagram illustrating the seasonal correction filling effect provided in an embodiment of the present disclosure, wherein, Figure 7 For based on Figure 6Seasonal coefficient in Figure 5 Showcase of seasonal fill effects from July to September 2022.

[0082] It should be noted that, Figure 7 The interim output results of the above-described process are presented. First, through... Figure 5 By plotting the original data process curve, a significant periodic variation pattern was observed. Based on this finding, the monthly seasonal coefficient was calculated and generated, and its numerical results are as follows: Figure 6 As shown in the figure, the monthly coefficient distribution is concentrated, exhibiting a significant and stable periodic pattern. Furthermore, regarding the missing data segment identified in historical data from July to September 2022, see... Figure 7 The identifier, using the aforementioned seasonal coefficient and the filling formula disclosed herein, generates a reasonable filling value. Figure 7 Finally, the connection between the filled values ​​and the original data was demonstrated. The results showed that the filled data perfectly continued the original trend and periodicity, proving the completeness and practicality of this method.

[0083] Based on this scheme, missing segments in the data can be identified, and differentiated weighted average filling strategies can be adopted according to the type of monitoring indicator. For non-periodic monitoring indicators, the filling can be performed by combining the mean before and after the missing position, the length of the missing data, and the position of the missing point in the continuous missing segment. For periodic monitoring indicators, a seasonal correction factor can be incorporated, that is, the seasonal coefficient before and after the missing segment can be introduced, so that the filling value gradually changes with the missing position, conforming to the slow change trend. This scheme creatively introduces a seasonal coefficient. Compared with the limitations of traditional linear interpolation, which only relies on local endpoint trends and completely ignores the inherent periodicity of the indicator, or complex seasonal decomposition methods, which have high requirements for model and data integrity and insufficient computational stability, this scheme directly embeds the statistically obtained seasonal coefficient into the determination process of the weighted missing point filling value. This makes the filling value not only reflect the local trend before and after the missing point, but also conform to the historical change pattern of the same period. This method is particularly suitable for monitoring indicators with significant annual cycle characteristics, such as hydropower station seepage flow and temperature deformation. It can restore the periodic fluctuation pattern that conforms to physical laws within long missing segments, thereby significantly improving the accuracy and reliability of filling missing points of periodic monitoring indicators.

[0084] Optionally, in the hydropower station safety monitoring data processing method provided in the embodiments of this disclosure, before the above-mentioned S203, the following steps S207 to S210 may also be included.

[0085] S207. The monitoring data processing system acquires historical monitoring data of the target indicators within the second preset time period.

[0086] For example, the second preset duration is at least greater than 3 years. For instance, complete historical monitoring data of 3 to 5 years can be selected.

[0087] S208. The monitoring data processing system calculates the baseline value for each month of each year based on the actual monitoring values ​​of each month.

[0088] S209. The monitoring data processing system calculates the monthly coefficient for each year based on the actual monitoring values ​​and benchmark values ​​for each month within a year.

[0089] For example, the monthly coefficient for each year is calculated using the ratio of the actual monitored value to the baseline value.

[0090] S210. The monitoring data processing system obtains the seasonal coefficient corresponding to each month based on the average of the monthly coefficients of the same month in all years within the second preset time period.

[0091] Specifically, the coefficients for the same month across all years are averaged to obtain 12 coefficients: January coefficient, February coefficient, and so on up to December coefficient. These coefficients represent the fluctuation pattern of each month relative to the average level. For example, the coefficient for summer months (such as July) is usually greater than 1 (above average). The coefficient for winter months (such as January) is usually less than 1 (below average).

[0092] In actual filling: Use the historical average coefficient of the month corresponding to the left of the missing segment. Use the historical average coefficient of the month corresponding to the right of the missing segment.

[0093] For example, Table 1 shows the seasonal coefficients calculated based on strain gauge data from a certain power station. The seasonal coefficients for different parameters from different power stations are different.

[0094] Table 1 - Seasonal Coefficients

[0095]

[0096] Based on this scheme, the baseline value for each month can be calculated using the actual monitoring values ​​of each month based on historical monitoring data. Then, based on the monitoring values ​​and the baseline values, the monthly coefficient corresponding to the historical monitoring data for each year can be calculated. Finally, the monthly coefficients of all available years can be averaged to obtain a seasonal coefficient that accurately represents the trend of each month. This seasonal coefficient can be used to adjust the missing values ​​of periodic monitoring indicators, thereby making the determined missing values ​​closer to the true values.

[0097] Optionally, in the hydropower station safety monitoring data processing method provided in the embodiments of this disclosure, the above-mentioned S208 may specifically include S208a, S208b and S208c, or S208a, S208b and S208d.

[0098] S208a The monitoring data processing system arranges the complete daily historical monitoring data within the second preset time period in chronological order to obtain a one-dimensional time series.

[0099] It is understandable that sorting historical monitoring data in chronological order can ensure that the monitoring value sequence is not out of time.

[0100] S208b The monitoring data processing system groups the daily historical monitoring data by natural month, calculates the arithmetic mean of all daily historical monitoring data in each month, and obtains the monthly mean sequence.

[0101] That is, to calculate the monthly average series based on the daily series.

[0102] S208c, the monitoring data processing system calculates the baseline value for each month by moving the current month as the center window for months that meet the center window conditions in the monthly mean series.

[0103] The central window indicates a sliding window with the current month as the center, extending to M months before and M months after the current month, for a total of 2M+1 months, where M is a positive integer.

[0104] That is, the width of the center window is 2M+1.

[0105] The central window condition indicates that the target quantity is greater than the width of the central window. The target quantity is the total quantity of the current month, the months before the current month, and the months after the current month.

[0106] For example, assuming the width of the center window is 11 months, then M=5. In this case, the first 5 months and the last 5 months in the monthly mean sequence are the marginal months that do not meet the center window condition, while the other months are the middle months that meet the center window condition.

[0107] Specifically, for the middle month, the monitoring data processing system takes the current month as the center and uses the monthly averages of the M months before the current month, the current month, and the M months after the current month (a total of 2M+1 months) to calculate the arithmetic average as the benchmark value of the target indicator for the current month.

[0108] For example, for the months in the monthly mean sequence excluding the first 5 months and the last 5 months of the sequence, take the monthly mean of the 5 months before the current month, the 5 months after the current month, and the current month as the center, and calculate the arithmetic mean of the monthly mean within the window, which is used as the baseline value for the current month.

[0109] It is understandable that, for the middle month in the monthly mean series, using the arithmetic mean within the center-moving sliding calculation window as the benchmark value for each month in each year can make the monthly coefficient determined based on this benchmark value more closely match the changing trend.

[0110] S208d, the monitoring data processing system calculates the baseline value for each month by moving the window based on the current month as the origin for months that do not meet the center window condition in the monthly mean series.

[0111] It should be noted that, in the embodiments of this disclosure, the above-mentioned center-based movement means that the reference value of each month is calculated by sliding with the target month as the center and the window width as the preset width, and the above-mentioned origin-based movement means that the reference value of each month is calculated by sliding with the target month as the origin and the window width as the preset width.

[0112] Specifically, for the first M months of the monthly mean sequence, the monitoring data processing system calculates the arithmetic mean of the monthly means for the next 2M+1 consecutive months, using the current month as the starting point, as the benchmark value for the target indicator in the target month. For the last M months of the monthly mean sequence, the monitoring data processing system calculates the arithmetic mean of the monthly means for the next 2M+1 consecutive months, using the target month as the ending point, as the benchmark value for the target indicator in the target month.

[0113] For example, for the first 5 months in the monthly mean sequence, starting from the current month, select the monthly mean of 11 consecutive months and calculate the arithmetic mean of the set of monthly mean values ​​as the base value of the current month. For the last 5 months in the monthly mean sequence, starting from the current month, select the monthly mean of 11 consecutive months and calculate the arithmetic mean of the set of monthly mean values ​​as the base value of the current month.

[0114] Based on this scheme, for the middle months in the monthly mean sequence, the arithmetic mean of the monthly mean within the center moving sliding calculation window can be used as the benchmark value for each month in each year. For the edge months in the monthly mean sequence, the arithmetic mean of the monthly mean within the origin moving sliding calculation window can be used as the benchmark value for each month in each year. By supplementing the benchmark values ​​of edge months that cannot meet the center window adjustment, it can be ensured that the final benchmark value sequence has no missing data, making the monthly coefficient determined based on the benchmark value more in line with the actual change trend of each month.

[0115] Optionally, in the hydropower station safety monitoring data processing method provided in the embodiments of this disclosure, the above-mentioned S205 may specifically include S205a or S205b.

[0116] S205a. Determine the width of the first sliding window that is adaptively adjusted in two dimensions based on the data length of the filled target monitoring data and the number of valid data in the sliding window.

[0117] S205b. Determine the width of the second sliding window that is adaptively adjusted in three dimensions based on the data length, fluctuation type, and number of valid data in the dynamic window after the target monitoring data is filled.

[0118] Based on this scheme, the width of the adaptively adjustable sliding window with different numbers of dimensions can be flexibly selected as needed. It can be flexibly selected according to requirements and can adapt to monitoring parameters with different total lengths, monitoring parameters with different fluctuation types, and monitoring parameters with different local effective data. It has stronger versatility and makes the spur removal processing method disclosed in this paper more generalizable.

[0119] Optionally, in the hydropower station safety monitoring data processing method provided in the embodiments of this disclosure, the above-mentioned S205a may specifically include S205a1 to S205a4.

[0120] S205a1, The monitoring data processing system adaptively selects the width of the base window based on the data length of the filled target monitoring data.

[0121] The width of the base window is an odd number.

[0122] For example, the width of the base window is selected based on the following formula (3).

[0123] ; Formula (3)

[0124] in, This indicates the length of the target monitoring data after padding. This indicates the width of the base window.

[0125] S205a2. For target data points in the filled target monitoring data, using the target data point as the center of the sliding window, determine whether the number of valid data points in the basic sliding window of the target data point is less than half the width of the basic sliding window.

[0126] The half-width of the basic sliding window is a positive integer rounded down.

[0127] Specifically, the half-width of the window can be calculated based on the width of the window and the following formula (4).

[0128] ; Formula (4)

[0129] Among them, if This indicates the width of the base sliding window. This represents the half-width of the base sliding window.

[0130] S205a3. If the number of valid data in the basic sliding window is less than half the width of the basic sliding window, then the basic sliding window is gradually expanded along both sides of the time axis with the target data point as the center until the number of valid data in the first sliding window after expansion is not less than half the width of the first sliding window.

[0131] Specifically, the initial forward and backward expansion step sizes are first set, and optionally, the maximum expansion range or maximum number of expansions can be set. Using the boundary of the previous sliding window as a reference, on the timeline, the starting time point of the window is shifted backward (backward) by one backward expansion step, and the ending time point of the window is shifted forward (forward) by one forward expansion step, thus generating a new candidate window. It is determined whether the number of valid historical data points within the candidate window has reached a preset number. If it has, the current candidate window is determined as the final sliding window, and expansion stops. If not, it is determined whether the predetermined maximum expansion range or maximum number of expansions has been reached. If not, a new round of bidirectional expansion is performed based on the candidate window from the previous round. If it has been reached, expansion stops, and the candidate window obtained from the last expansion is used as the second sliding window, or an exception handling mechanism is triggered.

[0132] Based on this scheme, the monitoring data processing system can adaptively select the width of the basic window according to the length of the monitoring data. On this basis, the system uses "data scale adaptation" and adopts a dynamic completion strategy of "center anchoring and bidirectional expansion" to generate an effective window that meets the statistical requirements when there is insufficient effective data in the window. This makes the sliding window for initial glitch removal more flexible in adapting to monitoring data of different lengths, so that the data can still maintain an accurate trend after glitch removal. This fundamentally solves the filtering failure problem caused by insufficient samples in the data missing area and at the beginning and end of the sequence in traditional methods.

[0133] Optionally, in the hydropower station safety monitoring data processing method provided in the embodiments of this disclosure, the above-mentioned S205b may specifically include S205b1 to S205b4.

[0134] S205b1. Based on the data length of the filled target monitoring data, adaptively select the width of the base window.

[0135] It should be noted that in this scheme, the method for determining S205b1 is the same as the method for determining S205a1. For a detailed description, please refer to S205a1 above.

[0136] S205b2. If the fluctuation type indicator is a fluctuation indicator, the monitoring data processing system will reduce the width of the basic window by a preset percentage. If the fluctuation type indicator is a slow-changing indicator, the system will increase the width of the basic window by a preset percentage to obtain the width of the third sliding window.

[0137] It is understandable that by matching the fluctuation characteristics of the actual monitoring data with the generation process of the sliding window for glitch removal, the best balance can be achieved between glitch removal and retaining real data.

[0138] For example, with a preset percentage of 20%, for volatility indicators, the width of the base window, determined based on the data length, can be reduced by 20%, while for slow-changing indicators, the width of the base window, determined based on the data length, can be increased by 20%.

[0139] S205b3. For the filled target monitoring data, the monitoring data processing system uses the target data point as the center of the third sliding window and determines whether the number of valid data in the third sliding window of the target data point is less than half the width of the third sliding window.

[0140] Specifically, the half-width of the third sliding window can be calculated based on the formula (4) above. Where, if Indicates the width of the third sliding window, then This represents the half width of the third sliding window.

[0141] S205b4. If the number of valid data points is less than half the width of the third sliding window, the monitoring data processing system shall gradually expand the third sliding window in both directions along the time axis with the target data point as the center until the number of valid data points in the second sliding window obtained after expansion is not less than half the width of the second sliding window.

[0142] Specifically, the third sliding window in this scheme, which expands bidirectionally along the time axis with the target data point as the center, can be referred to as the bidirectional expansion method described above, and will not be repeated here.

[0143] Based on this scheme, the monitoring data processing system's first layer adaptively selects the width of the base window according to the length of the monitoring data. The second layer adaptively adjusts the width of the base window according to the fluctuation type of the monitoring indicators to obtain the width of the sliding window. This allows the sliding window, after initial spurious removal, to more flexibly adapt to monitoring data of different lengths and monitoring indicators with different fluctuation types, thus ensuring that the data maintains accurate trends after spurious removal. The third layer, when there is insufficient effective data within the window, employs a dynamic completion strategy of center anchoring and bidirectional expansion to generate an effective sliding window that meets statistical requirements. This fundamentally solves the filtering failure problem caused by insufficient samples in data missing areas and at the beginning and end of sequences in traditional methods. The monitoring data processing system uses a three-layer adjustment logic of adaptive data scale, adaptive fluctuation characteristics, and adaptive supplementation of effective data: first, the base window scale is determined based on the total length of the monitoring data sequence; then, directional scaling is performed according to the fluctuation type of the indicators. For slowly changing indicators such as dam displacement, the window is expanded to enhance spurious suppression capabilities and extract smooth and stable long-term deformation trends; for fluctuating indicators such as seepage flow, the window is contracted to maintain its inherent periodic characteristics and rapid response details. Compared to the rigid parameters and poor adaptability of traditional fixed-window methods—namely, their inability to simultaneously handle scenarios with incomplete data and the conflicting demands of eliminating spikes in slowly changing indicators and preserving details in volatile indicators—this solution achieves accurate maintenance of the true changing patterns of various indicators while completely eliminating sudden spikes through dynamic window generation and three-dimensional parameter adaptation. This avoids coarsening of trend lines in slowly changing data and prevents excessive smoothing of characteristic oscillations in volatile data. This design overcomes the limitations of traditional algorithms that require manual parameter tuning and individual indicator development for multi-indicator monitoring systems. Its unified adaptive framework significantly improves the intelligence and reliability of data preprocessing, providing a high-quality data foundation for accurate assessment and early warning of hydropower station safety status.

[0144] Optionally, in the hydropower station safety monitoring data processing method provided in this embodiment, after S203 and before S204, that is, after filling in the missing points and before removing the burrs, a second preprocessing can be performed on the filled target monitoring data.

[0145] The second preprocessing includes: removing data points based on a set threshold and removing data points based on the standard Laida criterion.

[0146] Specifically, the monitoring data processing system, based on a preset set of maximum and minimum thresholds, removes data outside the threshold range. Then, it applies Layda's rule to the data sequence after threshold removal to eliminate outliers. For the data sequence after threshold removal, if the deviation between the monitored value and the true value of a data point exceeds 3σ, where σ represents the deviation threshold determined by Layda's rule, the monitored value of the data point is considered an outlier and is removed. In this disclosure, the variance of the data sequence is used. Instead of σ, use the mean of the data series. Instead of the truth value, for the first value in the data sequence If there are 1 data point, Then remove the first one. There are 10 data points; among them, Indicates if the first Monitoring values ​​of each data point Deviation from the mean of the data sequence.

[0147] Furthermore, in S205 above, the monitoring data processing system determines the burr removal sliding window based on the data length of the target monitoring data after the second preprocessing.

[0148] Based on this scheme, by first removing abnormal data that clearly exceeds the preset threshold range, and then removing statistical outliers identified based on 3σ, also known as the Laida criterion, a two-stage data cleaning process from coarse to fine is achieved, making the effect after subsequent removal of spikes closer to the true value.

[0149] Optionally, in the hydropower station safety monitoring data processing method provided in the embodiments of this disclosure, the above-mentioned S206 may specifically include S206a to S206c.

[0150] S206a. The monitoring data processing system acquires the median sequence of each burr removal sliding window, including the target data points.

[0151] Specifically, after determining the burr removal sliding window, the monitoring data processing system can determine the total set of burr removal sliding windows based on the length of the monitoring data of the burrs to be removed, and determine the window subset of the burr removal sliding window that includes the target data point according to the time axis.

[0152] After calculating the median of the spurious removal sliding window for each data point, for the target data point, a window subset including the target data point is determined, and the medians of multiple spurious removal sliding windows in the window subset are sorted in chronological order to obtain the median sequence corresponding to the target data point.

[0153] Based on formula (5), determine the data index in the burr removal sliding window corresponding to each data point.

[0154] ; Formula (5)

[0155] in, Indicates the first A sliding window for scrambling data points as the center Indicates the burr removal sliding window Data index in This indicates taking the maximum value. This indicates taking the minimum value. Indicates the burr removal sliding window half the width, This indicates the number of valid sequences in the monitoring data to be removed (excluding spikes). This represents the set of window indices for all glitch removal sliding windows in the monitoring data to be glitch removed. .

[0156] Remove burrs via sliding window All valid monitoring data within the time frame are arranged in ascending order to form a sequence. , ,in, Indicates the first The burr removal sliding window corresponding to each data point The total number of valid monitoring data actually included. Indicates the burr removal sliding window The sorting position is The numerical values ​​of the data points.

[0157] Based on the following formula (6), calculate the first... The median operator for each glitch removal sliding window for each data point.

[0158] ; Formula (6)

[0159] in, Indicates the first A sliding window for removing glitch data points. express the median of express The first in the median sequence The median, Indicates the burr removal sliding window The first in The value of each valid data point. Indicates the burr removal sliding window The first in The value of each valid data point.

[0160] S206b: The monitoring data processing system determines whether there exists a burr removal sliding window in each burr removal sliding window of the target data point whose median is equal to the monitoring value of the target data point within the calculation tolerance range.

[0161] S206c. If the median of the non-burr removal sliding window is equal to the monitoring value of the target data point within the calculation tolerance range, the monitoring data processing system determines that the target data point is a burr point, marks it, and removes the target data point.

[0162] For example, suppose it contains the first The index set of the glitch removal sliding window for each data point is: ,in, , Indicates containing the first The number of the sliding window for glitch removal for each data point. Indicates the first A sliding window for scrambling data points as the center Indicates the first The data point falls within the first... A glitch removal sliding window centered on a data point This represents the center of all possible windows in the valid sequence of monitoring data.

[0163] The anomaly determination function can be based on the following formula (7) to determine the first... Are the data points spurious?

[0164] ; Formula (7)

[0165] in, This indicates that the exception detection function is used to determine the first... Are the data points spurious? Indicates all, This indicates that it exists. The window contains the first A set of sliding windows for spurious data point removal. Indicates the first Monitoring values ​​of each data point Indicates the first A burr removal sliding window express the median of This indicates the calculation tolerance. , It represents a power of 10.

[0166] Traditional spurious point removal methods rely solely on the median of the target point's own window for judgment, making them susceptible to window size selection biases and local data distribution anomalies, leading to misjudgments or omissions. This disclosure creatively overcomes this technical bias by proposing a multi-window context-verified anomaly identification mechanism: instead of being limited to the target point's own window, it constructs a set of all sliding windows containing the point; the anomaly judgment logic changes from "whether the target point is abnormal within its own window" to "whether the target point is consistently abnormal within all windows containing it"; only when the target point significantly deviates from the median in all windows containing it is it identified as a spurious point; if it behaves normally in any window, it is considered valid data. This concept completely changes the traditional "one-vote veto" anomaly identification paradigm. Through multi-window cross-validation, it significantly improves the algorithm's robustness to window size biases and local data anomalies, effectively avoiding the limitations of a single window and achieving more accurate and reliable identification of spurious points in monitoring data. Based on this scheme, by using the aforementioned glitch removal sliding window, glitch removal can be effectively achieved by combining the length of the data and the type of data fluctuation, while accurately maintaining the actual trend of data change.

[0167] Optionally, in the hydropower station safety monitoring data processing method provided in the embodiments of this disclosure, the above-mentioned S206 may specifically include S206e to S205h.

[0168] S206e The monitoring data processing system determines the width of the statistical sliding window based on the length and fluctuation type of the monitoring data after the initial burr removal.

[0169] For example, the width of the statistical sliding window is calculated based on the length of the monitoring data after the initial burr removal and formula (8).

[0170] ; Formula (8)

[0171] in, This indicates the width of the statistical sliding window. This indicates the length of the monitoring data after the initial glitch removal. If the target indicator is a volatile indicator, the upper limit of the statistical sliding window width is used; if the target indicator is a slowly changing indicator, the lower limit of the statistical sliding window width is used.

[0172] For example, the seepage flow rate and opening / closing degree are taken as the upper limit.

[0173] Assumption ,but , , The base result is 100, but since it is a volatility indicator, the upper limit is taken, which is 365 days.

[0174] For slow-changing indicators, the lower limit is taken, which is 30 days.

[0175] S206f, the monitoring data processing system calculates the moving mean and moving standard deviation of the data in each statistical sliding window after the initial spurious removal based on the size of the statistical sliding window, and generates an adaptive threshold range for each statistical sliding window based on the moving mean and moving standard deviation in each statistical sliding window.

[0176] For example, the adaptive threshold range of the statistical sliding window is: .in, This represents the moving average of the data after the initial glitch removal within the statistical sliding window. It represents the sliding standard deviation of the data after the initial glitch removal within the statistical sliding window.

[0177] S206g, the monitoring data processing system determines the target data points outside the adaptive threshold interval in each statistical sliding window.

[0178] It is understandable that by using an adaptive threshold range, the local statistical characteristics of the data can be tracked in real time, and the number of spikes that exceed the required differential correction with median filtering can be accurately filtered out.

[0179] S206h Monitoring data processing system: If the target data point is an isolated point, the moving average is used to replace the value of the target data point. If the target data point is not an isolated point, the average value of the monitoring data before and after the target data point is used to replace the value of the target data point, and the target data point is marked as a point to be verified.

[0180] Specifically, if the two adjacent data points before and after the target point are both within the adaptive threshold range, the target point is determined to be an isolated point, which is a sudden spike. In this case, the moving average is used to replace the monitoring data of the isolated point.

[0181] If one of the two adjacent data points before and after the target point is outside the adaptive threshold range, the target point is determined to be a non-isolated point and a possible trend change (such as abnormal settlement of a dam). In this case, the monitoring data of the target point is replaced by the linear interpolation of the two adjacent data points before and after it, and the data point is marked as a point to be verified for subsequent manual composite analysis.

[0182] Figure 8 This is one of the schematic diagrams of the original data process line provided in the embodiments of this disclosure, wherein there are a total of 1828 data points and 590 glitch points. Figure 9 This is one of the process flow diagrams provided in the embodiments of this disclosure after deburring, such as... Figure 9 As shown, for Figure 8 The process line is composed of 1238 data points remaining after removing 590 puncture points from the original data.

[0183] As Figure 9 As shown: The processing results intuitively verify the technological innovation and significant advantages of this disclosure in spur removal of monitoring data. Figure 8 The original process line contained a large number of spikes, accounting for 30% according to statistics. Most of these spikes fluctuated back and forth between the two sides of the data trend. However, after adaptive spike removal by this solution, Figure 9 The process line not only effectively filtered out these abnormal interferences, but also completely preserved the continuous and smooth decline trend of the monitoring indicators from 2018 to 2024. The curve composed of 1238 valid data points showed good physical consistency and trend continuity.

[0184] Traditional processing methods are generally ineffective at removing the burrs shown in the image. If traditional methods are used, these burrs can only be removed manually one by one, which is time-consuming and labor-intensive.

[0185] Figure 10 This is the second schematic diagram of the original data process line provided in the embodiments of this disclosure, wherein there are a total of 1093 data points and 210 glitch points. Figure 11 This is a second schematic diagram of the process line after deburring provided in the embodiments of this disclosure, as shown below. Figure 11 As shown, for Figure 10 The process line consists of 883 data points remaining after removing 210 puncture points from the original data.

[0186] Figure 12 This is the third schematic diagram of the original data process line provided in the embodiments of this disclosure, wherein there are a total of 1092 data points and 344 glitch points. Figure 13 This is the third schematic diagram of the process line after deburring provided in the embodiments of this disclosure, as shown in Figure 3. Figure 13 As shown, for Figure 12 The process line consists of 748 data points remaining after removing 344 puncture points from the original data.

[0187] Figure 14 This is the fourth schematic diagram of the original data process line provided in the embodiments of this disclosure, wherein there are a total of 1070 data points and 548 glitch points. Figure 15 This is the fourth schematic diagram of the process line after deburring provided in the embodiments of this disclosure, as shown in Figure 4. Figure 15 As shown, for Figure 14 The process line consists of 522 data points remaining after removing 548 puncture points from the original data.

[0188] It can be seen that this burr removal method can effectively remove burrs from the original material while accurately preserving the original trend.

[0189] Figures 6 to 13The multiple processing results shown intuitively verify the technological innovation and significant advantages of this disclosure in removing glitch from monitoring data. The original process lines generally contain a large number of dense glitches distributed on both sides of the trend, with glitch proportions reaching 22.7%, 19.2%, 22.7%, and 51.2%, respectively. These glitches severely interfere with the judgment of the true trend. If conventional thresholding or fixed-window filtering methods are used, it is easy to lead to the dilemma of "incomplete glitch removal" or "trend distortion," and even require time-consuming and laborious manual point-by-point screening. After the adaptive glitch removal processing of this disclosure, all result process lines show significant improvement. While effectively removing glitches, it completely preserves the continuous and smooth long-term evolution trend of the monitoring indicators. The curves formed by all valid data points show good physical consistency and trend continuity.

[0190] The above results fully demonstrate the core breakthrough of the method disclosed in this publication: by constructing a collaborative mechanism of "three-dimensional sliding window based on fluctuation characteristics adaptation" and "median-statistical feature secondary verification," high-sensitivity identification and high-fidelity filtering of spikes in monitoring indicators of different fluctuation types are achieved; at the same time, relying on the "unified processing framework for multiple indicators," the same algorithm can be adapted to various hydropower station monitoring scenarios without manual parameter tuning, and the spike residue rate is continuously controlled at a low level while ensuring the authenticity of trends. This publication not only significantly improves the quality and reliability of monitoring data for hydropower station safety, but also greatly reduces the complexity of system operation and maintenance and labor costs through a fully automatic and adaptive processing method, laying a solid data foundation for subsequent accurate safety assessments, intelligent fault early warning, and scientific operation and maintenance decisions.

[0191] Based on this scheme, a second burr removal can be performed on the basis of the initial burr removal, which can further reduce burrs in the monitoring data.

[0192] Optionally, in the monitoring data processing method for hydropower station safety provided in the embodiments of this disclosure, after S206 above, it may also include S211 to S214 below.

[0193] S211. The monitoring data processing system constructs a Kalman filter state equation based on the target monitoring data after secondary burr removal and the rate of change of the target monitoring data.

[0194] For example, the Kalman filter state equation is constructed by following the steps below.

[0195] Specifically, considering the slow-varying characteristics of hydropower station monitoring data, a state vector at time m is constructed based on the monitored value at time m and the rate of change at time m. , , This represents the monitoring value at time m. This represents the rate of change at time m. Based on the state vector at time m and the state vector at time m-1, the state transition matrix is ​​used to construct the state transition equation. .in, Represents the state transition matrix. , Let m represent the state vector at time m-1. Let m represent the process noise at time m-1. That is, the state transition equation is: The observation matrix is ​​as follows: The observation equation is: .

[0196] S212. The monitoring data processing system adaptively initializes the Kalman filter parameters based on the statistical characteristics of the data and the fluctuation type.

[0197] Specifically, S212 can be performed by the following S212a and S212b.

[0198] S212a The monitoring data processing system adaptively selects the number of valid data to generate the initial state based on the fluctuation type of the monitoring indicator.

[0199] Assume the initial state is: .

[0200] If the target monitoring indicator is a fluctuation indicator, then This represents the mean of the first 15 valid data points. If the target monitoring indicator is a slowly changing indicator, then... This represents the mean of the first 30 valid data points.

[0201] S212b: The monitoring data processing system adaptively selects the first coefficient in the initial covariance matrix, the second and third coefficients in the process noise covariance matrix, and the fourth coefficient of the observation noise based on the fluctuation type of the monitoring index.

[0202] For example, if the target indicator is a volatility indicator, then the first coefficient in the initial covariance matrix is ​​1 / 15, and thus the initial covariance matrix... , This represents the initial standard deviation; if the target indicator is a slowly varying indicator, then the first coefficient in the initial covariance matrix is ​​1 / 30, and thus the initial covariance matrix... .

[0203] For example, if the target index is a volatility index, then the second coefficient in the process noise covariance matrix is ​​0.01, the third coefficient is 0.0005, and thus the process noise covariance matrix... If the target index is a slowly varying index, then the second coefficient in the process noise covariance matrix is ​​0.005, the third coefficient is 0.0001, and thus the process noise covariance matrix... .

[0204] For example, if the target indicator is a volatility indicator, the fourth coefficient of the observation noise is 0.02, and therefore the observation noise is... If the target index is a slowly varying index, the fourth coefficient of the observation noise is 0.01, and therefore the observation noise is... .

[0205] S213. The monitoring data processing system iteratively executes the Kalman filter prediction step and the update step.

[0206] Specifically, the prediction steps include: , .in, Let m represent the state vector at time m. Let m represent the state vector at time m-1.

[0207] The update steps include: If the current data point is missing, directly use the predicted value: The updated status is: Where z represents the observed value, The Kalman gain represents a weighting factor that primarily serves as a trade-off; the updated covariance is: .

[0208] S214. The monitoring data processing system extracts the monitoring value components from the Kalman-filtered state vector based on the updated state and updated covariance as the trend extraction result of the target monitoring data.

[0209] Based on this scheme, the Kalman filter parameters are adaptively initialized based on the monitoring index type and data statistical characteristics, which can accurately capture the slow changing trend of hydropower station facilities and has good trend extraction adaptability.

[0210] Corresponding to the embodiments of the foregoing methods, this disclosure also provides embodiments of the apparatus and the computer equipment on which it is applied.

[0211] Embodiments of the disclosed device can be applied to computer equipment, such as servers or terminal devices. The device embodiments can be implemented through software, hardware, or a combination of both. Taking software implementation as an example, a hydropower station safety monitoring data processing device, in a logical sense, is formed by a processor that reads the corresponding computer program instructions from non-volatile memory into memory and executes them. From a hardware perspective, such as... Figure 16 The diagram shown is a hardware structure diagram of the computer equipment housing the monitoring data processing system for hydropower station safety according to an embodiment of this disclosure. (Except for...) Figure 16In addition to the processor 1610, memory 1630, network interface 1620, and non-volatile memory 1640 shown, the server or electronic device where the hydropower station safety monitoring data processing system is located in the embodiment may also include other hardware depending on the actual function of the computer device, which will not be described in detail here.

[0212] like Figure 17 As shown, Figure 17 This disclosure provides a hydropower station safety monitoring data processing device 1700, comprising: a data acquisition module 1701, a missing point identification module 1702, a missing point filling module 1703, a burr identification module 1704, a burr removal sliding window construction module 1705, and a burr removal module 1706. The data acquisition module 1701 acquires target monitoring data for target indicators of hydropower station safety. The missing point identification module 1702 identifies whether missing points exist in the target monitoring data. The missing point filling module 1703, if missing points exist in the target monitoring data, fills the missing locations with values ​​based on the filling rules corresponding to the period type. The period type indicator is periodic monitoring data. The system includes: a measurement index or a non-periodic monitoring index; a spur identification module 1704, used to identify whether there are spurs in the filled target monitoring data; a spur removal sliding window construction module 1705, used to adaptively determine the width of the spur removal sliding window in multiple dimensions based on the data length of the filled target monitoring data and the number of valid data in the sliding window, or the data length, fluctuation type and the number of valid data in the sliding window, where the fluctuation type indicator is a slow-changing index or a fluctuation index; and a spur removal module 1706, used to calculate the median of the valid data in the spur removal sliding window corresponding to each data point, perform initial spur removal based on the median of each spur removal sliding window, and perform secondary spur removal based on the statistical characteristics after the initial spur removal.

[0213] Optionally, the spur removal module 1706 is specifically used to: obtain the median sequence of each spur removal sliding window including the target data point; determine whether there is a spur removal sliding window whose median is equal to the monitored value of the target data point within the calculation tolerance range; if there is no spur removal sliding window whose median is equal to the monitored value of the target data point within the calculation tolerance range, then the target data point is determined to be a spur point, and the target data point is marked and removed.

[0214] Optionally, the spurious data removal module 1706 is specifically used for: determining the size of the statistical sliding window based on the length and fluctuation type of the monitoring data after the initial spurious data removal; calculating the moving mean and moving standard deviation of the data in each statistical sliding window after the initial spurious data removal based on the size of the statistical sliding window; generating an adaptive threshold interval for each statistical sliding window based on the moving mean and moving standard deviation of each statistical sliding window; determining the target data point outside the adaptive threshold interval in each statistical sliding window; if the target data point is an isolated point, replacing the monitoring value of the target data point with the moving mean within the statistical sliding window corresponding to the target data point; if the target data point is not an isolated point, replacing the monitoring value of the target data point with the mean value of the monitoring data adjacent to the target data point before and after the target data point, and marking the target data point as a point to be verified.

[0215] Optionally, the missing point filling module 1703 is specifically used for: if the target indicator is a non-periodic monitoring indicator, then filling values ​​for each missing point in the missing segment based on the mean of the effective data of the first time period to the left of the missing segment, the mean of the effective data of the second time period to the right of the missing segment, the length of the missing segment, and the position of the missing point within the missing segment; if the target indicator is a periodic monitoring indicator, then filling values ​​for each missing point in the missing segment based on the mean of the effective data of the first time period to the left of the missing segment, the mean of the effective data of the second time period to the right of the missing segment, the first seasonal coefficient corresponding to the first time period, and the second seasonal coefficient corresponding to the second time period; the duration of the first time period and the second time period are both the first preset duration.

[0216] Optionally, the monitoring data processing device for hydropower station safety further includes: a seasonal coefficient generation module; the seasonal coefficient generation module is used to acquire historical monitoring data of target indicators within a second preset time period, the second preset time period being earlier than the first preset time period, and the length of the second preset time period being greater than the length of the first preset time period; calculate the benchmark value corresponding to each month of each year based on the actual monitoring value of each month; calculate the monthly coefficient of each year based on the actual monitoring value and benchmark value of each month within a year; and obtain the seasonal coefficient corresponding to each month based on the average of the monthly coefficients of the same month in all years within the second preset time period.

[0217] Optionally, the seasonal coefficient generation module is specifically used to: arrange the complete daily historical monitoring data within the second preset time period in chronological order to obtain a one-dimensional time series; group the continuous daily data by natural month, calculate the arithmetic mean of all daily monitoring values ​​in each month, and obtain a monthly mean series; for months in the monthly mean series that meet the center window condition, calculate the baseline value of each month by moving the window with the current month as the center; for months in the monthly mean series that do not meet the center window condition, calculate the baseline value of each month by moving the window with the current month as the origin; wherein, the center window condition indicates that the target number is greater than the width of the center window, and the target number is the total number of the current month, the months before the current month, and the months after the current month.

[0218] Optionally, the burr removal sliding window construction module 1705 is specifically used to: determine the width of the first sliding window with two-dimensional adaptive adjustment based on the data length of the filled target monitoring data and the number of valid data in the sliding window; or, determine the width of the second sliding window with three-dimensional adaptive adjustment based on the data length, fluctuation type, and number of valid data in the sliding window of the filled target monitoring data.

[0219] Optionally, the burr removal sliding window construction module 1705 is specifically used for: adaptively selecting the width of the base window based on the data length of the filled target monitoring data, wherein the width of the base window is an odd number; for the target data point in the filled target monitoring data, taking the target data point as the center of the sliding window, determining whether the number of valid data in the base sliding window of the target data point is less than half the width of the base sliding window; if the number of valid data in the base sliding window is less than half the width of the base sliding window, then gradually expanding the base sliding window along the time axis with the target data point as the center to generate a first sliding window centered on the target data point; wherein the number of valid data in the first sliding window is not less than half the width of the first sliding window.

[0220] Optionally, the spur removal sliding window construction module 1705 is specifically used for: adaptively selecting the width of the base window based on the data length of the filled target monitoring data, wherein the width of the base window is an odd number; if the fluctuation type indicates that the target indicator is a fluctuating indicator, then the width of the base window is reduced by a preset percentage; if the fluctuation type indicates that the target indicator is a slow-changing indicator, then the width of the base window is increased by a preset percentage to obtain the width of the third sliding window; for the target data point in the filled target monitoring data, taking the target data point as the center of the third sliding window, determining whether the number of valid data points in the third sliding window of the target data point is less than half the width of the third sliding window; if the number of valid data points in the third sliding window is less than half the width of the third sliding window, then gradually expanding the third sliding window bidirectionally along the time axis based on the target data point as the center, until the number of valid data points in the expanded second sliding window is equal to half the width of the second sliding window.

[0221] Optionally, the monitoring data processing device for hydropower station safety further includes: a trend extraction module; the trend extraction module is used to construct a Kalman filter state equation based on the target monitoring data after secondary spur removal and the rate of change of the target monitoring data; adaptively initialize the Kalman filter parameters based on data statistical characteristics and fluctuation type; iteratively execute the Kalman filter prediction step and update step; and extract the monitoring value component in the Kalman filtered state vector as the trend extraction result of the target monitoring data based on the updated state and the updated covariance.

[0222] Optionally, the trend extraction module is specifically used to: adaptively select the number of valid data for generating the initial state based on the fluctuation type of the monitoring indicator; and adaptively select the first coefficient in the initial covariance matrix, the second and third coefficients in the process noise covariance matrix, and the fourth coefficient of the observation noise based on the fluctuation type of the monitoring indicator.

[0223] The hydropower station safety monitoring data processing device provided in this embodiment first acquires target monitoring data for target indicators of hydropower station safety. Then, it identifies whether there are missing points in the target monitoring data. If missing points exist, it fills the missing locations with values ​​based on a weighted average filling rule corresponding to the period type. In other words, for both periodic and non-periodic indicators, different weighted average filling rules can be adaptively selected for various monitoring indicators according to the period type to fill missing points, thereby improving the filling accuracy. After filling, it identifies whether there are spikes in the filled target monitoring data. If spikes exist, it adaptively determines the width of the spike removal sliding window based on multiple dimensions, including the data length and the number of valid data points within the sliding window, or the data length, fluctuation type, and the number of valid data points within the sliding window. By adaptively constructing the width of the spike removal sliding window across dimensions, the determined spike removal sliding window can better match the characteristics of the actual monitoring data, thereby avoiding the bottleneck problem of spike removal accuracy caused by inappropriate selection of the sliding window width. Initial spurious removal based on the median of the spurious removal sliding window better reflects the short-term trend characteristics of the monitoring data. Subsequent secondary spurious removal based on statistical characteristics further refines the short-term distribution of the data. This two-round spurious removal process, employing different methods, allows for the precise removal of sudden spurs while preserving key trend changes such as abnormal dam settlement. Verification with real monitoring data shows that the spur residual rate is below 3%. This multi-indicator differentiated processing approach, based on the periodicity and fluctuation type of the indicators, is adaptable to various core monitoring indicators of hydropower stations, such as dam displacement, seepage flow, seepage pressure, and opening / closing degree. It eliminates the need for developing separate algorithms for individual indicators, reducing system deployment costs and providing precise data support for hydropower station safety assessment, fault early warning, and operation and maintenance decisions.

[0224] Accordingly, this disclosure also provides a monitoring data processing device for hydropower station safety, the device including a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the various steps as described in the above embodiments of the monitoring data processing method for hydropower station safety.

[0225] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps described in the above embodiments of the monitoring data processing method for hydropower station safety.

[0226] This disclosure also provides a computer device, the computer device including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein when the computer-readable instructions are executed by the processor, they implement the various steps in the above-described embodiments of the monitoring data processing method for hydropower station safety.

[0227] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0228] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0229] The foregoing has described specific embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0230] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention applied herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not claimed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0231] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

[0232] The above description is merely a preferred embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for processing monitoring data on the safety of a hydropower station, characterized in that, The method includes: Acquire target monitoring data for the target indicators of hydropower station safety, wherein the target indicators include at least one of the following: dam displacement, seepage rate, seepage pressure, opening and closing degree, and strain; Identify whether there are missing points in the target monitoring data; If there are missing points in the target monitoring data, and the target indicator is a non-periodic monitoring indicator, then the missing points are filled with values ​​based on the mean of the effective data within the first preset time period on both sides of the missing segment, the position of the missing point within the segment, and the length of the missing segment. If the target indicator is a periodic monitoring indicator, then the missing points are filled with values ​​based on the mean of the effective data within the first preset time period on both sides of the missing segment, the corresponding seasonal coefficients on both sides, the position of the missing point within the segment, and the length of the missing segment. Identify whether there are burrs in the target monitoring data after filling; If glitches exist, the width of the glitches removal sliding window is adaptively determined from multiple dimensions based on the data length of the filled target monitoring data and the number of valid data within the sliding window, or the data length, fluctuation type, and the number of valid data within the sliding window. This includes: adaptively selecting a base window with an odd width based on the data length of the filled target monitoring data; if fluctuation type is not considered, using the base window as the sliding window; if fluctuation type is considered, and the fluctuation type indicates that the target indicator is a fluctuating indicator, reducing the width of the base window by a preset percentage; if the fluctuation type indicates that the target indicator is a slow-changing indicator, increasing the width of the base window by a preset percentage to obtain the sliding window; for the target data point in the filled target monitoring data, taking the target data point as the center, determining whether the number of valid data within the sliding window is less than half the width of the sliding window; if so, gradually expanding the sliding window bidirectionally along the time axis until the number of valid data in the expanded sliding window is not less than half the width of the expanded sliding window. Calculate the median of valid data within the glitch removal sliding window corresponding to each data point. Perform initial glitch removal based on the median of each glitch removal sliding window. Perform secondary glitch removal based on the statistical characteristics after the initial glitch removal.

2. The method according to claim 1, characterized in that, The initial burr removal based on the median of each burr removal sliding window includes: Obtain the median sequence of each spur removal sliding window, including the target data points; Determine whether there exists a burr removal sliding window for the target data point whose median value is equal to the monitored value of the target data point within the calculation tolerance range; If the median of the non-scratching sliding window is equal to the monitoring value of the target data point within the calculation tolerance range, then the target data point is determined to be a scrat point, and the target data point is marked and removed.

3. The method according to claim 1 or 2, characterized in that, The secondary burr removal based on the statistical characteristics after the initial burr removal includes: The size of the statistical sliding window is determined based on the length and fluctuation type of the monitoring data after the initial burr removal. Based on the size of the statistical sliding window, the moving mean and moving standard deviation of the data in each statistical sliding window after the initial spurious removal are calculated. Based on the moving mean and moving standard deviation of each statistical sliding window, an adaptive threshold range for each statistical sliding window is generated. Determine the target data point outside the adaptive threshold interval in each statistical sliding window; If the target data point is an isolated point, the moving average within the statistical sliding window corresponding to the target data point is used to replace the monitoring value of the target data point. If the target data point is not an isolated point, the average value of the monitoring data points before and after the target data point is used to replace the monitoring value of the target data point, and the target data point is marked as a point to be verified.

4. The method according to claim 1, characterized in that, Before acquiring the target monitoring data for the target indicators of hydropower station safety, the method further includes: Obtain historical monitoring data of the target indicator within a second preset time period, wherein the second preset time period is earlier than the first preset time period, and the length of the second preset time period is greater than the length of the first preset time period; Calculate the baseline value for each month of each year based on the actual monitoring values ​​for each month. Calculate the monthly coefficient for each year based on the actual monitoring values ​​and benchmark values ​​for each month within a year; Based on the average of the monthly coefficients of the same month in all years within the second preset time period, the seasonal coefficient corresponding to each month is obtained.

5. The method according to claim 4, characterized in that, The calculation of the baseline value for each month of each year based on the actual monitoring values ​​of each month includes: Arrange the complete daily historical monitoring data within the second preset time period in chronological order to obtain a one-dimensional time series; Grouping consecutive daily data by natural month, calculating the arithmetic mean of all daily monitoring values ​​within each month, yields a monthly mean sequence; For the months in the monthly mean sequence that meet the center window condition, the baseline value of each month is calculated by sliding the moving window with the current month as the center. For months in the monthly mean sequence that do not meet the central window condition, the baseline value for each month is calculated by moving the window with the current month as the origin. The central window indicates a sliding window of 2M+1 months centered on the current month, with M months before and M months after the current month, where M is a positive integer. The first M months and the last M months in the monthly average sequence are marginal months that do not meet the conditions of the central window, while the other months are intermediate months that meet the conditions of the central window.

6. A monitoring data processing device for the safety of a hydropower station, characterized in that, The monitoring data processing device includes: a data acquisition module, a missing point identification module, a missing point filling module, a burr identification module, a burr removal sliding window construction module, and a burr removal module; The data acquisition module is used to acquire target monitoring data of target indicators for monitoring the safety of hydropower stations. The target indicators include at least one of the following: dam displacement, seepage rate, seepage pressure, opening and closing degree, and strain. The missing point identification module is used to identify whether there are missing points in the target monitoring data; The missing point filling module is used to fill in missing points if there are missing points in the target monitoring data and the target indicator is a non-periodic monitoring indicator, based on the mean of the effective data within a first preset time period on both sides of the missing segment, the position of the missing point within the segment, and the length of the missing segment; if the target indicator is a periodic monitoring indicator, it fills in missing points based on the mean of the effective data within the first preset time period on both sides of the missing segment, the seasonal coefficient, the position of the missing point within the segment, and the length of the missing segment. The burr recognition module is used to identify whether there are burrs in the filled target monitoring data; The glitch removal sliding window construction module is used to adaptively determine the width of the glitch removal sliding window in multiple dimensions based on the data length of the filled target monitoring data and the number of valid data in the sliding window, or the data length, fluctuation type, and the number of valid data in the sliding window, if glitch exists. This includes: adaptively selecting a base window with an odd width based on the data length of the filled target monitoring data; if fluctuation type is not considered, using the base window as the sliding window; if fluctuation type is considered, and the fluctuation type indicates that the target indicator is a fluctuating indicator, reducing the width of the base window by a preset percentage; if the fluctuation type indicates that the target indicator is a slow-changing indicator, increasing the width of the base window by a preset percentage to obtain the sliding window; for target data points in the filled target monitoring data, determining whether the number of valid data in the sliding window is less than half the width of the sliding window, centered on the target data point; if so, gradually expanding the sliding window bidirectionally along the time axis until the number of valid data in the expanded sliding window is not less than half the width of the expanded sliding window. The burr removal module is used to calculate the median of the effective data within the burr removal sliding window corresponding to each data point, perform initial burr removal based on the median of the burr removal sliding window, and perform secondary burr removal based on the statistical characteristics after the initial burr removal.

Images