Hydropower station dispatching operation data cleaning method considering data physical connection and related device

Abstract

The invention discloses a hydropower station dispatching operation data cleaning method considering data physical relation and a related device, and the method comprises the steps: carrying out the preprocessing of an original dispatching operation data set S based on the physical relation of the hydropower station dispatching operation data, and carrying out the calculation to obtain a data condition evaluation index CRI at different moments; on the basis of the data condition evaluation index CRI, multiple abnormal data detection models are constructed, abnormal detection results of different models are comprehensively considered, and the abnormal moment when abnormal data occur is judged; aiming at an abnormal moment with abnormal data, considering the abnormal moment and variable distribution in a previous time period and a later time period, and determining an abnormal variable value of the abnormal moment; and correcting the identified abnormal variable value according to the physical relationship among the scheduling operation data. According to the method, anomaly detection and cleaning correction can be carried out on the cascade power station dispatching operation data set, the data quality is improved, and basic data support is provided for dispatching operation rule analysis, prediction model construction and safe and economical operation of a power station.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular, to a method and related device for cleaning hydropower station dispatching operation data considering the physical connection of data. Background Art

[0002] Hydropower stations take into account functions such as the utilization of water energy resources and the prevention of flood disasters, and play an important role in China's water security and energy security. High-quality dispatching operation data of hydropower stations is the basis for formulating power station dispatching plans, analyzing the "output - water level - flow" relationship, and constructing system state prediction models, and is crucial for the safe and economic operation of power stations. However, due to various reasons such as possible generator set failures, electromagnetic signal interference, and damage to transmission equipment, during the actual operation of power stations, monitoring data is prone to abnormal situations, and the occurrence of abnormal situations is random. In addition, the data sources of hydropower stations are extensive and the data volume is huge, making it difficult to make manual judgments.

[0003] Currently, simple mathematical statistics methods or traditional data analysis methods are often used to process data. For example, statistically analyzing whether a certain data conforms to the data distribution law, analyzing whether the time variation of the data is reasonable, etc. Such methods have the following defects: (1) A single data anomaly monitoring method is difficult to apply to complex data anomaly situations. In the historical dispatching operation data of cascade power stations, the causes of abnormal data are complex, and multiple data anomaly modes coexist and interact with each other as cause and effect. The existence of such data anomaly characteristics makes it difficult to use a unified method to process all abnormal data; (2) The water level, flow, and output data in dispatching operations are all variables with physical meanings, and the physical relationships between them are clear. Even if the data of a certain variable at a certain moment conforms to the distribution and variation law of the variable, the relationship between it and other physical variables may still show a significant violation of the physical relationship, which makes the applicability of the processing method that does not consider the physical relationship between variables poor.

[0004] Generally speaking, the dispatching operation data of hydropower stations involves many variables, the physical relationships between the variables are clear, the data anomaly characteristics are complex, and a single method that does not consider its internal physical relationship is difficult to effectively achieve the goal of data cleaning. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and related device for cleaning hydropower station dispatching operation data considering the physical connection of data, which can comprehensively consider the physical relationships between hydropower station dispatching operation data and the characteristics of different data anomaly monitoring methods during the data cleaning process, realize rapid anomaly detection and interpolation correction of dispatching operation data, construct a high-quality dispatching operation data set, and thus provide basic data support for formulating hydropower station dispatching plans and safe and economic operation.

[0006] To achieve the above object, the present invention provides a method for cleaning hydropower station dispatching operation data considering the physical connection of data, including the following steps:

[0007] S1. Based on the physical relationship of hydropower station dispatching operation data, preprocess the original dispatching operation data set S, and calculate the data situation evaluation index CRI at different times;

[0008] S2. Based on the data situation evaluation index CRI, construct multiple abnormal data detection models, comprehensively consider the abnormal detection results of different models, and judge the abnormal time when abnormal data appears;

[0009] S3. For the abnormal time with abnormal data, consider the distribution of each variable in the abnormal time and the time periods before and after it, and determine the abnormal variable value at the abnormal time;

[0010] S4. According to the physical relationship between dispatching operation data, correct the abnormal variable value identified in step S3.

[0011] Further, step S1 includes the following steps:

[0012] S11. Set the time period and time scale of the data, collect the corresponding water level, flow rate, and output data, and construct the original dispatching operation data set S = {H, Q, N}, where H represents water level data, Q represents flow rate data, and N represents output data;

[0013] S12. Calculate the data situation evaluation index according to the water level-flow rate-output relationship, water volume balance relationship, and hydraulics relationship:

[0014]

[0015] Among them, n represents the number of moments, m represents the number of evaluation indexes, n and m are natural numbers, and the specific calculation steps include: calculating the power generation flow rate of the power station based on the water level-flow rate-output relationship, and then subtracting the measured power generation flow rate to obtain the power generation flow rate deviation value; calculating the downstream water level of the power station based on the hydraulics relationship, and then subtracting the actual downstream water level to obtain the deviation value of the downstream water level, and the hydraulics relationship includes the power station outlet flow rate and the power station tail water level curve; calculating the end-of-period water level of the power station upstream based on the water volume balance relationship, the initial water level upstream of the power station, the inflow rate during the period, and the outflow rate during the period, and subtracting the actual water level to obtain the deviation value of the upstream water level.

[0016] Further, step S2 includes the following steps:

[0017] S21. Based on the calculated data situation evaluation index, use k unsupervised learning algorithms to construct data abnormal detection models respectively, denoted as M1, M2,..., M k ;

[0018] S22. Based on different types of data situation evaluation indicators, apply the constructed k types of data anomaly detection models to calculate the abnormal data monitoring results of the k models like A value of 1 indicates that the data is abnormal, and a value of 0 indicates that the data is normal, where 1≤i≤n, 1≤j≤m;

[0019] S23. For any time and any evaluation indicator, if all models If both are 1, then the moment is determined to be an abnormal moment when abnormal data occurs.

[0020] Further, step S3 includes the following steps:

[0021] S31, for each abnormal time i where abnormal data occurs and the data situation evaluation index CRI=f(A, B, C), collect the variable values involved in the calculation of the data situation evaluation index at the abnormal time and the T moments before and after [A i-T ,…,A i ,…,A i+T ],[B i-T ,…,B i ,…,B i+T ],[C i-T ,…,C i ,…,C i+T ]];

[0022] S32, if there are other moments determined to be abnormal data between the ith abnormal moment and the T moments before and after it, remove the data of the other abnormal moments except the ith abnormal moment, and postpone them forward / backward;

[0023] S33. According to the data distribution of each variable in the time period, an abnormal data detection model is used to determine the abnormal situation of each variable at the abnormal moment, and the variable that is abnormal at the abnormal moment is determined as the abnormal variable value at the abnormal moment.

[0024] Further, step S4 includes the following steps:

[0025] S41. Set all abnormal variable values to null values;

[0026] S42. For any null value, the water level-flow-output relationship, water balance relationship, and hydraulic relationship used in step S1 are used to calculate the null value based on other variable elements at that moment. If other variable elements related to it are also null values and cannot be calculated, interpolation is performed based on the variable values at the moments before and after it.

[0027] A hydropower station dispatching operation data cleaning device considering the physical connection of data, comprising:

[0028] An evaluation index calculation module, which is used to preprocess the original scheduling operation data set S based on the physical relationships of the hydropower station scheduling operation data, and calculate the evaluation index CRI of the data conditions at different times;

[0029] An abnormal time judgment module, which is used to construct multiple abnormal data detection models based on the evaluation index CRI of the data conditions, and comprehensively consider the abnormal detection results of different models to judge the abnormal time when the abnormal data appears;

[0030] An abnormal variable value determination module, which is used to determine the abnormal variable value at the abnormal time for the abnormal time with abnormal data, considering the distributions of various variables at and around the abnormal time;

[0031] An abnormal variable value correction module, which is used to correct the identified abnormal variable value according to the physical relationships between the scheduling operation data.

[0032] Furthermore, the evaluation index calculation module specifically is used for:

[0033] Set the time period and time scale of the data, collect the corresponding water level, flow rate, and output data, and construct the original scheduling operation data set S = {H, Q, N}, where H represents the water level data, Q represents the flow rate data, and N represents the output data;

[0034] Calculate the evaluation index of the data conditions according to the water level-flow rate-output relationship, water balance relationship, and hydraulics relationship:

[0035]

[0036] Where n represents the number of times, m represents the number of evaluation indexes, and n and m are natural numbers. The specific calculation steps include: calculating the power generation flow rate of the power station based on the water level-flow rate-output relationship, and then subtracting the measured power generation flow rate to obtain the power generation flow rate deviation value; calculating the downstream water level of the power station based on the hydraulics relationship, and then subtracting the actual downstream water level to obtain the deviation value of the downstream water level. The hydraulics relationship includes the power station outlet flow rate and the power station tail water level curve; calculating the water level at the end of the upstream time period of the power station based on the water balance relationship, the initial water level of the upstream of the power station, the inflow rate during the time period, and the outflow rate during the time period, and subtracting the actual water level to obtain the deviation value of the upstream water level.

[0037] Furthermore, the abnormal time judgment module specifically is used for:

[0038] Based on the calculated evaluation index of the data conditions, use k unsupervised learning algorithms to construct data abnormal detection models respectively, denoted as M1, M2,..., M k ;

[0039] Based on different types of data situation evaluation indicators, the constructed k types of data anomaly detection models are applied to calculate the abnormal data monitoring results of k models like A value of 1 indicates that the data is abnormal, and a value of 0 indicates that the data is normal, where 1≤i≤n, 1≤j≤m;

[0040] For any time and any evaluation indicator, if all models If both are 1, then the moment is determined to be an abnormal moment when abnormal data occurs.

[0041] Furthermore, the abnormal variable value determination module is specifically used to:

[0042] For each abnormal time i where abnormal data occurs and the data situation evaluation index CRI = f(A, B, C), collect the variable values involved in the calculation of the data situation evaluation index at the abnormal time and the T moments before and after [A i-T ,…,A i ,…,A i+T ],[B i-T ,…,B i ,…,B i+T ],[C i-T ,…,C i ,…,C i+T ]];

[0043] If there are other moments determined to be abnormal data between the ith abnormal moment and the T moments before and after it, the data of other abnormal moments except the ith abnormal moment are removed and postponed forward / backward;

[0044] According to the data distribution of each variable in this period, the abnormal data detection model is used to judge

[0045] The abnormal situation of each variable at the abnormal moment is judged, and the variable that is abnormal at the abnormal moment is judged as the abnormal variable value at the abnormal moment.

[0046] Furthermore, the abnormal variable value correction module is specifically used to:

[0047] Set all abnormal variable values to null;

[0048] For any null value, the water level-flow-output relationship, water balance relationship, and hydraulic relationship are used to calculate the null value based on other variable elements at that moment. If other variable elements related to it are also null values and cannot be calculated, interpolation is performed based on the variable values at the previous and next moments.

[0049] The technical effects of the present invention are as follows:

[0050] (1) In view of the characteristic that there are physical relationships among different types of data in the hydropower station dispatching operation dataset, data situation evaluation indicators are calculated based on the physical relationships among water level, flow rate, and output, and are used as the basis for data anomaly detection, so as to ensure the consistency of the physical relationships among different variables;

[0051] (2) In view of the characteristics that different data anomaly detection models have their own advantages, by comprehensively considering the detection results of different anomaly data detection models on different data situation evaluation indicators, the moment when abnormal data appears is judged, and the accuracy of the anomaly detection results is guaranteed. Brief Description of the Drawings

[0052] Figure 1 is a flowchart of a method for cleaning hydropower station dispatching operation data considering data physical connections according to an embodiment of the present invention;

[0053] Figure 2 is a schematic diagram of the specific implementation steps of the method for cleaning hydropower station dispatching operation data considering data physical connections according to an embodiment of the present invention;

[0054] Figure 3 is a schematic diagram of the calculation results of the selected data situation evaluation indicator as the power generation flow deviation value within the selected time period according to an embodiment of the present invention;

[0055] Figure 4 is a diagram of the anomaly detection results of 6 data anomaly detection models according to an embodiment of the present invention;

[0056] Figure 5 is a diagram of the anomaly detection results comprehensively considering 6 data anomaly detection models according to an embodiment of the present invention;

[0057] Figure 6 is a comparison diagram of the flow rate before and after correction according to an embodiment of the present invention. Detailed Embodiment

[0058] The technical solutions of the present invention will be further described below with reference to the drawings and embodiments.

[0059] Unless otherwise defined, the technical terms or scientific terms used in the present invention should have the ordinary meanings understood by those of ordinary skill in the field to which the present invention belongs.

[0060] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above-described exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the gist or basic characteristics of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present invention, and any reference signs in the claims should not be regarded as limiting the claims involved.

[0061] In addition, it should be understood that although this specification is described according to embodiments, not every embodiment only contains an independent technical solution. This narrative way of the specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art. These other embodiments are also covered by the protection scope of the present invention.

[0062] It should also be understood that the above-described specific embodiments are only used to explain the present invention, and the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, makes equivalent substitutions or changes, and should be covered by the protection scope of the present invention.

[0063] Technologies, methods, and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and devices should be regarded as part of the specification.

[0064] The content disclosed in the prior art documents cited in the specification of the present invention is incorporated into the present invention by reference in its entirety, and thus is part of the disclosure content of the present invention.

[0065] Embodiment

[0066] As Figure 1 and Figure 2 shown, the embodiment of the present invention provides a method for cleaning hydropower station dispatching operation data considering the physical connection of data. It analyzes the deviation value of the data pair from the physical relationship through the physical relationship of the dispatching operation data, comprehensively determines the data anomaly situation through a variety of abnormal data detection models and a variety of evaluation indicators, and corrects the abnormal data through the physical relationship. The method includes the following steps:

[0067] Step S1: Based on the physical relationship of the hydropower station dispatching operation data, preprocess the original dispatching operation data set S, and calculate the evaluation index CRI of the data situation at different times. Step S1 includes the following steps:

[0068] Step S11: Set the time period and time scale of the data, collect the corresponding water level, flow rate, and output data, and construct the original dispatching operation dataset S = {H, Q, N}, where H represents the water level data, Q represents the flow rate data, and N represents the output data.

[0069] In this embodiment, the dispatching operation data of a certain power station is cleaned and corrected. The time period of the data is from January 1, 2020 to April 30, 2020, and the time scale of the data is set to 1 hour. The dataset includes water level data H (including the water level in front of the power station dam and the water level downstream of the power station), flow rate data Q (including the inflow rate, outflow rate, waste water flow rate, power generation flow rate, etc. of the power station), and output data N (the output of the power station).

[0070] Step S12: Calculate the evaluation indexes of the data situation according to the water level-flow rate-output relationship, water balance relationship, hydraulics relationship, etc.

[0071] (There are n moments in total and m evaluation indexes).

[0072] In this embodiment, taking the calculation of the deviation value of the power generation flow rate as an example, the head is calculated based on the water levels upstream and downstream of the power station. The water consumption rate at this head is obtained according to the water consumption rate curve, and then the power generation flow rate is calculated by combining the output. The deviation value of the power generation flow rate is obtained by subtracting the measured power generation flow rate from the calculated power generation flow rate. The calculated deviation value of the power generation flow rate is as Figure 3 shown. Among them, m = 1 evaluation index is adopted, and the evaluation indexes at n = 121 days * 24 hours / day * 1 moment / hour = 2904 moments are calculated.

[0073] Step S2: Based on the evaluation indexes of the data situation, construct multiple abnormal data detection models, comprehensively consider the abnormal detection results of different models, and judge the abnormal moments when abnormal data appears. Step S2 includes the following steps:

[0074] Step S21: Based on the calculated evaluation indexes of the data situation, use k unsupervised learning algorithms to construct data abnormal detection models, which are respectively denoted as M1, M2,..., M k .

[0075] In this embodiment, k = 6 data abnormal detection models are adopted, namely the 3-Sigma method based on distribution, the box plot (Box-plot) method based on quantiles, the CBLOF model, the KNN model, the AvgKNN model, and the Isolation Forest (IF) model. A brief description of each model is as follows:

[0076] 1) 3-Sigma method based on distribution: Denote the data mean as μ and the data standard deviation as σ, then the data outside the interval (μ - 3σ, μ + 3σ) is judged as an outlier.

[0077] 2) Quantile-based box plot method: Arrange all the data in the data set in ascending order, and determine the upper quartile Q3 (75% of the data in ascending order), the lower quartile Q1 (25% of the data in ascending order), and the interquartile range IQR (equal to the difference between Q3 and Q1) respectively. Data outside the interval (Q1 + 1.5 * IQR, Q3 - 1.5 * IQR) is judged as an outlier.

[0078] 3) CBLOF model: Reflect the distance from outliers by solving the reachability density of samples in a small range. Samples beyond a certain range are judged as outliers.

[0079] 4) KNN model and AvgKNN model: Identify outliers by calculating the statistical value of the distance from L nearest neighbor samples to the current sample. In this method, the maximum value, average value, and median of the distance can be used as the discrimination basis. In this study, the maximum value and average value are used as the discrimination basis respectively, and the formed methods are denoted as KNN and AvgKNN respectively.

[0080] 5) IF model: An isolation-based abnormal data diagnosis method. Determine the average division path length of a certain data point through multiple random forest divisions, and then perform outlier scoring.

[0081] For the specific calculation methods of the CBLOF model, KNN model, AvgKNN model, and Isolation Forest (IF) model, please refer to the relevant literature.

[0082] Step S22: Based on different types of data situation evaluation indicators, apply the constructed k data anomaly detection models. If (1 ≤ i ≤ n, 1 ≤ j ≤ m) has a value of 1, it indicates that the data is abnormal; if it is 0, it indicates that the data is normal.

[0083] Based on the calculated flow deviation value, the outlier detection results obtained by using 6 models are as Figure 4 shown. In the figure, those with a value of 1 are marked as "x", and those with a value of 0 are marked as ".".

[0084] S23. For any moment and any evaluation indicator, if all models' are all 1, then determine that this moment is the abnormal moment when abnormal data appears.

[0085] In this embodiment, the abnormal situations determined according to the F values of all models are as Figure 5As shown in the figure, it can be seen that there is 1 abnormal data point, which is the 2193th moment (9:00 on April 1, 2020).

[0086] Step S3: For the abnormal moment with abnormal data, consider the distribution of each variable in the abnormal moment and the time period before and after, and determine the abnormal variable value at the abnormal moment. It includes the following steps:

[0087] S31: For each abnormal time i where abnormal data occurs and the data situation evaluation index CRI=f(A, B, C), collect the variable values involved in the calculation of the data situation evaluation index at the abnormal time and T times before and after.

[0088] In this embodiment, the data situation evaluation indicator is the power generation flow deviation value, and the variables involved in its calculation process include power generation flow, water level in front of the dam, downstream water level, and output. T is set to 24, and the time i when abnormal data occurs is 2193.

[0089] S32: If there are other moments determined to be abnormal data between the ith abnormal moment and the T moments before and after it, remove the data of the other abnormal moments except the ith abnormal moment and postpone them forward / backward.

[0090] In this embodiment, there is no other abnormal data within the 2193rd moment and the 24 moments before and after it, and no processing is required.

[0091] S33: According to the data distribution of each variable in the time period, an abnormal data detection model is used to determine the abnormal situation of each variable at the abnormal moment, and the variable that is abnormal at the abnormal moment is determined as the abnormal variable value at the abnormal moment.

[0092] In this embodiment, the 3-Sigma method described above is used to analyze the time series of different variables, and the abnormal detection value at this moment is obtained as shown in Table 1. The abnormal variable at this moment is the power generation flow.

[0093] Table 1

[0094]

[0095]

[0096] Step S4: Correct the abnormal variable value identified in step S3 according to the physical relationship between the scheduling operation data. It includes the following steps:

[0097] S41. Set all abnormal variable values to null values.

[0098] In this embodiment, the power generation flow at 9:00 on April 1, 2020 is set to a null value.

[0099] S42. For any null value, adopt the water level-discharge-output relationship, water balance relationship, and hydraulic relationship used in step S1, calculate the null value based on other variable elements at this moment. If other variable elements related to it are also null and cannot be calculated, then perform interpolation based on the variable values at the moments before and after it.

[0100] In this embodiment, for the case where the generated discharge at 9:00 on April 1, 2020 is a null value, adopt the water level-discharge-output relationship, and calculate the generated discharge as 13428 m 3 / s from the water level in front of the dam, the downstream water level, and the output. The comparison of the discharge before and after correction is as Figure 6 shown. It can be seen from Figure 6 that the obvious abnormal data in the original discharge data has been replaced by the correction result, and the corrected discharge process is smoother than the measured discharge process.

[0101] Therefore, by adopting the above method for cleaning hydropower station dispatching operation data considering data physical connections, the physical relationships between hydropower station dispatching operation data and the results of different data anomaly detection models can be comprehensively considered during the data cleaning process, realizing accurate anomaly detection and correction of dispatching operation data, constructing a high-quality dispatching operation data set, and thus providing basic data support for the compilation of hydropower station dispatching plans and safe and economic operation.

[0102] Another embodiment of the present invention provides a device for cleaning hydropower station dispatching operation data considering data physical connections, including:

[0103] An evaluation index calculation module, which is used to preprocess the original dispatching operation data set S based on the physical relationship of hydropower station dispatching operation data, and calculate the evaluation index CRI of data conditions at different moments;

[0104] An abnormal moment judgment module, which is used to construct multiple abnormal data detection models based on the evaluation index CRI of data conditions, and comprehensively consider the abnormal detection results of different models to judge the abnormal moment when abnormal data appears;

[0105] An abnormal variable value determination module, which is used to consider the distribution of each variable at the abnormal moment and the time periods before and after it for the abnormal moment with abnormal data, and determine the abnormal variable value at the abnormal moment;

[0106] An abnormal variable value correction module, which is used to correct the identified abnormal variable value according to the physical relationship between dispatching operation data.

[0107] Another embodiment of the present invention provides a system for cleaning hydropower station dispatching operation data considering data physical connections, including: a computer-readable storage medium and a processor;

[0108] The computer-readable storage medium is used to store executable instructions;

[0109] The processor is used to read the executable instructions stored in the computer-readable storage medium and execute the method for cleaning the hydropower station scheduling operation data considering the physical connection of data.

[0110] Another embodiment of the present invention provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method for cleaning the hydropower station scheduling operation data considering the physical connection of data is implemented.

[0111] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0112] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of processes and / or blocks in the flowchart and / or block diagram can also be implemented. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0113] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured product including an instruction device, and the instruction device implements the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0114] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide for implementing the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1Steps of the functions specified in one or more boxes.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications or equivalent replacements can still be made to the specific implementation manners of the present invention, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A method for cleaning hydropower station scheduling operation data considering the physical connection of data, characterized in that The following steps are involved: S1. Based on the physical relationship of the hydropower station dispatching and operation data, the original dispatching and operation data set S is preprocessed to calculate the data situation evaluation index CRI at different times; S2. Based on the data situation evaluation index CRI, build multiple abnormal data detection models, comprehensively consider the abnormal detection results of different models, and determine the abnormal time when the abnormal data appears; S3. For an abnormal moment with abnormal data, consider the distribution of each variable in the abnormal moment and the time period before and after, and determine the abnormal variable value at the abnormal moment; S4. Correct the abnormal variable value identified in step S3 according to the physical relationship between the scheduling operation data.

2. The method for cleaning hydropower station dispatching operation data considering the physical connection of data according to claim 1, characterized in that Step S1 includes the following steps: S11. Set the time period and time scale of the data, collect the corresponding water level, flow rate, and output data, and construct the original scheduling operation data set S = {H, Q, N}, where H represents water level data, Q represents flow rate data, and N represents output data; S12. Evaluation indicators based on the water level-flow-output relationship, water balance relationship, and hydraulic relationship calculation data: Wherein n represents the number of moments, m represents the number of evaluation indicators, n and m are natural numbers, and the specific calculation steps include: calculating the power generation flow of the power station based on the water level-flow-output relationship, and then subtracting the measured power generation flow to obtain the power generation flow deviation value; calculating the downstream water level of the power station based on the hydraulic relationship, and then subtracting the actual downstream water level to obtain the deviation value of the downstream water level, wherein the hydraulic relationship includes the power station outflow and the power station tail water level curve; calculating the upstream end water level of the power station based on the water balance relationship and the initial water level upstream of the power station, the time period inflow, and the time period outflow, and subtracting the actual water level to obtain the upstream water level deviation value.

3. The method for cleaning hydropower station dispatching operation data considering the physical connection of data according to claim 1, characterized in that, Step S2 includes the following steps: S21. Based on the data situation evaluation indicators of the calculation, k unsupervised learning algorithms are used to construct data anomaly detection models, denoted as M1, M2, ……, M k ; S22. Based on the evaluation metrics for different types of data conditions, apply the constructed k data anomaly detection models to calculate the anomaly data monitoring results of the k models. If the value is 1, it indicates that the data is abnormal; if it is 0, it indicates that the data is normal, where 1 ≤ i ≤ n and 1 ≤ j ≤ m. S23. For any moment and any evaluation index, if the of all models are all 1, then determine that this moment is an abnormal moment when abnormal data appears.

4. The method for cleaning hydropower station dispatching operation data considering physical data connections as claimed in claim 1, wherein Step S3 includes the following steps: S31. For each abnormal moment i with abnormal data and the data situation evaluation index CRI = f(A, B, C), collect the variable values involved in calculating the data situation evaluation index at this abnormal moment and the T moments before and after it [[A i-T , …, A i , …, A i+T , [B i-T , …, B i , …, B i+T , [C i-T , …, C i , …, C i+T ; S32, if there are other moments determined to be abnormal data between the ith abnormal moment and the T moments before and after it, remove the data of the other abnormal moments except the ith abnormal moment, and postpone them forward / backward; S33. According to the data distribution of each variable in the time period, an abnormal data detection model is used to determine the abnormal situation of each variable at the abnormal moment, and the variable that is abnormal at the abnormal moment is determined as the abnormal variable value at the abnormal moment.

5. The method for cleaning the operation data of a hydropower station considering the physical connection of data according to claim 2, wherein Step S4 includes the following steps: S41. Set all abnormal variable values to null values; S42. For any null value, the water level-flow-output relationship, water balance relationship, and hydraulic relationship used in step S1 are used to calculate the null value based on other variable elements at that moment. If other variable elements related to it are also null values and cannot be calculated, interpolation is performed based on the variable values at the moments before and after it.

6. A device for cleaning the dispatching operation data of a hydropower station considering the physical connection of data, characterized in that include: The evaluation index calculation module is used to pre-process the original scheduling operation data set S based on the physical relationship of the hydropower station scheduling operation data, and calculate the evaluation index CRI of the data situation at different times; The abnormal moment judgment module is used to build multiple abnormal data detection models based on the data situation evaluation index CRI, comprehensively consider the abnormal detection results of different models, and judge the abnormal moment when the abnormal data appears; An abnormal variable value determination module, which is used to determine the abnormal variable value at the abnormal moment in the presence of abnormal data, considering the variable distributions within the abnormal moment and the time periods before and after it. An abnormal variable value correction module, which is used to correct the identified abnormal variable values according to the physical relationships among the dispatching operation data.

7. The device for cleaning hydropower station scheduling operation data considering the physical connection of data according to claim 6, wherein The evaluation index calculation module is specifically used for: Setting the time period and time scale of the data, collecting the corresponding water level, flow rate, and output data, and constructing the original dispatching operation data set S = {H, Q, N}, where H represents the water level data, Q represents the flow rate data, and N represents the output data. Calculating the data situation evaluation indexes according to the water level-flow rate-output relationship, water volume balance relationship, and hydraulic relationship. Where n represents the number of moments, m represents the number of evaluation indexes, and n and m are natural numbers. The specific calculation steps include: calculating the power generation flow rate of the power station based on the water level-flow rate-output relationship, and then subtracting the measured power generation flow rate to obtain the power generation flow rate deviation value; calculating the downstream water level of the power station based on the hydraulic relationship, and then subtracting the actual downstream water level to obtain the deviation value of the downstream water level. The hydraulic relationship includes the power station's discharge flow rate and the power station's tail water level curve; calculating the water level at the end of the upstream time period of the power station based on the water volume balance relationship, the initial upstream water level, the inflow rate during the time period, and the outflow rate during the time period, and then subtracting the actual water level to obtain the upstream water level deviation value.

8. The device for cleaning the operation data of a hydropower station considering the physical connection of data according to claim 6, wherein The abnormal moment judgment module is specifically used for: Based on the evaluation metrics of computational data, k unsupervised learning algorithms are used to construct data anomaly detection models, denoted as M1, M2, ……, M k ; Based on different types of data situation evaluation metrics, apply the constructed k data anomaly detection models, and calculate the anomaly data monitoring results of the k models If The value of 1 indicates that the data is abnormal, and the value of 0 indicates that the data is normal, where 1 ≤ i ≤ n, 1 ≤ j ≤ m; For any moment and any evaluation metric, if the of all models is 1, then it is determined that this moment is an abnormal moment when abnormal data appears.

9. The device for cleaning hydropower station dispatching operation data considering physical data connection according to claim 6, characterized in that The abnormal variable value determination module is specifically used for: For each abnormal moment i with abnormal data and the data situation evaluation index CRI = f(A, B, C), collect the variable values involved in calculating the data situation evaluation index at this abnormal moment and the previous and next T moments [[A i-T , …, A i , …, A i+T , [B i-T , …, B i , …, B i+T , [C i-T , …, C i , …, C i+T ; If there are other moments determined to have abnormal data within the i-th abnormal moment and the T moments before and after it, then remove the data of other abnormal moments except the i-th abnormal moment, and shift forward / backward. According to the data distribution of each variable within this time period, using the abnormal data detection model to respectively judge the abnormal situation of each variable at this abnormal moment, and judge the variable that shows abnormality at this abnormal moment as the abnormal variable value at this abnormal moment.

10. The device for cleaning hydropower station scheduling operation data considering physical data connections according to claim 7, characterized in that The abnormal variable value correction module is specifically used for: Setting all the abnormal variable values to null values. For any null value, using the water level-flow rate-output relationship, water volume balance relationship, and hydraulic relationship, calculating this null value according to other variable elements at this moment. If the other variable elements related to it are also null values and cannot be calculated, then perform interpolation according to the variable values at its previous and next moments.

Images