Water transfer channel operation data processing method, system and device and storage medium

Abstract

The invention discloses a water transfer channel operation data processing method, system and device and a storage medium, and relates to the technical field of water transfer channel operation data optimization. The method comprises the steps of obtaining water transfer channel operation data, and performing data cleaning on the water transfer channel operation data to obtain cleaned data; determining decomposition components based on a variational mode decomposition algorithm and the cleaned data; using a DBSCAN algorithm and a K-means algorithm to determine steady state data of the decomposition component; the steady-state data comprises the number of categories and steady-state points of each category of data; and according to the steady-state data, an overflowing coefficient is calculated by using an overflowing formula, an overflowing curve is fitted by using an outline analysis method, and the relationship among the flow, the water level and the opening degree is obtained. According to the method, data information can be deeply mined while the accuracy of original data is ensured, direct calling and use of a channel operation control system are facilitated, and the efficiency and the accuracy of channel real-time scheduling are greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of water diversion channel operation data optimization, and in particular to a water diversion channel operation data processing method, system, equipment and storage medium. Background Art

[0002] In channel water resource scheduling, monitoring of operational data is a key step in ensuring the safe operation of water conservancy projects and improving water resource utilization efficiency. Cleaning, analyzing, and mining operational data are of great practical significance for automatic channel operation control.

[0003] In the past, channel operation data was only used as a reference for manual scheduling, resulting in inefficient data utilization. However, the channel operation control system is used to improve the overall scheduling and operation of the canal system, adjusting water delivery efficiency to ensure timely and appropriate water supply. Consequently, the current scheduling data resulted in insufficient and wasted water supply, high scheduling and operation costs, and low water resource utilization. Summary of the Invention

[0004] The object of the present invention is to provide a method, system, device and storage medium for processing water diversion channel operation data, aiming to solve or improve at least one of the above-mentioned technical problems.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A method for processing water diversion channel operation data, comprising:

[0007] Acquiring water diversion channel operation data and performing data cleaning on the water diversion channel operation data to obtain cleaned data; the water diversion channel operation data includes valve opening, water level before the gate, and flow through the gate;

[0008] Determining decomposition components based on a variational mode decomposition algorithm and the cleaned data; the decomposition components include a high-frequency component, a low-frequency component, and a trend component;

[0009] Determine the steady-state data of the decomposition components using the DBSCAN algorithm and the K-means algorithm; the steady-state data includes the number of categories and the steady-state points of each category;

[0010] According to the steady-state data, the flow coefficient is calculated using the flow formula, and the flow curve is fitted using the dimensional analysis method to obtain the relationship between flow rate, water level and opening.

[0011] Optionally, performing data cleaning on the water diversion channel operation data to obtain cleaned data specifically includes:

[0012] Missing value detection and processing are performed on the water diversion channel operation data: when there is no missing data in the data of the previous set time interval, and there is missing data in the data of the next set time interval, an upward interpolation method is adopted to fill the missing value with the data of the previous set time interval; when there is a period of missing data within the set detection period, the data mean of the same missing time period is used to fill the missing value;

[0013] The water diversion channel operation data is detected and processed for outliers: the box plot method and the Laida criterion are used to judge anomalies and screen out outliers; wherein, the box plot method is based on quartiles and interquartile ranges, and points outside the interquartile range of 1.5 times the lower quartile are judged as outliers, and points outside the interquartile range of 1.5 times the upper quartile are also judged as outliers if they exist; the Laida criterion is to determine the corresponding confidence interval under the premise of a given confidence level, and errors exceeding the confidence interval are judged as not belonging to the pre-assumed random error range, and observed values ​​with errors within the confidence interval are judged as outliers.

[0014] Optionally, the DBSCAN algorithm and the K-means algorithm are used to determine the steady-state data of the decomposition components, specifically including:

[0015] The decomposition components are clustered using the DBSCAN algorithm to calculate the number of categories. Then, the K-means algorithm is used to input the number of categories, determine the central data of each category data, and obtain the steady-state point of each category data.

[0016] Optionally, the formula for fitting the overflow curve using dimensional analysis is:

[0017]

[0018] Where N is the number of holes involved in the operation of the control gate, B is the gate width of the control gate, e is the gate opening, H e is the water level difference between upstream and downstream of the gate, i and j are empirical coefficients related to the gate type and the flow rate through the gate, and g represents the acceleration of gravity.

[0019] The present invention also provides a water diversion channel operation data processing system, comprising:

[0020] a data cleaning unit, configured to obtain water diversion channel operation data and perform data cleaning on the water diversion channel operation data to obtain cleaned data; the water diversion channel operation data includes valve opening, water level before the gate, and flow through the gate;

[0021] A modal decomposition unit, configured to determine decomposition components based on a variational modal decomposition algorithm and the cleaned data; the decomposition components include a high-frequency component, a low-frequency component, and a trend component;

[0022] A clustering unit, configured to determine steady-state data of the decomposed components using a DBSCAN algorithm and a K-means algorithm; the steady-state data includes the number of categories and steady-state points of each category;

[0023] The fitting unit is used to calculate the flow coefficient using the flow formula according to the steady-state data, and to fit the flow curve using the dimensional analysis method to obtain the relationship between the flow rate, water level and opening.

[0024] The present invention also provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the above-mentioned water diversion channel operation data processing method.

[0025] The present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the water diversion channel operation data processing method as described above.

[0026] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0027] The present invention discloses a method, system, device and storage medium for processing water diversion channel operation data. The method comprises obtaining water diversion channel operation data and performing data cleaning on the water diversion channel operation data to obtain cleaned data; the water diversion channel operation data comprises valve opening, water level before the gate and flow through the gate; decomposition components are determined based on a variational mode decomposition algorithm and the cleaned data; the decomposition components comprise high-frequency components, low-frequency components and trend components; the steady-state data of the decomposition components are determined using a DBSCAN algorithm and a K-means algorithm; the steady-state data comprises the number of categories and the steady-state points of each category of data; based on the steady-state data, the flow coefficient is calculated using a flow formula, and the flow curve is fitted using a dimensional analysis method to obtain the relationship between flow, water level and opening. The present invention can, while ensuring the accuracy of the original data, deeply mine data information, facilitate direct call and use by the channel operation control system, and greatly improve the efficiency and accuracy of real-time channel scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0029] Figure 1 This is a schematic diagram comparing the opening data of the 2# gate of the Tanghe culvert aqueduct inlet before and after cleaning;

[0030] Figure 2 This is a schematic diagram comparing the water level data before and after cleaning in front of the 3# gate of the inlet regulating gate of the Tanghe culvert aqueduct in this embodiment;

[0031] Figure 3 This is a schematic diagram comparing the flow rate data before and after cleaning of the 3# gate of the inlet control gate of the Tanghe culvert aqueduct in this embodiment;

[0032] Figure 4 Schematic diagram of the variational modal decomposition of the water level in front of the 1# gate of the inlet regulating gate of the Tanghe culvert aqueduct in this embodiment;

[0033] Figure 5 This is a schematic diagram of the clustering effect of the Tanghe River culvert aqueduct inlet control gate data in this embodiment;

[0034] Figure 6 This is a fitting diagram of the flow curve of the inlet regulating gate of the Tanghe culvert aqueduct in this embodiment;

[0035] Figure 7 Schematic diagram of the principle flow in this embodiment;

[0036] Figure 8 Schematic diagram of the steps of the water diversion channel operation data processing method in this embodiment. DETAILED DESCRIPTION

[0037] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0038] The object of the present invention is to provide a method, system, device and storage medium for processing water diversion channel operation data, aiming to solve or improve at least one of the above-mentioned technical problems.

[0039] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] like Figure 1 As shown, the present invention provides a method for processing water diversion channel operation data, comprising:

[0041] Step 100: Acquire water diversion channel operation data, and clean the water diversion channel operation data to obtain cleaned data; the water diversion channel operation data includes valve opening, water level before the gate, and flow through the gate.

[0042] Step 200: Determine decomposition components based on a variational mode decomposition algorithm and the cleaned data; the decomposition components include high-frequency components, low-frequency components, and trend components.

[0043] Step 300: Determine the steady-state data of the decomposed components using the DBSCAN algorithm and the K-means algorithm; the steady-state data includes the number of categories and the steady-state points of each category.

[0044] Step 400: Calculate the flow coefficient using the flow formula according to the steady-state data, and fit the flow curve using dimensional analysis to obtain the relationship between flow rate, water level and opening.

[0045] As a specific implementation method, a specific processing process of each of the above steps is provided.

[0046] S1, real-time data cleaning;

[0047] Based on methods such as missing value detection and processing, outlier detection and processing, and mean filtering in data cleaning, real-time data such as gate opening, water level in front of the gate, and flow rate through the gate are cleaned.

[0048] S2. Variational mode decomposition of real-time data (water level before the gate, flow through the gate)

[0049] Based on variational mode decomposition, the water level in front of the gate and the flow rate data are decomposed into high-frequency components, low-frequency components and trend components.

[0050] S3. Data cluster analysis and steady-state data extraction

[0051] The DBSCAN algorithm is used to cluster the gate opening, water level in front of the gate, flow rate and other data in a certain time period (daily or monthly) to calculate the number of categories; then, the K-means algorithm is used to input the number of categories to obtain the central data of each category, which is the steady-state point of each category.

[0052] S4. Overcurrent curve fitting

[0053] According to the steady-state data, the flow coefficient is calculated using the flow formula, and the flow curve (the relationship between flow rate, water level and opening) is fitted using the dimensional analysis method.

[0054] Combining the above steps, it can be found that the method of the present invention mainly includes four parts, namely real-time data cleaning, variational mode decomposition of real-time data (water level before the gate, flow through the gate), data clustering analysis and steady-state data extraction, and flow curve fitting.

[0055] 1. Real-time data cleaning

[0056] This part corresponds to step S1, specifically:

[0057] Missing value detection and processing: judge the missing values. If the data of the previous time interval is not missing, but the data of the next time interval is missing, then the upward interpolation method is adopted, that is, the missing value is filled with the data of the previous time interval; if the data is missing for a period of time, the mean value of the data of the previous period (such as 10 minutes) is used to fill it.

[0058] Outlier detection and processing, box plot method, the standard for judging outliers by box plot is based on quartiles and interquartile ranges. Points outside the lower quartile that are 1.5 times the interquartile range are called outliers, and points outside the upper quartile that are 1.5 times the interquartile range are also called outliers if they exist; the Laida criterion, also known as the criterion, first assumes that there are only some random errors in the data, and under the premise of a given confidence level, determines the corresponding confidence interval. Errors exceeding the confidence interval are considered to be outside the range of random errors, and observations containing such errors are considered to be outliers.

[0059] Use a mean filter to reduce noise data.

[0060] 2. Variational mode decomposition of real-time data (water level before the gate, flow through the gate)

[0061] This part corresponds to step S2, specifically,

[0062] The variational mode decomposition (VMD) process is essentially a variational problem solution. By iteratively finding the optimal solution, the frequency center and bandwidth of each decomposition component are determined. VMD decomposition is used to stabilize the gate water level (gate flow) and decompose it into multiple IMF components that share the characteristics of the original data.

[0063] Each modal function h k (t) Obtain its analytical function B through Hilbert transform:

[0064]

[0065] Where δ(t) represents the unit pulse function; * represents convolution; and j is the imaginary unit.

[0066] The analytical function B of each mode obtained from the above formula is combined with the exponential coordination term Multiply to achieve h k (t) The center frequency band moves to the base frequency band:

[0067]

[0068] From this we get h k (t), and the corresponding constraints:

[0069]

[0070]

[0071] In the formula, {h k}={h1,h2,…,h k} is the set of modal functions; {ω k}={ω1,ω2,…,ω k} is the center frequency set corresponding to each mode; f is the original sequence.

[0072] After obtaining the variational equation, it needs to be solved. The quadratic penalty factor α and the Lagrange multiplier λ(t) are introduced. To ensure the convergence of the formula and make the constrained variational problem unconstrained, the constrained variational formula is converted to an unconstrained variational formula. Since the quadratic penalty factor α is a sufficiently large positive number, the signal reconstruction accuracy can be guaranteed, while the Lagrange multiplication operator ensures the strictness of the constraints. The expanded Lagrange multiplication operator expression is as follows:

[0073]

[0074] Use the multiplication operator alternating direction method to solve the above equation and continuously iterate and update The value of , until the accuracy E is greater than the discrimination accuracy ε, the iteration is stopped, and finally the component decomposition combination {h1,h2,…,h k}. The calculation is as follows:

[0075]

[0076] Where m is the number of iterations; and They are and the Fourier transform result corresponding to λ(t).

[0077] 3. Data Cluster Analysis and Steady-State Data Extraction

[0078] This part corresponds to step S3, specifically:

[0079] K-means algorithm: First, K initial centroids are selected, where K is a user-specified parameter representing the desired number of clusters. Each point is assigned to the nearest centroid, and the set of points assigned to a centroid constitutes a cluster. Then, the centroid of each cluster is updated based on the points assigned to the cluster. Clustering is complete by repeating the assignment and updating steps until the number of clusters remains constant, or equivalently, until the centroids remain constant.

[0080] DBSCAN algorithm: Any two core points that are close enough (within a given radius) are placed in the same cluster. Similarly, any border points that are close enough to a core point are also placed in the same cluster as the core point. (If a border point is close to core points of different clusters, tie resolution may be required.) Noise points are discarded.

[0081] 1. Mark all points as core points, boundary points or noise points.

[0082] 2. Delete noise points.

[0083] 3. Assign an edge between all core points within a given radius.

[0084] 4. Each group of connected core points forms a cluster.

[0085] 5. Assign each boundary point to a cluster of core points associated with it.

[0086] 4. Overcurrent curve fitting

[0087] This part corresponds to step S4, specifically:

[0088] A method for distinguishing weir water flow is introduced. Weir water flow can be divided into four types: free hole flow, submerged hole flow, free weir flow and submerged weir flow.

[0089] Method for judging water flow in weir gate: when h s When / H0>0.8, it is submerged outflow, h s When e / H0≤0.8, it is free outflow; when e / H>0.65, it is weir flow, and e / H≤0.65 is hole flow.

[0090] (1) Flow rate formula for free hole flow:

[0091]

[0092] Where M1 represents the comprehensive flow coefficient of free hole flow, B g is the total width of water flowing through the control gate, e is the gate opening, and H is the water head above the weir.

[0093] (2) Flow rate formula for free weir flow:

[0094]

[0095] Where M2 represents the comprehensive discharge coefficient of free weir flow, B g is the total width of water flowing through the control gate, and H is the water head above the weir.

[0096] (3) Flow rate formula for submerged hole flow:

[0097]

[0098] Where M3 represents the comprehensive discharge coefficient of submerged hole flow, B g is the total width of the regulating gate, e is the gate opening, H is the water head above the weir, h s The water depth behind the sluice gate.

[0099] (4) Flow formula for submerged weir flow:

[0100]

[0101] Where M4 represents the comprehensive discharge coefficient of submerged weir flow, B g is the total width of the regulating gate, H is the water head above the weir, h s The water depth behind the sluice gate.

[0102] The combined flow coefficients M1, M2, M3, and M4 in flow formulas are closely related to the weir type, operating conditions, and building characteristics. Due to the influence of various objective factors such as scale effects, equipment aging, and uncertain disturbances, existing empirical formulas or charts often fail to meet the accuracy requirements in actual engineering applications.

[0103] Introducing flow formula calibration and curve drawing. Flow formula calibration was conducted under orifice flow conditions. Based on this, curves were drawn showing the relationship between flow through the gate and gate opening under different water level differences before and after the gate. This helps dispatchers understand the gate's operating status and conduct daily gate operations in a timely manner.

[0104] Dimensional analysis method flow formula:

[0105]

[0106] Where N is the number of holes involved in the operation of the control gate, B is the gate width of the control gate, e is the gate opening, H e is the water level difference between upstream and downstream of the gate, i and j are empirical coefficients related to the gate type and the flow rate passing through the gate, respectively.

[0107] The empirical coefficients i and j of the formula are obtained by linear fitting.

[0108] As a specific embodiment, a specific experimental process as shown below is provided.

[0109] In order to better illustrate the execution process of the method of the present invention, this embodiment takes the Tanghe culvert-type aqueduct inlet regulating gate of the South-to-North Water Diversion Project as an example to demonstrate the results of the water diversion channel operation data cleaning, analysis and mining method.

[0110] First, based on the methods of missing value detection and processing, outlier detection and processing, mean filtering and other methods in data cleaning, the real-time data such as gate opening, water level before the gate, and flow through the gate are cleaned. The data cleaning effect of the present invention is as follows: Figure 1-Figure 3 shown.

[0111] Then, based on variational mode decomposition, the water level before the gate and the flow rate data are decomposed into high-frequency components, low-frequency components and trend components. The effect of the variational mode decomposition of the data of the present invention is as follows: Figure 4 shown.

[0112] Then, the DBSCAN algorithm is used to cluster the gate opening, water level in front of the gate, flow rate and other data from 00:00:00 on October 1, 2020 to 23:59:00 on October 7, 2020, and the number of categories is calculated; then, the K-means algorithm is used to input the number of categories to obtain the central data of each category of data, which is the steady-state point of each category of data. The data clustering steady-state data effect of the present invention is as follows: Figure 5 shown.

[0113] Finally, according to the steady-state data, the flow coefficient is calculated using the flow formula, and the flow curve (the relationship between flow rate, water level and opening) is fitted using the dimensional analysis method. Figure 6 shown.

[0114] By adopting the above technical solution disclosed in the present invention, the following beneficial effects are obtained:

[0115] This method uses data cleaning methods to clean real-time data to ensure the accuracy of calibration of channel water diversion related parameters; introduces DBSCAN algorithm and K-means algorithm to perform cluster analysis on real-time data to obtain channel water diversion steady-state data, so as to better calibrate channel water diversion related parameters; uses the overflow formula to calculate the overflow coefficient, and uses the dimensional analysis method to fit the overflow curve, so as to guide dispatching personnel to understand the gate operation status and carry out daily gate scheduling in a timely manner.

[0116] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0117] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A method for processing water diversion channel operation data, characterized in that: include: Acquiring water diversion channel operation data and performing data cleaning on the water diversion channel operation data to obtain cleaned data; the water diversion channel operation data includes valve opening, water level before the gate, and flow through the gate; Determining decomposition components based on a variational mode decomposition algorithm and the cleaned data; the decomposition components include a high-frequency component, a low-frequency component, and a trend component; Determine the steady-state data of the decomposition components using the DBSCAN algorithm and the K-means algorithm; the steady-state data includes the number of categories and the steady-state points of each category; According to the steady-state data, the flow coefficient is calculated using the flow formula, and the flow curve is fitted using the dimensional analysis method to obtain the relationship between flow rate, water level and opening.

2. The water diversion channel operation data processing method according to claim 1, characterized in that: The water diversion channel operation data is cleaned to obtain cleaned data, specifically including: Missing value detection and processing are performed on the water diversion channel operation data: when there is no missing data in the data of the previous set time interval, and there is missing data in the data of the next set time interval, an upward interpolation method is adopted to fill the missing value with the data of the previous set time interval; when there is a period of missing data within the set detection period, the data mean of the same missing time period is used to fill the missing value; The water diversion channel operation data is detected and processed for outliers: the box plot method and the Laida criterion are used to judge anomalies and screen out outliers; wherein, the box plot method is based on quartiles and interquartile ranges, and points outside the interquartile range of 1.5 times the lower quartile are judged as outliers, and points outside the interquartile range of 1.5 times the upper quartile are also judged as outliers if they exist; the Laida criterion is to determine the corresponding confidence interval under the premise of a given confidence level, and errors exceeding the confidence interval are judged as not belonging to the pre-assumed random error range, and observed values ​​with errors within the confidence interval are judged as outliers.

3. The water diversion channel operation data processing method according to claim 1, characterized in that: The DBSCAN algorithm and the K-means algorithm are used to determine the steady-state data of the decomposition components, specifically including: The decomposition components are clustered using the DBSCAN algorithm to calculate the number of categories. Then, the K-means algorithm is used to input the number of categories, determine the central data of each category data, and obtain the steady-state point of each category data.

4. The water diversion channel operation data processing method according to claim 1, characterized in that: The formula for fitting the overflow curve using the dimensional analysis method is: Where N is the number of holes involved in the operation of the control gate, B is the gate width of the control gate, e is the gate opening, H e is the water level difference between upstream and downstream of the gate, i and j are empirical coefficients related to the gate type and the flow rate through the gate, and g represents the acceleration of gravity.

5. A water diversion channel operation data processing system, characterized in that: include: a data cleaning unit, configured to obtain water diversion channel operation data and perform data cleaning on the water diversion channel operation data to obtain cleaned data; the water diversion channel operation data includes valve opening, water level before the gate, and flow through the gate; A modal decomposition unit, configured to determine decomposition components based on a variational modal decomposition algorithm and the cleaned data; the decomposition components include a high-frequency component, a low-frequency component, and a trend component; A clustering unit, configured to determine steady-state data of the decomposed components using a DBSCAN algorithm and a K-means algorithm; the steady-state data includes the number of categories and steady-state points of each category; The fitting unit is used to calculate the flow coefficient using the flow formula according to the steady-state data, and to fit the flow curve using the dimensional analysis method to obtain the relationship between the flow rate, water level and opening.

6. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the water diversion channel operation data processing method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that It stores a computer program, which, when executed by a processor, implements the water diversion channel operation data processing method according to any one of claims 1 to 4.

Images