A sewer network monitoring point optimization arrangement method and system based on improved K-means

Abstract

The application provides a kind of drainage pipe network monitoring point optimization arrangement method and system based on improved K-means, the method comprises: S1, gather monitoring point data as original data;S2, original data pre-processing;S3, BIRCH pre-clustering;S4, determine the optimal parameter combination of pre-clustering;S5, pre-clustering clustering result and parameter are substituted into K-means;S6, K-means clustering obtains clustering result;S7, whether the clustering result is reasonable in combination with actual verification;S8, if yes, then according to the clustering result output drainage pipe network monitoring point optimization arrangement scheme, otherwise, the preceding steps S4 to step S7 are executed cyclically.The application solves the technical problems that K value is randomly selected in K-means clustering and is sensitive to initial clustering center sample point, which affects optimization effect and optimization scheme is unreasonable.

Description

Technical Field

[0001] This invention relates to an optimization technique for the layout of monitoring points, specifically to a method and system for optimizing the layout of monitoring points in drainage pipe networks based on improved K-means. Background Technology

[0002] To more intuitively and quickly identify problems in drainage systems and monitor their daily operation, a digital management model has gradually emerged, utilizing a large number of pipeline monitoring instruments to dynamically monitor and manage the operation of urban drainage systems in real time. In the early stages of this digital management model, a large number of monitoring instruments were often required to operate at various key nodes to comprehensively understand the system's daily water quality and quantity patterns. However, as the management model matures and managers have sufficient pipeline data, to save on manpower and equipment operating costs, the layout of monitoring points in the drainage system can be optimized using statistical and operations research theories. This aims to achieve the goal of using as few monitoring points as possible to comprehensively reflect the drainage network information.

[0003] Currently, the optimization methods for monitoring points can be broadly categorized into two types: one based on statistical theory and the other based on operations research theory. Furthermore, research on the optimal layout of monitoring points in my country primarily focuses on water supply systems, with relatively less research on drainage systems, and the technical methodology is still incomplete. Yuan Jingdong, based on the hydraulic model of GZ City constructed by InfoWorksCS, optimized the layout of monitoring points in the drainage system using fuzzy clustering and dynamic proximity methods, and compared the optimization results of the two methods. Cao Bing, considering the characteristics of the drainage network, analyzed and proposed principles for monitoring point layout for different monitoring objectives. Guo Xiaochen et al. used SWMM software to construct a water quality and quantity model for the study area. Based on the time series curves of each node under different rainfall scenarios, they clustered the nodes in the area and selected representative monitoring points from the clustering results of each area using an automatic identification method. The invention patent CN201911383715.9, entitled "A Method for Arranging Pressure Monitoring Points in Water Supply Networks for Pipe Burst Monitoring," analyzes the node pressure sensitivity matrix of the network under normal and burst conditions using a clustering method based on spatial density clustering and noise points. It performs node clustering, divides the network into monitoring zones, and focuses on generated noise points. Then, it uses an improved Pareto evolutionary algorithm based on reference vectors to solve the optimal arrangement model of the water supply network pressure monitoring points, generating more feasible solution spaces. It considers the overlap and cluster coverage of monitoring points and noise points under burst conditions to obtain the final monitoring point arrangement scheme. This invention combines clustering and optimization algorithms, compares and analyzes normal and burst conditions, and fully considers the special characteristics of clustered noise points, enabling more comprehensive monitoring of the water supply network. The aforementioned prior art patents do not use the pre-clustering technology disclosed in this application, and therefore cannot overcome the shortcomings of K-means clustering in which the K value is randomly selected and sensitive to the initial cluster center sample point. The invention patent application CN201911025643.0, entitled "A Method for Segmenting Electricity Customers Based on BIRCH and Random Forest Algorithms," includes the following steps: S1: Selecting electricity customer characteristics as clustering analysis objects, collecting electricity customer information based on the selected characteristics to construct a customer dataset, and preprocessing the customer dataset; S2: Using the BIRCH algorithm to perform preliminary clustering on the preprocessed customer dataset to obtain cluster sub-clusters; S3: Using the random forest algorithm to perform secondary clustering on the cluster sub-clusters to achieve electricity customer segmentation. This invention, through electricity customer segmentation based on random forest and BIRCH algorithms, can effectively cluster customer data of power companies and, through feature analysis within customer groups, derive the overall value level of each customer group, providing support for formulating corresponding service standards.While the aforementioned existing patents utilize the BIRCH algorithm for preliminary clustering, this pre-clustering technique differs from the technical solution disclosed in this application. Furthermore, the output data type of this pre-clustering technique is significantly different from that of this application. Moreover, the existing application is not applied to the optimization of monitoring points in drainage pipe networks, and its technical solution and application field differ from those of this application. Existing technologies optimize monitoring points based on hydraulic and water quality models. During the modeling process, parameter selection inevitably deviates from actual conditions, resulting in node time series curves that do not match reality. Optimizing monitoring points based on this data may lead to unreasonable optimization schemes, impacting project operation. Existing technologies also suffer from technical problems such as the random selection of the K value in K-means clustering, sensitivity to the initial cluster center sample points affecting optimization effectiveness, and unreasonable optimization schemes. Summary of the Invention

[0004] The technical problem to be solved by this invention is how to address the issues of random selection of K values ​​in K-means clustering, the sensitivity of initial cluster center sample points to the optimization effect, and unreasonable optimization schemes.

[0005] This invention solves the above-mentioned technical problems by employing the following technical solution: A method for optimizing the layout of drainage pipe network monitoring points based on improved K-means includes:

[0006] S1. Collect data from monitoring points in the drainage network as raw data, and preprocess the raw data to obtain preprocessed data for the drainage network.

[0007] S2. Based on the pre-processing data of the drainage network, key monitoring point data and non-key monitoring point data are obtained through classification and processing. The BIRCH hierarchical clustering method is used to pre-cluster the key monitoring points and the non-key monitoring points, and the optimal parameter combination is determined with preset index data to obtain the pre-clustering results.

[0008] S3. Use the pre-clustering results as K-means input data, and input them into K-means to obtain K-means clustering results through K-means clustering.

[0009] S4. Verify the rationality of the K-means clustering results in conjunction with actual conditions, obtain clustering result verification data, and process the clustering result verification data to obtain the optimal layout scheme of the drainage pipe network.

[0010] This invention patent addresses the shortcomings of existing methods by proposing an optimized layout method for monitoring points in drainage pipe networks based on improved K-means clustering. Since the value of K in K-means clustering is random and sensitive to the initial cluster sample points, this invention patent proposes to first use BIRCH hierarchical clustering to pre-cluster the data, thereby optimizing the K-means clustering method and making the monitoring point optimization scheme more reasonable and reliable.

[0011] In a more specific technical solution, step S1 includes:

[0012] S11. Collect data from drainage network monitoring points as raw data;

[0013] S12. Obtain normalization parameters and normalize the original data accordingly to obtain normalized original data;

[0014] S13. Remove missing values ​​from the normalized data to obtain the preprocessed data of the drainage network.

[0015] In a more specific technical solution, in step S1, the drainage network monitoring point data includes: current monitoring point water volume data, current monitoring point water quality data, monitoring point geographical distribution data, monitoring point location function data, and network topology data.

[0016] This invention uses real monitoring data during node operation as raw data, which closely matches the actual operation status of the drainage network. This avoids the problem of parameter selection not matching the actual situation in the modeling process of existing technologies, making the node time series curves output by the monitoring system more consistent with the actual situation. Based on this data, the monitoring points are optimized to improve the rationality of the optimization scheme and ensure the accuracy of the engineering operation monitoring point layout scheme.

[0017] In a more specific technical solution, the normalized parameter in step S12 is the comprehensive pipeline network data.

[0018] In a more specific technical solution, step S2 includes:

[0019] S21. Based on the water quantity and quality data of the existing monitoring points, the geographical distribution and function of the monitoring points, and the pipeline layout, the existing monitoring points are divided into key monitoring points and non-key monitoring points.

[0020] S22. Use the BIRCH hierarchical clustering method to pre-cluster the key monitoring points and the non-key monitoring points. Based on the optimal parameter combination determined by the preset index data, obtain the number of clusters and cluster centers of the key monitoring points and the non-key monitoring points respectively.

[0021] S23. The number of clusters and the cluster centers of the key monitoring points and the non-key monitoring points are used as the pre-clustering result.

[0022] In a more specific technical solution, the preset index data in step S22 is the CH index.

[0023] This invention divides the actual monitoring points in the drainage pipe network into two categories: key monitoring points and non-key monitoring points. First, it performs pre-clustering through BIRCH cluster analysis to obtain the number of clusters and cluster centers for key and non-key monitoring points, respectively. Then, it inputs these as K values ​​and initial cluster centers into K-means to obtain the final cluster centers, thereby optimizing the layout of the monitoring points.

[0024] In a more specific technical solution, step S3 includes:

[0025] S31. Input the pre-clustering results as K values ​​and initial cluster center sample points into K-means;

[0026] S32. Based on the optimal parameter combination, K-means is used to process the pre-clustering results to obtain the K-means clustering results.

[0027] This invention determines the optimal number of clusters and cluster center sample points based on the CH index corresponding to different parameter combinations in BIRCH clustering, and substitutes them into K-means clustering to further improve the clustering accuracy of each monitoring point in the drainage network.

[0028] In a more specific technical solution, step S4 includes:

[0029] S41. Obtain actual layout data of drainage pipe network to determine whether the clustering results are reasonable;

[0030] S42. Determine whether the K-means clustering results are reasonable based on the preset criteria;

[0031] S43. If so, the parameters need to be adjusted again to obtain new clustering results;

[0032] S44. If not, output the K-means clustering result as the final optimization scheme.

[0033] In a more specific technical solution, the preset criteria in step S42 include: removing the impact information of non-cluster center monitoring points on the monitoring data in the K-means clustering results.

[0034] In a more specific technical solution, a drainage network monitoring point optimization layout system based on improved K-means includes:

[0035] The preprocessing module is used to collect data from monitoring points of the drainage pipe network as raw data, and preprocess the raw data to obtain preprocessed data of the drainage pipe network.

[0036] The BIRCH hierarchical pre-clustering module is used to classify and process the pre-processed drainage network data to obtain key monitoring point data and non-key monitoring point data. The BIRCH hierarchical clustering method is used to pre-cluster the key monitoring points and non-key monitoring points, and the optimal parameter combination is determined with preset index data to obtain the pre-clustering results. The BIRCH hierarchical pre-clustering module is connected to the pre-processing module.

[0037] The K-means clustering module takes the pre-clustering results as K-means input data and inputs them into the K-means system. The K-means clustering results are obtained through K-means clustering. The K-means clustering module is connected to the BIRCH hierarchical pre-clustering module.

[0038] The optimal solution acquisition module is used to verify the rationality of the K-means clustering results in conjunction with actual conditions, thereby obtaining clustering result verification data, and processing the clustering result verification data to obtain the optimal layout scheme of the drainage pipe network. The optimal solution acquisition module is connected to the K-means clustering module.

[0039] Compared with the prior art, the present invention has the following advantages: In view of the shortcomings of the existing methods, the present invention proposes an optimized layout method for monitoring points of drainage pipe network based on improved K-means. Since the value of K in K-means clustering is random and sensitive to the initial cluster sample points, the present invention proposes to first use BIRCH hierarchical clustering to pre-cluster the data, thereby optimizing the K-means clustering method and making the monitoring point optimization scheme more reasonable and reliable.

[0040] This invention uses real monitoring data during node operation as raw data, which closely matches the actual operation status of the drainage network. This avoids the problem of parameter selection not matching the actual situation in the modeling process of existing technologies, making the node time series curves output by the monitoring system more consistent with the actual situation. Based on this data, the monitoring points are optimized to improve the rationality of the optimization scheme and ensure the accuracy of the engineering operation monitoring point layout scheme.

[0041] This invention divides the actual monitoring points in the drainage pipe network into two categories: key monitoring points and non-key monitoring points. First, it performs pre-clustering through BIRCH cluster analysis to obtain the number of clusters and cluster centers for key and non-key monitoring points, respectively. Then, it inputs these as K values ​​and initial cluster centers into K-means to obtain the final cluster centers, thereby optimizing the layout of the monitoring points.

[0042] This invention determines the optimal number of clusters and cluster center samples based on the CH index corresponding to different parameter combinations in BIRCH clustering, and substitutes these values ​​into K-means clustering to further improve the clustering accuracy of monitoring points in drainage pipe networks. This invention solves the technical problems in existing technologies where the K-means clustering method involves random selection of the K value, which is sensitive to the initial cluster center samples and affects the optimization effect, as well as unreasonable optimization schemes. Attached Figure Description

[0043] Figure 1 A schematic diagram of the process for optimizing the layout of monitoring points in drainage pipe networks based on improved K-means;

[0044] Figure 2 A schematic diagram illustrating the specific process of preprocessing raw data. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] Example 1

[0047] like Figure 1 As shown, S1 collects monitoring point data as raw data. Optionally, monitoring point optimization is performed based on the hydraulic and water quality model. During the modeling process, there will inevitably be problems with parameter selection not matching the actual situation, which will also output node time series curves that do not match the actual situation. Optimizing monitoring points based on this data may output unreasonable optimization schemes, which will affect the operation of the project. Therefore, the present invention uses real monitoring data during node operation as raw data. Optionally, in order to understand the operating status of the drainage pipe network, the present invention uses a large amount of water quantity and water quality data from existing monitoring points, based on their geographical distribution and point functions, pipe network layout, data characteristics, etc.

[0048] S2. Raw data preprocessing: Optionally, this invention first selects suitable monitoring dates and monitoring indicators as raw data according to the monitoring purpose. Before performing BIRCH clustering, the raw data needs to be preprocessed.

[0049] like Figure 2 As shown, the steps for raw data preprocessing also include:

[0050] S21. Normalize the original data;

[0051] S22. Eliminate missing values ​​in the standardized data. Optionally, missing values ​​and zero values ​​in the original data can be eliminated through preprocessing. However, there are some special events in the pipeline network, such as: the flow rate in the pipeline is too small, which is lower than the monitoring threshold of the monitoring equipment, which may also lead to zero values. Therefore, the handling of zero values ​​needs to be considered in conjunction with the pipeline network situation. At the same time, in order to eliminate the influence of the units, the data also needs to be normalized or standardized.

[0052] S3, BIRCH pre-clustering, optional, divides the actual monitoring points into two categories: key monitoring points and non-key monitoring points. In view of the shortcomings of K-means clustering, which randomly selects the K value and is sensitive to the initial cluster center sample points, this invention first performs pre-clustering through BIRCH clustering analysis to obtain the number of clusters and cluster centers of key monitoring points and non-key monitoring points respectively.

[0053] S4. Determine the optimal parameter combination for pre-clustering. Optionally, in BIRCH clustering, the CH index can be used as an indicator to determine the optimal parameter combination.

[0054] S5. Substitute the pre-clustering results and parameters into K-means;

[0055] S6. K-means clustering yields clustering results. Optionally, the number of clusters and cluster center sample points determined in BIRCH clustering can be used as the K value and the initial cluster center sample points as input to K-means to obtain the final clustering results.

[0056] S7. Verify the rationality of the clustering results in conjunction with actual conditions. Optionally, the final determination should also be based on actual conditions to determine whether the clustering results are reasonable. Will removing monitoring points other than the cluster center monitoring point lead to serious data loss? If removing monitoring points affects the operation of the project, the parameters need to be adjusted again to obtain new clustering results. If it will not affect the operation of the project or cause serious data loss, it can be output as the final optimization solution.

[0057] S8. If yes, then output the optimized layout scheme of drainage network monitoring points according to the clustering results; otherwise, repeat steps S4 to S7.

[0058] In summary, this invention addresses the shortcomings of existing methods by proposing an improved K-means-based method for optimizing the layout of monitoring points in drainage networks. Since the value of K in K-means clustering is random and sensitive to the initial cluster sample points, this invention proposes to first use BIRCH hierarchical clustering to pre-cluster the data, thus optimizing the K-means clustering method and making the monitoring point optimization scheme more reasonable and reliable.

[0059] This invention uses real monitoring data during node operation as raw data, which closely matches the actual operation status of the drainage network. This avoids the problem of parameter selection not matching the actual situation in the modeling process of existing technologies, making the node time series curves output by the monitoring system more consistent with the actual situation. Based on this data, the monitoring points are optimized to improve the rationality of the optimization scheme and ensure the accuracy of the engineering operation monitoring point layout scheme.

[0060] This invention divides the actual monitoring points in the drainage pipe network into two categories: key monitoring points and non-key monitoring points. First, it performs pre-clustering through BIRCH cluster analysis to obtain the number of clusters and cluster centers for key and non-key monitoring points, respectively. Then, it inputs these as K values ​​and initial cluster centers into K-means to obtain the final cluster centers, thereby optimizing the layout of the monitoring points.

[0061] This invention determines the optimal number of clusters and cluster center samples based on the CH index corresponding to different parameter combinations in BIRCH clustering, and substitutes these values ​​into K-means clustering to further improve the clustering accuracy of monitoring points in drainage pipe networks. This invention solves the technical problems in existing technologies where the K-means clustering method involves random selection of the K value, which is sensitive to the initial cluster center samples and affects the optimization effect, as well as unreasonable optimization schemes.

[0062] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for optimizing the layout of monitoring points in drainage pipe networks based on improved K-means, characterized in that, The method includes: S1. Collect data from monitoring points in the drainage network as raw data, and preprocess the raw data to obtain preprocessed data for the drainage network. S2. Based on the pre-processing data of the drainage network, key monitoring point data and non-key monitoring point data are obtained through classification and processing. The key monitoring points and non-key monitoring points are pre-clustered using the BIRCH hierarchical clustering method, and the optimal parameter combination is determined with preset index data to obtain the pre-clustering results. S2 includes: S21. Based on the water quantity and quality data of the existing monitoring points, the geographical distribution and function of the monitoring points, and the pipeline layout, the existing monitoring points are divided into key monitoring points and non-key monitoring points. S22. Use the BIRCH hierarchical clustering method to pre-cluster the key monitoring points and the non-key monitoring points, and determine the optimal parameter combination based on the preset index data to obtain the number of clusters and cluster centers of the key monitoring points and the non-key monitoring points respectively. S23. The number of clusters and the cluster centers of the key monitoring points and the non-key monitoring points are used as the pre-clustering result; S3. Use the pre-clustering results as K-means input data, and input them into K-means to obtain K-means clustering results through K-means clustering. S4. Verify the rationality of the K-means clustering results in conjunction with actual conditions, obtain clustering result verification data, and process the clustering result verification data to obtain the optimal layout scheme of the drainage pipe network.

2. The method for optimizing the layout of drainage pipe network monitoring points based on improved K-means as described in claim 1, characterized in that, Step S1 includes: S11. Collect data from drainage network monitoring points as raw data; S12. Obtain normalization parameters and normalize the original data accordingly to obtain normalized original data; S13. Remove missing values ​​from the normalized data to obtain the preprocessed data of the drainage network.

3. The method for optimizing the layout of drainage pipe network monitoring points based on improved K-means as described in claim 1, characterized in that, In step S1, the drainage network monitoring point data includes: current monitoring point water volume data, current monitoring point water quality data, monitoring point geographical distribution data, monitoring point location function data, and network topology data.

4. The method for optimizing the layout of drainage pipe network monitoring points based on improved K-means as described in claim 2, characterized in that, The normalized parameter in step S12 is the comprehensive pipeline network data.

5. The method for optimizing the layout of drainage pipe network monitoring points based on improved K-means as described in claim 1, characterized in that, The preset index data in step S22 is the CH index.

6. The method for optimizing the layout of drainage pipe network monitoring points based on improved K-means as described in claim 1, characterized in that, Step S3 includes: S31. Input the pre-clustering results as K values ​​and initial cluster center sample points into K-means; S32. Based on the optimal parameter combination, K-means is used to process the pre-clustering results to obtain the K-means clustering results.

7. The method for optimizing the layout of drainage pipe network monitoring points based on improved K-means as described in claim 1, characterized in that, Step S4 includes: S41. Obtain actual layout data of drainage pipe network to determine whether the clustering results are reasonable; S42. Determine whether the K-means clustering results are reasonable based on the preset criteria; S43. If so, the parameters need to be adjusted again to obtain new clustering results; S44. If not, output the K-means clustering result as the final optimization scheme.

8. The method for optimizing the layout of drainage pipe network monitoring points based on improved K-means as described in claim 1, characterized in that, The preset criteria in step S42 include: removing the impact information of non-cluster center monitoring points on the monitoring data in the K-means clustering results.

9. A drainage network monitoring point optimization layout system based on improved K-means, used to execute the drainage network monitoring point optimization layout method based on improved K-means as described in any one of claims 1 to 8, characterized in that, The system includes: The preprocessing module is used to collect data from monitoring points of the drainage pipe network as raw data, and preprocess the raw data to obtain preprocessed data of the drainage pipe network. The BIRCH hierarchical pre-clustering module is used to classify and process the pre-processed drainage network data to obtain key monitoring point data and non-key monitoring point data. The BIRCH hierarchical clustering method is used to pre-cluster the key monitoring points and the non-key monitoring points, and the optimal parameter combination is determined by preset index data to obtain the pre-clustering result. The BIRCH hierarchical pre-clustering module is connected to the pre-processing module. The K-means clustering module takes the pre-clustering results as K-means input data and inputs them into the K-means system. The K-means clustering results are obtained through K-means clustering. The K-means clustering module is connected to the BIRCH hierarchical pre-clustering module. The optimal solution acquisition module is used to verify the rationality of the K-means clustering results in conjunction with actual conditions, thereby obtaining clustering result verification data, and processing the clustering result verification data to obtain the optimal layout scheme of the drainage pipe network. The optimal solution acquisition module is connected to the K-means clustering module.

Images