An Automatic Topology Simplification Method for SWMM Models Based on Graph Databases

Abstract

This invention discloses an automatic topology simplification method for SWMM models based on graph databases, comprising the following steps: reconstructing the drainage network topology model based on the graph database; deleting branch pipe links; deleting isolated nodes and isolated pipelines; identifying necessary nodes; identifying necessary links; and deleting some nodes and fields corresponding to pipes and channels in the model file. Addressing the difficulties and time-consuming nature of solving micro-level drainage network topology SWMM models, this invention proposes an automatic topology simplification method based on graph databases. This method can delete branch pipes, isolated nodes, and isolated pipelines from the SWMM micro-level drainage network topology model while retaining important nodes, effectively simplifying the drainage network topology model and improving the efficiency of model solving.

Description

Technical Field

[0001] This invention relates to the field of municipal drainage, and in particular to an automatic topology simplification method for SWMM models based on graph databases. Background Technology

[0002] Microscopic drainage network topology models reflect the detailed topological connections of urban drainage networks as much as possible, and can restore the original appearance of urban drainage network systems to the greatest extent. However, when performing optimization scheduling and optimization and transformation analysis of drainage networks, the huge amount of data in microscopic drainage network topology models can lead to difficulties in solving the model and an astonishingly long time consumption. Therefore, it is necessary to simplify the microscopic drainage network topology model to improve the model's analysis efficiency.

[0003] The SWMM model is a stormwater runoff management model developed by the U.S. Environmental Protection Agency to address the increasingly serious urban drainage problems. This model can dynamically simulate stormwater runoff generated by single or continuous rainfall events, thereby resolving water quantity and quality issues related to urban drainage systems. It possesses powerful hydrological, hydrodynamic, and water quality simulation capabilities, providing a solid basis for the management of urban drainage networks and the protection of the water environment and aquatic ecosystems. If the SWMM model can be used to simplify the microscopic drainage network topology model, the analytical efficiency of this model will be significantly improved. Summary of the Invention

[0004] To address the aforementioned problems in existing technologies, this invention provides an automatic topology simplification method for SWMM models based on graph databases, which improves the analysis efficiency of the models and lays the foundation for the optimization scheduling and renovation analysis of drainage pipe networks.

[0005] The technical solution of the present invention is as follows:

[0006] An automatic topology simplification method for SWMM models based on graph databases includes the following steps:

[0007] S1. Rebuild the drainage network topology model based on the graph database;

[0008] S2. Delete the branch link;

[0009] S3. Delete isolated nodes; an isolated node is a node in the graph database that has neither an upstream nor a downstream channel; delete all queried isolated nodes in the graph database and record the deleted isolated nodes.

[0010] S4. Delete isolated pipelines; Isolated pipelines refer to pipelines where: first, a threshold is set, i.e., the isolated pipeline threshold; then, a weak connection algorithm is used to perform community detection on the topology model, dividing the drainage network topology model into multiple communities where nodes within each community are not interconnected; finally, a Python script is used to count the number of nodes in each community, identifying communities with a node count less than the isolated pipeline threshold. Pipelines connected to these communities are then considered isolated pipelines. All communities with a node count less than a certain set value are deleted from the graph database, i.e., all isolated pipelines are deleted, while the remaining communities are retained. The nodes and relationships in all deleted isolated pipelines are recorded.

[0011] S5. Determine the necessary nodes;

[0012] S6. Determine the necessary links;

[0013] S7. Traverse all nodes and pipes in the SWMM model INP file, delete some of the fields corresponding to nodes and pipes in the model file, and obtain the simplified topology model SWMM model file.

[0014] Furthermore, S1 includes the following steps:

[0015] S1-1. Use a Python script to read the SWMM model file and obtain the following data:

[0016] (1) The number attribute and type of all nodes;

[0017] (2) The inflow node number, outflow node number, and type of all pipes and channels;

[0018] (3) The numbering attribute of all inflow nodes in the sub-catchment areas;

[0019] S1-2. Use a Python script to organize the data obtained in S1-1 into a CSV file as specified in the Neo4j graph database administrator import method, and perform the following operations in the file:

[0020] (1) Use the nodes in the SWMM model as nodes in the graph database;

[0021] (2) Treat the pipes and channels in the SWMM model as relations in the graph database;

[0022] (3) Use the type of node in the SWMM model as the label of the node in the graph database;

[0023] (4) Use the pipe and channel types in the SWMM model as the types of relations in the graph database;

[0024] (5) Add the attribute "Sub-catchment inflow point" to the inflow node of the sub-catchment area;

[0025] (6) Construct topological connections in the graph database by using the node numbering attribute and the numbering attribute of the inlet and outlet nodes of the pipeline;

[0026] S1-3. Use the Neo4j graph database administrator import method to import the sorted data into an empty Neo4j graph database and build a city drainage network topology model based on the graph database.

[0027] Furthermore, S2 includes the following steps:

[0028] S2-1, Query multi-connection nodes; a multi-connection node refers to a node in the topology model whose number of pipes it is connected to is defined as the number of pipes it is connected to. A node is called a multi-connection node if it has three or more pipes connected to it.

[0029] Query a graph database to find all nodes that satisfy any one of the following three conditions:

[0030] (1) There is a node with two upstream pipelines and one downstream pipeline;

[0031] (2) There is a node with one upstream pipeline and two downstream pipelines;

[0032] (3) There is a node with three upstream pipelines;

[0033] Based on the node number, all the nodes found in the query are deduplicated to obtain all the multi-path nodes in the topology model;

[0034] S2-2. Query all nodes in the graph database that have no upstream pipes but have downstream pipes to obtain all the starting nodes of the drainage network.

[0035] S2-3. Set a threshold, i.e., threshold A, based on the user's requirements for branch pipe length. Then, use a Python script to traverse all the starting nodes of the drainage network found in S2-2. Query the graph database for all nodes whose downstream depth from the traversed starting node is not greater than threshold A, and perform the following operations:

[0036] If no multi-path nodes are found in the query, no operation is performed on the starting node of the traversal; otherwise, the number of multi-path nodes is checked and the following operations are performed:

[0037] If the number of multi-path nodes is equal to 1, then delete all links between the starting node and the multi-path node in the traversal, that is, all nodes and relationships, and record the deleted nodes and relationships. Otherwise, find the multi-path node with the smallest depth to the starting node in the traversal, delete all links between the starting node and the multi-path node, and record the deleted nodes and relationships.

[0038] Furthermore, S5 includes the following steps:

[0039] S5-1. Re-query all multi-connection nodes in the drainage network topology model according to the method in S2-1;

[0040] S5-2. Re-query all starting nodes in the drainage network topology model according to the method in S2-2;

[0041] S5-3. Query all nodes in the graph database labeled as drainage outlet, sub-catchment inlet, and reservoir.

[0042] S5-4. Query all inflow or outflow nodes of relationships with the categories of pump, weir and orifice in the graph database.

[0043] S5-5: Count all nodes found in S5-1 to S5-4 and perform deduplication on the nodes.

[0044] Furthermore, S6 includes the following steps:

[0045] S6-1. First, set a threshold, namely threshold B, and count all communities with more than the threshold B. Based on the numbers of all necessary nodes queried in S5 and the numbers of all nodes in each community obtained in S4, count the necessary nodes in the communities corresponding to all large pipelines and group the necessary nodes.

[0046] S6-2. Add the attribute "Necessary Link Node" to all nodes in a community where the number of nodes is greater than threshold A and less than threshold B, and add the "Necessary Link Relationship" attribute to all relationships.

[0047] S6-3. Taking the necessary node group within each community with a node count greater than the threshold B as the unit, query the longest undirected link between any two necessary nodes within each community's necessary node group, and perform the following operations:

[0048] (1) Add the attribute "necessary link node" to the nodes on the longest link found;

[0049] (2) Add the attribute "necessary link relationship" to the relationship on the longest queried link;

[0050] This results in a simplified topology model consisting of the longest undirected link between any two necessary nodes in each community's necessary node group.

[0051] Furthermore, the fields corresponding to the nodes and pipes deleted in S7 include:

[0052] (1) The INP file does not have the following attribute fields marked in the graph database: necessary link nodes;

[0053] (2) The INP file does not have the following attribute fields marked in the graph database for the corresponding pipes and channels: necessary link relationship;

[0054] (3) Fields corresponding to nodes and pipes recorded in S2 and S3.

[0055] Furthermore, the SWMM micro-drainage network topology model in S1-1 was built based on data obtained from on-site surveys of urban drainage networks, and the parameters related to head loss were not calibrated.

[0056] Furthermore, the undirected link mentioned in S6-2 refers to a link between two nodes that are connected by a relationship, without considering the direction of the relationship in the link, i.e., the direction of water flow in the pipe.

[0057] The beneficial technical effects of this invention are as follows:

[0058] Branch pipes, isolated nodes, and isolated pipelines in the SWMM micro-drainage network topology model can be deleted, while important nodes are retained, effectively simplifying the drainage network topology model and improving the efficiency of model operation and solution. Attached Figure Description

[0059] Figure 1 This is a flowchart of the present invention;

[0060] Figure 2 This is a microscopic drainage network topology SWMM model of the embodiment;

[0061] Figure 3 This is a schematic diagram of the branch link deletion in step 2;

[0062] Figure 4 The local SWMM model before deleting the branch link;

[0063] Figure 5 The local SWMM model after removing the branch links;

[0064] Figure 6 A schematic diagram of isolated nodes and isolated pipelines;

[0065] Figure 7 A schematic diagram illustrating the necessary links between the necessary nodes;

[0066] Figure 8 The example is a simplified SWMM model. Detailed Implementation

[0067] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0068] The example uses Figure 1 The steps shown illustrate the SWMM micro-topological model of the drainage pipe network in the Fenghuangqiao area of ​​Lu'an City. Figure 2 Perform automatic topology simplification:

[0069] S1. Rebuild the drainage network topology model based on the graph database, including the following two sub-steps:

[0070] S1-1. Use a Python script to read the SWMM model file and obtain the following data:

[0071] (1) The numbering attributes and node types of all nodes; There are 11,206 ordinary nodes and 157 drainage outlet type nodes in the SWMM model;

[0072] (2) The inflow node number, outflow node number, and pipe type of all pipes and channels; There are a total of 11,277 circular pipe types in the SWMM model;

[0073] (3) Numbering attributes of all sub-catchment inflow nodes; there are no sub-catchments and their corresponding nodes in the SWMM model.

[0074] S1-2. Using a Python script, organize the data obtained in S1-1 into a CSV file as specified by the Neo4j graph database administrator import method. In the file, use the nodes in the SWMM model as nodes in the graph database and the pipes and channels in the SWMM model as relations in the graph database. Use the type of the node in the SWMM model as the label of the node in the graph database, and use the type of the pipe and channel in the SWMM model as the type of relation in the graph database. Add the attribute "sub-catchment inflow point" to the sub-catchment inflow node. Construct the topological connections in the graph database through the node number attribute and the number attribute of the pipe and channel inflow and outflow nodes. Use the Neo4j graph database administrator import method to import the organized data into an empty Neo4j graph database and build a topological model of the urban drainage network based on the graph database. The built topological model contains 11,363 nodes and 11,277 relations.

[0075] S2. Delete the branch link, which includes the following 3 sub-steps:

[0076] S2-1. Querying multi-connection nodes: The number of pipes connected to a node in the topology model is called the node's "connectivity," and nodes with three or more connections are called multi-connection nodes. Query all nodes that satisfy any one of the following three conditions using a graph database:

[0077] (1) There is a node with two upstream pipelines and one downstream pipeline;

[0078] (2) There is a node with one upstream pipeline and two downstream pipelines;

[0079] (3) There is a node with three upstream pipelines;

[0080] By deduplicating all the nodes found based on their node numbers, we can obtain all the multi-path nodes in the topology model. A total of 2296 multi-path nodes were found.

[0081] S2-2. Query all nodes in the graph database that have no upstream pipes but have downstream pipes to obtain all the starting nodes of the drainage network. A total of 3043 starting nodes were found.

[0082] S2-3. Using a Python script, all starting nodes of the drainage network queried in S2-2 are traversed. The graph database is searched for all nodes within a depth of 3 downstream of the starting node. If no multi-connection node is found, no operation is performed on the starting node. If a multi-connection node exists and the number of multi-connection nodes is 1, all links between the starting node and this multi-connection node are deleted, i.e., all nodes and relationships, and the deleted nodes and relationships are recorded. If a multi-connection node exists and the number of multi-connection nodes is greater than 1, the multi-connection node with the smallest depth to the starting node is found, and all links between the starting node and this multi-connection node are deleted, and the deleted nodes and relationships are recorded. A total of 2619 starting nodes and their corresponding branch pipe links were deleted.

[0083] Figure 3 This diagram illustrates the deletion of branch links. Circles in the diagram represent nodes in the graph database, and arrows represent relationships within the graph database. As shown, nodes I and J are the starting nodes queried in step 22, and nodes B and E are the multi-way nodes queried in step 21. Since the depth between nodes I and B, and between nodes J and E, is less than 3, the links highlighted in the diagram are the branch links that need to be deleted. Figure 4 and Figure 5 This is a schematic diagram of the local SWMM model before and after the removal of the branch link.

[0084] S3. Delete isolated nodes. Query the graph database for all nodes that have neither upstream nor downstream connections (i.e., isolated nodes). Delete all the found isolated nodes from the graph database. Delete a total of 10 isolated nodes and record the deleted nodes.

[0085] S4. Delete isolated pipelines. Using a weak connection algorithm, the topology model is divided into multiple communities where nodes within each community are not interconnected. A total of 305 communities are formed. A Python script is used to count the number of nodes in each community. All communities with fewer than 10 nodes are deleted from the graph database, i.e., all isolated pipelines are deleted. The deleted nodes and their relationships are recorded, and the remaining communities are retained.

[0086] Figure 6 This diagram illustrates isolated nodes and isolated pipelines in S3 and S4. Node A in the diagram is an isolated node as it has no connected pipelines. The two pipelines highlighted in the diagram represent two communities defined by weak connectivity community detection. Nodes within the two communities are not connected to each other. One community has more than 10 nodes and is defined as a non-isolated pipeline, while the other community has less than 10 nodes and is defined as an isolated pipeline.

[0087] S5. Determine the necessary nodes, including the following 5 sub-steps:

[0088] S5-1. Re-query the multi-connection nodes. Using the same method as in S2-1, re-query all multi-connection nodes in the drainage network topology model. A total of 424 multi-connection nodes were found.

[0089] S5-2. Re-query the starting node and query all nodes that do not have an upstream pipe but have a downstream pipe. A total of 541 non-branch starting nodes were found.

[0090] S5-3. Query all nodes labeled as drainage outlet, sub-catchment inlet, and reservoir in the graph database. A total of 157 nodes of the drainage outlet type were found.

[0091] S5-4. Query the graph database for all inflow nodes of the types pump, weir, and orifice. There are no relationships of the types pump, weir, and orifice in the graph database.

[0092] S5-5: Count all nodes found in S5-1 to S5-4, and perform deduplication on the nodes. After deduplication, there are a total of 1122 necessary nodes.

[0093] S6. Determine the necessary links, which consists of the following 5 sub-steps:

[0094] S6-1. Query communities with more than 200 nodes. A total of four communities were found, with 1216, 208, 1896 and 817 nodes in each community, respectively. Based on the IDs of all necessary nodes found in S5 and the IDs of all nodes in each community obtained in S3, the communities to which each necessary node belongs are counted. The necessary nodes are grouped, and the number of necessary nodes in the four communities is 157, 27, 281 and 117, respectively.

[0095] S6-2. Add the "Necessary Link Node" attribute to all nodes in a community with more than 10 nodes but less than 200 nodes, and add the "Necessary Link Relationship" attribute to all relationships.

[0096] S6-3. Taking the necessary node groups within each community as units, query the longest undirected link between any two necessary nodes within each community's necessary node group. Add the attribute "necessary link node" to the nodes on the longest link to distinguish them from non-necessary link nodes. Add the attribute "necessary link relationship" to the relationships on the longest link to distinguish them from non-necessary link relationships. This yields a simplified topology model composed of the longest undirected links between any two necessary nodes within each community's necessary node group. The simplified drainage network topology model consists of nodes with the "necessary link node" attribute and relationships with the "necessary link relationship" attribute, containing a total of 7927 nodes and 7832 relationships. An undirected link refers to a link where there is a relationship connecting two nodes, without considering the direction of the relationship in the link, i.e., the direction of water flow in the pipe.

[0097] Figure 7 To illustrate the necessary links between necessary nodes, assume that nodes A and B are necessary nodes, and there are three links between them: ACDB, AHIB, and AEFGB. Among them, AEFGB is the longest undirected link. Therefore, add the "necessary link node" attribute to node AEFGB and the "necessary link relationship" attribute to the pipeline between the nodes.

[0098] S7. Traverse all nodes and pipes in the SWMM model INP file, delete the fields corresponding to nodes in the INP file that do not have the "necessary link node" attribute in the graph database, delete the fields corresponding to pipes in the INP file that do not have the "necessary link relationship" attribute in the graph database, and delete the fields corresponding to nodes and pipes recorded in S2-3, S3-1, and S3-2, to obtain the simplified topology model SWMM model file as follows: Figure 8 As shown.

[0099] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, and for those of ordinary skill in the art, various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention. Therefore, the present invention is not limited to the specific details without departing from the general concept defined by the claims and their equivalents.

Claims

1. An automatic topology simplification method for SWMM models based on graph databases, characterized in that, Includes the following steps: S1. Rebuild the drainage network topology model based on the graph database; S2. Delete the branch link; including the following steps: S2-1, Query multi-connection nodes; a multi-connection node refers to a node in the topology model whose number of pipes it is connected to is defined as the number of pipes it is connected to. A node is called a multi-connection node if it has three or more pipes connected to it. Query a graph database to find all nodes that satisfy any one of the following three conditions: (1) There is a node with two upstream pipelines and one downstream pipeline; (2) There is a node with one upstream pipeline and two downstream pipelines; (3) There is a node with three upstream pipelines; Based on the node number, all the nodes found in the query are deduplicated to obtain all the multi-path nodes in the topology model; S2-2. Query all nodes in the graph database that have no upstream pipes but have downstream pipes to obtain all the starting nodes of the drainage network. S2-3. Set a threshold A based on the user's requirements for branch pipe length. Then, use a Python script to traverse all the starting nodes of the drainage network found in S2-2. Query the graph database for all nodes whose downstream depth from the starting node is not greater than the threshold A, and perform the following operations: If no multi-path nodes are found in the query, no operation is performed on the starting node in the traversal; otherwise, the number of multi-path nodes is checked and the following operations are performed: If the number of multi-path nodes is equal to 1, then delete all links between the starting node and the multi-path node in the traversal, and record the deleted nodes and relationships. Otherwise, find the multi-path node with the smallest depth to the starting node in the traversal, delete all links between the starting node and the multi-path node in the traversal, and record the deleted nodes and relationships. S3. Delete isolated nodes; an isolated node is a node in the graph database that has neither an upstream nor a downstream channel; delete all queried isolated nodes in the graph database and record the deleted isolated nodes. S4. Delete isolated pipelines; Isolated pipelines refer to: first, setting an isolated pipeline threshold; then, using a weak connection algorithm to perform community detection on the topology model, dividing the drainage network topology model into multiple communities where nodes within each community are not interconnected; finally, using a Python script to count the number of nodes in each community, identifying communities with fewer nodes than the isolated pipeline threshold; these communities are the isolated pipelines; delete all communities with fewer nodes than the isolated pipeline threshold in the graph database, retaining the remaining communities, and recording the nodes and relationships in all deleted isolated pipelines; S5. Determine the necessary nodes; including the following steps: S5-1. Re-query all multi-connection nodes in the drainage network topology model according to the method in S2-1; S5-2. Re-query all starting nodes in the drainage network topology model according to the method in S2-2; S5-3. Query all nodes in the graph database labeled as drainage outlet, sub-catchment inlet, and reservoir. S5-4. Query all inflow or outflow nodes of relationships with the categories of pump, weir and orifice in the graph database. S5-5. Count all nodes found in S5-1 to S5-4 and remove duplicate nodes; S6. Determine the necessary links; including the following steps: S6-1. First, set a threshold B, count all communities with more than the threshold B, and count the communities to which each necessary node belongs based on the numbers of all necessary nodes queried in S5 and the numbers of all nodes in each community obtained in S4. Group the necessary nodes. S6-2. Add the attribute "Necessary Link Node" to all nodes in a community where the number of nodes is greater than threshold C and less than threshold B, and add the "Necessary Link Relationship" attribute to all relationships. S6-3. Taking the necessary node group within each community with a node count greater than the threshold B as the unit, query the longest undirected link between any two necessary nodes within each community's necessary node group, and perform the following operations: (1) Add the attribute "necessary link node" to the nodes on the longest link found; (2) Add the attribute "necessary link relationship" to the relationship on the longest queried link; This results in a simplified topology model consisting of the longest undirected link between any two necessary nodes in each community's necessary node group; S7. Traverse all nodes and pipes in the SWMM model INP file, delete some fields corresponding to certain nodes and pipes in the model file, and obtain the simplified SWMM model file of the topology model; the fields corresponding to certain nodes and pipes include: (1) The INP file does not have the following attribute fields marked in the graph database: necessary link nodes; (2) The INP file does not have the following attribute fields marked in the graph database for the corresponding pipes and channels: necessary link relationship; (3) Fields corresponding to nodes and pipes recorded in S2 and S3.

2. The automatic topology simplification method for SWMM models based on graph databases according to claim 1, characterized in that, S1 includes the following steps: S1-1. Use a Python script to read the SWMM model file and obtain the following data: (1) The number attribute and type of all nodes; (2) The inflow node number, outflow node number, and type of all pipes and channels; (3) The numbering attribute of all inflow nodes in the sub-catchment areas; S1-2. Use a Python script to organize the data obtained in S1-1 into a CSV file as specified in the Neo4j graph database administrator import method, and perform the following operations in the file: (1) Use the nodes in the SWMM model as nodes in the graph database; (2) Treat the pipes and channels in the SWMM model as relations in the graph database; (3) Use the type of node in the SWMM model as the label of the node in the graph database; (4) Use the pipe and channel types in the SWMM model as the types of relations in the graph database; (5) Add the attribute "Sub-catchment inflow point" to the inflow node of the sub-catchment area; (6) Construct topological connections in the graph database by using the node numbering attribute and the numbering attribute of the inlet and outlet nodes of the pipes and channels; S1-3. Use the Neo4j graph database administrator import method to import the sorted data into an empty Neo4j graph database and build a city drainage network topology model based on the graph database.

3. The automatic topology simplification method for SWMM models based on graph databases according to claim 1, characterized in that, The undirected link mentioned in S6-2 refers to a link between two nodes that are connected by a relationship, without considering the direction of the relationship in the link.

Images