Distributed road network connectivity calculation method

Abstract

The invention discloses a distributed road network connectivity calculation method. The method comprises the following steps: S1, preparing a proper administrative division boundary data source and corresponding road coordinate data; s2, defining a division unit boundary for the prepared administrative division boundary data source; s3, preprocessing the prepared road coordinate data; s4, obtaining road coordinates, and constructing a road model; s5, carrying out space segmentation on the road model data; s6, carrying out the construction of a directed graph; s7, calculating a strong connectivity component of each zoning unit in a distributed manner; s8, simplifying a strong connected component; s9, reconstructing a global simplified graph; s10, calculating a strong connectivity component of the global simplified graph; according to the method, connectivity analysis can be efficiently and accurately carried out under the condition of large data volume, the calculation time is greatly shortened, and the calculation cost is saved.

Description

Technical Field

[0001] The present invention relates to a road network connectivity analysis technology, and in particular to a distributed road network connectivity calculation method. Background Art

[0002] With the popularity of in-vehicle navigation systems, people's dependence on navigation travel continues to deepen. The quality of map data is the basis of the navigation system and affects the user's travel experience. In addition to factors such as road coverage and accuracy, the connectivity of the vehicle road network also plays a vital role in the navigation effect. If a road is not connected to the next road end to end during the path finding calculation, it will cause path finding failure or detour, which will greatly affect the user experience. Therefore, it is particularly necessary to calculate the connectivity of the vehicle road network. However, with the continuous increase in data volume, traditional graph theory algorithms are often unable to cope with large-scale data and cannot meet the growing demand. Therefore, developing more efficient distributed connectivity calculation methods to adapt to the big data environment has become a topic that needs to be solved urgently. Summary of the invention

[0003] The present invention aims to provide a distributed road network connectivity calculation method, which can efficiently and accurately perform connectivity analysis in the case of large amounts of data.

[0004] The present invention is achieved through the following technical solutions:

[0005] A distributed road network connectivity calculation method comprises the following steps:

[0006] S1: Prepare appropriate administrative division boundary data sources and corresponding road coordinate data according to business needs;

[0007] S2: The prepared administrative division boundary data source is divided into administrative regions, and then the open source geographic information processing library JTS is used to uniformly construct the MultiPolygon type as the division unit, and each division unit is constructed into a mapping relationship of the type "division unit → division unit name" F c ;

[0008] S3: pre-processing the prepared road coordinate data to remove duplicate and illegal parts;

[0009] S4: Obtain road coordinates, build a road model, and convert the road coordinates into a data form suitable for graph calculation;

[0010] S5: perform spatial segmentation on the road model data, calculate the intersection of each road and boundary, and combine the mapping relationship F c Assign roads to corresponding zoning units;

[0011] S6: Create a graph object G for each road in each zoning unit, insert the starting and ending points of each road as the node information of the graph, and insert the travel direction as the edge information to obtain a directed graph;

[0012] S7: Distributed calculation of the strongly connected components of each partition unit;

[0013] S8: Simplify the strongly connected components and create a new directed graph to obtain the simplified graph G of each partition unit s ;

[0014] S9: Reconstruct the global simplified graph G g ;

[0015] S10: Calculate the global simplified graph G g The strongly connected component of .

[0016] Preferably, in step S3, the prepared road coordinate data is processed on the cluster using the Spark framework, and the collected road data is distributed to the executor for reading, cleaning and formatting.

[0017] Preferably, in step S7, a flatMap operator is added to each executor to support the output of multiple strongly connected components.

[0018] Furthermore, the strongly connected components in the operator in step S7 and the global simplified graph G in step S10 g For the calculation, Kosaraju algorithm or Tarjan algorithm can be used.

[0019] Preferably, the step S8 further includes allocating a virtual point to each strongly connected component in each actuator, and recording the key-value pair mapping relationship F of "node→virtual point" v , which is persisted and summarized on the cluster's driver side for later use.

[0020] Furthermore, the simplified graph G of each zoning unit obtained in step S8 is s The way is to first declare a new empty graph G s， Traverse each node of the original directed graph G:

[0021] If the number of nodes in the strongly connected component where the node is located is greater than 1, go to G s Insert the virtual point corresponding to the strongly connected component. If the virtual point already exists, there is no need to insert it again.

[0022] If the number of nodes in the strongly connected component where the node is located is equal to 1, and the in-degree and out-degree of the node are both 1, then the point is generally an intermediate point that is artificially interrupted during the digitization process due to its excessive length, and is directly discarded;

[0023] The remaining nodes are all isolated nodes, all inserted into G s .

[0024] After all nodes of graph G are traversed, G s That is a simplified diagram of each zoning unit.

[0025] Furthermore, in step S9, a new map G is created on the driver side. g , collect the simplified graph of the partition units in each actuator obtained in step S8 back to the driver side and insert it into G g And the corresponding mapping relationship F v The nodes of the keys in are converted to virtual points of the mapping.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] (1) Using a distributed strategy, we first perform graph theory calculations locally, then simplify the global graph, and finally achieve the effect of calculating the global graph. Compared with traditional methods, this method can efficiently and accurately perform connectivity analysis in the case of large amounts of data, and is suitable for the big data environment.

[0028] (2) This technical solution uses the Spark framework to perform parallel computing in a cluster, which can significantly reduce computing time and save computing costs. The larger the amount of data that needs to be processed, the more obvious the advantage. In addition, the rental fee for on-demand computing can also save expenses and reduce a large amount of hardware procurement costs.

[0029] (3) The global simplified graph finally summarized by this technical solution after distributed processing is a full calculation of all data. Compared with other multi-point path-finding calculations, it can more comprehensively discover map data problems and improve calculation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a flow chart of the method of the present invention;

[0031] Figure 2 A schematic diagram of constructing a directed graph of the present invention;

[0032] Figure 3 It is a schematic diagram of constructing a simplified diagram in the present invention. DETAILED DESCRIPTION

[0033] The present invention is further described in detail below in conjunction with specific embodiments, which are intended to explain the present invention rather than to limit it.

[0034] refer to Figure 1 As shown, the present invention provides a distributed road network connectivity calculation method, comprising the following steps:

[0035] S1: Define the boundaries of the division units. According to business needs, select appropriate administrative division boundary data sources, such as open data sources such as OpenStreetMap or other commercial data sources, divide the administrative regions according to actual conditions, and then use the open source geographic information processing library JTS to uniformly construct them into MultiPolygon types as division units.

[0036] For countries with large land areas and many roads, such as China and the United States, they can be divided according to the first-level administrative divisions. Other countries can choose national border divisions according to actual conditions. If custom boundaries are required, specific regional division methods should be set according to business needs. Grid division, division according to terrain features, etc. can be used. Regardless of the division method, the open source geographic information processing library JTS can be used to check the boundaries to ensure that the boundaries are closed. Finally, a string name is assigned to each division unit to construct a mapping relationship of the "division unit → division unit name" type F c For subsequent use.

[0037] S2: Road data preprocessing: Obtain road coordinate data through OpenStreetMap, supplier purchases and other channels. The data format can be GeoJSON, Shapefile, etc., or a custom format. The amount of this data is generally very large, and the Spark framework can be used to process it on the cluster. The collected road data is distributed to the executor for reading, cleaning and formatting, and duplicate and illegal parts are removed to ensure that the data meets the usage requirements.

[0038] S3: Obtain road coordinates, build a road model, and convert the road coordinates into a data format suitable for graph calculation, which at least needs to include the road type, starting point, and end point.

[0039] S4: Data segmentation: perform spatial segmentation on the road model data according to the defined boundaries, and use the intersects method in JTS and the boundary to calculate the intersection of each road, combined with the mapping relationship F c Assign the roads to the corresponding zoning units. If a road object intersects with multiple zoning units near the boundary, make multiple copies and assign them to multiple zoning units respectively. Use the flatMapPairs operator in the Spark computing framework to construct the key-value pair of "zoning unit → road object", and then assign them to each executor by key group for the next step of calculation.

[0040] S5: To construct a directed graph, first create a graph object G in each executor, then insert the starting and ending points of each road as the node information of the graph, and insert the travel direction of each road as the edge information. If the road is a one-way road, only one edge is inserted according to the driving direction. If it is a two-way road, one edge is inserted in each of the forward and reverse directions. See the example. Figure 2 .

[0041] Theoretically, there may be overlap in the longitude and latitude of the division points of non-intersecting roads in real life at places such as interchanges and tunnels during digital collection. This can be distinguished through the altitude field in the road data source, the special vertical level field, etc. Some data sources even directly provide the previous and next road numbers of each road, which can also be used to distinguish them.

[0042] S6: Distributed calculation of the strongly connected components of each partition unit.

[0043] In graph theory, a strongly connected component (SCC) of a graph refers to a subgraph in which every pair of nodes are reachable from each other, which means that in a strongly connected component, you can travel to any other place from any road.

[0044] For the transportation road network, except for special cases such as closed roads and islands without ferries, a complete road network should have only one strongly connected component, which can cover all open roads and be accessible to vehicles. If there are more than one strongly connected components, except for the largest main road network connected component, other connected components cannot form two-way connectivity with the main road network connected component. These unconnected connected components are where the connectivity loss points need to be analyzed and repaired. In addition, for distributed computing, there will be many in and out roads on the boundaries of the division units. When each independently running executor is running, the end connection information of these roads (especially one-way roads) is unknown, so many strongly connected components at the boundaries will also be calculated.

[0045] Since there may be multiple strongly connected components calculated from the data of each input division unit, a flatMap operator can be added to each executor to support the output of multiple strongly connected components. For the calculation of the strongly connected components in the operator, the Kosaraju algorithm or the Tarjan algorithm can be used.

[0046] Taking the Kosaraju algorithm as an example, it is an efficient algorithm for finding strongly connected components in a directed graph. It is implemented through two depth-first searches. The first depth-first search is performed on the original graph and the nodes are sorted according to the completion time. The second depth-first search is performed on the reverse graph in reverse order of the node completion time of the first depth-first search to obtain each strongly connected component. The time complexity of the algorithm is O(V+E), where V is the number of nodes and E is the number of edges. After calculation, each node will be classified into a strongly connected component.

[0047] S7: Simplify strongly connected components.

[0048] In each executor, a virtual point is assigned to each strongly connected component, and the key-value mapping relationship F of "node→virtual point" is recorded. v , output it persistently and summarize it on the driver side for later use.

[0049] Create a new directed graph as the simplified graph G of each partition unit s , traverse each node of the original directed graph G:

[0050] If the number of nodes in the strongly connected component where the node is located is greater than 1, go to G s Insert the virtual point corresponding to the strongly connected component. If the virtual point already exists, there is no need to insert it again.

[0051] If the number of nodes in the strongly connected component where the node is located is equal to 1, and the in-degree and out-degree of the node are both 1, then the point is generally an intermediate point that is artificially interrupted during the digitization process due to its excessive length, and is directly discarded;

[0052] The remaining nodes are all isolated nodes, all inserted into G s .

[0053] After all nodes of graph G are traversed, G s This is a simplified diagram of each zoning unit. Figure 3 .

[0054] In actual operation, for the map data sources available on the market (including many free open source data of poorer quality), most of the roads are connected, so the number of nodes in the simplified graph will be significantly reduced. However, if not, it means that there are huge problems with the quality of the map data source. At this time, more resources should be invested in the collection and processing of map data to ensure the accuracy and completeness of the data.

[0055] S8: Reconstruct the global simplified graph G g , create a new graph G on the driver side g , simplify the partition unit graph G in each actuator in the previous step sCollected back to the driver side and inserted into G g In each insertion process, check whether the current node is the mapping relationship F of the previous step. v If it is the key in , convert the current node into a mapped virtual point.

[0056] S9: Calculate the global simplified graph G g The strongly connected components of G are consistent with the method of calculating the strongly connected components in the partition unit, because the global simplified graph G g It has been greatly simplified, and one driver is sufficient to complete the calculation in a relatively short time. After the calculation is completed, the results can be analyzed subsequently, thereby continuously improving the map quality.

[0057] An embodiment performs connectivity analysis on road data throughout China, using the distributed road network connectivity calculation method proposed in the present invention, and the specific implementation steps are as follows:

[0058] 1) Data preparation: Use Amap as the road data source to obtain road coordinate data and provincial boundary data within the area in GeoJSON format, with a data volume of about 100 GB.

[0059] 2) Data preprocessing: Use the Spark framework to distribute road data to multiple executors in the cluster for deduplication, cleaning, and formatting to remove redundant points and invalid road entries in the data to ensure that the data meets the analysis requirements.

[0060] 3) Division of regional units: Choose to divide according to provincial administrative boundaries, use the JTS geographic information processing library to construct the boundaries into MultiPolygon type, and assign the province name as a unique identifier for each regional unit.

[0061] 4) Road data partitioning: Road data is spatially divided according to provincial boundaries to ensure that each road belongs to a clear division unit. For roads that cross divisions, multiple copies are used to assign them to corresponding division units.

[0062] 5) Directed graph construction: In each zoning unit, the starting point and end point of the road section are used as nodes, and edges are inserted according to the direction of travel to construct a directed graph. A one-way road is inserted with a one-way edge, and a two-way road is inserted with two forward and reverse edges.

[0063] 6) Calculation of strongly connected components: The Kosaraju algorithm is used to calculate the strongly connected components in each division unit. The calculation results show that there are several small strongly connected components near some newly built roads in some provinces, which are not connected to the main road network. In addition, there are some strongly connected components formed by the division of division units on some cross-provincial roads at the edges of provinces.

[0064] 7) Simplify strongly connected components: Assign virtual points to each strongly connected component and create a simplified graph of the zoning unit. In the simplified graph, the number of nodes of the strongly connected components is greatly reduced, which optimizes the performance of subsequent global calculations.

[0065] 8) Reconstruct the global simplified graph G g : Merge the simplified maps of each zoning unit into a global simplified map.

[0066] 9) Calculate the global simplified graph G g Strongly connected components: By calculating the strongly connected components of the global graph, cross-provincial roads can be connected back to the main road network after splicing the data of neighboring provinces, and the corresponding isolated strongly connected components are eliminated, while the strongly connected components of the newly built roads continue to be retained. Finally, the coverage rate of the main road network connected components reached 99.3%, but some newly added roads had connectivity problems, which needed to be analyzed and data repaired.

[0067] Evaluation of the computing performance and effect of this solution: A 10-node Spark cluster was used, with each node configured with a 16-core CPU and 128GB of memory. The total computing time was 1 hour, which saved about 90% of the time compared to the traditional single-machine method. At the same time, the accuracy of the unconnected areas discovered was significantly improved compared to the traditional method.

[0068] The distributed road network connectivity calculation method proposed in the present invention is also applicable to the connectivity analysis of the European road network and small-scale urban road networks.

[0069] The above contents are further detailed descriptions of the present invention in combination with specific embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these inventions. For ordinary technicians in the technical field of the present invention, without departing from the concept of the present invention, they can also make several simple deductions or substitutions, which should be regarded as belonging to the protection scope of the present invention.

Claims

1. A distributed road network connectivity calculation method, characterized in that: The following steps are involved: S1: Prepare appropriate administrative division boundary data sources and corresponding road coordinate data according to business needs; S2: The prepared administrative division boundary data source is divided into administrative regions, and then the open source geographic information processing library JTS is used to uniformly construct the MultiPolygon type as the division unit, and each division unit is constructed into a mapping relationship of the "division unit→division unit name" type F c ; S3: pre-processing the prepared road coordinate data to remove duplicate and illegal parts; S4: Obtain road coordinates, build a road model, and convert the road coordinates into a data form suitable for graph calculation; S5: perform spatial segmentation on the road model data, calculate the intersection of each road and boundary, and combine the mapping relationship F c Assign roads to corresponding zoning units; S6: Create a graph object G for each road in each zoning unit, insert the starting and ending points of each road as the node information of the graph, and insert the travel direction as the edge information to obtain a directed graph; S7: Distributed calculation of the strongly connected components of each partition unit; S8: Simplify the strongly connected components and create a new directed graph to obtain the simplified graph G of each partition unit s ; S9: Reconstruct the global simplified graph G g ; S10: Calculate the global simplified graph G g The strongly connected component of .

2. A distributed road network connectivity calculation method according to claim 1, characterized in that: In step S3, the prepared road coordinate data is processed on the cluster using the Spark framework, and the collected road data is distributed to the executor for reading, cleaning and formatting.

3. A distributed road network connectivity calculation method according to claim 1, characterized in that: In step S7, a flatMap operator is added to each executor to support the output of multiple strongly connected components.

4. A distributed road network connectivity calculation method according to claim 3, characterized in that: The strongly connected components in the operator in step S7 and the global simplified graph G in step S10 g Calculation uses Kosaraju's algorithm or Tarjan's algorithm.

5. A distributed road network connectivity calculation method according to claim 1, characterized in that: The step S8 also includes allocating a virtual point to each strongly connected component in each actuator, and recording the key-value pair mapping relationship F of "node→virtual point" v , which is persisted and summarized on the cluster's driver side for later use.

6. A distributed road network connectivity calculation method according to claim 5, characterized in that: The simplified graph G of each zoning unit obtained in step S8 s The way is to first declare a new empty graph G s , traverse each node of the original directed graph G: If the number of nodes in the strongly connected component where the node is located is greater than 1, go to G s Insert the virtual point corresponding to the strongly connected component. If the virtual point already exists, there is no need to insert it again. If the number of nodes in the strongly connected component where the node is located is equal to 1, and the in-degree and out-degree of the node are both 1, then the point is generally an intermediate point that is artificially interrupted during the digitization process due to its excessive length, and is directly discarded; The remaining nodes are all isolated nodes, all inserted into G s . After all nodes of graph G are traversed, G s That is a simplified diagram of each zoning unit.

7. A distributed road network connectivity calculation method according to claim 5, characterized in that: In step S9, a new map G is created on the driver side. g , collect the simplified graph of the partition units in each actuator obtained in step S8 back to the driver side and insert it into G g And the corresponding mapping relationship F v The nodes of the keys in are converted to virtual points of the mapping.

Images