Urban drainage system pipe section de-weighting and rapid splicing method

Abstract

The invention provides an urban drainage system pipe section de-weighting and rapid splicing method, and relates to the field of urban infrastructure data processing, the method comprises four steps of data preprocessing, envelope surface calculation, pipe section overlapping calculation and pipe section splicing: firstly, preprocessing. Shp vector data into a standard Excel file containing attributes of pipe section coordinates, elevation, pipe diameter and the like; generating outer envelope surfaces of the two detection areas, calculating an overlapping area, and extracting pipe section data of the overlapping area; then identifying overlapped pipe sections through transverse, longitudinal and attribute triple calculation; and finally, positioning the most downstream pipe section by using a recursion method, replacing end point information, deleting repeated pipe sections, and integrating to obtain non-repeated data. The system is based on Mapbox visualization and Turf.js space calculation, the processing efficiency reaches 20-30 times that of manual work, the labor cost is reduced by 95% or above, the duplicate removal error is smaller than or equal to 0.5%, the data overlapping and splicing problems caused by boundary extension detection are effectively solved, high-quality data support is provided for intelligent water affairs, and the method can be popularized to other underground pipeline data processing fields.

Description

Technical Field

[0001] This invention relates to the field of urban infrastructure data processing technology, specifically to a method for rapid splicing and deduplication of pipe sections in an urban drainage system. Background Technology

[0002] With the rapid advancement of urbanization in my country, the area of ​​urban built-up areas is constantly expanding, and population density is continuously increasing, placing higher demands on the coverage, carrying capacity, and operational stability of drainage systems. Geophysical exploration technology, as a core means of data acquisition for urban drainage systems, can quickly acquire key information such as the spatial location, pipe diameter, pipe bottom elevation, pipe material, defects, and surrounding geological environment of underground drainage pipes through technologies such as ground-penetrating radar (GPR), pipeline robots (CCTV), and sonar detection. This provides data support for the digital modeling, operation and maintenance management, and upgrading of drainage systems. With continuous innovation in geophysical exploration technology, data acquisition accuracy has reached the centimeter level. The information acquisition dimensions for a single pipe have expanded from the traditional binary attribute of "location-diameter" to a multi-dimensional data system encompassing "spatial coordinates-structural parameters-operating status-defect information," resulting in an exponential increase in data volume.

[0003] In practical geophysical exploration, to ensure the integrity of subsequent data splicing between different zones, geophysical exploration teams typically adopt a "boundary extension exploration" strategy. This involves extending the data collection beyond the boundary area of ​​each exploration zone by an additional 2-3 manholes. This strategy stems from the linear topological characteristics of urban drainage systems—drainage pipes form a continuous network system with manholes as nodes. If exploration stops only at the boundary between adjacent zones, it can easily lead to missing boundary node information, causing "discontinuities" during subsequent data splicing and preventing the formation of a complete network topology. In the boundary area between new and old urban areas, if the geophysical exploration range in the new area is bounded by the planned road red line, and the geophysical exploration range in the old area is bounded by the existing pipeline end, and neither is extended, it can easily lead to the omission of information for 1-2 sections of pipeline in the boundary area. This results in a "disjointed" overall network model, affecting subsequent hydraulic simulations, hazard identification, and other work.

[0004] Therefore, this invention provides a rapid deduplication and splicing method for urban drainage system pipe segments. Utilizing the Mapbox open-source map engine service and the Turf.js spatial computing tool, it performs spatial topology data combination, deduplication, and splicing processing. Its processing efficiency is 20-30 times that of manual labor, reducing labor costs by over 95%, and achieving a deduplication error rate of ≤0.5%, meeting the high-quality data requirements of smart water management. Furthermore, this technology can be extended to the processing of pipe segments in other underground pipelines such as water supply networks and gas pipelines, possessing broad industrial application prospects and economic value. It can promote the digital construction of urban underground infrastructure and provide technical support for refined urban management. Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide a pipe segment overlap identification and stitching system based on an open source map service engine, which solves the problems of low efficiency, large error and high cost of traditional manual deduplication methods, and realizes automatic identification, deduplication and seamless stitching of pipe segments in multiple areas.

[0006] The technical solution adopted in this invention is to provide a method for rapid splicing of urban drainage system pipe sections to remove duplicates, including the following steps: S11. Data preprocessing: Convert common vector data formats such as .shp into standard Excel files containing attribute values ​​such as serial number, unique identifier of pipe segment, unique identifier of starting pipe point, longitude of starting pipe point, latitude of starting pipe point, elevation of pipe bottom of starting point, unique identifier of ending pipe point, longitude of ending pipe point, latitude of ending pipe point, elevation of pipe bottom of ending point, pipe diameter, and pipe length. S12. Envelope surface calculation: Generate outer envelope surfaces for the two imported regions respectively, use intersection calculation to generate corresponding envelope surface overlap areas, and then extract pipe segment data from the two regions in the envelope surface to perform pipe segment overlap calculation analysis. S13. Pipe segment overlap calculation: Perform horizontal overlap, vertical overlap and attribute similarity calculation on pipe segment overlap. When all three conditions of horizontal overlap, vertical overlap and attribute similarity are met at the same time, the pipe segment is considered to be overlapping. S14. Pipe segment splicing involves using a recursive approach to find the end point of the downstream pipe segment in overlapping pipe segments in area A that have upstream and downstream relationships, and then finding the end point information of the corresponding downstream pipe segment in area B and replacing it.

[0007] In a preferred embodiment, the present invention provides a pipeline overlap calculation method, including lateral overlap, longitudinal overlap, and attribute similarity calculation. Lateral overlap calculation involves generating 0.5m buffer zones at the start and end points of two pipe segments according to their flow direction and performing intersection calculations. Longitudinal overlap calculation involves creating buffer zones with radii equal to half the diameter of each pipe segment and determining if they intersect. If they do intersect, the difference in their bottom elevations is calculated, using the lower of the two bottom elevations as the baseline. If the difference in elevations is not greater than the diameter of the segment with the lower bottom elevation, then the two segments overlap longitudinally, and the next step, attribute similarity calculation, is performed. Attribute similarity calculation involves applying a ±5% tolerance threshold to the pipe diameter, pipe length, starting point bottom elevation, and ending point bottom elevation. If the values ​​are within the threshold, the two segments are considered to have similar attributes.

[0008] In a preferred embodiment, the present invention also provides a pipeline overlap detection and stitching system, which includes map development using the Mapbox open-source map service. First, the Mapbox map is initialized, setting the initial style, center point, and zoom level. A function is written to read two Excel files from different regions, parsing the pipeline data within them. For each parsed pipeline data line, line segment features are created based on the start and end coordinates. The imported data from the two regions are processed separately to obtain their respective pipeline feature sets. Spatial calculation methods are used to detect line segment intersections to determine the horizontal overlap, vertical overlap, and attribute similarity of pipelines. For detected overlapping pipelines, the pipelines to be retained are determined, duplicate pipelines are deleted, and the processed non-overlapping pipeline data is integrated into a new pipeline feature set. After the map is loaded, the integrated pipeline feature set is added to the map's data source. Attached Figure Description

[0009] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 This is a structural diagram of the method for rapid splicing and deduplication of drainage system pipe sections according to the present invention; Figure 2 This is a flowchart of the pipeline overlap detection and splicing system of the present invention; Figure 3 This is the data map display of area 1 of the present invention; Figure 4 This is the data map display for area 2 of the present invention; Figure 5 This is the overlapping data map display diagram of the present invention; Figure 6 This is a diagram showing the data stitching effect of the present invention. Detailed Implementation

[0010] To better understand the purpose, system architecture, and functional implementation of this embodiment, the embodiments and features in the embodiments of this application can be combined with each other without conflict. The exemplary embodiments disclosed in this application will be described below with reference to the accompanying drawings, which include specific technical details disclosed in this embodiment to aid understanding; however, these details should be considered exemplary rather than restrictive. Therefore, those skilled in the art should understand that various improvements and adjustments can be made to the embodiments described herein without departing from the scope and core ideas of the invention. Similarly, for clarity, detailed descriptions of well-known technologies, functions, and structures (such as standard image processing algorithms and common communication protocols) are omitted in the following description.

[0011] Example 1 Figure 1 This is a structural diagram of the method for rapid splicing and deduplication of drainage system pipe sections according to the present invention.

[0012] like Figure 1As shown, a method 100 for rapid splicing and deduplication of urban drainage system pipe sections includes steps S110 to S140: In step S110, data preprocessing converts common vector data formats such as .shp into standard Excel files containing attribute values; In step S120, the corresponding envelope surface overlap area is generated by intersection calculation, and then the pipe segment overlap calculation analysis is performed. In step S130, the pipe segments are calculated for lateral overlap, longitudinal overlap, and attribute similarity to determine whether the pipes overlap. In step S140, for pipe segments with overlapping sections, a recursive approach is used to splice the pipe segments.

[0013] According to an embodiment of the present invention, in operation S110, common vector data such as .shp format is read, and spatial geometric information such as the coordinates of the pipe segment's start and end points, as well as attribute table data such as pipe segment identification and pipe diameter, are parsed from the vector file using a geographic information extraction tool. Next, the extracted information is mapped and matched according to preset standard field numbers, unique pipe segment identifiers, latitude and longitude of the start / end pipe points, pipe bottom elevation, pipe diameter, and pipe length. The number is automatically generated by incrementing, and the pipe length is calculated using Turf.js to complete the spatial distance between the start and end points. Finally, the matched structured data is written to an Excel file using the SheetJS library, ensuring that each field is accurately associated with its corresponding attribute in the vector data, forming a standard Excel file containing complete attribute values, providing a unified data format for subsequent spatial calculations and overlap detection.

[0014] According to an embodiment of the present invention, in operation S120, two detection zones, zone A and zone B, to be spliced ​​are selected through the system interface. A pre-processed standard Excel file is called, and the spatial coordinates of the pipe segments, the latitude and longitude of the starting pipe point, the latitude and longitude of the ending pipe point, and attribute data are read. All pipe segments in zone A are traversed, and the starting and ending coordinates of each pipe segment are extracted and integrated into a coordinate set for calculation to generate the smallest rectangular outer envelope surface A1 that can encompass all pipe segments in zone A. Similarly, the smallest rectangular outer envelope surface A2 of all pipe segments in zone B is generated. Spatial intersection operation is performed on A1 and A2 to obtain the overlapping area A3 of the two outer envelope surfaces. For each pipe segment in zone A, it is determined whether its starting or ending point is located within A3. All pipe segments that meet the conditions are selected to form pipe segment data B1. At the same time, the same operation is performed on each pipe segment in zone B to obtain pipe segment data B2. This can accurately locate the spatial overlap range and pipe segment data of the two zones, providing a target dataset for subsequent overlapping pipe segment identification.

[0015] According to an embodiment of the present invention, in operation S130, the overlapping pipe segments are calculated for lateral overlap, longitudinal overlap, and attribute similarity. Lateral overlap calculation involves generating 0.5m buffer zones at the starting and ending points of two pipe segments C1 and C2 based on the flow direction of the pipe segments, and performing intersection calculations accordingly. If the buffer zones at the starting and ending points of C1 and C2 intersect, it is considered that there is lateral overlap, and the longitudinal overlap analysis continues. Longitudinal overlap calculation involves creating buffer zones for C1 and C2 with radii equal to half the diameter of their respective pipes, and determining whether they intersect. If there is an intersection, the difference in the bottom elevations of the two pipe segments is calculated, taking the lower of the bottom elevations as the benchmark. If the difference in pipe elevations is not greater than the diameter of the pipe segment with the lower bottom elevation, it indicates that there is longitudinal overlap, and the next step, attribute similarity calculation, is performed. Attribute similarity calculation involves applying a threshold with a tolerance of ±5% to the pipe diameter, pipe length, starting point bottom elevation, and ending point bottom elevation. If the values ​​are within the threshold, the two pipe segments are considered to have similar attributes. Pipe segments are considered to overlap when all three conditions—horizontal overlap, vertical overlap, and similar attributes—are met simultaneously.

[0016] According to an embodiment of the present invention, in operation S140, for overlapping pipe segments in area A and area B, the starting and ending pipe points of the pipe segments are used as nodes, and an association relationship is established through the unique identifier of the pipe points. A recursive method is used to traverse the topology network of overlapping pipe segments in area A. First, starting with any overlapping pipe segment, the downstream associated pipe segments are recursively queried. The starting pipe point of the next pipe segment is matched through the ending pipe point until no downstream pipe segment is found. This pipe segment is the downstreammost pipe segment among the overlapping pipe segments in area A, thereby extracting its longitude, latitude, and pipe bottom elevation, etc., as endpoint information. In the overlapping pipe segments of area B, the pipe segment corresponding to the downstreammost pipe segment of area A is found by matching the similarity of pipe segment attributes such as pipe diameter and length error ±5% with the topological relationship. Its endpoint information is extracted and replaced with the original endpoint information of the downstreammost pipe segment of area A, ensuring that the spatial coordinates of the splicing nodes are consistent. Traverse all pipe segments in area B and delete the identified overlapping pipe segments; merge the pipe segment data of area A after updating the endpoint information with the pipe segment data of area B after deleting the overlapping segments to form a final pipe segment dataset with no duplicates and continuous topology, thus completing the splicing.

[0017] Figure 2 This is a flowchart of the pipeline overlap detection and splicing system of the present invention.

[0018] like Figure 2As shown, during the Mapbox map initialization phase, the Mapbox GL JS JavaScript library and corresponding style files are first imported into the HTML page to provide basic support for map functionality. Simultaneously, a container is created to hold the map, ensuring its complete display. The Mapbox map instance is initialized via JavaScript code, key parameters are configured, the previously created container ID is bound to the map instance, the map display location is determined, and a gray background is used to highlight the blue pipe data, clearly showing the distribution of pipes within each area.

[0019] Write a geographic feature conversion function that receives parsed JSON formatted pipeline data, iterates through each pipeline data line, constructs line segment geometry information based on the latitude and longitude coordinates of the pipeline's starting and ending points, and binds attribute information to the line segments, including a unique identifier, pipe diameter, starting and ending pipe bottom elevations, pipeline length, and area identifier to distinguish between data from areas A and B. Finally, convert all pipeline data into a collection of line segment features supported by Turf.js, such as... Figure 3 and Figure 4 As shown.

[0020] Write a lateral overlap detection function that receives two pipe elements to be detected. It generates 0.5-meter buffer zones at the start and end points of each pipe, and determines whether the start buffer zone of area A intersects with the start buffer zone of area B, and simultaneously determines whether the end buffer zone of area A intersects with the end buffer zone of area B. If both intersection points are true, the two pipes are considered to have lateral overlap. Based on the pipe diameters of the two pipes, generate buffer zones with a radius of half the pipe diameter for each. If the function detects that the two buffer zones do not intersect, then there is no longitudinal overlap. If they intersect, calculate the difference in the bottom elevation of the two pipe termination points, and take the lower of the two bottom elevations as the benchmark value. If the elevation difference does not exceed 5% of the benchmark value, the two pipes are considered to have longitudinal overlap. Write an attribute similarity verification function that compares the core attributes of area A and area B, calculating the ratio of the difference in pipe diameter to the pipe diameter of area A, and the ratio of the difference in length to the length of area A. If both ratios for area A and area B do not exceed 5%, the two pipes are considered to have similar attributes. Figure 5 As shown.

[0021] Write a comprehensive overlap determination function that receives feature set A and feature set B from regions A and B. Iterate through each pipe A in feature set A and each pipe B in feature set B, calling the horizontal overlap detection, vertical overlap detection, and attribute similarity verification functions in sequence. If all three functions return true, record this pair of pipes as overlapping pipes, and finally form a list of overlapping pipes and return it.

[0022] Write a function to integrate pipe segments, receiving feature set A, feature set B, and a list of overlapping pipes. First, extract the unique identifiers of overlapping pipes in area A, construct the topological relationships of these overlapping pipes, and clarify the upstream and downstream connections between pipe segments. Second, traverse the topological relationships recursively to find the downstreammost pipe segment among the overlapping pipe segments in area A. Then, match the pipe in area B corresponding to this downstreammost pipe segment in the overlapping pipe list, and replace the endpoint coordinates and endpoint bottom elevation of the pipe in area B with the corresponding attributes of the downstreammost pipe segment in area A. Fourth, extract the unique identifiers of overlapping pipes in area B, delete these overlapping pipe segments from feature set B, and finally merge the updated feature set of area A with the deduplicated feature set of area B to obtain the integrated pipe data, such as... Figure 6 As shown.

[0023] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0024] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method for rapid splicing and deduplication of pipe sections in an urban drainage system, characterized in that, Includes the following steps: S1. Data preprocessing: Convert common vector data formats such as .shp into standard Excel files containing attribute values ​​such as serial number, unique identifier of pipe segment, unique identifier of starting pipe point, longitude of starting pipe point, latitude of starting pipe point, elevation of pipe bottom of starting point, unique identifier of ending pipe point, longitude of ending pipe point, latitude of ending pipe point, elevation of pipe bottom of ending point, pipe diameter, and pipe length. S2. Envelope surface calculation: Select two detection areas to be spliced ​​as area A and area B. Based on the standard Excel file obtained in step S1, extract the coordinates of the starting point and ending point of the pipe segment. Generate an outer envelope surface A1 that can wrap all pipe segments in area A and an outer envelope surface A2 that can wrap all pipe segments in area B. Perform intersection calculation on A1 and A2 to obtain the overlapping area A3 of the envelope surface. Extract the pipe segment data with the starting point or ending point located in A3 from area A as pipe segment data B1. Extract the pipe segment data with the starting point or ending point located in A3 from area B as pipe segment data B2. S3. Pipe segment overlap calculation: For any pipe segment C1 in pipe segment data B1 and any pipe segment C2 in pipe segment data B2, perform horizontal overlap calculation, vertical overlap calculation, and attribute similarity calculation in sequence. When the three conditions of horizontal overlap, vertical overlap, and attribute similarity are met at the same time, C1 and C2 are determined to be overlapping pipe segments. S4. Pipe segment splicing operation: For overlapping pipe segments, a recursive method is used to construct the topological relationship of overlapping pipe segments in area A. The topological network is traversed to find the downstream pipe segment in the overlapping pipe segments of area A, and its endpoint information is extracted. In the overlapping pipe segments of area B, pipe segments with similar attributes to the downstream pipe segment are matched, their endpoint information is extracted and replaced with the original endpoint information of the downstream pipe segment of area A. All overlapping pipe segments in area B are deleted. The pipe segment data of area A after updating the endpoint information is integrated with the pipe segment data of area B after deleting the overlaps to obtain the deduplicated and spliced ​​pipe segment data.

2. The method for rapid splicing and deduplication of urban drainage system pipe sections according to claim 1, characterized in that, In step S1, the specific process of data preprocessing is as follows: The spatial geometry information and attribute table data of pipe segments in .shp vector files are analyzed using geographic information extraction tools. Information is mapped and matched according to the fields of a standard Excel file, where the serial number is generated automatically by incrementing, and the pipe length is completed by calculating the spatial distance between the start and end points using Turf.js. Use the SheetJS library to write the matched structured data to an Excel file.

3. The method for rapid splicing and deduplication of urban drainage system pipe sections according to claim 1, characterized in that, in In step S2, the envelope surface calculation is based on the coordinates of the starting and ending points of all pipe segments in the area. The smallest rectangular outer envelope surface that can wrap all pipe segments in the area is calculated, and the overlapping area of ​​the two outer envelope surfaces is obtained through spatial intersection operation.

4. The method for rapid splicing and deduplication of urban drainage system pipe sections according to claim 1, characterized in that, in In step S3, the specific method for calculating the horizontal overlap is as follows: Based on the flow direction of the pipe section, a 0.5m buffer zone is generated at the starting point and ending point of C1, and a 0.5m buffer zone is generated at the starting point and ending point of C2. If the starting buffer of C1 intersects with the starting buffer of C2, and the ending buffer of C1 intersects with the ending buffer of C2, then it is determined that C1 and C2 have a horizontal overlap.

5. The method for rapid splicing and deduplication of urban drainage system pipe sections according to claim 1, characterized in that, in In step S3, the characteristic is that the specific method for calculating the vertical overlap is as follows: Buffer zone C1 is generated with a radius of half the diameter of pipe C1, and buffer zone C2 is generated with a radius of half the diameter of pipe C2. If buffer zone C1 intersects with buffer zone C2, calculate the difference in the bottom elevation of the pipe between C1 and C2, and take the lower of the two bottom elevations as the benchmark value. If the difference in the bottom elevation of the pipe is not greater than the pipe diameter corresponding to the benchmark value, then it is determined that C1 and C2 have longitudinal overlap.

6. The method for rapid splicing and deduplication of urban drainage system pipe sections according to claim 1, characterized in that, in In step S3, the specific method for calculating attribute similarity is as follows: Calculate the pipe diameter difference, pipe length difference, starting point bottom elevation difference, and ending point bottom elevation difference between C1 and C2 respectively; If the ratio of each difference to the corresponding attribute value of C1 is within the ±5% tolerance range, then C1 and C2 are determined to have similar attributes.

7. The method for rapid splicing and deduplication of urban drainage system pipe sections according to claim 1, characterized in that, In step S4, the recursive method constructs the topological relationship using the starting and ending pipe points of the overlapping pipe segments in area A as nodes, and establishes upstream and downstream connections between pipe segments through the unique identifier of the pipe points, thereby forming a topological network.

8. A rapid splicing system for deduplication of urban drainage system pipe sections, characterized in that, include: The data preprocessing module is used to read .shp vector data, parse spatial geometry and attribute information, and convert it into a standard Excel file; The spatial calculation module, based on the Turf.js tool, realizes the generation of the outer envelope surface, intersection calculation, and extraction of overlapping pipe segments; The overlap detection module performs calculations for horizontal overlap, vertical overlap, and attribute similarity to identify overlapping pipe segments; The splicing processing module analyzes the topological relationship of overlapping pipe segments using a recursive method, completes the replacement of endpoint information and deletion of overlapping pipe segments, and integrates pipe segment data; The map visualization module initializes the map based on the Mapbox open-source map service, adds the integrated pipeline data to the map data source, and realizes the visualization display of the pipeline data.

9. The rapid splicing system for deduplication of urban drainage system pipe sections according to claim 8, characterized in that, The map visualization module also supports: The standard Excel file generated in step S1 is read using a function, the pipeline data is parsed and converted into Turf.js line segment features; Once the map is loaded, the pipe segments are displayed in different colors according to the area identifier. Pipe segments in area A and area B use different colors, and overlapping pipe segments use a preset highlight color.

10. The rapid splicing system for deduplication of urban drainage system pipe sections according to claim 8, characterized in that, The system is applicable to the deduplication and splicing of pipe sections in urban water supply networks or gas pipeline networks.

Images