Automatic processing method for multi-source spatial data topology consistency for fast update of vector database

Abstract

The application provides a multi-source spatial data topology consistency automatic processing method for quick update of a vector database, and belongs to the field of topology relation checking. The method comprises the following steps: pre-processing a fusion data layer graph spot, processing the topology consistency of the fusion data layer graph spot and a basic database graph spot, and post-processing the fusion data layer graph spot; setting a topology consistency processing threshold value of the fusion data layer graph spot and the basic database graph spot, and obtaining a processing result according to the processing. The application uses an R-tree spatial index method to search for geometric graph spots with common points, common edges or overlaps, so that the accuracy is not sacrificed while meeting the quick search. The application processes and aligns the boundaries of points, lines and surfaces of the geometric graph spots in all directions, and has high correction accuracy and high speed.

Description

Technical Field

[0001] This invention relates to the field of topology relationship checking, and more particularly to an automatic method for processing the topology consistency of multi-source spatial data for rapid updates of vector databases. Background Technology

[0002] Geospatial databases are crucial infrastructure supporting numerous fields such as natural resource management, urban planning, and environmental monitoring. The timeliness and accuracy of these databases are paramount, requiring regular updates and maintenance based on business needs. Typical update processes (e.g., annual change surveys, quarterly monitoring updates, and database construction for special censuses) usually involve merging changed features with historical versions of the baseline geodatabase to generate a new database version reflecting the current situation. However, a key challenge in this merging process is ensuring that the boundaries of changed features are strictly topologically consistent with the boundaries of their corresponding baseline geodatabase features (i.e., no overlapping, gaps, or other topological errors). Such topological inconsistencies may stem from data acquisition errors, differences in processing procedures, or legacy issues. For regions with wide coverage (e.g., large districts, counties, provinces, and cities) or numerous legacy issues, the number of topological errors during initial database construction or incremental updates is often enormous (tens of thousands or more), making manual inspection and repair extremely labor-intensive, inefficient, and prone to errors. While some general-purpose topology checking and batch repair tools exist in the market, their effectiveness is often unsatisfactory when dealing with this specific type of problem involving complex boundary matching and topology consistency between old and new data layers: either the repair is incomplete, leaving hidden dangers; or over-processing destroys valid data; or the efficiency still cannot meet the needs of large-scale data processing. Therefore, there is a need to design an automatic topology consistency processing method for multi-source spatial data that is rapidly updated for vector databases. Summary of the Invention

[0003] The purpose of this invention is to provide an automatic method for handling topological consistency of multi-source spatial data in rapid updates of vector databases, solving the technical problem of maintaining topological consistency between old and new data layers during incremental updates of geospatial databases. Through in-depth analysis of relevant data specifications and processing procedures, this method can intelligently identify and accurately repair topological inconsistencies between changed polygons and their corresponding base polygon boundaries, significantly improving the quality and efficiency of data fusion, and providing core support for building and maintaining a high-precision, highly consistent geospatial database.

[0004] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0005] An automatic method for handling topology consistency of multi-source spatial data for rapid updates of vector databases, the method comprising the following steps:

[0006] Step 1: Preprocess the mosaic patterns in the fused data layer;

[0007] Step 2: Ensure topological consistency between the merged data layer patches and the base database patches;

[0008] Step 3: Post-process the fused data layer patches;

[0009] Step 4: Set the threshold for topological consistency processing between the fused data layer patches and the basic database patches, and obtain the processing results according to Steps 1-3.

[0010] Furthermore, the specific process of step 1 is as follows:

[0011] Step 1.1: Read the data of the data layer to be fused as a Shapefile or gdb format file, obtain the corresponding basic data, including the projection coordinates of each patch. All correction processes only need to modify the geometry of the patches in the features, i.e., the outline coordinate points.

[0012] Step 1.2: Obtain the patch communities and independent patches in the data of the data layer to be fused. Based on the R-tree spatial indexing method, find all the patch communities in the data of the data layer to be fused, obtain the index number of all patches and independent patches in all patch communities, and obtain the set of all index numbers and corresponding patch outline coordinates to ensure that the data storage structure after processing is consistent with that before processing.

[0013] Step 1.3: Process the polygon communities and independent polygons separately. For independent polygons, duplicate point deletion, self-intersection check and correction, and sharp corner deletion are required. For polygon communities, the same processing as for independent polygons is required, followed by replacement of neighboring points between polygons in the community.

[0014] Furthermore, the specific process of step 2 is as follows:

[0015] Step 2.1: Load the data of the data layer to be merged and the basic database, and check the consistency of the geographic coordinate system and projected coordinate system of the two data. The coordinate systems of the two data must be consistent. If the coordinate systems of the basic database of the two coordinate systems of the data layer to be merged are inconsistent, the coordinate system of the basic database shall be used as the standard. Assign the coordinate system of the basic database to the data layer to be merged, obtain the outline coordinates of the basic database patches and the patches of the data layer to be merged, and store them in different arrays for later use.

[0016] Step 2.2: First, obtain the set of base database patches that intersect with the patches of the data layer to be merged using the R_tree indexing method. Determine which patches are reference patches based on the percentage threshold of the area of ​​the data layer patches to be merged relative to the area of ​​the intersecting base database patches. Then, perform operations such as neighbor point replacement, perpendicular point replacement, and patch overlap processing based on the data layer patches to be merged and the reference patches of the base database. The reference patches remain unchanged; only the points, lines, and surfaces of the data layer patches to be merged need to be corrected. This allows the boundaries of the data layer patches to be merged to be aligned with the boundaries of the reference patches of the base database, improving the accuracy and efficiency of data updates.

[0017] Furthermore, the specific process of step 3 is as follows: self-intersection processing, which is to check and correct the self-intersection of the data layer patches to be fused after step 2; small acute angle processing, which is to check and correct the small internal angles; small area fragmented patch processing, which is to check and correct the small area patches; and multiple patch gap filling processing, which is to check and fill the gaps formed between patches in the patch cluster.

[0018] Furthermore, in step 4, the manually set threshold is an acute angle threshold a = 15°.

[0019] Furthermore, in step 4, the distance threshold between two points or between a point and a line segment is m = 0.8m, the reference patch judgment threshold is w = 0.1, and the patch area threshold is s = 30.

[0020] The present invention, by adopting the above-described technical solution, has the following beneficial effects:

[0021] (2) The present invention uses the R-tree spatial indexing method to retrieve geometric patch sets that have common points, common edges or overlaps, which can satisfy fast retrieval without sacrificing accuracy;

[0022] (2) The present invention processes and aligns the boundaries of geometric patches in all aspects, including points, lines and surfaces, with high accuracy and fast speed.

[0023] (3) The present invention is divided into three major steps: preprocessing of the data layer patches to be merged, topological consistency processing of the data layer patches to be merged and the basic database patches, and postprocessing of the data layer patches to be merged. The preprocessing of the data layer patches to be merged is performed before the topological consistency processing of the data layer patches to be merged and the basic database patches, which allows the data layer patches to be merged to perform self-checks and reduce obvious topological errors. The preprocessing of the data layer patches to be merged is performed after the topological consistency processing of the data layer patches to be merged and the basic database patches, which reduces new errors that occur after the consistency processing and improves the accuracy and pass rate of the consistency processing. Attached Figure Description

[0024] Figure 1This is a flowchart of the method of the present invention;

[0025] Figure 2 This is a flowchart of the R-tree spatial indexing method for retrieving map community data in this invention;

[0026] Figure 3 This is a schematic diagram of the patch community and independent patches of the present invention;

[0027] Figure 4 This is a schematic diagram of the preprocessing of the variation pattern in this invention;

[0028] Figure 5 This is the base database of variant and intersecting patterns for this invention;

[0029] Figure 6 This is a schematic diagram illustrating the consistency processing between the changed patterns and the basic library topology in this invention;

[0030] Figure 7 This is a schematic diagram of the post-processing and merging of small patches and gaps in the modified patch of the present invention;

[0031] Figure 8 This is an experimental result diagram of the consistency processing of the variation pattern and the basic library topology in this invention. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and preferred embodiments. However, it should be noted that many details listed in the specification are merely to provide the reader with a thorough understanding of one or more aspects of the present invention, and these aspects of the invention can be implemented even without these specific details.

[0033] Please see Figures 1-8 An automatic method and system for handling topology consistency of multi-source spatial data in a vector database with rapid updates, comprising the following steps:

[0034] S1, Preprocessing of data layer patches to be merged.

[0035] (1) In this example, the data to be merged is a Shapefile or gdb format file. Read the Shapefile or gdb file to obtain the corresponding basic data. The file contains basic information such as geometryType, features, fields, and spatialReference. Features contain the coordinates of each polygon outline. All correction processes only need to modify the polygon geometry in features, while other attributes remain unchanged.

[0036] (2) Obtain the patch communities and independent patches in the data layer to be fused, such as Figure 2 This invention presents a technical approach for retrieving polygon communities using the R-tree spatial indexing method. The R-tree spatial indexing method is introduced to find all polygon communities in the data to be merged. First, the outermost rectangles of all polygons are obtained and used as nodes in the R-tree index. Then, a query range is set. Assuming any polygon A is denoted as query range A, according to geometric relationships, if query range A intersects with other polygons B, then A must intersect with the outermost rectangle of polygon B. Next, each polygon (query range A) is traversed to check if it intersects with the outermost rectangle (R-tree index node) of other polygons B. If they intersect, they are grouped as [A, B], with individual A and B polygon numbers, and stored in an array denoted as I. Based on geometric relationships, since the query range A intersects with the outermost rectangle of a certain patch B, but range A does not necessarily intersect with a certain patch B, we only need to further search in array I for all other intersecting patches of the query range A to finally obtain all patch clusters and the index numbers of all patches in each cluster. After removing the patch clusters, the remaining patches in the entire layer data are independent patches, and their index numbers can be easily obtained, such as... Figure 3 The diagram shows the community and individual patches. The set of patch community indices in the diagram is {[19322, 22390,29834, 32187], [4394, 9009, 40147, 41948]}, and the set of individual patch indices is {26550, 39694}. In addition, all index numbers and the corresponding set of patch outline coordinates are obtained.

[0037] (3) Separate processing of polygon clusters and independent polygons. First, traverse all polygon indices in the data to be merged in sequence to determine whether the index is in the polygon cluster index set or the independent polygon index set. If it is in the independent polygon index set, process it according to the following steps, which are divided into three steps: First, delete duplicate points. Store all point coordinates of a single polygon in the set and delete duplicate points using the deduplication method of the set; Second, perform self-intersection check and correction. Decompose the polygon of a single geometric polygon into rings and edges. Perform pairwise intersection tests on each edge (excluding adjacent edges). If an intersection point with non-segment endpoints is detected, mark it as a self-intersection point. For self-intersection cases, geometric reconstruction can be performed, that is, insert new nodes at all self-intersection points, split the "X" shaped intersection point into four segment endpoints, generate multi-part polygons, delete small-area geometry, and only retain the single-part polygon with the largest area, such as Figure 4 (a) and Figure 4(d) shows the third step: sharp angle deletion. First, calculate all interior angles of a single patch. If an interior angle is less than the acute angle threshold 'a', delete the angle. This involves using the corner point as the center, a radius equal to one-third of the length of the shorter included side, and replacing the corner point with the two intersection points of the circle and the two included sides. If the interior angle is a very small acute angle, such as 'a / 2', then delete the corner point directly. Figure 4 (b) and Figure 4 As shown in (e). If the traversed index is in the patch community index set, in addition to the above three steps, neighbor point replacement is also required. Assuming there are two patches x1 and x2 in the patch community, and the distance between point a1 in x1 and point a2 in x2 is less than a set distance threshold m, then point a2 in x2 is replaced with point a1 in x1, as shown in (e). Figure 4 (c) and Figure 4 As shown in (f).

[0038] S2, topological consistency processing between the data layer patches to be merged and the basic database patches.

[0039] (1) Load the data layer to be merged and the base database. The loading format of the data layer to be merged is the same as described in step S1(1). The base database can be a gdb format file or a Shapefile format file, but ultimately only the geometric contour coordinates of the features need to be read. Check the consistency of the geographic coordinate system and the projected coordinate system of the two data. The coordinate systems of the two data must be consistent. If the coordinate systems of the features in the data layer to be merged are inconsistent with the coordinate system of the base database, the coordinate system of the base database shall be used as the standard, and the coordinate system of the base database shall be assigned to the feature data of the data layer to be merged.

[0040] (2) First, obtain the set of base database patches that intersect with the patches in the data layer to be fused using the R_tree indexing method (the method is similar to the method for obtaining the patch community index set in step S1(2)): Since the base database is full land cover, each patch C in the data layer to be fused can find a set of database patches D{d1,d2,d3,…,di} (i≥1) that intersect with it, such as Figure 5 As shown, for the intersection area Scdi of patch C and D, which is w of the area of ​​patch C, where w is the reference patch determination percentage threshold, then di is used as the reference patch for C and stored in the reference patch set R{di, di+1,…dj} (j≥1). The second step is to replace the neighboring points of patch C with the reference patch di. Assuming the coordinates of point C are ci and the coordinates of any point di in set R are ki, if ci=ki, then point ci remains unchanged; if ci≠ki and the distance between ci and ki is less than the set distance threshold m, then point ci is replaced with point ki. Figure 6As shown in (a) and 6(f); the third step is perpendicular point replacement. If the distance between point ci and a line segment li in any patch di in set R is less than the distance threshold m, then point ci is replaced with the coordinate q of the perpendicular point from point ci to line segment li, as shown in (a) and (b) and (c). Figure 6 As shown in (b) and 6(g); the fourth step is the patch overlap processing. If the intersection area of ​​patch C and reference patch di is less than the area threshold s, then the intersection part of patch C is deleted. If the difference set between patch C and reference patch di is a single-part geometry and its area is less than the area threshold s, then patch C will merge the difference set. If the difference set is a multi-part geometry, it needs to be split into multiple single-part difference sets. If the area of ​​a single-part difference set is less than the area threshold s, patch C merges the single-part difference sets, such as... Figure 6 As shown in (c)~6(j).

[0041] S3 Post-processing of data layer patches to be merged.

[0042] After the four steps of S2(2), since the topological consistency processing of the data layer patches to be merged is based on the basic database patches, the shape of the data layer patches to be merged and the spatial geometric relationship between the data layer patch communities to be merged will change to varying degrees. New problems such as self-intersection, small acute angles, small-area fragmented patches, and overlap and gaps between multiple patches may occur, requiring further processing. The post-processing of the data layer patches to be merged is divided into four steps: First, self-intersection processing, which uses the processing method according to step S1(2); Second, small acute angle processing, which uses the processing method according to step S1(3); Third, small-area fragmented patch processing, if the patch area is less than the area threshold s, it is merged into the nearest data layer patch to be merged that is in contact with it; if the small-area fragmented patch has no data layer patch to be merged that is in contact with it, it is directly deleted, such as Figure 7 (a) shows the fourth step, handling the case of gaps between multiple patches. Assume a patch community in a data layer to be fused contains four patches, C1, C2, C3, and C4, with corresponding database reference patches R1, R2, R3, and R4. First, extract the outer contour surface of the patch community, denoted as X. Then, calculate the difference between X and each patch C1, C2, C3, and C4. The remaining part is the gap patch F. If geometry F is contained within any patch of R1, R2, R3, or R4 (e.g., contained within R1), then patch C1 will merge with F, while other patches remain unchanged. Figure 7 As shown in (b).

[0043] S4 Threshold and Result for Topological Consistency Processing between Data Layer Patches to be Merged and Base Database Patches

[0044] The thresholds that need to be manually set in this invention include acute angle threshold a, distance threshold m between two points or between a point and a line segment, reference patch judgment threshold w, and patch area threshold s. Based on experience, the thresholds set in this example are a = 15°, m = 0.8m, w = 0.1, and s = 30.

[0045] Please refer to the following for some experimental results. Figure 8 , Figure 8 In the four sets of experimental results, the blue line represents the outline of the basic database patches, the red line represents the outline of the patches in the data layer to be fused, and the green line represents the processed patch result of the data layer to be fused. It can be seen that the processing method of the present invention has achieved good results, realizing automatic processing of topological consistency of multi-source spatial data for natural resource management, ensuring that there are no problems such as intersection or mutual overlap between the boundary of the patch in the data layer to be fused and the boundary of the corresponding basic data patch, greatly saving labor costs and improving the quality and efficiency of data fusion.

[0046] In summary, this invention first preprocesses the patches in the data layer to be merged, including rapid R-tree retrieval and extraction of patch communities, while simultaneously obtaining independent patches. Processing these two types of patches separately ensures rationality while increasing processing efficiency and reducing topological issues between patches in the data layer to be merged, facilitating topological consistency processing with the base database patches. Secondly, the preprocessed patches in the data layer to be merged undergo topological consistency processing with the base database patches, including neighbor point replacement, perpendicular point replacement, and overlap correction. Fine-grained processing based on points, lines, and surfaces effectively improves the accuracy of topological consistency processing. Finally, further topological error processing is performed on the processed patches in the data layer to be merged, including checking and correcting self-intersections, small acute angles, small-area fragmented patches, and overlapping and gaps between multiple patches, further ensuring the quality of topological consistency processing. Experiments demonstrate that the method provided by this invention can effectively solve the underlying bottleneck of topological inconsistency between patches in the data layer to be merged and the base database patches, providing core support for the modernization of territorial spatial governance.

[0047] The above embodiments are for illustrative purposes only and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all equivalent technical solutions also fall within the scope of this invention, and the patent protection scope of this invention should be defined by the claims.

[0048] Matters not covered in this invention are common knowledge.

[0049] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An automatic method for handling topological consistency of multi-source spatial data for rapid updates in vector databases, characterized in that: The method includes the following steps: Step 1: Preprocess the mosaic patterns in the fused data layer; Step 2: Ensure topological consistency between the merged data layer patches and the base database patches; Step 3: Post-process the fused data layer patches; Step 4: Set the threshold for topological consistency processing between the fused data layer patches and the basic database patches, and obtain the processing results according to Steps 1-3.

2. The automatic processing method for topological consistency of multi-source spatial data for rapid updating of vector databases according to claim 1, characterized in that, The specific process of step 1 is as follows: Step 1.1: Read the data of the data layer to be fused as a Shapefile or gdb format file, obtain the corresponding basic data, including the projection coordinates of each patch. All correction processes only need to modify the geometry of the patches in the features, i.e., the outline coordinate points. Step 1.2: Obtain the patch communities and independent patches in the data of the data layer to be fused. Based on the R-tree spatial indexing method, find all the patch communities in the data of the data layer to be fused, obtain the index number of all patches and independent patches in all patch communities, and obtain the set of all index numbers and corresponding patch outline coordinates to ensure that the data storage structure after processing is consistent with that before processing. Step 1.3: Process the polygon communities and independent polygons separately. For independent polygons, duplicate point deletion, self-intersection check and correction, and sharp corner deletion are required. For polygon communities, the same processing as for independent polygons is required, followed by replacement of neighboring points between polygons in the community.

3. The automatic processing method for topological consistency of multi-source spatial data for rapid updating of vector databases according to claim 1, characterized in that, The specific process of step 2 is as follows: Step 2.1: Load the data of the data layer to be merged and the basic database, and check the consistency of the geographic coordinate system and projected coordinate system of the two data. The coordinate systems of the two data must be consistent. If the coordinate systems of the basic database of the two coordinate systems of the data layer to be merged are inconsistent, the coordinate system of the basic database shall be used as the standard. Assign the coordinate system of the basic database to the data layer to be merged, obtain the outline coordinates of the basic database patches and the patches of the data layer to be merged, and store them in different arrays for later use. Step 2.2: First, obtain the set of base database patches that intersect with the patches of the data layer to be merged using the R_tree indexing method. Determine which patches are reference patches based on the percentage threshold of the area of ​​the data layer patches to be merged relative to the area of ​​the intersecting base database patches. Then, perform operations such as neighbor point replacement, perpendicular point replacement, and patch overlap processing based on the data layer patches to be merged and the reference patches of the base database. The reference patches remain unchanged; only the points, lines, and surfaces of the data layer patches to be merged need to be corrected. This allows the boundaries of the data layer patches to be merged to be aligned with the boundaries of the reference patches of the base database, improving the accuracy and efficiency of data updates.

4. The automatic processing method for topological consistency of multi-source spatial data for rapid updates of vector databases according to claim 1, characterized in that, The specific process of step 3 is as follows: self-intersection processing, which is to check and correct the self-intersection of the data layer patches to be fused after step 2; small acute angle processing, which is to check and correct the small internal angles; small area fragmented patch processing, which is to check and correct the small area patches; and multiple patch gap filling processing, which is to check and fill the gaps formed between patches in the patch cluster.

5. The automatic processing method for topological consistency of multi-source spatial data for rapid updating of vector databases according to claim 1, characterized in that: In step 4, the manually set threshold is an acute angle threshold a = 15°.

6. The automatic processing method for topological consistency of multi-source spatial data for rapid updates of vector databases according to claim 1, characterized in that: In step 4, the distance threshold between two points or between a point and a line segment is m = 0.8m, the reference patch judgment threshold is w = 0.1, and the patch area threshold is s = 30.

Images