A parallel computing method and system for vector space data quality inspection

Abstract

The present invention discloses a parallel computing method and system for vector space data quality inspection, comprising the following steps: obtaining vector space data, and inspecting the vector space data according to a quality inspection scheme; extracting four types of quality inspection items from the vector space data that has passed the inspection according to a quality inspection item classification method to obtain four quality inspection item sets, and further decomposing the four quality inspection item sets to obtain a number of quality inspection sub-items corresponding to the four quality inspection item sets; assigning quality inspection tasks to the several quality inspection sub-items according to the fixed time step of the execution of a single quality inspection task and the type of quality inspection item set to generate quality inspection tasks; performing parallel quality inspection according to the assigned quality inspection tasks until all quality inspection tasks are completed. The beneficial effects of the present invention are: the time of quality inspection is greatly shortened under the premise of ensuring the correctness of the quality inspection results, and the efficiency of vector space data quality inspection is improved.

Description

Technical Field

[0001] The present invention relates to the field of vector space data quality inspection, and in particular to a parallel computing method for vector space data quality inspection. Background Art

[0002] With the rapid development of information technology, in addition to the traditional natural resources industry, the demand for vector space data in multiple industries such as housing and urban and rural construction, agriculture and rural areas, emergency management, transportation, energy and electricity is increasing. Quality is the life of vector space data, which directly affects the correctness and reliability of all application analysis and decision-making based on the vector space data. As the update cycle of vector space data becomes shorter and shorter, the requirements for the quality of vector space data are getting higher and higher, and higher requirements are also put forward for the efficiency of vector space data quality inspection. Due to the characteristics of vector space data with large data volume, rich data types and many inspection contents, the quality inspection method for vector space data is also very complicated. At the same time, due to the inconsistent database construction specifications of different vector space data, the corresponding quality inspection methods are also inconsistent.

[0003] At present, the most widely used vector spatial data storage forms in the surveying and mapping industry are: ESRI Shapefile, ESRI Personal Geographic Database (.mdb), ESRI Spatial Database (.gdb), and these three data formats are all developed by Esri. For these data, the current technical means of vector spatial data quality inspection are mainly based on desktop stand-alone quality inspection software, which realizes the element graphics inspection, attribute inspection, topology inspection, edge inspection, etc. of vector spatial data. Limited by the storage form of vector spatial data, the large amount of vector spatial data and the computing power of desktop computers, it takes a long time to complete the vector spatial data quality inspection in this way, and it is impossible to complete the quality inspection of large amounts of vector spatial data. For example, when there are more than 1 million elements in the vector spatial data, the time for the attribute inspection of a single field in a single table is often more than 30 seconds, and the time for the topological inspection is generally tens of minutes to several hours. The overall time to complete the vector spatial data quality inspection is more than several hours, and the stability is not high, and it is very easy to get stuck. This quality inspection efficiency cannot meet the current demand for quality inspection due to the gradual increase in the frequency of vector spatial data updates and the gradual increase in the amount of vector spatial data.

[0004] The present invention provides a parallel computing method for vector space data quality inspection. Under the premise of ensuring the correctness of the quality inspection results, the method makes full use of parallel computing to solve the performance bottleneck problem when performing quality inspection on large amounts of vector space data, shortens the time of vector space data quality inspection, and improves the efficiency of vector space data quality inspection. Summary of the invention

[0005] The present invention provides a parallel computing method for vector space data quality inspection, which mainly solves the following technical problems:

[0006] On the premise of ensuring the correctness of the quality inspection results, full use is made of parallel computing to solve the performance bottleneck problem when conducting quality inspection on large amounts of vector space data, shorten the time of vector space data quality inspection, and improve the efficiency of vector space data quality inspection.

[0007] A parallel computing method for vector space data quality inspection, characterized by comprising the following steps:

[0008] Acquire vector space data, and inspect the vector space data according to a quality inspection plan;

[0009] According to the quality inspection item classification method, four categories of quality inspection items are extracted from the qualified vector space data to obtain four quality inspection items.

[0010] The four quality inspection item sets are further decomposed to obtain a number of quality inspection sub-items corresponding to the four quality inspection item sets;

[0011] After allocating quality inspection tasks to several quality inspection sub-items according to the fixed time step of execution of a single quality inspection task and the quality inspection item type, quality inspection tasks are generated;

[0012] Carry out quality inspection in parallel according to the assigned quality inspection tasks until all quality inspection tasks are completed.

[0013] In the above-mentioned parallel computing method for vector spatial data quality inspection, the database correctness inspection items include submission directory correctness inspection, file naming correctness inspection, data format correctness inspection, data validity inspection, structure correctness inspection, attribute correctness inspection, spatial reference system inspection, layer integrity inspection, and attribute integrity inspection.

[0014] In the above-mentioned parallel computing method for vector space data quality inspection, the four types of quality inspection items are:

[0015] Element graphic inspection includes: composite graphic inspection, graphic geometry anomaly inspection, line discount inspection, face edge inspection, short line inspection, micro face inspection, element node number excess inspection, element direction correctness inspection, narrow face inspection, face gap inspection.

[0016] Attribute checks include: attribute null value check, attribute association check, attribute value range check, attribute unique value check, attribute correctness SQL check, ID card code verification, code length check, decimal place correctness check, coding rule correctness check, contour elevation value correctness check, map consistency check (area), map consistency check (length).

[0017] Topological tests include: intersection of lines on the same layer, intersection of lines on different layers, intersection of surfaces on the same layer, intersection of surfaces on different layers, self-intersection of lines, self-intersection of surfaces, overlap of points, overlap of lines, overlap of surfaces on the same layer, overlap of surfaces on different layers, pseudo nodes, hanging points on the same layer, hanging points on different layers, points must fall on the line, surfaces are inside another surface, lines and surfaces must not intersect, points fall into surfaces, points do not fall into surfaces, lines fall into surfaces, lines do not fall into surfaces, line intersection interruption test, spatial constraint test, line and surface boundary consistency test, and contour point and line contradiction test.

[0018] Edge inspection includes: attribute edge inspection between map sheets, attribute edge inspection between regions, graphic edge inspection between map sheets, and graphic edge inspection between regions.

[0019] In the above-mentioned parallel computing method for vector space data quality inspection, when decomposing,

[0020] For the quality inspection of a single layer, the inspection sub-item accurately describes a rule for multiple fields of a specific layer;

[0021] For quality inspection of multiple associated layers, the inspection sub-items accurately describe a rule of multiple fields or conditions of multiple associated layers;

[0022] An inspection sub-item does not contain the parameters of other inspection sub-items. After the decomposition is completed, the quality inspection sub-items are classified into the classification set of quality inspection sub-items according to the classification of quality inspection items.

[0023] In the above-mentioned parallel computing method for vector space data quality inspection, when allocating quality inspection tasks, the number of element graphics inspection sub-items is set to n1, the number of topology inspection sub-items is n2, the number of edge inspection sub-items is n3, and the number of attribute inspection sub-items is n4. The time consumption parameter of each inspection sub-item in the element graphics inspection is t1, the time consumption parameter of each inspection sub-item in the topology inspection is t2, the time consumption parameter of each inspection sub-item in the edge inspection is t3, and the time consumption parameter of each inspection sub-item in the attribute inspection is t4;

[0024] Total time consumed for linear calculation

[0025] T = n1t1+n2t2+n3t3+n4t4

[0026] Get the number of computing servers C; calculate the shortest parallel quality inspection time as

[0027]

[0028] The fixed time step of a single task execution is set to one quarter of Tmin, that is,

[0029]

[0030] All quality inspection sub-items are assigned to quality inspection tasks according to the quality inspection item type according to the fixed time step of executing a single quality inspection task. Each quality inspection task contains multiple quality inspection sub-items of the same type.

[0031] In the above-mentioned parallel computing method for vector space data quality inspection and quality inspection task generation method,

[0032] If the inspection evaluation time of each inspection sub-item is greater than or equal to Ta, then each inspection sub-item is divided into an inspection task Task;

[0033] If the inspection evaluation time of each inspection sub-item is less than Ta, each inspection sub-item is aggregated according to type;

[0034] Each type of quality inspection sub-item is aggregated separately, and inspection sub-items of different types are not put into the same quality inspection task;

[0035] Each type of quality inspection sub-item is aggregated to form a quality inspection task. The aggregation method is to create a quality inspection task Task, extract a quality inspection sub-item from the quality inspection sub-item set and add it to the quality inspection task, and delete the quality inspection sub-item from the quality inspection sub-item set at the same time, and accumulate the evaluation time of all quality inspection sub-items contained in the quality inspection task to obtain the evaluation time of the quality inspection task. If the evaluation time of the quality inspection task is greater than or equal to Ta; save the inspection task Task; and then continue to create new tasks until the number of quality inspection sub-item sets of this type is 0.

[0036] In the above-mentioned parallel computing method for vector space data quality inspection, during quality inspection,

[0037] Extract all layers associated with the quality inspection sub-items in the topology inspection quality inspection sub-item set, remove duplicate layers, and obtain the layer set participating in the topology inspection.

[0038] Then the scheduling server uses the PostGIS engine of the open source spatial database PostgreSQL to build the topological relationship of the vector spatial data only for the layer set involved in the topology verification.

[0039] The scheduling server distributes the vector space data containing topological relationships to the computing servers.

[0040] The scheduling server obtains multiple idle computing servers and assigns multiple quality inspection tasks to the multiple computing servers for calculation. Each computing server can only execute one quality inspection task at a time.

[0041] The computing server performs calculations using Spatial SQL that complies with the OGC SimpleFeature Access SQL specification through the PostGIS engine of the open source spatial database PostgreSQL. When a computing server completes the calculation, it notifies the scheduling server that the computing task is completed; the scheduling server reads the quality inspection results from the computing server and assigns the quality inspection tasks that have not yet started to be executed to the computing server.

[0042] The quality inspection results are output after all quality inspection tasks are completed.

[0043] A parallel computing system for vector space data quality inspection, comprising:

[0044] The first module is configured to obtain vector space data and perform quality inspection on the vector space data according to the quality inspection scheme.

[0045] test;

[0046] The second module is configured to extract four categories of quality inspection items from the qualified vector space data according to the quality inspection item classification method to obtain four quality inspection items.

[0047] The four quality inspection item sets are further decomposed to obtain a number of quality inspection sub-items corresponding to the four quality inspection item sets;

[0048] The third module is configured to generate quality inspection tasks after allocating quality inspection tasks to a number of quality inspection sub-items according to the fixed time step of execution of a single quality inspection task and the quality inspection item type;

[0049] The fourth module is configured to perform quality inspection in parallel according to the assigned quality inspection tasks until all quality inspection tasks are completed.

[0050] The beneficial effects provided by the present invention are as follows: the parallel computing method for vector spatial data quality inspection provided by the present invention preferentially performs the "database correctness" inspection before performing the quality inspection, thereby avoiding the problem of being unable to obtain effective quality inspection results due to incorrect vector spatial data structure. The parallel computing method for vector spatial data quality inspection provided by the present invention only extracts the layers involved in the topological inspection to construct topological relationships when performing topological inspection, thereby reducing the time for constructing topological relationships, reducing the data storage space, and shortening the time for data distribution. The parallel computing method for vector spatial data quality inspection provided by the present invention greatly shortens the time for overall quality inspection while ensuring the correctness of the quality inspection results, thereby improving the efficiency of vector spatial data quality inspection. The parallel computing method for vector spatial data quality inspection provided by the present invention uses the PostGIS engine of the open source spatial database PostgreSQL to perform quality inspection calculations using Spatial SQL that complies with the OGC Simple Feature Access SQL specification, thereby reducing dependence on industry commercial software and being suitable for promotion and application. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a flow chart of a parallel computing method for vector space data quality inspection of the present invention;

[0052] Figure 2 It is a flowchart generated by the quality inspection task. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the present invention clearer, the quality inspection of the vector space database of the third national land survey will be taken as an example, and the implementation mode of the present invention will be further described in conjunction with the accompanying drawings.

[0054] The invention mainly comprises the following steps: copying vector spatial data to a scheduling server, and the scheduling server submitting the vector spatial data to an open source spatial database PostgreSQL; extracting "database correctness" inspection items in a quality inspection scheme and inspecting them first; classifying and decomposing the quality inspection items in the quality inspection scheme in a fine-grained manner to form a quality inspection sub-item set; aggregating the quality inspection sub-items into a quality inspection sub-item set according to a fixed time step, and assigning the quality inspection sub-item set to a quality inspection task; the scheduling server constructing a topological relationship for only the layers participating in the topological inspection, and distributing the data after the topological relationship is constructed to a computing server; the scheduling server respectively assigning a plurality of quality inspection tasks to a plurality of computing servers for calculation; the computing server uses a PostGIS engine of the open source spatial database PostgreSQL to perform calculations using Spatial SQL that complies with the OGC Simple Feature Access SQL specification, and when a computing server completes the calculation, notifies the scheduling server that the calculation task is completed; the scheduling server reads the quality inspection result from the computing server, and assigns the quality inspection task that has not yet been executed to the computing server; repeating the previous step until all the quality inspection tasks are executed; and the scheduling server merges the quality inspection results.

[0055] The above steps are explained one by one below.

[0056] Please refer to Figure 1 , the present invention comprises the following steps:

[0057] S101: Copy the vector space data to the scheduling server, and the scheduling server submits the vector space data to the spatial database PostgreSQL.

[0058] S102: Submit the quality inspection plan to the scheduling server, which extracts the inspection items of "submission directory correctness, file naming correctness, data format correctness, data validity, structure correctness, attribute correctness, spatial reference system, layer integrity, attribute integrity" from the quality inspection plan as inspection items of the "database correctness inspection" type, and the scheduling server performs a quality inspection on the database correctness of the vector spatial data; if the database correctness inspection fails, subsequent quality inspection cannot be performed, and the scheduling server outputs the quality inspection result; if the database correctness inspection passes, proceed to S103. For example, the "spatial reference system test" in the quality inspection plan for vector spatial data of the third national land survey needs to check whether the geographic coordinate system name of the CJDCQJX, TTQ, XZQ, XZQJX, ZYXMYD, CJDCQ, ZRYCBHQ, JZKZD, KFYQ, LSYD, PDT, PZWJSTD, QTJZKFQ, STBHHX, ZRBHQ, GJGY, SLGY, ZJ, and CLKFQ layers is CGCS2000, and whether the WKID is 4490.

[0059] S103: Classify and decompose the quality inspection items in the quality inspection plan. Extract four categories of quality inspection items from the quality inspection plan according to the quality inspection item classification method to obtain four quality inspection item sets. The quality inspection classification method is:

[0060] Element graphic inspection includes: conformity inspection, graphic geometry inspection, line discount inspection, surface edge inspection, micro-short line inspection, micro-face inspection, element node number over-limit inspection, element direction correctness inspection, narrow and long face inspection, and face gap inspection. For example, the "micro-face inspection" in the third national land survey vector space data needs to inspect the micro-faces with an area of ​​less than 200 square meters in the TTQ, XZQ, ZYXMYD, CJDCQ, ZRYCBHQ, KFYQ, LSYD, PDT, PZWJSDTD, QTJZKFQ, STBHHX, ZRBHQ, GJGY, SLGY, ZJ, CSKFBJ, DZGY, and FJMSQ layers. The "face gap inspection" needs to inspect the faces with a distance of less than 0.0001 meters in the XZQ, CJDCQ, and DLTB layers. The "micro-short line inspection" needs to inspect the micro-short lines with a length of less than 0.2 meters in the CJDCQJX, XZQJX, and DGX layers. Attribute checks include: attribute control check, attribute association check, attribute value range check, attribute unique value check, attribute correctness SQL check, ID card code verification, code length check, decimal point correctness check, code rule correctness check, contour elevation value correctness check, map attribute consistency check (area), map attribute consistency check (length); for example, the "cultivated land planting attribute code value range check" in the third national land survey vector space data needs to check whether the value of the ZZSXM field of DLTB is one of the values ​​in {"LYFL", "WG", "XG", "LS", "FLS", "LLJZ", "JKHF", "GCHF"}; the "area field value greater than 0 check" (a case of attribute value range check) needs to check whether the values ​​of fields such as XZQ.JSMJ, XZQ.DCMJ, LSYD.TBMJ, PZWJSTD.TBMJ, and PZWJSTD.PZMJ are greater than or equal to 0.Topological tests include: same-layer line intersection test, different-layer line intersection test, same-layer surface intersection test, different-layer surface intersection test, line self-intersection test, surface self-intersection test, point overlap test, line overlap test, same-layer surface overlap test, different-layer surface overlap test, pseudo-node test, same-layer hanging point test, different-layer hanging point test, point must fall on line test, surface is in another surface test, line and surface must not intersect test, point falls into surface test, point does not fall into surface test, line falls into surface test, line does not fall into surface test, line intersection interruption test, spatial constraint test, line and surface boundary test Consistency test, contour line point line contradiction test; for example, the "intersection test of lines on the same layer" in the vector spatial data of the third national land survey needs to check the intersection of lines in the three layers of CJDCQJX, XZQJX, and DGX; "the elements must be tested within the administrative area" (a case of intersection test at different levels) needs to check whether all the elements of the TTQ, ZYXMYD, CJDCQ, ZRYCBHQ, KFYQ, LSYD, PDT, PZWJSTD, QTJZKFQ, STBHHX, and ZRBHQ layers are within the XZQ layer. (4) Edge test, including: attribute edge test between sheets, attribute edge test between regions, graphic edge test between sheets, and graphic edge test between regions. For example, the "edge test of land attribute consistency between sheets" in the vector spatial data of the third national land survey needs to check whether the DLBM and DLMC fields of the DLTB layer with the same BSM field value in adjacent sheets are consistent.

[0061] Each quality inspection item in the quality inspection item set is further decomposed into quality inspection sub-items, and the decomposition rules are as follows:

[0062] (1) For the quality inspection of a single layer, the inspection sub-item accurately describes a rule for multiple fields of a specific layer;

[0063] (2) For quality inspection of multiple associated layers, the inspection sub-item accurately describes a rule of multiple fields (or conditions) of multiple associated layers;

[0064] (3) The test item does not contain the parameters of other test items;

[0065] (4) According to the first three decomposition rules, the test sub-item cannot be further decomposed.

[0066] After the decomposition is completed, the quality inspection sub-items are classified into the classification set of quality inspection sub-items according to the classification of the quality inspection items. For example, after the inspection items given in step S103 are decomposed, the following classification sets of 4 quality inspection sub-items are obtained:

[0067] Element graphic inspection: Inspection sub-item 1 (inspect the surface of the TTQ layer with an area less than 200 square meters), inspection sub-item 2 (inspect the surface of the XZQ layer with an area less than 200 square meters), inspection sub-item 3 (inspect the surface of the ZYXMYD layer with an area less than 200 square meters), inspection sub-item 4 (inspect the surface of the CJDCQ layer with an area less than 200 square meters), inspection sub-item 5 (inspect the surface of the ZRYCBHQ layer with an area less than 200 square meters), inspection sub-item 6 (inspect the surface of the KFYQ layer with an area less than 200 square meters), inspection sub-item 7 (inspect the surface of the LSYD layer with an area less than 200 square meters) , check sub-item 8 (check the areas of PZWJSDTD layer with an area of ​​less than 200 square meters), check sub-item 9 (check the areas of XZQ layer with a spacing of less than 0.0001 meters), check sub-item 10 (check the areas of XZQJX layer with a spacing of less than 0.0001 meters), check sub-item 11 (check the areas of DLTB layer with a spacing of less than 0.0001 meters), check sub-item 12 (check the lines of CJDCQJX layer with a length of less than 0.2 meters), check sub-item 13 (check the lines of XZQJX layer with a length of less than 0.2 meters), check sub-item 14 (check the lines of DGX layer with a length of less than 0.2 meters). (2) Attribute verification: Verify sub-item 15 (verify whether the value of the ZZSXM field of DLTB is one of the values ​​in {"LYFL", "WG", "XG", "LS", "FLS", "LLJZ", "JKHF", "GCHF"}), verify sub-item 16 (verify whether XZQ.JSMJ is greater than 0), verify sub-item 17 (verify whether XZQ.DCMJ is greater than 0), verify sub-item 18 (verify whether LSYD.TBMJ is greater than 0), verify sub-item 19 (verify whether PZWJSTD.TBMJ is greater than 0), and verify sub-item 20 (verify whether PZWJSTD.PZMJ is greater than 0).

[0068] (3) Topological test: Test item 21 (test the intersecting lines in the CJDCQJX layer), test item 22 (test the intersecting lines in the XZQJX layer), test item 23 (test the intersecting lines in the DGX layer), test item 24 (test whether all the patches in the TTQ layer are within the XZQ layer), test item 25 (test whether all the patches in the ZYXMYD layer are within the XZQ layer), test item 26 (test whether all the patches in the TTQ layer are within the XZQ layer), test item 27 (test whether all the patches in the CJDCQ layer are within the XZQ layer), test item 28 (test whether all the patches in the ZRYCBHQ layer are within the XZQ layer) within the scope of the XZQ layer), check sub-item 29 (check whether all the spots of the KFYQ layer are within the scope of the XZQ layer), check sub-item 30 (check whether all the spots of the LSYD layer are within the scope of the XZQ layer), check sub-item 31 (check whether all the spots of the PDT layer are within the scope of the XZQ layer), check sub-item 32 (check whether all the spots of the PZWJSTD layer are within the scope of the XZQ layer), check sub-item 33 (check whether all the spots of the QTJZKFQ layer are within the scope of the XZQ layer), check sub-item 34 (check whether all the spots of the STBHHX layer are within the scope of the XZQ layer), check sub-item 35 (check whether all the spots of the ZRBHQ layer are within the scope of the XZQ layer).

[0069] (4) Edge inspection: Check sub-item 36 (check whether the DLBM of the DLTB layer with the same BSM field value in adjacent map sheets is consistent) and check sub-item 37 (check whether the DLMC of the DLTB layer with the same BSM field value in adjacent map sheets is consistent).

[0070] The number of decomposed quality inspection sub-item sets is defined as follows:

[0071] Element graphic test: Set the number of this type of test items to n1.

[0072] Topological test: Set the number of this type of test items to n2,

[0073] Edge inspection: Set the number of this type of inspection items to n3.

[0074] Attribute inspection: Set the number of inspection items of this type to n4.

[0075] S104: Calculate the time step of allocating quality inspection tasks when the scheduling server performs scheduling:

[0076] The time consumption of each inspection item is directly related to the type of inspection item. The time consumption parameters for each type of inspection item are set as follows:

[0077] (1) Element graphic inspection: Set the time-consuming parameter of each inspection sub-item of this type of inspection item to t1.

[0078] (2) Topological test: Set the time consumption parameter of each test sub-item of this type of test item to t2.

[0079] (3) Edge inspection: Set the time consumption parameter of each inspection sub-item of this type of inspection item to t3.

[0080] (4) Attribute inspection: Set the time-consuming parameter of each inspection sub-item of this type of inspection item to t4.

[0081] According to the number of each type of test sub-item obtained in step S103, the total linear calculation time is calculated.

[0082] T = n1t1+n2t2+n3t3+n4t4

[0083] The scheduling server obtains the number of computing servers C; the shortest parallel quality inspection duration is calculated as

[0084]

[0085] The fixed time step of a single task execution is set to one quarter of Tmin, that is,

[0086]

[0087] All quality inspection sub-items are assigned to quality inspection tasks according to the quality inspection item type according to the fixed time step of a single quality inspection task. Each quality inspection task contains multiple quality inspection sub-items of the same type. The method for generating quality inspection tasks is:

[0088] (1) If the inspection evaluation time of each inspection sub-item is greater than or equal to Ta, then each inspection sub-item is divided into an inspection task Task; if the inspection evaluation time of each inspection sub-item is less than Ta, then each inspection sub-item is aggregated according to type;

[0089] (2) Each type of quality inspection sub-item is aggregated separately, and inspection sub-items of different types are not placed in the same quality inspection task;

[0090] (3) Each type of quality inspection sub-item is aggregated to form a quality inspection task, refer to Figure 2The aggregation method is to create a quality inspection task Task, extract a quality inspection sub-item from the quality inspection sub-item set and add it to the quality inspection task (and delete the quality inspection sub-item from the quality inspection sub-item set at the same time), and accumulate the evaluation time of all quality inspection sub-items contained in the quality inspection task to obtain the evaluation time of the quality inspection task. If the evaluation time of the quality inspection task is greater than or equal to Ta; save the inspection task Task; and then continue to create new tasks until the number of quality inspection sub-item sets of this type is 0.

[0091] Taking the 37 quality inspection sub-items decomposed in S103 as an example, the inspection tasks that can be created are:

[0092] Task 1 (element graphics verification task), including verification sub-item 1, verification sub-item 2, verification sub-item 3, verification sub-item 5, verification sub-item 6, verification sub-item 7, and verification sub-item 8; Task 2 (element graphics verification task), including verification sub-item 9, verification sub-item 10, verification sub-item 11, verification sub-item 12, verification sub-item 13, and verification sub-item 14; Task 3 (attribute verification task): including verification sub-item 15, verification sub-item 16, verification sub-item 17, verification sub-item 18, verification sub-item 19, and verification sub-item 20; Task 4 (topology verification task): including verification sub-item 1 Task 21, inspection sub-item 22, inspection sub-item 23, inspection sub-item 24; Task 5 (topology inspection task): including inspection sub-item 25, inspection sub-item 26, inspection sub-item 27, inspection sub-item 28; Task 6 (topology inspection task): including inspection sub-item 29, inspection sub-item 30, inspection sub-item 31, inspection sub-item 32; Task 7 (topology inspection task): including inspection sub-item 33, inspection sub-item 34, inspection sub-item 35; Task 8 (edge ​​connection inspection task): including inspection sub-item 36; Task 9 (edge ​​connection inspection task): including inspection sub-item 37. S105: The scheduling server extracts the layers associated with all quality inspection sub-items in the topology inspection quality inspection sub-item set, removes duplicate layers, and obtains the layer set participating in the topology inspection, and then the scheduling server constructs the topological relationship of the vector space data only for the layer set participating in the topology inspection through the spatial database. Taking the topology check in S103 as an example, the layers that need to be extracted and the topological relationships constructed are CJDCQJX, XZQJX, DGX, TTQ, ZYXMYD, CJDCQ, ZRYCBHQ, KFYQ, LSYD, PDT, PZWJSTD, QTJZKFQ, STBHHX, and ZRBHQ.

[0093] S106: The scheduling server distributes the PostreSQL database containing the topological relationship to the computing server.

[0094] S107: The scheduling server obtains multiple idle computing servers, and assigns multiple quality inspection tasks to the multiple computing servers for computing. Each computing server executes only one quality inspection task at a time. For example, when there are three idle computing servers, Task 1 can be assigned to computing server A, Task 2 to computing server B, and Task 3 to computing server C.

[0095] S108: The computing server uses Spatial SQL that complies with the OGC Simple Feature Access SQL specification to perform data calculations through the spatial data engine PostGIS of the open source spatial database PostgreSQL. When a computing server completes the calculation, it notifies the scheduling server that the computing task is completed; the scheduling server reads the quality inspection results from the computing server and assigns the quality inspection tasks that have not yet been executed to the computing server. For example, when Task 1 is completed on computing server A, it notifies the scheduling server, and the scheduling server reads the quality inspection results from computing server A and assigns Task 5 to computing service area A for continued execution.

[0096] S109: Repeat S108 until all quality inspection tasks are completed.

[0097] S110: The scheduling server merges the quality inspection results.

[0098] At this point, the vector space data quality inspection is completed.

[0099] The innovation of the present invention mainly lies in that a computing server directly uses the spatial data engine PostGIS of an open source spatial database PostgreSQL and uses Spatial SQL that complies with the OGC Simple Feature Access SQL specification to perform calculations, thereby greatly improving the calculation efficiency; only layers participating in topological verification are extracted to construct topological relationships, thereby reducing the time for constructing topological relationships, reducing data storage space, and thereby shortening the time for data distribution; quality inspection sub-items are aggregated according to fixed time steps and then assigned to quality inspection tasks, thereby enabling a computing server to perform calculations for quality inspection tasks with consistent evaluation time consumption, thereby improving the scheduling efficiency of quality inspection tasks and shortening the total time for quality inspection.

[0100] The beneficial effects brought by the present invention are:

[0101] (1) The present invention provides a parallel computing method for vector space data quality inspection. Before performing the quality inspection, the "database correctness" inspection is performed first, thereby avoiding the problem of being unable to obtain effective quality inspection results due to incorrect vector space data structure.

[0102] (2) The present invention provides a parallel computing method for vector space data quality inspection. When performing topology inspection, only the layers involved in the topology inspection are extracted to construct topological relationships, which reduces the time for constructing topological relationships, reduces data storage space, and shortens the time for data distribution.

[0103] (3) The present invention provides a parallel computing method for vector space data quality inspection, which greatly shortens the overall quality inspection time while ensuring the correctness of the quality inspection results, and improves the efficiency of vector space data quality inspection.

[0104] (4) The present invention provides a parallel computing method for vector spatial data quality inspection, which uses the PostGIS engine of the open source spatial database PostgreSQL and Spatial SQL that complies with the OGC Simple Feature Access SQL specification to perform quality inspection calculations, thereby reducing dependence on industry commercial software and being suitable for promotion and application.

[0105] The above shows and describes the basic principles, main features and advantages of the present invention. However, those skilled in the art may still make various substitutions and modifications based on the disclosure of the present invention without departing from the spirit of the present invention. Therefore, the protection scope of the present invention should not be limited to the contents disclosed in the embodiments, but should include various substitutions and modifications without departing from the present invention, and should be covered by the claims of this patent application.

Claims

1. A parallel computing method for vector space data quality inspection, characterized in that: The following steps are involved: Acquire vector space data, and inspect the vector space data according to a quality inspection plan; Extracting four categories of quality inspection items from the qualified vector space data according to the quality inspection item classification method to obtain four quality inspection item sets, and further decomposing the four quality inspection item sets to obtain a number of quality inspection sub-items corresponding to the four quality inspection item sets; The four categories of quality inspection items are: Element graphics inspection, including: composite graphics inspection, graphic geometry anomaly inspection, line discount inspection, surface edge inspection, short line inspection, micro surface inspection, element node number over-limit inspection, element direction correctness inspection, narrow surface inspection, surface gap inspection; Attribute verification, including: attribute null value verification, attribute association verification, attribute value range verification, attribute unique value verification, attribute correctness SQL verification, ID card code verification, code length verification, decimal point correctness verification, coding rule correctness verification, contour elevation value correctness verification, area map consistency verification, length map consistency verification; Topological tests, including: intersection of lines on the same layer, intersection of lines on different layers, intersection of planes on the same layer, intersection of planes on different layers, self-intersection of lines, self-intersection of planes, overlap of points, overlap of lines, overlap of planes on the same layer, overlap of planes on different layers, pseudo nodes, hanging points on the same layer, hanging points on different layers, points must fall on a line, a plane is inside another plane, lines and planes must not intersect, points fall into a plane, points do not fall into a plane, lines fall into a plane, lines do not fall into a plane, line intersection interruption test, spatial constraint test, line and plane boundary consistency test, contour line point and line contradiction test; Edge inspection, including: attribute edge inspection between sheets, attribute edge inspection between regions, graphic edge inspection between sheets, and graphic edge inspection between regions; After allocating quality inspection tasks to several quality inspection sub-items according to the fixed time step of a single quality inspection task and the quality inspection item type, quality inspection tasks are generated; during quality inspection, Extract all layers associated with the quality inspection sub-items in the topology inspection quality inspection sub-item set, remove duplicate layers, and obtain the layer set participating in the topology inspection. Then the scheduling server uses the PostGIS engine of the open source spatial database PostgreSQL to build the topological relationship of the vector spatial data only for the layer set involved in the topology verification; The scheduling server distributes the vector space data containing the topological relationship to the computing server; The scheduling server obtains multiple idle computing servers and assigns multiple quality inspection tasks to the multiple computing servers for calculation. Each computing server can only execute one quality inspection task at a time. The computing server uses the PostGIS engine of the open source spatial database PostgreSQL to perform calculations using Spatial SQL that complies with the OGC SimpleFeature Access SQL specification. When a computing server completes the calculation, it notifies the scheduling server that the computing task is complete. The scheduling server reads the quality inspection results from the computing server and assigns the quality inspection tasks that have not yet been executed to the computing server. The quality inspection results are output after all quality inspection tasks are completed; Carry out quality inspection in parallel according to the assigned quality inspection tasks until all quality inspection tasks are completed.

2. The parallel computing method for vector space data quality inspection according to claim 1, characterized in that: The database correctness check items include submission directory correctness check, file naming correctness check, data format correctness check, data validity check, structure correctness check, attribute correctness check, spatial reference system check, layer integrity check, and attribute integrity check.

3. The parallel computing method for vector space data quality inspection according to claim 1, characterized in that When decomposed, For the quality inspection of a single layer, the inspection sub-item accurately describes a rule for multiple fields of a specific layer; For quality inspection of multiple associated layers, the inspection sub-items accurately describe a rule of multiple fields or conditions of multiple associated layers; A test item does not contain parameters of other test items; After the decomposition is completed, the quality inspection sub-items are classified into the classification set of quality inspection sub-items according to the classification of the quality inspection items.

4. The parallel computing method for vector space data quality inspection according to claim 1, characterized in that Therefore, when allocating quality inspection tasks, the number of element graphics inspection sub-items is set to n1, the number of topology inspection sub-items is set to n2, the number of edge inspection sub-items is set to n3, and the number of attribute inspection sub-items is set to n4; the time consumption parameter of each inspection sub-item in element graphics inspection is t1, the time consumption parameter of each inspection sub-item in topology inspection is t2, the time consumption parameter of each inspection sub-item in edge inspection is t3, and the time consumption parameter of each inspection sub-item in attribute inspection is t4; Total time consumed for linear calculation T = n1t1+n2t2+n3t3+n4t4 Get the number of computing servers C; calculate the shortest parallel quality inspection time as The fixed time step of a single task execution is set to one quarter of Tmin, that is, All quality inspection sub-items are assigned to quality inspection tasks according to the quality inspection item type according to the fixed time step of executing a single quality inspection task. Each quality inspection task contains multiple quality inspection sub-items of the same type.

5. The parallel computing method for vector space data quality inspection according to claim 1, Its characteristics are When generating quality inspection tasks, If the inspection evaluation time of each inspection sub-item is greater than or equal to Ta, then each inspection sub-item is divided into an inspection task Task; If the inspection evaluation time of each inspection sub-item is less than Ta, each inspection sub-item is aggregated according to type; Each type of quality inspection sub-item is aggregated separately, and different types of inspection sub-items are not placed in the same quality inspection task; Each type of quality inspection sub-item is aggregated to form a quality inspection task. The aggregation method is to create a quality inspection task Task, extract a quality inspection sub-item from the quality inspection sub-item set and add it to the quality inspection task, and delete the quality inspection sub-item from the quality inspection sub-item set at the same time, and accumulate the evaluation time of all quality inspection sub-items contained in the quality inspection task to obtain the evaluation time of the quality inspection task. If the evaluation time of the quality inspection task is greater than or equal to Ta; save the inspection task Task; and then continue to create new tasks until the number of quality inspection sub-item sets of this type is 0.

6. A parallel computing system for vector space data quality inspection, using the method described in any one of claims 1 to 5, characterized in that: include: The first module is configured to obtain vector space data and perform quality inspection on the vector space data according to the quality inspection scheme. test; The second module is configured to extract four categories of quality inspection items from the qualified vector space data according to the quality inspection item classification method to obtain four quality inspection items. The four quality inspection item sets are further decomposed to obtain a number of quality inspection sub-items corresponding to the four quality inspection item sets; The third module is configured to generate quality inspection tasks after allocating quality inspection tasks to a number of quality inspection sub-items according to the fixed time step of execution of a single quality inspection task and the quality inspection item type set; The fourth module: is configured to perform quality inspection in parallel according to the assigned quality inspection tasks until all quality inspection tasks are completed.

Images