A multi-layer vector data overlay method based on Spark big data

Through the Spark platform and the partition storage method of quad-tree encoding, the problem of low efficiency of multi-layer vector data overlay analysis is solved, and efficient data processing and uniform computing load allocation are achieved.

CN115391351BActive Publication Date: 2025-08-26CHINA UNICOM (GUANGDONG) IND INTERNET CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202211113903.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-09-14
Publication Date
2025-08-26
Estimated Expiration
2042-09-14

AI Technical Summary

Technical Problem

The existing technology cannot efficiently perform the overlay analysis of multi-layer vector data. Traditional tools such as ArcGis are based on a single computer and are unable to effectively process massive multi-layer vector data.

Method used

The Spark big data platform is adopted to generate distributed data sets by minimally including rectangular and quad-tree encoding, and partition storage and grid processing of vector data are performed, and data allocation is optimized in combination with random number partitioning to achieve efficient superposition and fusion of multi-layer vector data.

Benefits of technology

The efficiency of multi-layer vector data overlay analysis is improved, the calculation amount is reduced, the data processing load is evenly distributed, and efficient data overlay and fusion is achieved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115391351B_ABST
    Figure CN115391351B_ABST
Patent Text Reader

Abstract

This invention provides a multi-layer vector data overlay method based on Spark big data. The method encodes each layer using a quadtree, selects the optimal quadtree as the standard quadtree for all layers, and then slices the layers using this quadtree. The sliced ​​layers are then distributed across Spark executors for overlay calculations, and finally, the graphics are fused. Compared to existing technologies, this method leverages Spark's distributed computing capabilities and optimizes Spark's distribution using quadtrees, enabling efficient overlay of multi-layer vector data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of vector data overlay calculation, and more specifically, to a multi-layer vector data overlay method based on Spark big data. Background Art

[0002] With the increasing attention paid to land resources and the development of vector data processing and analysis, vector data processing has become increasingly important, and the amount of data that needs to be processed has become increasingly larger, resulting in the overlay processing of multiple layers of vector data. For the analysis of multiple large data layers, depending on the amount of vector data, it often takes a long time to obtain the results, making it impossible to achieve efficient overlay analysis of multiple layers of vector data. Traditional overlay analysis of massive vector data usually uses ArcGis as a tool to perform overlay analysis on vector data. However, ArcGis is based on a single computer for data overlay analysis, and the computing power of a single computer cannot efficiently perform data overlay analysis.

[0003] Based on this demand, Spark's high-performance, scalable computing capabilities can be used to analyze massive multi-layer vector data. The invention patent application with publication number CN111563081A discloses a vector element parallel computing method, device, storage medium, and terminal. The method includes: building a distributed element dataset model based on the Spark computing framework; reading external data based on the distributed element dataset model; repartitioning the data after reading; combining quadtree indexes and binary tree indexes to create a local spatial index for the repartitioned data; and processing and analyzing the data after creating the local spatial index. Extended development based on the Spark module allows users to freely combine various interfaces. The processing and analysis interfaces of the distributed element dataset model can include filtering, obtaining geographic and time ranges, clipping, spatial query, attribute aggregation, grid overlay, polygon overlay, extracting columns, appending columns, etc.

[0004] Although the above patent discloses that massive vector data can be processed and analyzed based on Spark, it does not disclose how to overlay multiple layers of vector data, and the processing method for vector data is not disclosed in detail. Summary of the Invention

[0005] The present invention aims to overcome at least one of the above-mentioned defects of the prior art and provide a multi-layer vector data overlay method based on Spark big data to solve the problem of superposition analysis and calculation of massive multi-layer vector data.

[0006] The technical solution adopted by the present invention is:

[0007] The present invention provides a multi-layer vector data overlay method based on Spark big data, the method comprising:

[0008] S1: Merge the layers of the multi-layer vector graphics to be processed and generate the distributed dataset RDD1 in Spark;

[0009] S2: Establish a minimum containing rectangle z, which is a rectangular layer containing all RDD1 vector graphics;

[0010] S3: Traverse all layers in RDD1, generate a shortest quadtree for each layer based on the minimum containing rectangle z as the boundary, and record its maximum encoding length len;

[0011] S4: Select the length of the layer with the most occurrences in the maximum length len in S3, and use this length as the standard quadtree encoding length len1;

[0012] S5: Split the minimum containing rectangle z according to the standard quadtree code length len1 to generate a grid structure, and generate RDD2 with the quadtree code as the index value;

[0013] S6: Repartition the data in RDD2, redistribute the data in the Spark partitions and generate RDD3;

[0014] S7: Decompile the quadtree encoding of the vector graphic in RDD3, obtain the specific position of the grid of the vector graphic in the vector graphic and cut the grid, and generate RDD4 with the data in the grid;

[0015] S8: overlay the vector graphics in RDD4 according to the grid area data encoded by the quadtree, then decompose the data and generate RDD5 from the decomposed data;

[0016] S9: Fuse the vector layer data in RDD5 and generate RDD6 from the fused data.

[0017] Spark is used to partition and store multi-layer vector data, splitting massive amounts of data to reduce the amount of data in each partition. On this basis, vector data is overlaid and analyzed to improve computational efficiency. Quadtree encoding is introduced to select the optimal partitioning method. Vector data is gridded and partitioned and cropped using quadtree encoding so that each grid has a corresponding quadtree encoding. This allows a large amount of computation to be spread across each grid, making the computational load of each grid partition as uniform as possible. By allocating data to each grid and overlaying and disassembling data using each grid as a unit, since the amount of data in each grid partition is small, the computational load of overlaying and disassembling multi-layer vector data in each grid is small, enabling efficient overlay and fusion of vector data, and efficient analysis of massive multi-layer vector data.

[0018] Furthermore, the data format of the RDD1 is: (vector graphic element ID, layer code, graphic WKT, minimum coordinate x, maximum coordinate x1, minimum coordinate y, maximum coordinate y1, original element json information).

[0019] Encode each layer, recording it as the layer code. Assign an ID to each vector graphic in all layers and record it as the vector graphic feature ID. Each vector graphic corresponds to a vector graphic feature ID. Establish a unified coordinate system for all layers, recording the minimum x coordinate, maximum x1 coordinate, minimum y coordinate, and maximum y1 coordinate of each vector graphic in this coordinate system. Generate the raw feature JSON information for each vector graphic and record this data in RDD1 using the vector graphic feature ID as the index value. Determine the vector graphic data recording format to facilitate subsequent traversal and various operations on the vector graphic data.

[0020] Furthermore, selecting the standard quadtree encoding length len1 in step S4 also includes:

[0021] When the maximum lengths appear the same number of times, the longest one is selected as the standard quadtree encoding length len1.

[0022] In step S4, the generated quadtree code is selected, and the maximum code length len with the highest number of occurrences is chosen to ensure that the grid generated on the minimum containing rectangle z can completely enclose the vector graphics as much as possible, avoiding further division of monotonous data in the grid, thereby reducing the computational complexity of cropping. If there are different lengths with the same number of occurrences, the longest length is used as the standard quadtree code length len1. While ensuring the reasonable division of the grid, the number of grid division levels is increased as much as possible to prevent a large number of vector graphics from being included in a single grid, which may cause data skew.

[0023] Furthermore, the data format of the RDD2 is: (quadtree encoding, (vector graphic element ID, layer encoding, graphic WKT, original element json)).

[0024] By quadtree encoding the minimum containing rectangle z with the standard quadtree encoding length len1, z is divided into grids and cropped according to the grid structure. Each grid corresponds to a quadtree code. Since z is the minimum containing rectangle of the vector data of all layers, the quadtree code is used as the spatial index value to generate a new RDD2. The spatial index value can be used to determine the data of each layer's vector graphics in the corresponding grid of z, which facilitates data search and overlay processing.

[0025] Furthermore, step 6 repartitions the data in RDD2, redistributes the data in the Spark partitions and generates RDD3, specifically:

[0026] Traverse the RDD2, add random numbers to the index values ​​in RDD2 to generate new index values, repartition RDD2 according to the index values ​​to generate RDD3;

[0027] The data format of the RDD3 is: (quadtree code plus random number, (vector layer element ID, layer code, graphic WKT, original element json information)).

[0028] Since Spark may have data skew when creating partitions or transformations, that is, more data is concentrated on the same executor, resulting in uneven resource allocation and reduced data computing efficiency. Specifically, because the number of layers with vector graphics under different grids is different, if the quadtree encoding is used for repartitioning, the amount of data in each partition will be different. Then, when the partition tasks are assigned to Spark's executors, data skew will occur. Therefore, a random number is added to the index value in RDD2. In one embodiment, the addition of the random number can be done by taking values ​​from 0 to 9 through a random function to construct a four-digit random number, and the random number and a one-digit identification connector are combined. The quadtree encoding is added to form a new index value, and then RDD2 is repartitioned based on the generated new index value. Due to the randomness of random numbers, the number of occurrences of random numbers within the defined range can be balanced. Repartitioning makes the index value of the data in each partition the same. After adding the random number, the quadtree and random number of the data in each partition are made the same. At this time, because the random number has a certain degree of uniformity, the index value with the random number can ensure that the data volume of each partition of the vector graphics data is even after repartitioning. This allows the data of the RDD in Spark to be distributed as evenly as possible to each executor after shuffling. In addition, for vector graphics data, the shuffle process in Spark consumes less space calculations than cropping. Choosing the shuffle method to evenly distribute the data to each executor spreads the computational load.

[0029] Furthermore, step 7 decompiles the quadtree code of the vector graphics in RDD3, obtains the position of the code in the vector graphics, generates a specific grid of the vector graphics and cuts it to generate RDD4, which is specifically:

[0030] Traverse the RDD3, remove the random number from the index value to restore the quadtree code, decompile the quadtree code, generate a specific grid graphic for each vector graphic, cut the vector graphic WKT according to the grid to form a cut graphic WKT for each grid, and generate RDD4 using the quadtree code as the index value;

[0031] The data format of the RDD4 is: (quadtree encoding, (layer encoding, vector graphic element ID, cropped graphic WKT, original element json information)).

[0032] At this time, the vector layer of each layer in RDD3 is cut into grids. The specific grid of the vector graphics can be determined according to the quadtree code and the layer code, and the data in the grid can be obtained, which facilitates the superposition of vector graphics of each layer in units of grids.

[0033] Furthermore, the step 8 superimposes and decomposes the vector layer data of each layer according to the grid in S7, specifically:

[0034] Group RDD4 according to the quadtree code, superimpose the vector graphics under the same grid, decompose the superimposed vector graphics into overlapping and non-overlapping parts, generate fusion feature codes, and generate RDD5 with the decomposed vector graphics;

[0035] The data format of the RDD5 is (fusion element encoding, cropped graphic WKT, JSON information).

[0036] The specific steps of stacking and disassembling are:

[0037] Group the vector graphics under the same grid according to the layer code;

[0038] When only one group appears, no operation is performed;

[0039] When multiple groups appear, traverse all objects under the grid, sort them according to their layer codes, arrange and combine them in pairs, and overlay the overlapping vector graphics. Cut out the overlaid parts, then merge their layer codes, and repeat this step until all layer codes are the same.

[0040] Specifically, the layer codes are merged by taking the layer code of the top layer in the merged layers as the merged layer code.

[0041] Furthermore, the fusion element coding is divided into overlapping part coding and non-overlapping part coding;

[0042] The overlapping portion is encoded as a concatenated combination of vector graphic element IDs of all overlapping portions;

[0043] The non-overlapping portion is encoded as a vector graphic element ID of a single non-overlapping portion;

[0044] The JSON information is a combination of multiple overlapping raw element JSON information or a single non-overlapping raw element JSON information.

[0045] Specifically, the superimposed data is divided into overlapping parts and non-overlapping parts, and fusion element codes are used to distinguish them. In subsequent operations, the operations are grouped according to the fusion element codes, which makes data operations more convenient and improves data processing efficiency.

[0046] Furthermore, step S9 fuses the vector layer data in RDD5 and generates RDD6 from the fused data, specifically:

[0047] Traverse the vector layer data in the RDD5, group them according to the fusion element code, determine whether the data in the group is connected, and if so, fuse the vector layer data to generate a fusion graph WKT and fusion JSON information, and generate RDD6 with the fused data;

[0048] The data format of the RDD6 is (fusion element encoding, fusion graphic WKT, fusion JSON information).

[0049] Specifically, in one embodiment, touch may be used to determine whether vector data is connected. If the vector graphics are connected, the vector graphics are fused; if they are not connected, no operation is performed.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] 1. The method of the present invention can efficiently process multi-layer vector data by superimposing multi-layer vector data.

[0052] 2. A quadtree approach is used to determine the optimal grid division, and random numbers are used to distribute vector graphics data to Spark executors based on the grid conditions, improving the efficiency of massive data processing.

[0053] 3. Perform overlay operations on multi-layer vector data according to the grid structure, reduce the overlay calculation of vector data, improve the efficiency of data overlay, and facilitate the processing of vector data. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 1 is a schematic diagram of a specific flow chart of step S5 of an embodiment of the present invention.

[0055] Figure 2 FIG. 1 is a schematic diagram of a specific flow chart of step S7 according to an embodiment of the present invention.

[0056] Figure 3 1 is a schematic diagram of a specific flow chart of step S8 according to an embodiment of the present invention.

[0057] Figure 4 1 is a schematic diagram of a specific flow chart of step S9 according to an embodiment of the present invention. Specific implementation methods

[0058] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting the present invention. To better illustrate the following embodiments, some components in the accompanying drawings may be omitted, enlarged, or reduced in size, and do not represent actual product dimensions. Those skilled in the art will appreciate that some well-known structures and their descriptions may be omitted from the accompanying drawings. Example

[0059] like Figure 1 As shown, in this embodiment, the outer frame rectangle is the minimum containing rectangle z, and the two vector graphics therein are superimposed.

[0060] Specifically, first, read the multi-layer vector graphics data in the same coordinate system: encode each layer, recorded as layer code; assign an ID to each vector graphic in all layers and record it, recorded as vector graphic feature ID, and each vector graphic corresponds to a vector graphic feature ID; establish a unified coordinate system for all layers, and record the minimum coordinate x, maximum coordinate x1, minimum coordinate y, and maximum coordinate y1 of each vector graphic in all layers in the coordinate system; generate the original feature json information of each vector graphic, and then merge multiple layers to generate a distributed dataset RDD1 with the same data format containing all layers in Spark. The data format of RDD1 is (vector graphic feature ID, layer code, graphic WKT, minimum coordinate x, maximum coordinate x1, minimum coordinate y, maximum coordinate y1, original feature json information).

[0061] For the vector data in RDD1, determine the spatial range, that is, the spatial range of the quadtree space crop. Specifically, traverse all vector graphics in all layers, obtain the maximum coordinates x1, y1 and minimum coordinates x, y of each vector graphic, and by comparing the maximum coordinates x1, y1 and minimum coordinates x, y of each vector graphic in the same layer, obtain the minimum rectangle z1 of the layer that can contain all vector graphics in the layer. Compare the z1 of all layers to determine the minimum rectangle that can contain all z1s. This rectangle is the minimum containing rectangle z.

[0062] After determining the minimum containing rectangle z, traverse all the vector graphics in the minimum containing rectangle z and generate a shortest quadtree encoding for each layer of vector graphics based on the minimum containing rectangle z. The shortest quadtree is the quadtree with the minimum depth formed when all areas cannot be further divided. The maximum encoding length len of each vector graphics is recorded, such as Figure 1 As shown in the figure, the quadtree code of the grid "00" is "00", the code length is 2, and the length len that appears the most times is selected from the maximum code length len of each recorded vector graphic. If there are multiple lengths len with the same number of times, the longest len ​​is selected from the lengths len as the standard quadtree code length len1 of the minimum containing rectangle z.

[0063] like Figure 1As shown, after the minimum containing rectangle z is quadtree-encoded according to the standard quadtree encoding length len1, a grid structure of z is generated. z is then cropped according to the grid structure so that the vector graphics data of each layer is also divided according to the grid. At this time, the quadtree code can be used as an index value to generate a new RDD2. The data format of RDD2 is (quadtree code, (vector graphic element ID, layer code, graphic WKT, original element json)). In this embodiment, when we use the index value "03", we can obtain the data of two vector graphics in the corresponding grid, such as the graphic WKT.

[0064] In order to evenly distribute the data on the executors in the RDD, a random number is added to the index value in RDD2. In this embodiment, the data in RDD2 is traversed, and a random function is used to obtain a random value from 0 to 9 to construct a 4-bit random number, such as "6120". A separator identifier, such as "_", is set, and the random number is added to the quadtree code of RDD2 to form a new index value, such as "03_6120". Then, repartitioning is performed according to the new index value. At this time, all data with the index value of "03_6120" are divided into the same partition. Due to the introduction of random numbers, A value with the quadtree encoding of "03" but a different random number is assigned to a different partition. Since random numbers can be kept evenly distributed within a certain range, the amount of data in these partitions is relatively evenly distributed. Without the random number, using the quadtree encoding as the index, data with index "03" would be assigned to the same partition, while data with index "20" would be assigned to another partition. Because different indexes might correspond to different amounts of data, this partitioning method results in varying computational load when partitioned data is distributed to executors, which can easily lead to data skew. Spark then shuffles the data to generate an RDD3 whose format is (quadtree encoding + random number, (vector feature ID, layer encoding, WKT image, raw feature JSON)).

[0065] Traverse RDD3, remove the separator and random number from the index value, and restore the index value to the quadtree code. Decompile the quadtree code according to the minimum containing rectangle z, such as Figure 2 As shown, since the data stored in RDD3 is image WKT, the specific vector graphics indexed by the quadtree encoding and vector element ID are actually the WKT data of the entire vector graphics (i.e. Figure 2 The first step from the left in the figure), and according to the quadtree code decompilation, the specific grid graphics of each layer vector graphics can be generated, each grid corresponds to a quadtree code (corresponding Figure 2The second step from the left) is to cut the vector graphics according to the grid, which is specifically to generate a cut graphic WKT according to the grid situation (corresponding to Figure 2 The third step from the left) and the cut shape WKT is used to generate RDD4. The data format of RDD4 is (quadtree encoding, (layer encoding, vector graphic element ID, cut shape WKT, original element json information)). In step S6, the data is shuffled. At this time, the data index in the partition remains unchanged, and the data content changes from shape WKT to cut shape WKT (corresponding to Figure 2 (picture of the fourth step from the left).

[0066] After the above operations, multiple grids containing multi-layer vector graphics data can be obtained. These grids are grouped according to the quadtree encoding using Spark's groupByKey, and the multi-layer vector data under the same grid are superimposed and disassembled.

[0067] The superposition and disassembly process is to group according to the layer code under the same grid. If only one group appears, no operation is performed; if multiple groups appear, all objects in the group are traversed, the vector graphics of the objects are sorted, and then arranged and combined in pairs to determine whether there is any graphic overlap. If there is no overlap, no further operation is performed. If there is overlap, the graphics are overlapped and the overlapping parts are cut out, and then the layer codes are merged. The processed results are subjected to the above operation again until there is no graphic overlap and the fused feature code is generated.

[0068] The above overlay and unpacking operations divide the vector graphics into two parts: one for the overlapping part of the layers and the other for the non-overlapping part of the graphics. The fused element code is also divided into overlapping part code and non-overlapping part code. The overlapping part code is the concatenated combination of the vector graphic element IDs of all overlapping parts, while the non-overlapping part code is the vector graphic element ID of a single non-overlapping part. The JSON information of the overlapping part is the combination of multiple original element JSON information, while the JSON information of the non-overlapping part is the JSON information of a single original element.

[0069] In this embodiment, specifically, Figure 3As shown, in Spark task 1, the grid with the quadtree code "20" is superimposed and disassembled. If it is determined that there are two layer codes, the two layers are superimposed, and the overlapping part is cut out, and the graphic is divided into one overlapping part and two non-overlapping parts. Similarly, in other executors, other grids are superimposed and disassembled, and an RDD5 is generated based on the disassembled vector graphics data. The data format of the RDD5 is (fused feature code, cut shape WKT, JSON information). At this time, the fused feature code is two superimposed vector graphics connected in the order of the layer codes, and the layer codes are merged. Specifically, the layer code is set to the layer code of the layer where the top vector graphic is located.

[0070] At this time, if there is data with different layer codes in the grid corresponding to the quadtree code "20", it is necessary to perform superposition and disassembly operations until the layer codes of all data are the same.

[0071] The vector layer data in RDD5 is traversed, grouped according to the fusion element code, and it is determined whether the data in the group is connected. If so, the vector layer data is fused.

[0072] Specifically, in this embodiment, Figure 4 As shown in the figure, after the overlay and disassembly operations, the vector graphics are still divided into various executors by the grid. The vector graphics with the same overlay relationship are grouped together through the fusion element coding. The touch function is used to determine whether the vector graphics in the group are connected. If they are connected, the vector graphics with the connected relationship are fused.

[0073] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the technical solutions of the present invention, and are not intended to limit the specific implementation methods of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A multi-layer vector data overlay method based on Spark big data, characterized in that: The method comprises: S1: Merge the layers of the multi-layer vector graphics to be processed and generate a distributed dataset RDD1 in Spark; S2: Establish a minimum containing rectangle z, which is a rectangular layer containing all RDD1 vector graphics; S3: Traverse all layers in RDD1, generate a shortest quadtree for each layer based on the minimum containing rectangle z as the boundary, and record its maximum encoding length len; S4: Select the length of the layer with the most occurrences in the maximum length len in S3, and use this length as the standard quadtree encoding length len1; S5: Split the minimum containing rectangle z according to the standard quadtree code length len1 to generate a grid structure, and generate RDD2 with the quadtree code as the index value; S6: Use random numbers to repartition the data in RDD2 and generate RDD3 with the repartitioned data; S7: Decompile the quadtree encoding of the vector graphic in RDD3, obtain the specific position of the grid of the vector graphic in the vector graphic and cut the grid, and generate RDD4 with the data in the grid; S8: Overlay and disassemble the vector graphics in RDD4 according to the grid area encoded by the quadtree, and generate RDD5 from the disassembled data; S9: Fuse the vector layer data in RDD5 and generate RDD6 from the fused data.

2. The multi-layer vector data overlay method based on Spark big data according to claim 1 is characterized in that: The data format of the RDD1 is: (vector graphic element ID, layer code, graphic WKT, minimum coordinate x, maximum coordinate x1, minimum coordinate y, maximum coordinate y1, original element json information).

3. The multi-layer vector data overlay method based on Spark big data according to claim 2 is characterized in that: The step S4 of selecting the standard quadtree encoding length len1 also includes: When the maximum lengths appear the same number of times, the longest one is selected as the standard quadtree encoding length len1.

4. The multi-layer vector data overlay method based on Spark big data according to claim 1 is characterized in that: The data format of the RDD2 is: (quadtree encoding, (vector graphic element ID, layer encoding, graphic WKT, original element json)).

5. The multi-layer vector data overlay method based on Spark big data according to claim 4 is characterized in that: Step 6 uses random numbers to repartition the data in RDD2 and generates RDD3 with the repartitioned data; specifically: Traverse the RDD2, add random numbers to the index values ​​in RDD2 to generate new index values, repartition RDD2 according to the index values ​​to generate RDD3; The data format of the RDD3 is: (quadtree code plus random number, (vector layer element ID, layer code, graphic WKT, original element json information)).

6. The multi-layer vector data overlay method based on Spark big data according to claim 5 is characterized in that: The step 7 decompiles the quadtree code of the vector graphics in RDD3, obtains the position of the code in the vector graphics, generates a specific grid of the vector graphics and cuts it to generate RDD4, specifically: Traverse the RDD3, remove the random number from the index value to restore the quadtree code, decompile the quadtree code, generate a specific grid graphic for each vector graphic, cut the vector graphic WKT according to the grid to form a cut graphic WKT for each grid, and generate RDD4 using the quadtree code as the index value; The data format of the RDD4 is: (quadtree encoding, (layer encoding, vector graphic element ID, cropped graphic WKT, original element json information)).

7. A multi-layer vector data overlay method based on Spark big data according to any one of claim 6, characterized in that: Step 8 superimposes and decomposes the vector graphics in RDD4 according to the grid area encoded by the quadtree, and generates RDD5 from the decomposed data, specifically: Grouping the RDD4 according to the quadtree code, superimposing the vector graphics under the same grid, decomposing the superimposed vector graphics into overlapping parts and non-overlapping parts, generating fusion element codes, and generating RDD5 with the decomposed vector graphics; The data format of the RDD5 is (fusion element encoding, cropped graphic WKT, JSON information).

8. The multi-layer vector data overlay method based on Spark big data according to claim 7 is characterized in that: The specific steps of stacking and disassembling are: Group the vector graphics under the same grid according to the layer code; When only one group appears, no operation is performed; When multiple groups appear, traverse all objects under the grid, sort them according to their layer codes, arrange and combine them in pairs, and overlay the overlapping vector graphics. Cut out the overlaid parts, then merge their layer codes, and repeat this step until all layer codes are the same.

9. The multi-layer vector data overlay method based on Spark big data according to claim 7 is characterized in that: The fusion element coding is divided into overlapping part coding and non-overlapping part coding; The overlapping portion is encoded as a concatenated combination of vector graphic element IDs of all overlapping portions; The non-overlapping portion is encoded as a vector graphic element ID of a single non-overlapping portion; The JSON information is a combination of multiple overlapping raw element JSON information or a single non-overlapping raw element JSON information.

10. The multi-layer vector data overlay method based on Spark big data according to claim 9 is characterized in that: The step S9 merges the vector layer data in RDD5 and generates RDD6 from the merged data, specifically: Traverse the vector layer data in the RDD5, group them according to the fusion element code, determine whether the data in the group is connected, and if so, fuse the vector layer data to generate a fusion graph WKT and fusion JSON information, and generate RDD6 with the fused data; The data format of the RDD6 is (fusion element encoding, fusion graphic WKT, fusion JSON information).

Citation Information

Patent Citations

  • Vector element parallel computing method and device, storage medium and terminal

    CN111563081A

  • Three-dimensional terrain model real-time smooth drawing method with combination of GPU technology

    CN105336003A

  • Spatial analysis method and device

    CN110597935A