3DTiles data storage method and map data query method

Abstract

The present invention discloses a 3D tile data storage method and a map data query method, wherein the 3D tile data storage method includes the following steps: 1) establishing a LOD hierarchical pyramid-shaped hierarchical table and creating an LOD range index; 2) generating proxy points for three-dimensional objects, segmenting tiles according to a KD-tree structure to construct a three-dimensional KD-tree index tree; 3) storing the data of the KD-tree leaf nodes in GridFS of MongoDB multimodal storage, mapping the file path to a unique hash value, writing the value into the corresponding node according to the KD-tree node storage structure, and creating a geospatial index. The present invention creatively solves the difficulties in file system management caused by a large number of files. The hierarchical structure based on the file directory is difficult to support efficient spatial range retrieval. By combining the three-dimensional KD-tree index with the MongoDB geospatial index, the spatial query response time is reduced to milliseconds.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional geographic space data storage and management, and in particular to a 3D Tiles data storage method and a map data query method. Background Art

[0002] 3DTiles is an open standard three-dimensional geospatial data format widely used in fields such as digital twins and smart cities. Traditional 3DTiles data is stored in file format, which has the following problems:

[0003] Large number of files: A single scene may contain millions of files, making file system management difficult.

[0004] Low transmission efficiency: When transferring a large number of small files, there is a significant delay due to the high network request overhead.

[0005] Poor spatial query performance: The hierarchical structure based on file directories is difficult to support efficient spatial range retrieval.

[0006] Existing technologies generally use folder management, or solutions based on HBase database management, or file systems combined with databases, but these solutions still suffer from storage redundancy and query performance bottlenecks. Therefore, there is an urgent need for a solution that supports the efficient storage, fast retrieval, and transmission of massive 3D Tiles data. Summary of the Invention

[0007] The present invention aims to overcome the shortcomings of existing technologies by providing a 3D Tiles data storage method and a map data query method, aiming to address efficiency issues in the transmission, storage, and management of massive amounts of 3D tile data. By deeply integrating spatial indexing with database storage, this solution provides an efficient and scalable management paradigm for massive 3D Tiles data.

[0008] A 3D Tiles data storage method comprises the following steps:

[0009] 1) Data preprocessing step: sort the tile data of different LOD levels according to the corresponding scale to build an LOD level pyramid table and create an LOD range index;

[0010] 2) Generate proxy points for the 3D object, segment the tiles according to the KD-tree structure to construct a 3D KD-tree index tree. Each node stores the tiles and child node references of the corresponding spatial range and merges adjacent low-detail tiles. At the same time, multi-target clustering is performed on the tiles in the spatial range and the clustering results are mapped back to the proxy points.

[0011] 3) Store the data of the KD-tree leaf nodes in MongoDB's multimodal storage GridFS, map the file path to a unique hash value, generate a file_index collection to record the path mapping, write it to the corresponding node according to the KD-tree node storage structure, and create a geospatial index.

[0012] Preferably, the above step 1) further comprises the steps of acquiring three-dimensional data and converting all data into the same geographic coordinate system and projection, aligning the Z-axis reference, and then cleaning the data.

[0013] Preferably, step 3) further includes a dynamic KD-tree adjustment step, which includes the following steps:

[0014] 31) If the maximum LOD level value of the current node is greater than the global threshold, or the data point density is greater than the density threshold, the split is triggered;

[0015] 32) Adjust the split point position according to the LOD level distribution gradient of the current node;

[0016] 33) Split and verify again whether the maximum LOD level value of the current node is ≤ the global threshold, and the data point density is ≤ the density threshold. If so, stop splitting, otherwise continue splitting and verification;

[0017] 34) Nodes that meet the stop splitting conditions are marked as leaf nodes, and the aggregation information of their coverage areas is stored. High LOD level nodes are not merged, and their original fine-grained KD-tree structure is maintained to achieve high-precision queries.

[0018] Preferably, the step 32) is to obtain a distribution gradient by statistically analyzing the distribution trend of the LOD level values on the segmentation axis, and convert the distribution gradient into a segmentation point offset.

[0019] Preferably, the KD-tree nodes of the high LOD level area are preloaded into the memory, the node pointers are stored using an efficient memory data structure, and the leaf nodes of the low LOD level area are persistently stored to the disk.

[0020] Preferably, a dynamic update step is also included. When new high LOD level data is added, only the affected KD-tree branches are updated, and the modification status of the nodes is marked using version control to avoid global reconstruction.

[0021] A map data query method, wherein the map data is stored by the data storage method, and the map data is deployed in cluster storage. The query method includes: if accessed directly from a map page, first calling the LOD range index, then the three-dimensional KD-tree tree index; if spatial operation is performed, first calling the three-dimensional KD-tree tree index, then the geographic space index;

[0022] If a partition is specified, first perform a 3D KD-tree index, then perform an LOD range index.

[0023] The advantages and beneficial effects of the present invention are:

[0024] This invention creatively solves the difficulties in file system management caused by the huge number of files. By storing massive small files through GridFS, the number of files is reduced by more than 90%. The hierarchical structure based on file directories is difficult to support efficient spatial range retrieval. Combining the three-dimensional KD-tree index with the MongoDB geospatial index, the spatial query response time is reduced to milliseconds. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the present invention are further described below with reference to specific embodiments.

[0026] A 3DTiles data storage method of the present invention is specifically a 3DTiles data storage method based on MongoDB and KD-tree, which includes the following steps:

[0027] 1) Data preprocessing step: sort the tile data of different LOD levels according to the corresponding scale to build an LOD level pyramid table and LOD range index, and create a spatial hash grid for the LOD level data; specifically, it includes:

[0028] 1. Structured annotation of LOD level information, including LOD level definition, which clarifies the correspondence between LOD levels and map scales (e.g., LOD level 12 corresponds to a 1:500 scale), and the establishment of an LOD level table. Each tile or geographic area is assigned a unique LOD level identifier to ensure consistent LOD level values within the same area. This also includes LOD level metadata integration, which embeds LOD level information into spatial data structures (e.g., attribute fields in vector layers or hierarchical naming conventions for raster data) and constructs LOD level metadata indexes (e.g., R-trees or range trees) to enable fast querying of the spatial extent corresponding to a specific LOD level.

[0029] 2. Hierarchical organization of spatial data, which includes,

[0030] Multi-scale data integration:

[0031] Sort the tile data of different LOD levels by scale and build an LOD level pyramid.

[0032] Ensure that higher LOD data covers details of lower LOD data (e.g. LOD 12 tile covers the entire area of LOD 5 tile).

[0033] Spatial index optimization:

[0034] Build an LOD range index (such as an interval tree) to support fast queries for areas with "LOD level ≥ N", establish a spatial hash grid for LOD level data, and accelerate local area queries.

[0035] 2) Generate proxy points for the 3D objects and segment the tiles of 3DTiles according to the KD-tree structure to construct a 3D KD-tree index tree. Each node stores the tile data and child node references of the corresponding spatial range. At the same time, multi-target clustering is performed on the tiles within the spatial range and the clustering results are mapped back to the proxy points. The multi-target clustering mainly integrates attributes such as geometric features (volume, surface area), semantic labels (building type, land use) and environmental parameters (sunshine duration, wind speed). The implementation method includes first extracting the proxy points and attribute matrix of the 3D objects, then running the K-means algorithm to generate clusters (such as zoning by function), and finally mapping the clustering results back to the original 3D entities.

[0036] Among them, proxy point generation includes the geometric center method and the surface sampling method. The geometric center method takes the center of mass of each three-dimensional object (such as a building) as the proxy point; the surface sampling method uniformly samples points on the surface of the three-dimensional model (applicable to grid models). The construction process of the three-dimensional KD-tree index tree includes recursive space segmentation, and each time the dimension with the largest span in the current area is selected for cutting. The stopping condition is when the number of points in the node is less than a threshold (such as 100) or the maximum depth is reached.

[0037] 3) Data from KD-tree leaf nodes is stored in MongoDB's multimodal storage, GridFS. File paths are mapped to unique hash values, and a file_index collection is generated to record the path mapping. Data is written to the corresponding nodes according to the KD-tree node storage structure, and a geospatial index is created. 3DTiles files (such as .b3dm and .pnts) from KD-tree leaf nodes (a single .b3dm 3DTiles file is typically between 1KB and 2MB) are stored in binary form in MongoDB GridFS (16MB), addressing the storage of massive small files. File paths are mapped to unique hash values, and the file_index collection maintains the association between logical paths and physical storage, enhancing spatial indexing. A MongoDB geospatial index (2dsphere) is added to KD-tree nodes to support fast spatial range queries. KD-tree hierarchy and LOD level information are combined to achieve dual filtering: spatial and level of detail.

[0038] Through the storage method of the present invention, storage efficiency is improved: by storing massive small files through GridFS, the number of files is reduced by more than 90%. Query performance is optimized, KD-tree is combined with geographic spatial index, and the spatial query response time is reduced to milliseconds. Transmission bandwidth is saved, and data block compression reduces network transmission volume by 20%-36%. Taking the smart city project as an example, the original 3DTiles data contains 2.2 million files with a total size of 1.3TB. After processing by the method of the present invention: MongoDB storage occupancy is reduced to 1TB (compression rate 23%); tile loading time is shortened from an average of 8s to 1.5s; and the server's concurrent support capability is increased to 10,000QPS. This solution provides an efficient and scalable management paradigm for massive 3DTiles data by deeply integrating spatial indexing and database storage, and has significant industrial application value.

[0039] Specifically, the following display can be quickly achieved:

[0040] (a) Range-based display

[0041] Define a 3D query window:

[0042] text

[0043] [xmin, xmax] × [ymin, ymax] × [zmin, zmax]

[0044] Example: Range query for a city development zone (x=113°~116°, y=22°~25°, z=0~150m)

[0045] Execute the query:

[0046] Use the three-dimensional KD-tree index tree to quickly locate all points / objects in the area.

[0047] Result output: a list of 3D entities that meet the conditions and their attributes.

[0048] (b) Division under administrative division constraints

[0049] Constraint design:

[0050] 1. Spatial overlap principle: Only three-dimensional objects that are completely located within the same administrative division are allowed to participate in the division.

[0051] 2. Vertical stratification rule: vertically divide the same administrative division according to height intervals (such as 0~50m / 50~100m).

[0052] Implementation method:

[0053] Find the intersection of the three-dimensional boundary of the administrative division and the three-dimensional KD-tree index tree query window.

[0054] Recursively subdivide the data within the intersection area.

[0055] Typical application scenarios of the above model include smart city management, such as demarcating fire rescue priority areas (combining building height and road network), monitoring the distribution of urban heat island effects (analyzing the relationship between surface temperature and building density); cultural heritage protection, such as three-dimensional zoning of ancient building complexes (according to historical period or architectural style), setting up virtual tour routes to avoid sensitive areas; disaster simulation analysis, such as predicting the scope of flood disasters (combining terrain elevation and administrative divisions), and assessing the threat posed to residential areas by wildfire spread paths.

[0056] Furthermore, data preprocessing steps also include 3D data acquisition, which involves generating point cloud data through lidar scanning or photogrammetry, such as using GIS tools (such as QGIS) to export the 3D boundaries of administrative divisions (such as converting .shp files into 3D polyhedra). This also includes coordinate system 1, such as converting all data to the same geographic coordinate system (such as CGCS2000) and projection (such as UTM), and aligning the Z-axis datum (such as using the same datum for altitude: the Yellow Sea Elevation System). Data cleaning is also performed to remove noise points (such as outlier height values) and redundant blank areas, and to perform interpolation and filling (such as kriging interpolation) on sparse 3D areas.

[0057] As a further optimization solution, it also includes the dynamic hierarchical construction of KD-tree, that is, dynamically adjusting the KD-tree depth according to the tile detail level (LOD level) to ensure fine-grained division of nodes in high-detail areas and merged storage of low-detail areas. It includes the following steps:

[0058] 31) If the maximum LOD level value of the current node is greater than the global threshold, or the data point density is greater than the density threshold, the split is triggered;

[0059] 32) Adjust the split point position according to the LOD level distribution gradient of the current node, that is, obtain the distribution gradient by statistically analyzing the distribution trend of the LOD level value on the split axis, and convert the distribution gradient into the split point offset.

[0060] 33) Before segmentation, verify the above conditions 31) until the maximum LOD level value of the current node is ≤ the global threshold, and the data point density is ≤ the density threshold. If so, stop segmentation, otherwise continue segmentation;

[0061] 34) Mark the nodes that meet the stopping conditions as leaf nodes, store the aggregate information of their coverage areas, do not merge high LOD level nodes, and maintain their original fine-grained KD-tree structure to achieve high-precision queries.

[0062] Specifically, the dynamic KD-tree construction and refinement includes the following steps:

[0063] 1. Dynamic calculation of segmentation conditions

[0064] First, the LOD level threshold is determined:

[0065] Set the global LOD level threshold (such as LOD level_THRESHOLD=10) as a hard condition for whether to continue segmentation. If it is not met, no segmentation will be performed directly.

[0066] Calculate the maximum LOD level value of the area covered by the current node (obtained through the pre-processed LOD range index).

[0067] Next, evaluate the data density:

[0068] Define a density threshold (e.g., DENSITY_THRESHOLD = 100 points / km²) and trigger splitting when the density of data points within a node exceeds the threshold.

[0069] Density calculation method: number of data points in a node ÷ node coverage area.

[0070] 2. Dynamic selection of split axes

[0071] While traditional methods prioritize the dimension with the largest variance (x→y→z cyclic priority), this invention employs a dynamic scaling approach, adjusting axis priorities based on LOD gradients. For example, dimensions with large LOD differences are prioritized for segmentation. The split point adjustment mechanism is based on the base split point, which is the median or mean calculated using the traditional KD-tree algorithm.

[0072] At the same time, it is also possible to adjust the LOD level weighted adjustment, that is, adjust the split point position according to the LOD level distribution gradient of the current node. Specifically, it includes gradient calculation, statistical distribution trend of LOD level values on the split axis (such as linear regression slope), that is, to analyze the distribution of LOD level values on the split axis and obtain the distribution gradient. The gradient can be approximated by numerical differentiation methods, such as calculating the difference between adjacent LOD level values;

[0073] The offset calculation involves converting the gradient into a split point offset (e.g., the larger the gradient, the larger the offset). This can be achieved through linear or nonlinear transformations. The proportional coefficient in the linear transformation formula controls the degree to which the gradient affects the offset, and this is adjusted based on the specific application scenario and requirements.

[0074] 3. The termination conditions of recursive segmentation include:

[0075] If the maximum LOD value of the current node is less than or equal to the global threshold, and the data point density is less than or equal to the density threshold, and the node coverage area is completely at the same LOD level and cannot be further split, the split is forced to terminate. At the same time, the above conditions are verified before each split to avoid unnecessary recursive calls.

[0076] Among them, parallel segmentation is adopted in the process of constructing the KD-tree structure, that is, multi-threading or distributed computing framework (such as Apache Spark) is used to parallelly process the segmentation tasks of sub-nodes, and a task allocation strategy is adopted to dynamically allocate computing resources according to the area size or LOD level complexity. At the same time, approximate segmentation is adopted, that is, an error range (such as slight deformation of geometric shapes) is allowed to be introduced in low LOD level areas to reduce the number of segmentations. The error control mechanism is to set the maximum allowable deviation (such as 5% shape similarity).

[0077] Furthermore, the segmentation also includes a complex scene processing refinement step, which includes:

[0078] 1. The coordination of horizontal and vertical LOD levels primarily considers 3D scene adaptation. This involves combining horizontal LOD levels (map scale) with vertical LOD levels (height precision). For example, a high horizontal LOD level (city center) + a high vertical LOD level (building facade details), or a low horizontal LOD level (suburban area) + a low vertical LOD level (terrain simplification). A hybrid LOD strategy dynamically adjusts the 3D space segmentation priority based on the 3D scene (e.g., segmenting by horizontal LOD level first, then by vertical LOD level).

[0079] 2. Dynamic LOD threshold adjustment, that is, runtime threshold modification. For example, an API interface is provided to allow users to adjust the LOD threshold at runtime (for example, based on the current network bandwidth or hardware performance). The threshold adjustment can automatically trigger local reconstruction of the KD-tree (such as merging or splitting nodes).

[0080] Furthermore, it also includes the step of refining the KD-tree dynamic storage strategy.

[0081] The aggregation steps of leaf nodes include:

[0082] Low-LOD regions are merged, marking nodes that meet the termination criteria as leaf nodes and storing aggregated information about their coverage areas (such as LOD range, maximum height, and minimum bounding box). This aggregation method can employ quadtree or gridding strategies to secondary index the internal data of leaf nodes. Meanwhile, high-LOD regions are retained, and high-LOD nodes are not merged, preserving their original fine-grained KD-tree structure to achieve high-precision queries.

[0083] 2. Hybrid storage architecture design, including:

[0084] Memory layer storage: Preload the KD-tree nodes in the high LOD layer area into memory (such as using object pool technology to manage the node life cycle), and use efficient memory data structures (such as array-based or linked list-based) to store node pointers.

[0085] Disk-tier storage: Persistently store leaf nodes in low-LOD regions on disk (e.g., using a file system directory tree to manage node files). Also optimize disk I / O performance by using block compression (e.g., Snappy) or columnar storage formats (e.g., Parquet).

[0086] 3. Dynamic update mechanism

[0087] The incremental build strategy updates only the affected KD-tree branches when new high-LOD data is added, using version control to mark the modified status of nodes to avoid global rebuilds. Lazy deletion technology is also used to delay deletion operations, record a list of nodes to be cleaned up, and process them in batches when queries are idle.

[0088] At the same time, the present invention also discloses a map data query method, wherein the map data is stored by the data storage method, and the map data is clustered and stored. The query method includes: if accessed directly from a map page, first calling the LOD range index, then the three-dimensional KD-tree index tree; if spatial operation is performed, first the three-dimensional KD-tree index tree, then the MongoDB geographic spatial index; if a region is specified, first the three-dimensional KD-tree index tree, then the LOD range index; if it is a composite query method or an unclear query method, then the last transmitted index mode is used, and one of the other two modes is selected at the same time, and a predetermined time or predetermined data volume is transmitted at the same time, and the transmission speeds under the two modes are compared and the index mode of one of the two modes is selected preferably. When a composite query or an unclear query mode is encountered next time, the last selected index mode is directly selected and compared with the third mode that was not compared last time, and the index mode of one of the two modes is selected preferably. This cycle can ensure fast decision-making and try to ensure the optimal transmission method.

[0089] The on-demand transmission and caching mechanism of the present invention optimizes transmission according to requests. The client sends a spatial query request to the server based on the view range and zoom level. The server quickly locates the required tile nodes through the KD-tree index tree and returns only the necessary data. When a user queries data, for example, when a user views a map and queries based on the view range, the server quickly queries the data required by the user through MongoDB GridFS, quickly locates the data within the view range based on the KD-tree index tree, and returns it to the user.

[0090] At the same time, LOD-sensitive queries can also be added. Specifically, a priority traversal strategy prioritizes access to child nodes that may contain high-LOD data during queries, using the node's pre-stored maximum LOD level for pruning. A traversal order adjustment strategy dynamically adjusts the order of child node access based on the LOD level requirements of the query area (e.g., prioritizing high-LOD child nodes). A range query optimization strategy combines LOD spatial indexes (e.g., R-tree) to quickly locate candidate nodes, and performs LOD-level conditional filtering on each candidate node (retaining only nodes with an LOD level ≥ the target LOD level).

[0091] Furthermore, there are also caches for frequently queried LOD levels and regions (e.g., Redis memory cache), a cache expiration policy that dynamically expires based on data update time or query popularity, and a prefetch mechanism that predicts the next likely accessed region based on query history and preloads relevant nodes into memory. In-memory caching is also enabled for frequently accessed KD-tree nodes to avoid repeated computations. KD-tree nodes for high-LOD regions are preloaded into memory, using efficient in-memory data structures to store node pointers. Leaf nodes for low-LOD regions are persistently stored to disk.

[0092] Furthermore, the present invention also adopts a data block compression strategy, merging tiles from the same KD-tree node into a single data block. For example, using the LZ4 compression algorithm to reduce transmission volume. Merging and compressing batches of data retrieved by users before sending them to users can improve efficiency.

[0093] This invention creatively solves the following technical problems: 1. Difficulty managing file systems due to a large number of files: GridFS stores a massive number of small files, reducing the number of files by over 90%. 2. Network latency caused by transferring large numbers of small files: Data block compression technology is used to reduce network transmission volume by 20%-36%. 3. Hierarchical file directory structures struggle to support efficient spatial range retrieval: Combining a three-dimensional KD-tree index with MongoDB geospatial indexing reduces spatial query response time to milliseconds.

[0094] The innovation of this invention lies in the deep integration of 3D tile data with spatial indexing to achieve efficient data management and optimized transmission. Through the dynamic construction of KD-tree and multimodal storage design, it solves the efficiency bottleneck in traditional 3D tile data storage and management, significantly improving data access performance and system scalability.

[0095] The above is an exemplary description of the present invention. It should be noted that, without departing from the core of the present invention, any simple deformation, modification or other equivalent replacement that can be made by other skilled in the art without expending creative labor falls within the scope of protection of the present invention.

Claims

1. A 3D Tiles data storage method, characterized in that: The following steps are included: 1) Data preprocessing step: sort the tile data of different LOD levels according to the corresponding scale to build an LOD level pyramid table and create an LOD range index; 2) Generate proxy points for the 3D object, segment the tiles according to the KD-tree structure to construct a 3D KD-tree index tree. Each node stores the tiles and child node references of the corresponding spatial range and merges adjacent low-detail tiles. At the same time, multi-target clustering is performed on the tiles in the spatial range and the clustering results are mapped back to the proxy points. 3) Store the data of the KD-tree leaf nodes into MongoDB's multimodal storage GridFS, map the file path to a unique hash value, generate a file_index collection to record the path mapping, write it to the corresponding node according to the KD-tree node storage structure, and create a geospatial index; Step 3) also includes a dynamic KD-tree adjustment step, which includes the following steps: 31) If the maximum LOD level value of the current node is greater than the global threshold, or the data point density is greater than the density threshold, the split is triggered; 32) Adjust the split point position according to the LOD level distribution gradient of the current node; 33) Split and verify again whether the maximum LOD level value of the current node is ≤ the global threshold, and the data point density is ≤ the density threshold. If so, stop splitting, otherwise continue splitting and verification; 34) Nodes that meet the stop splitting conditions are marked as leaf nodes, and the aggregation information of their coverage areas is stored. High LOD level nodes are not merged, and their original fine-grained KD-tree structure is maintained to achieve high-precision queries.

2. The 3D Tiles data storage method according to claim 1, wherein: The step 1) further includes the steps of acquiring three-dimensional data and converting all data into the same geographic coordinate system and projection, aligning the Z-axis reference, and then cleaning the data.

3. The 3D Tiles data storage method according to claim 1, wherein: The step 32) is to obtain a distribution gradient by statistically analyzing the distribution trend of the LOD level value on the segmentation axis, and convert the distribution gradient into a segmentation point offset.

4. The 3D Tiles data storage method according to claim 1, wherein: Preload the KD-tree nodes in the high LOD level area into memory, use efficient memory data structures to store node pointers, and persist the leaf nodes in the low LOD level area to disk.

5. The 3D Tiles data storage method according to claim 1, wherein: It also includes a dynamic update step. When new high-LOD level data is added, only the affected KD-tree branches are updated, and the modified status of the node is marked using version control to avoid global reconstruction.

6. A map data query method, characterized in that: The map data is stored by the 3DTiles data storage method according to any one of claims 1 to 5, the map data is deployed in cluster storage, and the query method includes: If accessed directly from the map page, first call the LOD range index, then the 3D KD-tree tree index; For spatial operations, first use the three-dimensional KD-tree index, then the geographic spatial index; If a partition is specified, first perform a 3D KD-tree index, then perform an LOD range index.