Storage and management method of multi-resolution block stacked grid based on HDF5

Abstract

The invention discloses an HDF5-based multi-resolution block stacked grid storage and management method, which comprises the following steps of: dividing into a plurality of three-dimensional models according to actual requirements and model complexity and different resolution grades, constructing a mixed data storage structure of a hierarchical K-way tree, fusing and managing the models of different scales, and performing cross-scale unified scheduling, so as to realize multi-scale multi-resolution block stacked grid storage and management. And then, after the three-dimensional model is converted into a sparse stacked grid model, the stacked grid model is stored and managed by utilizing the characteristics that an HDF5 file supports infinite multiple data types and is specially designed for flexible and efficient I / O and large-capacity and complex data.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional geological model data management, and more particularly to a storage and management method for multi-resolution block stacked grids based on HDF5. Background Art

[0002] Visualization helps extract and identify essential information from large amounts of spatiotemporal data and has become a hot topic in geoscience. Accessing, processing, and applying real-time data visualization of 3D geological models in web browsers is a major development trend. Online visualization is limited by hardware and software resources. Over the past few decades, researchers have proposed various data structures to represent 3D geological models, aiming to reduce the data size and make them easier to analyze and process online. Among them, the stacked grid model has been widely used in online visualization due to its advantages in data size and spatial querying (Natali et al. 2014; Tegtmeier et al. 2014; Graciano et al. 2018).

[0003] Mesh models need to be able to capture minute details of the solid geology. However, uniformly detailed volume representations require enormous memory usage. Processing large datasets for volume rendering is not a simple task, especially when interactivity is required, requiring significant computational effort. Although many studies have constructed stacked mesh models for various purposes, Graciano et al. (2018) also utilized stacked meshes for real-time visualization, which significantly saves memory compared to voxel models. However, current research focuses on single-block, single-scale stratigraphic and lithologic structural modeling (Prinds et al. 2020). Constructing high-precision spatial indexes for stacked mesh models to support rapid data updates remains an open problem. Summary of the Invention

[0004] The present invention provides a storage and management method for multi-resolution block stacked grids based on HDF5, which solves the technical problem of constructing a high-precision stacked grid model spatial index to support rapid data update in the prior art.

[0005] In order to solve the above technical problems, the technical solutions of the present invention are as follows: The present invention provides a method for storing and managing multi-resolution block stacked grids based on HDF5, comprising the following steps: Generating a plurality of three-dimensional models from the preset three-dimensional uniform grid model according to the resolution level, wherein the degree of detail expression of the three-dimensional models increases step by step with the resolution level; Based on the multiple three-dimensional models, a multi-level index structure is constructed, wherein the upper layer of the multi-level index structure adopts a multi-level resolution index to integrate and manage three-dimensional models of different resolution levels, and the lower layer of the multi-level index structure defines a plurality of K-ary trees, each K-ary tree recording a three-dimensional model of a resolution level; Based on the multi-level index structure, storing the multiple three-dimensional models; Converting each of the three-dimensional models into a stacked grid model to obtain a plurality of stacked grid models; The plurality of stacked grid models are converted into HDF5 files.

[0006] In the above technical means, first, according to actual needs and model complexity, the data is divided into multiple three-dimensional models at different resolution levels, and a hierarchical K-ary tree hybrid data storage structure is constructed to integrate and manage models of different scales for unified scheduling across scales. Then, after converting the three-dimensional model into a sparse stacked grid model, the stacked grid model is stored using the HDF5 file's support for an unlimited number of data types, which is designed for flexible and efficient I / O as well as large-capacity and complex data.

[0007] Furthermore, each K-ary tree records a three-dimensional model of a resolution level, including: The K-ary tree divides the three-dimensional uniform grid model into grids, each grid has the same time complexity and space complexity, the grid is expressed as an instance object of the StackedGridBlock class, and each node of the K-ary tree records the model information of the corresponding grid in the three-dimensional model; When the resolution level is n, the number of nodes in the K-ary tree is the number of grids into which the three-dimensional uniform grid model is divided. for: .

[0008] Furthermore, the K-ary tree uses a linear list to store leaf nodes and a compact array to implement continuous array storage. The compact array member type is a BlockInformation structure. The index position dataIndex(i) of the grid with a given index i in the linear list is calculated, where i is the index value of the full tree: dataIndex(i)= BlockInformation[i] Where, BlockInformation[ ] represents the BlockInformation structure array; If the return value is -2, the subspace is empty; otherwise, the return value is the index position of the grid with the given index i in the linear table.

[0009] Furthermore, based on the multi-level index structure, before storing the multiple three-dimensional models, the method further includes extracting data from physical storage into the multi-level index structure, including: The original data of the three-dimensional model at different resolution levels are stored in different computer disk spaces, and getLevelbyFile is used to extract detailed information corresponding to different resolution levels from the computer disk spaces, wherein the detailed information includes resolution level, grid scale, grid block spacing in three directions, and grid block width in three directions; Initializing a StackedGrid object based on the detailed information to store three-dimensional models at different resolution levels, calculating and setting spatial parameters of the object, wherein the spatial parameters include grid block position, grid block spacing in three directions, and grid block width in three directions; for each file in a computer disk space associated with a resolution level, creating a StackedGridBlock object to store the grid model, and reading block data from the computer disk space, wherein the block data includes PillarIndex and PillarData; The state of the grid is updated based on the presence of data, and pointers and flags are set accordingly. Finally, the populated StackedGrid object is appended to the LODStackedGrid array of the LODSStackedGrid object.

[0010] Furthermore, each of the three-dimensional models is converted into a stacked grid model, comprising: The three-dimensional model is evenly divided into a plurality of uniform grids according to the resolution level; Perform a binary tree search on all the uniform grids to obtain the top and bottom of each stratum, and arrange them in order from smallest to largest depth, without duplicating the top and bottom of adjacent strata, so as to form a linear array; Expressing the linear array as two one-dimensional arrays yields a stacked grid model: Array , Represents the stratigraphic unit index, and sequentially stores the stratigraphic objects that need to be stored, wherein the stratigraphic objects include stratigraphic attribute and depth data pairs; Array , Represents the grid cell index, records the location and data offset of the first stratigraphic object in the cell to clarify the relationship between the stratigraphic unit and the object.

[0011] Furthermore, converting the plurality of stacked grid models into HDF5 files comprises: The StackGrid array, StackGrid, and StackGridBlock are used to represent three-dimensional uniform grid models, single-resolution three-dimensional models, and stacked grid models, respectively. The arrays D and I in the stacked grid model correspond to PillarGridData and PillarIndex. In HDF5, the H5File object is used to store complete three-dimensional uniform grid models. The subordinate levels are organized by Group. The CommonFG class is used for Group and DataSet operations, DataSpace is used to apply for the space required for data storage, Attribute is used to store descriptive information of three-dimensional models at each level of resolution, and CompType is used to specify the stored data type. The System class manages the model data HDF5 file by calling instance objects of the four classes: Operate\Write\Reader\Query.

[0012] Furthermore, it also includes using the UpdateTileData function to update the data stored in the HDF5 file.

[0013] Furthermore, it also includes reading and writing large-scale data to HDF5 files, including the following steps: Initialization phase: Define the HDF5Tile structure, which contains three integer fields: PositionPointer, LODChildBloc, and Editable, which represent the position pointer, sub-block pointer, and editable flag respectively, and are used to describe the basic information and status of each grid; File creation and property setting: Create a new HDF5 file and write the attributes 'MaxLevel' and 'MinLevel' at the file level. These two attributes describe the highest and lowest level of detail of the data based on the resolution level of the grid. Data organization and writing: For each grid in the LODStackedGrid array, the program iterates and performs the following steps: a. Create a group for each resolution level of the grid to organize the spatial data at that level, including the grid block spacing in three directions and the grid width in three directions; b. Write group attributes. The group attributes are the spatial information of the stacked grid model. The spatial information describes the spatial data layout at each resolution level. Create a data space H5::DataSpace and an attribute variable H5::Attribute to store the spatial information of the stacked grid model and store it in the HDF5 file. c. Define a single-resolution model information dataset to store the attribute information of all stacked mesh models at that resolution level. Initialize an array of HDF5Tile structures to record the attribute data of the stacked mesh models, and then write it to the newly created group. d. Create and write the TileInfomationArray dataset to store the compact array; e. Create a subgroup within the group at each resolution level to store stacked grid model data. For each stacked grid model, define and create datasets of grid index PillarIndex and grid data PillarData, as well as the corresponding data space, to store arrays I and D. Finally, write the data to the HDF5 file.

[0014] Furthermore, it also includes cleaning up all dynamically allocated memory resources.

[0015] Furthermore, it also includes the use of capture and processing mechanisms to handle exceptions that may occur in operations on files, groups, data sets, data spaces and data types.

[0016] Furthermore, it also includes closing and deleting HDF5 files and dynamically allocated resources.

[0017] Compared with the prior art, the beneficial effects of the technical solution of the present invention are: The present invention generates several three-dimensional models according to the resolution level of the three-dimensional uniform grid model, displays the three-dimensional models of the corresponding resolution level according to actual needs, and converts the three-dimensional models into sparse stacked grid models to save storage space. Finally, the data management technology based on HDF5 is used to realize data management of the stacked grid models. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic flow chart of a method for storing and managing multi-resolution block stacked grids based on HDF5 provided in an embodiment of the present invention; Figure 2 A schematic diagram of a hierarchical K-ary tree index structure provided by an embodiment of the present invention; Figure 3 A UML class diagram of a hierarchical K-ary tree index provided by an embodiment of the present invention; Figure 4 A schematic diagram of a compact array provided by an embodiment of the present invention; Figure 5 A schematic diagram of a stacked grid model and a data storage method provided by an embodiment of the present invention, wherein: Figure 5 (a) is a stacked grid model. Figure 5 (b) is the data storage method; Figure 6 A UML class diagram of a multi-resolution block stacked grid based on HDF5 is provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting the present application; In order to better illustrate this embodiment, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product size; It is understandable to those skilled in the art that some well-known structures and descriptions thereof may be omitted in the drawings.

[0020] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0021] Example This embodiment provides a storage and management method for multi-resolution block stacked grids based on HDF5, such as Figure 1 As shown, the following steps are included: Generating a plurality of three-dimensional models from the preset three-dimensional uniform grid model according to the resolution level, wherein the degree of detail expression of the three-dimensional models increases step by step with the resolution level; Based on the multiple three-dimensional models, a multi-level index structure is constructed, wherein the upper layer of the multi-level index structure adopts a multi-level resolution index to integrate and manage three-dimensional models of different resolution levels, and the lower layer of the multi-level index structure defines a plurality of K-ary trees, each K-ary tree recording a three-dimensional model of a resolution level; Based on the multi-level index structure, storing the multiple three-dimensional models; Converting each of the three-dimensional models into a stacked grid model to obtain a plurality of stacked grid models; The plurality of stacked grid models are converted into HDF5 files.

[0022] In a further embodiment, each K-ary tree records a three-dimensional model of a resolution level, including: The K-ary tree divides the three-dimensional uniform grid model into grids, each grid has the same time complexity and space complexity, the grid is expressed as an instance object of the StackedGridBlock class, and each node of the K-ary tree records the model information of the corresponding grid in the three-dimensional model; When the resolution level is n, the number of nodes in the K-ary tree is the number of grids into which the three-dimensional uniform grid model is divided. for: .

[0023] In a specific embodiment, a K-ary tree records a single-scale 3D model, and each 3D model information is recorded in a leaf node associated with the K-ary tree. The hierarchical structure is as follows: Figure 2 As shown, the corresponding class diagram is shown in Figure 3 , showing the data structure composition of three main levels: LODStackedGrid, StackedGrid, and StackedGridBlock.

[0024] exist Figure 2 In the 3D model, the resolution levels range from 0 to N. N+1 3D models are generated according to the resolution level, referred to as models 0-N. The level of detail in each model increases regularly. The multi-resolution model corresponds to the LODStackedGrid class, which records the maximum and minimum resolutions of the model. Single-resolution models (StackedGrid) are stored sequentially in its defined member variable, LodStackedGrid.

[0025] In a further embodiment, the K-ary tree uses a linear table to store leaf nodes. Since the subspace may be empty during the level-by-level division process, this embodiment uses a non-full tree to avoid the generation of blank spaces. The linear structure used at the beginning of creation needs to be combined with encoding to complete positioning and query. In order to save memory, a compact array is used to implement continuous array storage, such as Figure 4 As shown, the compact array member type is a BlockInformation structure, which calculates the index position dataIndex(i) of the grid with a given index i in the linear list, where i is the index value of the full tree: dataIndex(i)= BlockInformation[i] Where, BlockInformation[ ] represents the BlockInformation structure array; If the return value is -2, the subspace is empty; otherwise, the return value is the index position of the grid with the given index i in the linear table.

[0026] In a further embodiment, based on the multi-level index structure, before storing the multiple 3D models, the process also includes extracting data from physical storage into the multi-level index structure, allowing users to specify different levels and block indexes to support multi-level and multi-granular data management. This is particularly important for processing large-scale geospatial data or 3D modeling data, where different levels of detail or accuracy are typically organized in different levels. The specific extraction process includes: The raw data of 3D models at different resolution levels is stored in separate computer disk spaces. GetLevelbyFile is used to extract detailed information corresponding to each resolution level from the disk space. This detailed information includes the resolution level, grid size, grid block spacing in three directions, and grid block width in three directions. First, memory allocation and StackedGrid data structure initialization are managed to store the grid data. The data hierarchy is handled to allow loading and accessing different levels of detail as needed. LOD levels are set based on the file contents.

[0027] Initializing a StackedGrid object based on the detailed information to store three-dimensional models at different resolution levels, calculating and setting spatial parameters of the object, wherein the spatial parameters include grid block position, grid block spacing in three directions, and grid block width in three directions; for each file in a computer disk space associated with a resolution level, creating a StackedGridBlock object to store the grid model, and reading block data from the computer disk space, wherein the block data includes PillarIndex and PillarData; The state of the grid is updated based on the presence of data, and pointers and flags are set accordingly. Finally, the populated StackedGrid object is appended to the LODStackedGrid array of the LODSStackedGrid object.

[0028] The pseudo code of the extraction process is as follows: InitLODLayeredPillarGrid(filePath){ levels[]←GetLevelbyFile(filePath) / / Get level information Foreach level in levels{ stackedGrid←new StackedGrid() / / Apply for memory space to store single-resolution models stackedGrid.SetStackedGrid(level) / / Set basic information of single-resolution model files←ReadBinaryFilename(filePath+"\\"+level) / / Read the binary file name of the mesh model For each file in files{ stackedGridBlock←new StackedGridBlock() / / Apply for memory space to store grid blocks pillarIndex←ReadPillarIndex(file) pillarData←ReadPillarData(file) If pillarData is not null stackedGridBlock.SetStackedGridBlock(pillarIndex,pillarData) / / If there is data in pillarData, store the data in the grid block. Otherwise, setting the information of the grid block is invalid. Else stackedGridBlock.SetInvalid() } } } Until now, most 3D grid models have used uniform grids, explicitly storing the properties of each grid. However, geological models often have larger uniform grid blocks, which suggests that using a sparse grid structure is more space-efficient. Because geological models generally have layered characteristics, there are often many grids with the same lithology or other properties in the vertical direction. Therefore, this embodiment uses the concept of a stacked grid model to construct a sparse grid.

[0029] The original mesh is a 3D model at the resolution Uniform grid model generated by uniform division, grid number Inversely proportional to the resolution, the global space is :

[0030] in is the number of grid cells in dimension i, is the scale of the bounding box of the global space grid in dimension i , is the scale of a single grid cell in dimension i .

[0031] Data is represented and managed using a stacked grid, which consists of two static data structures.

[0032] Converting each of the three-dimensional models into a stacked mesh model, comprising: The three-dimensional model is evenly divided into a plurality of uniform grids according to the resolution level; Perform binary tree search on all the uniform grids to obtain the top and bottom of each stratum, and arrange them in order from small to large according to depth. The top and bottom of adjacent strata that touch each other are not recorded repeatedly, forming a linear array. The intermediate data structure at this time is as follows: Figure 5As shown, the intermediate data structure stores pointers on each cell, resulting in a large amount of storage space waste on blank cells and is not suitable for physical storage. In order to achieve a compact grid representation in the global space and easy storage, the data structure is further improved and expressed as two one-dimensional arrays: Array , Represents the stratigraphic unit index, and sequentially stores the stratigraphic objects that need to be stored, wherein the stratigraphic objects include stratigraphic attribute and depth data pairs; Array , Represents the grid cell index, records the location and data offset of the first stratigraphic object in the cell to clarify the relationship between the stratigraphic unit and the object.

[0033] Recording only the data offset is sufficient to clearly identify the location of the data object. However, subsequent spatial analysis and dynamic updates of the grid model will require repeated calculation of the sum of the offsets. Recording the object position here can avoid the time complexity caused by repeated calculations.

[0034] In a further embodiment, converting the plurality of stacked grid models into HDF5 files comprises: HDF5 files are containers for storing a variety of scientific data using two basic data objects, Group and Dataset. HDF5 technology consists of data models, libraries, and file formats for storing and managing data. It supports an unlimited number of data types and is designed for flexible and efficient I / O as well as large and complex data. In the HDF5 data model, the container aspect of the HDF5 information set is represented by an HDF5 file with a specified root. HDF5 files contain array variables, groups, and types, which are respectively called HDF5 datasets, HDF5 groups, and HDF5 datatype objects in the HDF5 data model. The HDF5 data model defines simple and extended linking mechanisms for creating associations between HDF5 information items. Finally, the HDF5 data model defines tools for annotating HDF5 information items using HDF5 Attributes. The above features of HDF5 provide a storage solution worth exploring for the hierarchical K-ary tree-stacked grid hybrid index management of three-dimensional geological models in this study. Based on the characteristics of HDF5 file storage and the hybrid spatial index of this study, we designed a storage solution for three-dimensional geological structure models, corresponding to the class diagram ( Figure 6 ) as shown.

[0035] In this embodiment, the StackGrid array, StackGrid, and StackGridBlock are used to represent the three-dimensional uniform grid model, the single-resolution three-dimensional model, and the stacked grid model, respectively. The arrays D and I in the stacked grid model correspond to PillarGridData and PillarIndex. In HDF5, the H5File object is used to store the complete three-dimensional uniform grid model. The subordinate levels are organized by Group. The CommonFG class is used for Group and DataSet operations, DataSpace is used to apply for the space required for data storage, Attribute is used to store descriptive information of the three-dimensional model at each level of resolution, and CompType is used to specify the stored data type. The System class manages the model data HDF5 file by calling instance objects of the four classes: Operate\Write\Reader\Query.

[0036] In a further embodiment, the method further includes using the UpdateTileData function to update the data stored in the HDF5 file.

[0037] In a specific embodiment, the UpdateTileData function is used to update the data of a specific tile within a hierarchical tile grid model stored in the HDF5 file format. The UpdateTileData function receives user input to determine the location of the tile and its internal grid to be updated and updates it according to the user-provided data. By receiving coordinates and new data from the user, the function allows for dynamic modification and updating of specific portions of a dataset. This flexibility is crucial for real-time data analysis and editing tasks. The function's internal data processing logic (such as layer counting and data index management) allows for precise control over which data is read and modified, ensuring accurate data updates and reducing the risk of data redundancy and errors. Through its exception handling mechanism, the function handles errors that may occur during the reading or writing process and ensures that files and memory resources are properly closed and released after the operation is completed, thereby increasing program robustness and stability. The UpdateTileData function is more than just a technical data update tool; it also embodies a comprehensive consideration of efficiency, flexibility, and reliability when processing complex data structures, making it a critical component of the three-dimensional data processing and analysis of this embodiment.

[0038] Furthermore, it also includes reading and writing large-scale data to HDF5 files, storing spatial data in the form of multi-resolution three-dimensional geological models in the HDF5 file format, and supporting efficient reading and writing of large-scale spatial data, including the following steps: Initialization phase: First, the program attempts to shield the default exception printing mechanism to prevent abnormal information appearing in file operations from interfering with normal program output.

[0039] Define the HDF5Tile structure, which contains three integer fields: PositionPointer, LODChildBloc, and Editable, which represent the position pointer, sub-block pointer, and editable flag respectively, and are used to describe the basic information and status of each grid; H5::CompType tileInformation(sizeof(HDF5TileInfo)); tileInformation.insertMember(MEMBER6, HOFFSET(HDF5TileInfo,PositionPointer), H5::PredType::NATIVE_INT); tileInformation.insertMember(MEMBER7, HOFFSET(HDF5TileInfo,LODChildBlock), H5::PredType::NATIVE_INT); tileInformation.insertMember(MEMBER8, HOFFSET(HDF5TileInfo,Editable), H5::PredType::NATIVE_INT); File creation and property setting: Create a new HDF5 file and write the attributes 'MaxLevel' and 'MinLevel' at the file level. These two attributes describe the highest and lowest level of detail of the data based on the resolution level of the grid. hsize_t dimlod[1] = { 2}; H5::DataSpace file_attrspace(1, dimlod); / / Apply for file attribute space to store resolution information H5::Attribute file_attr = file->createAttribute(FILE_ATTR_NAME.c_str(), H5::PredType::STD_I16BE, file_attrspace); short attr_level[2]; attr_level[0] = stackedGrid.GetMinlevel(); attr_level[1] = stackedGrid .GetMaxlevel(); file_attr.write(H5::PredType::NATIVE_INT16, attr_level); / / Write resolution information to the file Data organization and writing: For each grid in the LODStackedGrid array, the program iterates and performs the following steps: a. Create a group for each resolution level of the grid to organize the spatial data at that level, including the grid block spacing in three directions and the grid width in three directions; H5::Group gridBlock_group = file->createGroup((" / level_" +level); b. Write group attributes. These group attributes contain the spatial information of the stacked grid model. This spatial information describes the spatial data layout at each resolution level. Create a data space H5::DataSpace and an attribute variable H5::Attribute to store the spatial information of the stacked grid model and store it in an HDF5 file. Perform the following operations on the five file attributes: hsize_t dimsize[1] = { 3}; H5::DataSpace group_attrspace(1, dimsize); / / Apply for the data space required for the attribute H5::Attribute gridBlock_group_attr; / / Declare HDF5 file attribute variables gridBlock_group_attr=gridBlock_group.createAttribute("TileGridSize",H5::PredType::STD_I16BE, group_attrspace); / / Define HDF5 file attribute variables gridBlock_group_attr.write(H5::PredType::NATIVE_INT16,BlockGridSize); / / Write attributes to HDF5 file c. Define a single-resolution model information dataset to store the attribute information of all stacked mesh models at that resolution level. Initialize an array of HDF5Tile structures to record the attribute data of the stacked mesh models, and then write it to the newly created group. d. Create and write the TileInfoOffset dataset to store the compact array (BlockInformation structure); e. Create a subgroup within the group at each resolution level to store stacked grid model data. For each stacked grid model, define and create datasets of grid index PillarIndex and grid data PillarData, as well as the corresponding data space, to store arrays I and D. Finally, write the data to the HDF5 file.

[0040] H5::Group tile_group = gridBlock_group.createGroup("TileData"); / / Create datasets PillarIndex and PillarData for (int j = 0; j <num_datasets; j += 2) { pillarIndex_dataset[j] = tile_group.createDataSet(("Pillarindex_" +std::to_string(j / 2)).c_str(), H5::PredType::NATIVE_INT, dataspace); pillarData_dataset[j + 1] = tile_group.createDataSet(("Pillardata_" +std::to_string(j / 2)).c_str(), H5::PredType::NATIVE_FLOAT, dataspace_); } In a further embodiment, after completing data writing, the program is responsible for clearing all dynamically allocated memory resources, including grid block information arrays and dataset resources. The program employs a mechanism to catch and handle exceptions that may arise during file, group, dataset, data space, and data type operations, ensuring the correct release of resources and stable program execution.

[0041] In a further embodiment, the method further includes closing and deleting the HDF5 file and dynamically allocated resources. In a specific embodiment, the memory usage of the HDF5 file obtained using this embodiment is shown in Table 1. The data source is a binary file. This embodiment compares the storage capacity difference between the HDF5 file and the source file. The results show that the memory compression ratio of the HDF5 file is about 31%, which proves the advantage of the HDF5 file management method in storing large-scale hierarchical and block data.

[0042] Table 1

[0043] The generated HDF5 file is queried. Global data, data sub-blocks, and grid unit data are randomly obtained according to the data hierarchical storage strategy. Multiple queries are performed and the average value is taken. The results show that all three query methods are at the millisecond level and have high query efficiency, as shown in Table 2.

[0044] Table 2

[0045] The same or similar reference numerals correspond to the same or similar components; The terms used in the drawings to describe positional relationships are for illustrative purposes only and are not to be construed as limiting the present application. Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A method for storing and managing multi-resolution block stacked grids based on HDF5, characterized in that: The following steps are involved: Generating a plurality of three-dimensional models from the preset three-dimensional uniform grid model according to the resolution level, wherein the detail expression degree of the three-dimensional model increases step by step with the resolution level; Based on the multiple three-dimensional models, a multi-level index structure is constructed, wherein the upper layer of the multi-level index structure adopts a multi-level resolution index to integrate and manage the three-dimensional models of different resolution levels, and the lower layer of the multi-level index structure defines a plurality of K-ary trees, each of which records a three-dimensional model of a resolution level; Based on the multi-level index structure, storing the multiple three-dimensional models; Convert each of the three-dimensional models into a stacked grid model to obtain a plurality of stacked grid models; The plurality of stacked mesh models are converted into HDF5 files.

2. The storage and management method of multi-resolution block stacked grids based on HDF5 according to claim 1, characterized in that: Each K-ary tree records a three-dimensional model of a resolution level, including: The K-ary tree divides the 3D uniform grid model into grids, each grid has the same time complexity and space complexity, the grid is expressed as an instance object of the StackedGridBlock class, and each node of the K-ary tree records the model information of the corresponding grid in the three-dimensional model; When the resolution level is n, the number of nodes in the K-ary tree is the number of grids into which the three-dimensional uniform grid model is divided. for: The K-ary tree uses a linear list to store leaf nodes and a compact array to implement continuous array storage. The compact array member type is a BlockInformation structure. The index position dataIndex(i) of the grid with a given index i in the linear list is calculated, where i is the index value of the full tree: dataIndex(i)= BlockInformation[i] In the formula, BlockInformation[ ] represents the BlockInformation structure array; When the return value is -2, the subspace is empty; otherwise, the return value is the index position of the grid with the given index i in the linear table.

3. The storage and management method of multi-resolution block stacked grids based on HDF5 according to claim 2, characterized in that: Based on the multi-level index structure, before storing the multiple three-dimensional models, the method further includes extracting data from physical storage into the multi-level index structure, including: The original data of the three-dimensional model of different resolution levels are stored in different computer disk spaces, and getLevelbyFile is used to extract the detail information corresponding to the different resolution levels from the computer disk space, wherein the detail information includes the resolution level, the grid scale, the grid block spacing in three directions, and the grid block width in three directions; Initialize a StackedGrid object according to the detail information to store three-dimensional models of different resolution levels, calculate and set the spatial parameters of the object, the spatial parameters include grid block position, grid block spacing in three directions, and grid block width in three directions; for each file in the computer disk space associated with the resolution level, create a StackedGridBlock object to store the grid model, read block data from the computer disk space, the block data includes PillarIndex and PillarData; Updates the state of the grid based on the presence of data and sets pointers and flags accordingly, and finally appends the populated StackedGrid object to the LODStackedGrid array of the LODSStackedGrid object.

4. The storage and management method of multi-resolution block stacked grids based on HDF5 according to claim 3 is characterized in that: Converting each of the three-dimensional models into a stacked mesh model, comprising: The three-dimensional model is evenly divided into a plurality of uniform grids according to the resolution level; Perform binary tree search on all the uniform grids attribute by attribute to obtain the top and bottom of each stratum, and arrange them in order from small to large according to depth, and do not record the top and bottom of adjacent strata in contact repeatedly, so as to form a linear array; Expressing the linear array as two one-dimensional arrays yields a stacked grid model: Arrays , Indicates the stratigraphic unit index, and sequentially stores the stratigraphic objects that need to be stored, wherein the stratigraphic objects include stratigraphic attribute and depth data pairs; Arrays , Represents the grid unit index, records the location and data offset of the first stratigraphic object in the unit to clarify the relationship between the stratigraphic unit and the object.

5. The storage and management method of multi-resolution block stacked grids based on HDF5 according to claim 4, characterized in that: Convert the multiple stacked grid models into HDF5 files, including: The StackGrid array, StackGrid, and StackGridBlock are used to represent the three-dimensional uniform grid model, the single-resolution three-dimensional model, and the stacked grid model respectively. The arrays D and I in the stacked grid model correspond to PillarGridData and PillarIndex. In HDF5, the H5File object is used to store the complete three-dimensional uniform grid model. The subordinate levels are organized by Group, in which the CommonFG class is used for Group and DataSet operations, DataSpace is used to apply for the space required for data storage, Attribute is used to store the description information of the three-dimensional model at each level of resolution, and CompType is used to specify the stored data type. The System class manages the model data HDF5 file by calling instance objects of the four classes Operate\Write\Reader\Query.

6. The storage and management method of multi-resolution block stacked grids based on HDF5 according to claim 5, characterized in that: It also covers updating data stored in HDF5 files using the UpdateTileData function.

7. The storage and management method of multi-resolution block stacked grids based on HDF5 according to claim 6, characterized in that: It also includes reading and writing large-scale data to HDF5 files, including the following steps: Initialization phase: Define the structure HDF5Tile, which contains three integer fields: PositionPointer, LODChildBloc and Editable, which represent the position pointer, sub-block pointer and editable flag respectively, and are used to describe the basic information and status of each grid; File creation and property setting: Create a new HDF5 file and write the attributes 'MaxLevel' and 'MinLevel' at the file level, which describe the highest and lowest level of detail of the data based on the resolution level of the grid; Data organization and writing: For each grid in the LODStackedGrid array, the program iterates and performs the following steps: a. Create a group for each resolution level of the grid to organize the spatial data at that level, including the grid block spacing in three directions and the grid width in three directions; b. Write group attributes, which are the spatial information of the stacked grid model. The spatial information describes the spatial data layout at each resolution level. Create data space H5::DataSpace and attribute variable H5::Attribute to store the spatial information of the stacked grid model and store it in the HDF5 file: c. Define a single-resolution model information dataset to store the attribute information of all stacked grid models at this resolution level, initialize an HDF5Tile structure array to record the attribute data of the stacked grid models, and then write it to the group just created; d. Create and write the TileInfomationArray dataset to store the compact array; e. Create a subgroup within each resolution level group specifically for storing stacked grid model data. For each stacked grid model, define and create datasets of grid index PillarIndex and grid data PillarData and corresponding data spaces to store arrays I and D. Finally, write the data into an HDF5 file.

8. The method for storing and managing multi-resolution block stacked grids based on HDF5 according to claim 7, characterized in that: This also includes cleaning up any dynamically allocated memory resources.

9. The method for storing and managing multi-resolution block stacked grids based on HDF5 according to claim 8, characterized in that: It also includes the use of capture and processing mechanisms to handle exceptions that may occur in operations on files, groups, data sets, data spaces, and data types.

10. The method for storing and managing multi-resolution block stacked grids based on HDF5 according to claim 8, characterized in that: It also includes closing and deleting HDF5 files and dynamically allocated resources.

Images