A dynamic geological mass data storage and high-speed retrieval method

Abstract

The present invention discloses a method for storing and retrieving dynamic massive geological data. The method includes globally dividing the three-dimensional geological data to be processed into multiple basic blocks; constructing a global spatial index based on the multiple basic blocks, and constructing a local index including a local spatial index and a local attribute index for each basic block based on the data distribution of each basic block; when it is determined that the current basic block meets the subdivision conditions, recursively subdividing the current basic block into multiple sub-blocks, and updating the local index of the current basic block based on the data distribution of the multiple sub-blocks; distributing the multiple basic blocks, the global spatial index, the local index of each basic block, and the multiple sub-blocks; and, in response to a user search request, retrieving the user's required data through the global spatial index, each local spatial index, and each local attribute index. The present invention improves the storage utilization rate of massive three-dimensional geological data, enhances data query efficiency, and can achieve efficient retrieval effects for global rough search and local detailed search.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method for storing and retrieving dynamic geological mass data. Background Art

[0002] In the fields of geological exploration, oil and gas development, earthwork engineering, etc., existing technologies are mostly based on HDFS (Hadoop Distributed File System), NoSQL (Not Only SQL, non-relational database) or object storage to simply block data for data storage and retrieval. In the scenario of massive data storage and retrieval, the existing technologies have low storage space utilization, slow query response, and inflexible data block management. Summary of the Invention

[0003] The present invention provides a dynamic geological massive data storage and high-speed retrieval method, which improves the storage utilization rate of massive three-dimensional geological data and improves the data query efficiency, and can achieve efficient retrieval effects of global rough search and local detailed search.

[0004] According to one aspect of the present invention, a method for dynamic geological mass data storage and high-speed retrieval is provided, the method comprising:

[0005] Globally divide the 3D geological data to be processed into multiple basic blocks;

[0006] Building a global spatial index based on the multiple basic blocks, and building a local index for each basic block based on data distribution of each basic block, wherein the local index includes a local spatial index and a local attribute index;

[0007] When it is determined that the current basic block meets the subdivision condition, recursively subdivide the current basic block into a plurality of sub-blocks, and update the local index of the current basic block based on data distribution of the plurality of sub-blocks;

[0008] Distributedly storing the multiple basic blocks, the global spatial index, the local index of each basic block, and the multiple sub-blocks;

[0009] In response to a user search request, user demand data is retrieved through the global spatial index, each local spatial index and each local attribute index.

[0010] According to another aspect of the present invention, a dynamic geological mass data storage and high-speed retrieval device is provided, the device comprising:

[0011] The data global partitioning module is used to globally partition the 3D geological data to be processed into multiple basic blocks;

[0012] A local index building module is used to build a global spatial index based on the multiple basic blocks, and to build a local index for each basic block based on the data distribution of each basic block, wherein the local index includes a local spatial index and a local attribute index;

[0013] a data subdivision module, configured to, if it is determined that the current basic block meets the subdivision condition, recursively subdivide the current basic block into a plurality of sub-blocks, and update a local index of the current basic block based on data distribution of the plurality of sub-blocks;

[0014] A data storage module, configured to perform distributed storage on the plurality of basic blocks, the global spatial index, the local index of each basic block, and the plurality of sub-blocks;

[0015] The data retrieval module is used to respond to a user's retrieval request and retrieve user demand data through the global spatial index, each local spatial index and each local attribute index.

[0016] According to another aspect of the present invention, an electronic device is provided, comprising:

[0017] at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the dynamic geological massive data storage and high-speed retrieval method described in any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the dynamic geological massive data storage and high-speed retrieval method described in any embodiment of the present invention when executed.

[0021] The technical solution of the embodiment of the present invention is to form multiple basic blocks by globally dividing the three-dimensional geological data to be processed; construct a global spatial index based on the multiple basic blocks, and construct a local index for each basic block based on the data distribution of each basic block, wherein the local index includes a local spatial index and a local attribute index; when it is determined that the current basic block meets the subdivision conditions, the current basic block is recursively subdivided into multiple sub-blocks, and the local index of the current basic block is updated based on the data distribution of the multiple sub-blocks; the multiple basic blocks, the global spatial index, the local index of each basic block, and the multiple sub-blocks are distributedly stored; and in response to a user search request, the user required data is retrieved through the global spatial index, the local spatial indexes, and the local attribute indexes. The technical means of hybrid indexing and dynamic block division are adopted to solve the problems of low storage space utilization, slow query response, and inflexible data block management of massive three-dimensional geological data in the prior art, thereby improving the storage utilization of massive three-dimensional geological data and the data query efficiency, and achieving efficient retrieval effects for global rough search and local detailed search.

[0022] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 A flowchart of a method for storing and retrieving dynamic geological mass data provided in the first embodiment of the present invention;

[0025] Figure 2 A schematic diagram of a block refinement process provided in this embodiment;

[0026] Figure 3 A schematic diagram of the overall process of a dynamic geological mass data storage and high-speed retrieval method provided in this embodiment;

[0027] Figure 4 A schematic structural diagram of a dynamic geological mass data storage and high-speed retrieval device provided in the second embodiment of the present invention;

[0028] Figure 5 The present invention is a schematic diagram of the structure of an electronic device for implementing the dynamic geological mass data storage and high-speed retrieval method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] Example 1

[0032] Figure 1 This is a flow chart of a method for dynamic geological mass data storage and high-speed retrieval provided in the first embodiment of the present invention. This embodiment is applicable to the storage and retrieval of massive three-dimensional geological data. The method can be executed by a dynamic geological mass data storage and high-speed retrieval device. The dynamic geological mass data storage and high-speed retrieval device can be implemented in the form of hardware and / or software. The dynamic geological mass data storage and high-speed retrieval device can be configured in a server. Figure 1 As shown, the method includes:

[0033] S110 , globally dividing the three-dimensional geological data to be processed into multiple basic blocks.

[0034] The three-dimensional geological data to be processed may be geological data in a three-dimensional space that needs to be stored in a standardized manner. The basic blocks may be data blocks obtained by dividing the three-dimensional geological data to be processed in the three-dimensional space.

[0035] In this embodiment, three-dimensional geological data with a large amount of data can be reasonably divided in space to facilitate storage and retrieval.

[0036] In an optional embodiment, globally dividing the three-dimensional geological data to be processed into multiple basic blocks may include: globally dividing the three-dimensional geological data to be processed according to an initial division strategy to form multiple basic blocks, and generating metadata for each basic block; wherein the metadata includes attribute statistics and access statistics; each basic block corresponds to a node of the first tree data structure.

[0037] The initial partitioning strategy may refer to a strategy for partitioning the three-dimensional geological data to be processed according to a grid of commonly used sizes. The first tree-like data structure may be an octree or a quadtree. For example, if the amount of three-dimensional geological data to be processed is large, the initial partitioning strategy may be an octree-based partitioning strategy; if the amount of three-dimensional geological data to be processed is small, the initial partitioning strategy may be a quadtree-based partitioning strategy. Attribute statistics may refer to statistical results such as the attribute type and quantity of each data within the basic block. Attributes may include lithology, porosity, and permeability. Access statistics may refer to statistics on the frequency of access to data within the basic block. When the three-dimensional geological data to be processed is initially stored, the access statistics may be an initial value set based on human experience. The access statistics may be subsequently updated in real time or periodically.

[0038] In this embodiment, the geological data to be processed may be globally recursively partitioned based on an octree or a quadtree to form a plurality of basic blocks, and metadata of the basic blocks may be generated. The metadata may also include data such as spatial boundaries.

[0039] S120 , constructing a global spatial index based on multiple basic blocks, and constructing a local index for each basic block based on data distribution of each basic block, where the local index includes a local spatial index and a local attribute index.

[0040] In this embodiment, the global spatial index can be an octree global index or a quadtree global index, which can efficiently eliminate data irrelevant to the query area, provide a preliminary coarse-grained organization of the data space, and quickly locate the approximate range of the user's area of ​​interest, laying the foundation for subsequent local fine-grained retrieval. The global spatial index also needs to be constructed based on the spatial boundaries of each basic block. The local spatial index can be constructed based on the spatial location of the data within the basic block. The local attribute index can be constructed based on the attributes of the data within the basic block.

[0041] In an optional embodiment, a local index is constructed for each basic block based on the data distribution of each basic block, which may include: obtaining the data distribution and data attributes of the current basic block; determining the target storage method of the current basic block based on the data distribution of the current basic block, using the target storage method to store the data of the current basic block and constructing a local spatial index; based on the data attributes, constructing a local attribute index for the current basic block through a second tree data structure.

[0042] Among them, determining the target storage mode of the current basic block according to the data distribution of the current basic block, using the target storage mode to store the data of the current basic block and constructing a local spatial index can include: when the data distribution is sparse, determining the target storage mode to be a first data structure, using the first data structure to recursively subdivide the current basic block to store non-empty voxels, and constructing a local spatial index for the current basic block; when the data distribution is uniform, determining the target storage mode to be a second data structure, using the second data structure to map the spatial position of the current basic block data and the local spatial index.

[0043] In this embodiment, an appropriate refined storage and indexing structure can be selected based on the data distribution within each basic block. Specifically, a current basic block can be determined from each basic block in turn. If the data distribution within the current basic block is sparse, a first data structure, such as a sparse voxel octree (SVO), can be used to recursively subdivide the basic block's space, storing only non-empty voxels with data, significantly reducing memory usage. If the data distribution within the current basic block is relatively uniform, a second data structure (such as a hash grid) can be used to directly map spatial locations to data indexes, enabling fast random access to any location. Furthermore, a geological attribute index (i.e., a local attribute index) can be established within each basic block. For example, a KD tree (K-Dimension tree) or R-tree can be used to organize attribute data such as lithology, porosity, and permeability. For continuous numerical multidimensional attribute data, KD trees can efficiently support nearest neighbor and range queries; while for spatial objects or discrete categorical attributes, R-trees can quickly perform range searches.

[0044] In this embodiment, a block-level hybrid index structure is constructed by combining the spatial positioning capabilities of SVOs or hash grids with the attribute retrieval capabilities of KD trees or R trees. Specifically, after the global octree provides coarse-grained positioning, the SVOs or hash grids within the block further pinpoint the precise spatial location. Finally, the KD tree or R tree performs precise attribute dimensionality screening, achieving efficient multi-dimensional queries based on spatial location and attribute features.

[0045] S130 : When it is determined that the current basic block meets the subdivision condition, recursively subdivide the current basic block into multiple sub-blocks, and update the local index of the current basic block based on data distribution of the multiple sub-blocks.

[0046] In an optional implementation, determining whether the current basic block meets the subdivision condition may include: calculating the current attribute entropy of the current basic block based on the current attribute statistics of the current basic block; calculating the current access frequency of the current basic block based on the current access statistics of the current basic block; and determining that the current basic block meets the subdivision condition when the current attribute entropy of the current basic block exceeds a preset attribute entropy threshold and the current access frequency of the current basic block exceeds a preset access frequency threshold.

[0047] In this embodiment, the data access and data attribute of each basic block can be monitored regularly or in real time, and a dynamic block partitioning strategy can be formulated based on the access frequency and attribute heterogeneity. Specifically, the access frequency can be defined for each basic block. and attribute entropy , which are used to measure the frequency of access to the block and the heterogeneity of the attributes of its internal data. Among them, the attribute entropy can be calculated as calculate, It can represent the occurrence ratio of the jth attribute in block i and set the access frequency threshold and attribute entropy threshold ,like and (i.e. the block is both an access hotspot and has complex and diverse attributes), it can be determined that the block meets the subdivision conditions, then block i can be subdivided into 2 n sub-blocks (where n = 2 or 3), and corresponding local spatial indexes (such as reconstructing the SVO or hash grid within the sub-block) and local attribute indexes (KD tree or R tree) can be established for the subdivided sub-blocks.

[0048] Optionally, after recursively subdividing the current basic block into multiple sub-blocks, and before updating the local index of the current basic block based on the data distribution of the multiple sub-blocks, the method may further include: assigning metadata of the current basic block as initial values ​​to the multiple sub-blocks.

[0049] In this embodiment, during the subdivision process, child blocks can be initially assigned values ​​according to the attributes of the parent block, so that historical access statistics can continue to guide new block adjustments. For areas with low access frequency or single attribute distribution, the larger block granularity is maintained unchanged, thereby reducing unnecessary index maintenance and storage overhead. The above dynamic re-blocking process can be performed periodically or in real time as needed, so that the block division granularity and index structure can be continuously optimized based on the latest data access and data distribution: hot spots are gradually refined and the index is updated to improve query efficiency, while non-hot spots are kept coarse-grained to save resources, achieving a balance between overall performance and cost.

[0050] In order to enable those skilled in the art to better understand the block refinement process of this embodiment, Figure 2This is a schematic diagram of the block refinement process provided by this embodiment. The data for block i (access logs and attribute data) is obtained, the access frequency and attribute entropy of block i are calculated, and a determination is made as to whether the access frequency exceeds the access frequency threshold or whether the attribute entropy exceeds the attribute entropy threshold. If either condition is not met, block i can be left in its current state. If both conditions are met, dynamic block refinement is triggered: a subdivision method is selected, the subdivision operation is executed to generate new sub-blocks, metadata is generated for the new sub-blocks (updating spatial boundaries, access statistics, and attribute statistics), and the local index is updated.

[0051] S140 , distributively storing multiple basic blocks, a global spatial index, a local index of each basic block, and multiple sub-blocks.

[0052] In this embodiment, distributed storage can be used to distribute and store each block of data and its index across multiple nodes through a distributed file system or object storage. A layered strategy combining memory cache, SSDs (Solid State Drives), and HDDs (Hard Disk Drives) can be used to improve data access efficiency. Furthermore, compression coding techniques (such as structured point cloud (SPC) or adaptive block storage) can be used to encode and compress the block data, reducing storage costs and accelerating data transmission across the network.

[0053] S150 , in response to a user search request, searching for user demand data through a global spatial index, local spatial indexes, and local attribute indexes.

[0054] In this embodiment, when a user's search request is received, the global octree index can be used to quickly determine the set of nodes containing the relevant data blocks. Each node then executes the query computation task in parallel. Leveraging GPU (Graphics Processing Unit) parallel computing technology, each node performs accelerated processing and local rendering of the data within its responsible block, fully leveraging the GPU's computing power to improve query processing throughput. This embodiment can also incorporate a pre-computation cache mechanism to cache frequently accessed query results or intermediate data, pre-load corresponding data blocks, or adjust indexes to reduce query response latency. To handle large-scale concurrent queries, an intelligent scheduling module can be implemented to monitor operational metrics such as CPU (Central Processing Unit) / GPU utilization, storage I / O load, and network bandwidth of each storage node in real time, and dynamically adjust the allocation of query tasks across nodes. When certain nodes are overloaded, the scheduling module can trigger the migration of hot data blocks within the cluster or temporarily add new compute nodes, thereby diverting some query requests. This achieves load balancing, ensuring stable system operation and good scalability. After all relevant nodes complete their respective subtasks, the system integrates the local results returned by multiple nodes and sends the final query results to the user terminal or visualization application, realizing real-time query and interactive analysis of massive geological data.

[0055] In order to enable those skilled in the art to better understand the dynamic geological massive data storage and high-speed retrieval method of this embodiment. Figure 3 This is a schematic diagram of the overall process of a dynamic geological mass data storage and high-speed retrieval method provided in this embodiment. The three-dimensional geological data to be processed is imported and preprocessed; initial spatial partitioning is performed to form basic blocks, and metadata is generated for the basic blocks; a global spatial index is constructed; local data is stored for the basic blocks. If the data is sparse, SVO storage is used; if the data is uniform, hash grid storage is used, and local attribute indexes are constructed based on KD trees or R trees. The basic blocks are monitored in real time to make dynamic block partitioning decisions. Based on the access frequency and attribute entropy of the basic blocks, whether the basic blocks meet the subdivision conditions is determined. If not, the current block state is maintained; if it is, block partitioning is performed, thereby updating the local index and metadata; distributed storage management is performed on the block data, and parallel query scheduling is performed during user retrieval, so that the query results are integrated and returned, and visualized.

[0056] The technical solution of the present invention forms a plurality of basic blocks by globally dividing the three-dimensional geological data to be processed; constructs a global spatial index based on the plurality of basic blocks, and constructs a local index for each basic block based on the data distribution of each basic block, wherein the local index includes a local spatial index and a local attribute index; recursively subdivides the current basic block into a plurality of sub-blocks when it is determined that the current basic block meets the subdivision condition, and updates the local index of the current basic block based on the data distribution of the plurality of sub-blocks; stores the plurality of basic blocks, the global spatial index, the local index of each basic block, and the plurality of sub-blocks in a distributed manner; and retrieves user-required data in response to a user search request through the global spatial index, the local spatial indexes, and the local attribute indexes. The technical solution adopts hybrid indexing and dynamic block division to solve the problems of low storage space utilization, slow query response, and inflexible data block management of massive three-dimensional geological data in the prior art, thereby improving the storage utilization of massive three-dimensional geological data and the data query efficiency, and achieving efficient search effects for global rough search and local fine search.

[0057] Example 2

[0058] Figure 4 This is a schematic diagram of the structure of a dynamic geological mass data storage and high-speed retrieval device provided in the second embodiment of the present invention. Figure 4 As shown, the device includes: a global data partitioning module 410, a local index building module 420, a data subdivision module 430, a data storage module 440 and a data retrieval module 450. Among them:

[0059] The data global partitioning module 410 is used to globally partition the 3D geological data to be processed into multiple basic blocks;

[0060] A local index building module 420 is configured to build a global spatial index based on the plurality of basic blocks, and to build a local index for each basic block based on data distribution of each basic block, wherein the local index includes a local spatial index and a local attribute index;

[0061] a data subdivision module 430 configured to, when determining that the current basic block satisfies the subdivision condition, recursively subdivide the current basic block into a plurality of sub-blocks, and update a local index of the current basic block based on data distribution of the plurality of sub-blocks;

[0062] A data storage module 440 is configured to perform distributed storage of the plurality of basic blocks, the global spatial index, the local index of each basic block, and the plurality of sub-blocks;

[0063] The data retrieval module 450 is configured to respond to a user retrieval request and retrieve user demand data through the global spatial index, each local spatial index, and each local attribute index.

[0064] The technical solution of the present invention forms a plurality of basic blocks by globally dividing the three-dimensional geological data to be processed; constructs a global spatial index based on the plurality of basic blocks, and constructs a local index for each basic block based on the data distribution of each basic block, wherein the local index includes a local spatial index and a local attribute index; recursively subdivides the current basic block into a plurality of sub-blocks when it is determined that the current basic block meets the subdivision condition, and updates the local index of the current basic block based on the data distribution of the plurality of sub-blocks; stores the plurality of basic blocks, the global spatial index, the local index of each basic block, and the plurality of sub-blocks in a distributed manner; and retrieves user-required data in response to a user search request through the global spatial index, the local spatial indexes, and the local attribute indexes. The technical solution adopts hybrid indexing and dynamic block division to solve the problems of low storage space utilization, slow query response, and inflexible data block management of massive three-dimensional geological data in the prior art, thereby improving the storage utilization of massive three-dimensional geological data and the data query efficiency, and achieving efficient search effects for global rough search and local fine search.

[0065] Optionally, the global data partitioning module 410 may be used to:

[0066] The three-dimensional geological data to be processed is globally divided according to an initial division strategy to form a plurality of basic blocks, and metadata is generated for each basic block; wherein the metadata includes attribute statistics and access statistics; each basic block corresponds to a node of the first tree data structure.

[0067] Optionally, the local index building module 420 may include:

[0068] A data status acquisition unit is used to obtain the data distribution and data attribute status of the current basic block;

[0069] a local spatial index building unit, configured to determine a target storage mode for the current basic block according to data distribution of the current basic block, store the data of the current basic block using the target storage mode, and build the local spatial index;

[0070] The local attribute index construction unit is configured to construct a local attribute index for the current basic block through a second tree data structure based on the data attribute situation.

[0071] Optional, local spatial index building unit, specifically used for:

[0072] When the data distribution condition is sparse, determining the target storage mode to be a first data structure, recursively subdividing the current basic block using the first data structure to store non-empty voxels, and constructing a local spatial index for the current basic block;

[0073] When the data distribution condition is uniform, the target storage mode is determined to be a second data structure, and the second data structure is used to map the spatial position of the current basic block data to a local spatial index.

[0074] Optionally, the data segmentation module 430 may be used to:

[0075] Calculating a current attribute entropy of the current basic block according to current attribute statistics of the current basic block;

[0076] Calculating a current access frequency of the current basic block according to current access statistics of the current basic block;

[0077] When the current attribute entropy of the current basic block exceeds a preset attribute entropy threshold and the current access frequency of the current basic block exceeds a preset access frequency threshold, it is determined that the current basic block meets the subdivision condition.

[0078] Optionally, the dynamic geological massive data storage and high-speed retrieval device also includes a data assignment module for assigning the metadata of the current basic block as an initial value to the multiple sub-blocks after recursively subdividing the current basic block into multiple sub-blocks and before updating the local index of the current basic block based on the data distribution of the multiple sub-blocks.

[0079] Optionally, the dynamic geological massive data storage and high-speed retrieval device also includes a data feedback module for responding to a user's retrieval request, retrieving the user's required data through the global spatial index, each local spatial index and each local attribute index, and then feeding back the user's required data in a visual form to the user.

[0080] The dynamic geological massive data storage and high-speed retrieval device provided in the embodiment of the present invention can execute the dynamic geological massive data storage and high-speed retrieval method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0081] Example 3

[0082] Figure 5 A schematic diagram of an electronic device 500 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers or various forms of mobile devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0083] like Figure 5As shown, electronic device 500 includes at least one processor 501 and memory, such as read-only memory (ROM) 502 and random access memory (RAM) 503, communicatively connected to the at least one processor 501. The memory stores computer programs executable by the at least one processor. Processor 501 can perform various appropriate actions and processes based on the computer programs stored in ROM 502 or loaded from storage unit 508 into RAM 503. RAM 503 can also store various programs and data required for the operation of electronic device 500. Processor 501, ROM 502, and RAM 503 are interconnected via bus 504. An input / output (I / O) interface 505 is also connected to bus 504.

[0084] Multiple components in the electronic device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a magnetic disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the electronic device 500 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0085] Processor 501 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Examples of processor 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors for running machine learning model algorithms, digital signal processors (DSPs), and any other suitable processor, controller, microcontroller, etc. Processor 501 executes the various methods and processes described above, such as the dynamic geological mass data storage and high-speed retrieval method.

[0086] In some embodiments, the dynamic geological mass data storage and high-speed retrieval method can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by processor 501, one or more steps of the dynamic geological mass data storage and high-speed retrieval method described above can be performed. Alternatively, in other embodiments, processor 501 can be configured to execute the dynamic geological mass data storage and high-speed retrieval method in any other suitable manner (e.g., via firmware).

[0087] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system comprising at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0088] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0089] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or apparatus. A computer-readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0090] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device that has: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0091] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0092] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0093] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0094] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for storing and retrieving dynamic geological mass data, characterized in that: include: The three-dimensional geological data to be processed is globally divided according to the initial division strategy to form a plurality of basic blocks, and metadata is generated for each basic block; wherein the metadata includes attribute statistics, access statistics, and spatial boundary data; each basic block corresponds to a node of the first tree data structure; the attribute statistics refer to the attribute type and attribute quantity statistics of each data in the basic block, and the attributes include lithology, porosity, and permeability; Building a global spatial index based on the plurality of basic blocks; building the global spatial index based on spatial boundary data of each basic block; Obtaining data distribution and data attributes of a current basic block; if the data distribution is sparse, determining a target storage mode as a first data structure, recursively subdividing the current basic block using the first data structure to store non-empty voxels, and constructing a local spatial index for the current basic block; if the data distribution is uniform, determining a target storage mode as a second data structure, mapping the spatial position of the current basic block data to the local spatial index using the second data structure; constructing a local attribute index for the current basic block using a second tree-like data structure based on the data attributes; the local spatial index is constructed based on the spatial position of the data in the basic block; and the local attribute index is constructed based on the attributes of the data in the basic block; Calculating a current attribute entropy of the current basic block based on current attribute statistics of the current basic block; calculating a current access frequency of the current basic block based on current access statistics of the current basic block; determining that the current basic block meets a subdivision condition when the current attribute entropy of the current basic block exceeds a preset attribute entropy threshold and the current access frequency of the current basic block exceeds a preset access frequency threshold, recursively subdividing the current basic block into a plurality of sub-blocks, assigning metadata of the current basic block as initial values ​​to the plurality of sub-blocks, so that the historical access statistics of the current basic block continue to guide new block adjustments, and updating a local index of the current basic block based on data distribution of the plurality of sub-blocks; Distributedly storing the multiple basic blocks, the global spatial index, the local index of each basic block, and the multiple sub-blocks; In response to a user search request, user demand data is retrieved through the global spatial index, each local spatial index and each local attribute index.

2. The method according to claim 1, characterized in that After responding to the user's search request and retrieving the user's required data through the global spatial index, each local spatial index, and each local attribute index, the method further includes: The user demand data is fed back to the user in a visual form.

Images