A point cloud data storage and retrieval method and system based on spatio-temporal hybrid coding

Abstract

This invention provides a point cloud data storage and retrieval method and system based on spatiotemporal hybrid coding, belonging to the field of geographic big data storage and spatial information technology. The method involves dividing the original point cloud data into time frame sequences based on two time scales: the macroscopic frame period and the peripheral pulse interval. A time offset is assigned to each point in the time frame sequence to obtain single-frame data. The single-frame data is then dynamically partitioned and spatiotemporally hybrid coded to obtain a point cloud data storage table. Finally, a spatiotemporal secondary index table is used to index the point cloud data storage table to obtain point cloud data retrieval results, thus completing the storage and retrieval of point cloud data. This invention solves the problem of low efficiency in point cloud data storage and retrieval.

Description

Technical Field

[0001] This specification relates to the field of geographic big data storage and spatial information technology, and in particular to a point cloud data storage and retrieval method and system based on spatiotemporal hybrid coding. Background Technology

[0002] With the rapid development of LiDAR and 3D scanning technologies, spatiotemporal point cloud data has become a core data carrier in fields such as autonomous driving, digital twins, and smart cities. A point cloud is a collection of discrete spatial points acquired through devices such as LiDAR and 3D scanners. Each data point contains attribute information such as 3D coordinates, timestamps, and reflection intensity, accurately representing the geometric features of an object's surface. With the development of mobile measurement platforms and sensor technologies, modern point cloud data has evolved from static scanning to dynamic continuous acquisition, forming a four-dimensional data stream with spatiotemporal correlation characteristics, playing a crucial role in fields such as autonomous driving, digital twins, and smart cities.

[0003] Faced with massive, highly dynamic point cloud data, existing technologies face severe challenges: traditional file storage lacks efficient indexing mechanisms, resulting in low query efficiency; relational databases are limited by fixed structures and struggle to adapt to dynamic data characteristics; distributed storage solutions suffer from problems such as spatiotemporal encoding fragmentation and uneven data distribution. Especially at data scales of hundreds of millions, the storage and retrieval performance of existing solutions significantly degrades, failing to meet the demands of real-time applications. Summary of the Invention

[0004] To address the aforementioned shortcomings in existing technologies, this invention provides a point cloud data storage and retrieval method and system based on spatiotemporal hybrid coding, which solves the problem of low efficiency in point cloud data storage and retrieval.

[0005] To achieve the aforementioned objectives, the present invention employs the following technical solution: a point cloud data storage and retrieval method based on spatiotemporal hybrid coding, comprising:

[0006] S1: Based on two time scales, the original point cloud data is divided to obtain a time frame sequence;

[0007] S2: Assign a time offset to each point in the time frame sequence to obtain single frame data;

[0008] S3: Perform dynamic partitioning and spatiotemporal hybrid encoding on the single frame data to obtain a point cloud data storage table;

[0009] S4: Using the spatiotemporal secondary index table, the point cloud data storage table is indexed to obtain the point cloud data retrieval results, thus completing the storage and retrieval of point cloud data.

[0010] The beneficial effects of this invention are as follows: This invention provides a point cloud data storage and retrieval method based on spatiotemporal hybrid coding. By introducing dynamic partitioning technology, it overcomes the data skew problem caused by fixed partitioning and achieves balanced storage load. Through spatiotemporal hybrid coding, high-dimensional spatiotemporal information is reduced to one-dimensional coding, achieving unified and efficient indexing of space and time. Finally, relying on the spatiotemporal secondary index table, the system can simultaneously support complex time range queries, spatial range queries, and efficient spatiotemporal joint queries, providing solid technical support for key applications such as real-time perception in intelligent driving and dynamic updates of digital twins.

[0011] Furthermore, the expression for the time base of each frame in the time frame sequence is:

[0012] ;

[0013] ;

[0014] in, Represents the time base of the k-th frame. Indicates the initial time base. Indicates the frame number. Indicates the time interval of a single frame. Indicates the scanning frequency.

[0015] Furthermore, the expression for the timestamp of each point in the single frame data is:

[0016] ;

[0017] in, This represents the precise time of the i-th point. This represents the start time reference for the k-th frame. Indicates the index of a point within the frame. This indicates the pulse interval time.

[0018] The method of discretizing continuous data streams into time frames was clarified, which established a unified time benchmark for subsequent data organization and indexing, and is the foundation for achieving efficient time-based queries.

[0019] Further, S3 includes:

[0020] An initial uniform network is constructed based on single-frame data and a fixed interval;

[0021] The normalized density of the initial uniform network is calculated to obtain the network with annotated abnormal density regions.

[0022] Through iterative optimization, the network in the annotated abnormal density region is dynamically partitioned to obtain the optimized network structure;

[0023] Spatial entropy calculation is performed on the root node of the octree in the optimized network structure to obtain point cloud distribution feature evaluation information.

[0024] The node splitting information is obtained by judging the point cloud distribution feature evaluation information using the dual splitting condition;

[0025] The spatial partitioning is recursively executed until the preset conditions are met. The leaf nodes are then subjected to spatiotemporal hybrid encoding and compression storage to obtain the point cloud data storage table.

[0026] By storing timestamps using a "base time + offset" method, data redundancy is greatly reduced (there is no need to store a complete absolute timestamp for each point), saving storage space, while still being able to restore the precise time information of each point without loss.

[0027] Furthermore, the expression for the normalized density is:

[0028] ;

[0029] in, Indicates the current grid density. Indicates the number of point clouds. Represents the grid area. Indicates the maximum value. Indicates density.

[0030] Through density detection and entropy assessment, the algorithm can intelligently identify dense and sparse regions of point clouds and implement a "coarse-to-fine" partitioning strategy to ensure that the amount of data in each storage unit is relatively balanced, thereby avoiding hotspot problems from the root and laying the foundation for efficient parallel retrieval.

[0031] Furthermore, the expression for the spatial entropy is:

[0032] ;

[0033] in, Represents the spatial entropy of the current node. This represents the number of point clouds in the i-th subspace. This represents the maximum number of point clouds across all subspaces. This represents the total number of nodes.

[0034] A quantified density index is defined, enabling the system to automatically and accurately identify areas exceeding the normal density range (dense areas that need further subdivision and sparse areas that need to be merged). This is the key judgment criterion for achieving automated dynamic partitioning.

[0035] Furthermore, the expression for the double splitting condition is:

[0036] ;

[0037] in, This represents the first weighting coefficient. Represents the spatial entropy of the current node. Represents the maximum entropy value. This represents the second weighting coefficient. This indicates the number of point clouds at the current node. This represents the minimum splitting capacity threshold. This represents the maximum splitting capacity threshold. This indicates the threshold for comprehensive judgment.

[0038] The concept of information entropy is introduced to quantify the spatial disorder of point cloud distribution. A higher entropy value indicates a more uniform and dispersed distribution of the point cloud within that node, and a greater benefit to query efficiency from splitting that node. Conversely, a lower entropy value indicates that the point cloud is clustered in one area, eliminating the need for splitting. This makes partitioning decisions no longer simply based on data volume, but rather on the "value" of data distribution, making it more intelligent.

[0039] Furthermore, the expression for the number of pre-partitions in the point cloud data storage table is:

[0040] ;

[0041] in, Indicates the number of pre-partitions. This represents the JVM heap memory allocated to a single RegionServer process. This indicates the data written to the temporary cache.

[0042] This invention provides a point cloud data storage and retrieval system based on spatiotemporal hybrid coding, comprising:

[0043] The data storage module is used to divide the original point cloud data into time frame sequences based on two time scales: macro frame period and peripheral pulse interval; to assign a time offset to each point in the time frame sequence to obtain single frame data; and to perform dynamic partitioning and spatiotemporal hybrid encoding on the single frame data to obtain a point cloud data storage table.

[0044] The timer module is used to time storage and retrieval times;

[0045] The data visualization module is used to visualize point cloud data and tabular data;

[0046] The data retrieval module is used to index the point cloud data storage table using a spatiotemporal secondary index table, obtain point cloud data retrieval results, and complete the storage and retrieval of point cloud data.

[0047] This condition takes into account both distribution complexity (entropy) and data capacity, avoiding the problems of "over-split in sparse but uniform regions" or "not splitting in dense but concentrated regions" caused by splitting based solely on data size. Attached Figure Description

[0048] This specification will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein:

[0049] Figure 1 This is a schematic diagram of a point cloud data storage and retrieval system based on spatiotemporal hybrid coding, as shown in some embodiments of this specification.

[0050] Figure 2 This is an exemplary flowchart of a point cloud data storage and retrieval method based on spatiotemporal hybrid coding, as shown in some embodiments of this specification.

[0051] Figure 3 This is an exemplary schematic diagram of the technical process of dynamic partitioning and spatiotemporal hybrid coding according to some embodiments of this specification;

[0052] Figure 4 This is an exemplary schematic diagram illustrating the construction process of a secondary index table according to some embodiments of this specification;

[0053] Figure 5 These are exemplary schematic diagrams shown according to some embodiments of this specification;

[0054] Figure 6 These are exemplary schematic diagrams shown according to some embodiments of this specification;

[0055] Figure 7 These are exemplary schematic diagrams shown according to some embodiments of this specification;

[0056] Figure 8 These are exemplary schematic diagrams shown according to some embodiments of this specification. Detailed Implementation

[0057] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0058] Example 1

[0059] Figure 1 This is a schematic diagram of a point cloud data storage and retrieval system based on spatiotemporal hybrid coding, as shown in some embodiments of this specification.

[0060] In some embodiments, the point cloud data storage and retrieval system based on spatiotemporal hybrid coding may include a data storage module, a timer module, a data visualization module, and a data retrieval module.

[0061] The data storage module is used to divide the original point cloud data based on two time scales: the macro frame period and the peripheral pulse interval, to obtain a time frame sequence; to assign a time offset to each point in the time frame sequence to obtain single frame data; and to perform dynamic partitioning and spatiotemporal hybrid encoding on the single frame data to obtain a point cloud data storage table.

[0062] The timer module is used to time storage and retrieval times.

[0063] The data visualization module is used to visualize point cloud data and tabular data.

[0064] The data retrieval module is used to index the point cloud data storage table using a spatiotemporal secondary index table, obtain point cloud data retrieval results, and complete the storage and retrieval of point cloud data.

[0065] In some embodiments, such as Figure 8 As shown, the data storage module includes a storage mode design unit and a data storage unit. Users can select the storage method for point cloud data, supporting HBase distributed database and HDFS distributed file storage system. Users can choose the appropriate storage method according to the data scale and application scenario, and then perform data storage. The timer module includes a storage timing unit and a retrieval timing unit, providing timing functions such as data import, retrieval and processing, recording the time of data upload, preprocessing, storage and retrieval, and displaying it in real time on the interface. The data visualization module includes a data tabulation unit and a 3D visualization unit. The former converts point cloud data into a tabular form, and the latter realizes 3D rendering of massive point clouds. The data retrieval module includes a spatiotemporal retrieval unit, a spatial retrieval unit and an attribute retrieval unit: spatiotemporal retrieval allows users to select a time range and a spatial range to retrieve point cloud data within that spatiotemporal range; spatial retrieval allows users to input 20,000 people or a 3D spatial range to retrieve point cloud data within that area; temporal retrieval allows users to input a time range or a time point to retrieve point cloud data within that area; attribute retrieval allows users to retrieve data based on attributes such as reflection intensity and classification information.

[0066] In some embodiments, such as Figure 5As shown, the processor can adopt a B / S architecture to realize the storage, retrieval, and other management functions of point cloud data. A six-layer architecture design is adopted: 1) Front-end interaction layer (presentation layer): The system is developed based on the Vue.js framework, combined with cutting-edge visualization technology (i.e., the front-end framework VUE) to complete various functions, including the Element-UI component library, the Mars3D 3D map framework, the Echarts visualization chart library, Vue-component components, the Vue-router routing manager, the Vuex state management tool, Axios interaction (Post Get), API interfaces, and the Node.js runtime environment. In the 3D display module, the Mars3D visualization platform is integrated to visualize spatial data; in the data analysis module, Echarts is used to chart the data; in the user interaction module, Element-UI is used to build the interface. 2) The control layer adopts the RESTful API specification, passing requests to the business layer according to the retrieval pattern triggered by the front end, such as HTTP requests to REST services. 3) The business layer implements timing functionality and different retrieval modes for point cloud data, transmitting retrieval results and generating logs to the control layer and front end. This includes: Entity entities, Controllers, Mappers, Services, and Config configurations. 4) The access layer utilizes dynamic SQL to match different index tables to support retrieval under various conditions. Combined with frameworks like MyBatis, it builds inverted indexes for attribute fields such as strength and classification, encapsulates the retrieved data, and generates logs, including MyBatis enhancement tools and the Elasticsearch distributed search engine. The control layer, business layer, and access layer are built on the Spring Boot backend framework and feature access control. 5) The storage layer uses HDFS to store point cloud files. The HBase table follows the table structure designed in the above research, storing point clouds after partitioning and spatiotemporal encoding, including spatiotemporally encoded point cloud data, 3D modeling data, raw point cloud data, strength-sorted ordered linked lists, transactions, and other data. 6) The database is deployed in a master-slave cluster with three nodes. Each node is configured with an HDFS environment and an HBase database, including a PC cluster (Master, Slave1, and Slave2, etc.) and a distributed database.

[0067] In some embodiments, a point cloud data storage and retrieval system based on spatiotemporal hybrid coding can be used to execute a point cloud data storage and retrieval method based on spatiotemporal hybrid coding, including: S1: dividing the original point cloud data based on two time scales, macroscopic frame period and peripheral pulse interval, to obtain a time frame sequence; S2: assigning a time offset to each point in the time frame sequence to obtain single frame data; S3: performing dynamic partitioning and spatiotemporal hybrid coding on the single frame data to obtain a point cloud data storage table; S4: indexing the point cloud data storage table using a spatiotemporal secondary index table to obtain point cloud data retrieval results, thus completing the storage and retrieval of point cloud data.

[0068] In some embodiments of this specification, the processor utilizes a point cloud data storage and retrieval system based on spatiotemporal hybrid coding to execute a point cloud data storage and retrieval method based on spatiotemporal hybrid coding. Through dynamic partitioning and spatiotemporal hybrid coding technology, the storage efficiency of massive point cloud data is significantly improved; the use of a spatiotemporal secondary index table enables efficient spatiotemporal joint queries, providing technical support for applications in multiple fields such as real-time perception for intelligent driving and dynamic urban monitoring.

[0069] Example 2

[0070] Figure 2 This is an exemplary flowchart illustrating a point cloud data storage and retrieval method based on spatiotemporal hybrid coding, according to some embodiments of this specification. Figure 2 As shown, the process includes the following steps. In some embodiments, the process may be executed by a processor.

[0071] S1: Based on two time scales, the original point cloud data is divided to obtain a time frame sequence.

[0072] Raw point cloud data consists of massive amounts of spatiotemporal point cloud data from LiDAR. For example, raw point cloud data can include point cloud data from intelligent driving systems, point cloud data from 3D geographic information systems, and point cloud data from digital twins.

[0073] In some embodiments, the processor can acquire point cloud data information scanned by LiDAR to obtain raw point cloud data.

[0074] A time frame sequence is a set of continuously time-stamped data generated by time-series simulation of statically scanned point cloud data, used to simulate the temporal characteristics of dynamic LiDAR scanning equipment.

[0075] In some embodiments, such as Figure 6As shown, the processor can segment the data into frames according to the scan cycle, allocate microsecond-level time offsets to the point cloud within each frame, and synchronize the timestamps to the global GPS time system. This simulates the scanning timing of a real LiDAR device, enabling static data to support dynamic scene analysis (such as moving object tracking and deformation monitoring), while ensuring that the timestamps strictly monotonically increase to meet spatiotemporal retrieval requirements. For example, the processor can analyze whether the input point cloud data is a dynamic scene. If so, it performs frame segmentation, calculates the frame reference time, and allocates time within the frame. If not, it allocates a unified timestamp; the allocated point cloud data is then rounded down to the nearest second and the point cloud is output.

[0076] In some embodiments, the expression for the time base of each frame in the time frame sequence can be:

[0077] ;

[0078] ;

[0079] in, Represents the time base of the k-th frame. Indicates the initial time base. The sequence number represents the frame number (the k-th frame, k=0,1,2,…), while the discrete frame number represents the discrete frame number. Indicates the time interval of a single frame. Indicates the scanning frequency.

[0080] S2: Assign a time offset to each point in the time frame sequence to obtain single frame data.

[0081] The time offset is the time offset between adjacent laser pulses.

[0082] A single frame of data is a collection of data from each frame in a time frame sequence after adding a time offset.

[0083] In some embodiments, the expression for the timestamp of each point in a single frame of data can be:

[0084] ;

[0085] in, This represents the precise time of the i-th point. This represents the start time reference for the k-th frame. Indicates the index of a point within the frame. This indicates the pulse interval time.

[0086] S3: Perform dynamic partitioning and spatiotemporal hybrid encoding on the single frame data to obtain a point cloud data storage table.

[0087] A point cloud data storage table is a data table used to store point cloud data. For example, a point cloud data storage table can be an HBase storage model table, and it can include hash values, time compression codes, and spatial compression codes.

[0088] In some embodiments, such as Figure 3 As shown, the processor achieves efficient organization of massive point cloud data through a two-level dynamic partitioning of "grid-octree" and spatiotemporal hybrid coding. This is achieved through four key steps: First, spatial coordinates of the original point cloud data are extracted based on a density adaptive strategy. The original point cloud is then dynamically gridded, initially with a uniform 0.1° partition. Normalized density is calculated. When the grid density is greater than 1.5 * average density, quadtree subdivision is performed. When the grid density is less than 0.5 * average density, it is merged with adjacent grids to obtain a finer-level coding. The grid partitioning is iteratively updated. When the number of iterations is not less than 10, iteration stops, maintaining the current grid to solve the problem of uneven data distribution. Second, within each grid, a system based on spatial entropy and nodes is constructed. A dynamic octree with point capacity is constructed by reading the grid partitioning results, calculating the node entropy value, and splitting the tree node when the number of nodes is greater than 3000. Otherwise, a split condition score is calculated. If the score is greater than 0.65, the node is merged with the adjacent grid; otherwise, it is marked as a leaf node. The dynamic octree is constructed iteratively. When the number of iterations is not less than 8 or the number of nodes is not less than the minimum split capacity, the dynamic octree is obtained, realizing the fine organization of local data. Then, the three-dimensional coordinates are normalized using the Z-space filling curve and mapped to one-dimensional Morton coding as spatial coding. Finally, the temporal information of the original point cloud data is extracted and temporal coding compression is performed. The spatial coding and differential temporal compression coding are combined to form a spatiotemporal hybrid coding, which serves as the basis for the row key design of HBase.

[0089] In some embodiments, the processor can construct an initial uniform network based on single-frame data and a fixed interval; perform normalized density calculation on the initial uniform network to obtain a network with labeled abnormal density regions; dynamically partition the network with labeled abnormal density regions through iterative optimization to obtain an optimized network structure; calculate the spatial entropy of the octree root node of the optimized network structure to obtain point cloud distribution feature evaluation information; use the double splitting condition to judge the point cloud distribution feature evaluation information to obtain node splitting information; recursively execute spatial partitioning until a preset condition is reached, and perform spatiotemporal hybrid encoding compression storage on the leaf nodes to obtain a point cloud data storage table.

[0090] A point cloud network with fixed intervals between initial uniform network point cloud data.

[0091] In some embodiments, the processor may load a point cloud data storage table and establish an initial uniform network at fixed intervals of 0.1°.

[0092] A network that labels regions of abnormal density is a point cloud network that labels regions of abnormal density.

[0093] In some embodiments, the expression for the normalized density can be:

[0094] ;

[0095] in, Indicates the current grid density. Indicates the number of point clouds. Represents the grid area. Indicates the maximum value. Indicates density.

[0096] The optimized network structure is a network structure that dynamically adjusts the interval of point cloud data.

[0097] In some embodiments, the processor can perform quadtree subdivision on regions with a density exceeding the threshold by 1.5 times, and merge neighbor networks on regions with a density below 0.5 times, to obtain an optimized network structure.

[0098] Point cloud distribution characteristic assessment information reflects the density of the current point cloud distribution.

[0099] In some embodiments, the processor can pre-divide the optimized network structure into grid cells to initialize the octree root node, calculate the node spatial entropy to evaluate the point cloud distribution characteristics, and obtain point cloud distribution characteristic evaluation information.

[0100] In some embodiments, the expression for spatial entropy can be:

[0101] ;

[0102] in, Represents the spatial entropy of the current node. This represents the number of point clouds in the i-th subspace. This represents the maximum number of point clouds across all subspaces. This represents the total number of nodes.

[0103] Node splitting information is information that reflects whether a node has split.

[0104] In some embodiments, the processor can determine the node splitting condition by using a dual splitting condition. When the number threshold Cmax or the weighted decision value is ≥0.65, the processor performs node splitting to obtain node splitting information.

[0105] In some embodiments, the expression for the double splitting condition can be:

[0106] ;

[0107] in, This represents the first weighting coefficient (satisfying α+β=1). Represents the spatial entropy of the current node. Represents the maximum entropy value. This represents the second weighting coefficient (satisfying α+β=1). This indicates the number of point clouds at the current node. This represents the minimum splitting capacity threshold. This represents the maximum splitting capacity threshold. This indicates the threshold for comprehensive judgment.

[0108] In some embodiments, the processor can adopt a structure of "<hash value>_<time compression coding>_<spatial compression coding>", and introduce the Spark computing framework to accelerate the coding process during row key encoding. The column family design follows logical grouping and scalability partitioning strategies, designing two column families: PositionAndReflectivity for storing the spatial coordinates and reflection intensity information of the point cloud data, and ClassificationAndTime for storing the classification, color, and time information of the point cloud data. Based on the cluster environment memory, the number of pre-partitions is determined, resulting in a point cloud data storage table. The specific contents of the point cloud data storage table are shown in Table 1.

[0109] In some embodiments, the expression for the number of pre-partitions in the point cloud data storage table can be:

[0110] ;

[0111] in, Indicates the number of pre-partitions. This represents the JVM heap memory allocated to a single RegionServer process. This indicates the data written to the temporary cache.

[0112] Table 1 Point Cloud Data Storage Table

[0113]

[0114] S4: Using the spatiotemporal secondary index table, the point cloud data storage table is indexed to obtain the point cloud data retrieval results, thus completing the storage and retrieval of point cloud data.

[0115] The spatiotemporal secondary index table is an index table used for retrieving point cloud data storage tables.

[0116] In some embodiments, such as Figure 7As shown, the processor can employ a "feature value-main table row key" mapping mechanism to parse the three-segment structure of the main table row key, extract spatiotemporal features, and generate a secondary index row key in a specific format. The index table is automatically maintained through HBaseCoprocessor, using a single-column structure of "idx:rk" to store the set of main table row keys compressed by RoaringBitmap. An asynchronous batch processing mechanism and a two-phase commit protocol are used, combined with WAL logs to ensure data consistency, resulting in a spatiotemporal secondary index table. For example, the processor can use timestamps for time encoding to obtain compressed Unix timestamps, and use point cloud x-coordinates, y-coordinates, and z-coordinates for spatial encoding to obtain grid numbers and Morton codes. Combining the compressed Unix timestamp, grid number, Morton code, and hash value yields the Rowkey row key, constructing the spatiotemporal secondary index table. The specific contents of the spatiotemporal secondary index table are shown in Table 2.

[0117] In some embodiments, such as Figure 4 As shown, the processor can construct a secondary index table. The construction mechanism decouples feature extraction and row keys, converting the spatiotemporal features of the data written to the main table into index row keys of a specific format. This establishes a mapping relationship between the index table row key and the main table row key, storing the data in a distributed queue for batch processing in the background. Asynchronous batch processing and a two-phase commit protocol are used to optimize index construction efficiency and data consistency. Feature extraction is then performed, decoupling (parsing) the three-segment structure of "<hash value>_<time compression encoding>_<spatial compression encoding>" used in the main table row key design. Spatiotemporal and attribute features are extracted to construct a key-value mapping, providing the foundational data for index construction. Secondly, secondary index row keys are generated based on the extracted features, establishing a mapping relationship between feature values ​​and the main table row keys. The index table uses the same pre-partitioning strategy as the main table and is automatically maintained and asynchronously updated via HBase Coprocessor. In terms of storage structure, the index table uses a single column family, "idx:rk," to store the set of index table row keys and main table row keys, and uses RoaringBitmap compression for storage to support efficient one-to-many mappings. To minimize the impact on the main table's write performance, the system employs an asynchronous batch processing mechanism. When main table data is written, index building tasks are stored in a distributed queue. The background service triggers batch processing when the cumulative number of tasks reaches 10,000, improving overall efficiency. To ensure data consistency, the system uses a two-phase commit protocol to achieve atomic updates to the main table and index tables, and utilizes a write-ahead log (WAL) to record changes and atomically update both tables. In case of a failure, the main table data is restored first, and the index is rebuilt based on the WAL. By establishing a mapping relationship between feature values ​​and main table row keys, the retrieval conditions for point clouds are transformed into index scan operations, significantly reducing the number of scans.

[0118] Table 2 Spatiotemporal Secondary Index Table

[0119]

[0120] In some embodiments of this specification, a point cloud data storage and retrieval method based on spatiotemporal hybrid coding is provided. Through dynamic partitioning and spatiotemporal hybrid coding technology, the storage efficiency of massive point cloud data is significantly improved. The use of a spatiotemporal secondary index table enables efficient spatiotemporal joint query, providing technical support for applications in multiple fields such as real-time perception for intelligent driving and dynamic urban monitoring.

[0121] Example 3

[0122] Existing technologies are limited at the hardware level by virtualized testing environments, making it difficult to support the high-frequency (e.g., above 10Hz) real-time processing needs of large-scale point clouds in autonomous driving scenarios; their intelligence level is insufficient, dynamic partitioning parameters require manual optimization, and they cannot adapt to the differences in point cloud distribution characteristics in different road scenarios; there are bottlenecks in real-time performance, and the existing architecture is difficult to meet the stringent requirements of autonomous driving for low-latency (<100ms) streaming point cloud processing, especially in terms of insufficient ability to cope with data surges in sudden traffic scenarios.

[0123] In some embodiments, a point cloud data storage and retrieval method based on spatiotemporal hybrid coding includes:

[0124] S1: Based on two time scales, macro frame period and peripheral pulse interval, the raw point cloud data of autonomous driving is divided to obtain a time frame sequence;

[0125] S2: Assign a time offset to each point in the time frame sequence to obtain single frame data;

[0126] S3: Perform dynamic partitioning and spatiotemporal hybrid encoding on the single frame data to obtain a point cloud data storage table;

[0127] S4: Using the spatiotemporal secondary index table, the point cloud data storage table is indexed to obtain the autonomous driving point cloud data retrieval results, thus completing the storage and retrieval of autonomous driving point cloud data.

[0128] In some embodiments, the expression for the time base of each frame in the time frame sequence is:

[0129] ;

[0130] ;

[0131] in, Represents the time base of the k-th frame. express, express, Indicates the time interval of a single frame. Indicates the scanning frequency.

[0132] In some embodiments, the expression for the timestamp of each point in the single frame data is:

[0133] ;

[0134] in, express, express, express, This indicates the pulse interval time.

[0135] In some embodiments, S3 includes:

[0136] An initial uniform network is constructed based on single-frame data and a fixed interval;

[0137] The normalized density of the initial uniform network is calculated to obtain the network with annotated abnormal density regions.

[0138] Through iterative optimization, the network in the annotated abnormal density region is dynamically partitioned to obtain the optimized network structure;

[0139] Spatial entropy calculation is performed on the root node of the octree in the optimized network structure to obtain point cloud distribution feature evaluation information.

[0140] The node splitting information is obtained by judging the point cloud distribution feature evaluation information using the dual splitting condition;

[0141] The spatial partitioning is recursively executed until the preset conditions are met. The leaf nodes are then subjected to spatiotemporal hybrid encoding and compression storage to obtain the point cloud data storage table.

[0142] In some embodiments, the expression for the normalized density is:

[0143] ;

[0144] in, express, Indicates the number of point clouds. Represents the grid area. express, express.

[0145] In some embodiments, the expression for the spatial entropy is:

[0146] ;

[0147] in, Represents the spatial entropy of the current node. This represents the number of point clouds in the i-th subspace. express, This represents the total number of nodes.

[0148] In some embodiments, the expression for the double splitting condition is:

[0149] ;

[0150] in, express, Represents the spatial entropy of the current node. Represents the maximum entropy value. express, This indicates the number of point clouds at the current node. This represents the minimum splitting capacity threshold. This represents the maximum splitting capacity threshold. express.

[0151] In some embodiments, the expression for the number of pre-partitions in the point cloud data storage table is:

[0152] ;

[0153] in, express, express, express.

[0154] In some embodiments of this specification, a point cloud data storage and retrieval method based on spatiotemporal hybrid coding is provided. This method processes raw point cloud data for autonomous driving based on two time scales: the macroscopic frame period and the surrounding pulse interval, to obtain the retrieval results. This approach enables efficient organization and rapid retrieval of point cloud data, significantly improving the autonomous driving system's perception of its surrounding environment. Specifically, this method ensures the complete preservation of the temporal characteristics of LiDAR point clouds through precise coding at dual time scales; the innovative spatiotemporal hybrid index structure significantly improves the retrieval efficiency for key scenes; and the adaptive storage partitioning strategy automatically optimizes the data structure according to road scene characteristics, ensuring high precision and providing more reliable and efficient environmental perception data support for autonomous driving decision-making systems.

Claims

1. A point cloud data storage and retrieval method based on spatiotemporal hybrid coding, characterized in that, include: S1: Based on two time scales, macroscopic frame period and microscopic pulse interval, the original point cloud data is divided to obtain a time frame sequence; S2: Assign a time offset to each point in the time frame sequence to obtain single frame data; S3: Perform dynamic partitioning and spatiotemporal hybrid encoding on the single-frame data to obtain a point cloud data storage table; including: An initial uniform network is constructed based on single-frame data and a fixed interval; The normalized density of the initial uniform network is calculated to obtain the network with annotated abnormal density regions. Through iterative optimization, the network in the annotated abnormal density region is dynamically partitioned to obtain the optimized network structure; Spatial entropy calculation is performed on the root node of the octree in the optimized network structure to obtain point cloud distribution feature evaluation information. The node splitting information is obtained by judging the point cloud distribution feature evaluation information using the dual splitting condition; The expression for the double splitting condition is: ； in, This represents the first weighting coefficient. Represents the spatial entropy of the current node. Represents the maximum entropy value. This represents the second weighting coefficient. This indicates the number of point clouds at the current node. This represents the minimum splitting capacity threshold. This represents the maximum splitting capacity threshold. This indicates the threshold for comprehensive judgment; The spatial partitioning is recursively executed until the preset conditions are met. The leaf nodes are then subjected to spatiotemporal hybrid encoding and compression storage to obtain the point cloud data storage table. S4: Using the spatiotemporal secondary index table, the point cloud data storage table is indexed to obtain the point cloud data retrieval results, thus completing the storage and retrieval of point cloud data.

2. The point cloud data storage and retrieval method based on spatiotemporal hybrid coding according to claim 1, characterized in that, The expression for the time base of each frame in the time frame sequence is: ； ； in, Represents the time base of the k-th frame. Indicates the initial time base. Indicates the frame number. Indicates the time interval of a single frame. Indicates the scanning frequency.

3. The point cloud data storage and retrieval method based on spatiotemporal hybrid coding according to claim 1, characterized in that, The expression for the timestamp of each point in the single frame data is: ； in, This represents the precise time of the i-th point. Represents the time base of the k-th frame. Indicates the index of a point within the frame. This indicates the pulse interval time.

4. The point cloud data storage and retrieval method based on spatiotemporal hybrid coding according to claim 1, characterized in that, The expression for the normalized density is: ； in, Indicates the current grid density. Indicates the number of point clouds. Represents the grid area. Indicates the maximum value. Indicates density.

5. The point cloud data storage and retrieval method based on spatiotemporal hybrid coding according to claim 1, characterized in that, The expression for the spatial entropy is: ； in, Represents the spatial entropy of the current node. This represents the number of point clouds in the i-th subspace. This represents the maximum number of point clouds across all subspaces. This represents the total number of nodes.

6. The point cloud data storage and retrieval method based on spatiotemporal hybrid coding according to claim 1, characterized in that, The expression for the number of pre-partitions in the point cloud data storage table is: ； in, Indicates the number of pre-partitions. This represents the JVM heap memory allocated to a single RegionServer process. This indicates the data written to the temporary cache.

7. A point cloud data storage and retrieval system based on spatiotemporal hybrid coding, used to execute the point cloud data storage and retrieval method based on spatiotemporal hybrid coding as described in any one of claims 1 to 6, characterized in that, include: The data storage module is used to divide the original point cloud data into time frame sequences based on two time scales: macroscopic frame period and microscopic pulse interval. Assign a time offset to each point in the time frame sequence to obtain single frame data; perform dynamic partitioning and spatiotemporal hybrid encoding on the single frame data to obtain a point cloud data storage table; The timer module is used to time storage and retrieval times; The data visualization module is used to visualize point cloud data and tabular data; The data retrieval module is used to index the point cloud data storage table using a spatiotemporal secondary index table, obtain point cloud data retrieval results, and complete the storage and retrieval of point cloud data.

Images