Compressed Tree Storage for Random Access Multi-Dimensional Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for storing and retrieving multi-dimensional data, such as octrees, are inefficient due to significant memory overhead from parent-to-child pointers and lack of random access, especially when data is sparsely defined, and compression methods often require decompressing the entire structure for access.
Innovation Solution
The method employs a bit vector representation using succinct data structures like Depth-First Unary Degree Sequence (DFUDS) and Elias-Fano encoding to store node structural information, allowing for efficient compression and random access of multi-dimensional data by mapping node positions to data stream locations, enabling decompression of only the required node without decompressing the entire structure.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional octree data structures with parent-to-child pointers are used, then data retrieval is straightforward, but memory overhead becomes significant especially for sparsely defined spatial data
Solution Approach 1:
The patent extracts and removes the parent-to-child pointer structure from the traditional octree implementation. Instead of maintaining explicit pointer relationships between parent and child nodes, the invention uses an implicit hierarchical structure where node relationships are derived from the sequential arrangement and bit mask representations in the compressed data stream, eliminating the memory overhead of storing pointer information.
Solution Approach 2:
The patent inverts the traditional approach by not storing explicit relationship information (pointers) but instead deriving relationships from the compressed structure itself. The bit mask vectors and sequential node arrangements encode the hierarchical relationships implicitly, allowing retrieval operations to reconstruct parent-child relationships during decompression without having stored them explicitly.
2Quantity of substance
If data is compressed to reduce storage size, then memory usage decreases, but random access becomes impossible as entire structure must be decompressed
Solution Approach 1:
The patent segments the compressed data structure into independent node representations that can be individually accessed. Each node is encoded with its position information and bit mask vector, allowing the data stream to be divided into discrete addressable units. This segmentation enables random access to specific nodes without requiring decomposition of the entire compressed structure.
Solution Approach 2:
The patent performs preliminary encoding of node position information and bit mask vectors during the compression phase. By pre-calculating and storing the sequential position of each node and its corresponding bit mask representation, the compressed structure inherently contains the addressing information needed for random access, eliminating the need for full decompression.
3Loss of energy
If data is stored in predetermined sequence for compression efficiency, then compression ratio improves, but data retrieval requires sequential navigation losing random access capability
Solution Approach 1:
The patent adds a positional dimension to the compressed data by encoding the sequential position of each node within the data stream. This positional information, combined with bit mask vectors, creates a multi-dimensional addressing scheme that allows direct calculation of node locations in the compressed stream, transforming sequential storage into randomly accessible structure.
Data Source
Figure 1~2
Figure 3A~4
Figure 5~7
AI summary
A method of storing multi-dimensional data, the method comprises encoding a structure of nodes of a tree data structure into an order representation; compressing data of nodes of the tree data structure to create a data stream of compressed data portions of variable length in a sequence corresponding to the order represented in the order representation. Each compressed data portion is formed from the data of each node. The method further comprises mapping the positions of the data portions in the data stream to the order representation to produce a location representation; and storing the order representation, data stream and location representation. A system of storing multi-dimensional data is also disclosed, as well as a method and system for retrieving stored multi-dimensional data.