Partial Compression of Tree-Based Index Structures
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional database systems face challenges in balancing memory consumption and update performance, particularly in cloud-based environments where high compression for memory efficiency hinders update speed and tree-based index structures consume more space, leading to performance trade-offs.
Innovation Solution
The proposed solution involves selectively compressing portions of a tree-based index structure at the leaf node level, triggered by insert operations, which merges sibling nodes and deallocates empty memory, allowing for reduced memory usage while maintaining access to other parts of the structure, thereby optimizing memory usage without compromising update performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored in a highly-compressed format, then memory consumption is reduced, but update performance deteriorates
Solution Approach 1:
The patent segments the tree-based index structure into two distinct parts: a compressed portion where leaf nodes are stored in a highly-compressed format to reduce memory consumption, and an uncompressed portion where leaf nodes remain in uncompress ed format to enable fast updates. This segmentation allows the system to simultaneously achieve both low memory usage and high update performance by applying different storage strategies to different parts of the same data structure.
2Productivity
If a tree-based index structure is used, then update performance is improved, but memory consumption increases
Solution Approach 1:
The patent applies local quality by making different parts of the tree-based index structure have different compression properties. Specifically, recently accessed or frequently updated leaf nodes are kept in an uncompress ed state in the uncompress ed portion, while less frequently accessed nodes are stored in the compressed portion. This local differentiation optimizes both memory usage and update performance by matching storage characteristics to access patterns.
3Quantity of substance
If the entire tree-based index structure is rewritten to conserve memory, then memory consumption is reduced, but insert performance deteriorates and access is blocked
Solution Approach 1:
The patent implements preliminary action by pre-allocating a separate uncompress ed portion in the memory structure before any compression operations occur. This uncompress ed portion is prepared in advance to receive newly inserted leaf nodes without requiring compression or rewriting operations. By having this buffer region ready beforehand, the system can accept inserts immediately without blocking access to the main index structure, thus maintaining insert performance while still enabling memory conservation through selective compression of the compressed portion.
Data Source
AI summary
A system includes storage of data into a target memory location allocated to a target leaf node of a tree-based index structure, the target leaf node being a child node of a parent node of the tree-based index structure, where the tree-based index structure comprises one or more other leaf nodes which are child nodes of the parent node, and each of the target leaf node and the one or more other leaf nodes is associated with a plurality of allocated memory locations, incremental identification of all unused allocated memory locations between a first allocated memory location of a left-most one of the target leaf node and the one or more other leaf nodes and a last used allocated memory location of a right-most one of the target leaf node and the one or more other leaf nodes, and movement of data stored in the target leaf node and the one or more other leaf nodes into the identified unused allocated memory locations.