Reference-Key Compression for Sorted Database Keys
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large database systems face significant storage footprint challenges due to the large size of database keys, which can be mitigated by compressing these keys while maintaining their sorted order, especially in log-structured merge-tree (LSM tree) systems that require key-sorted order for efficient data management.
Innovation Solution
A computer system compresses database keys by replacing portions with data identifying a reference key and the similarity between the key and the reference key, allowing for efficient storage and decompression, and implements a key compression engine to manage this process, ensuring that compressed keys maintain their sorted order.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If database keys are compressed by replacing portions with reference key data, then storage footprint is reduced, but key retrieval complexity increases
Solution Approach 1:
The system pre-computes and stores similarity metrics between database keys and reference keys during the compression phase. This preliminary action enables the retrieval process to simply look up pre-computed similarity data rather than performing complex comparisons during retrieval, thus reducing retrieval complexity while maintaining compression benefits
Solution Approach 2:
The patent introduces reference keys as intermediary elements that mediate between the original database keys and their compressed representations. The reference keys store only the necessary distinguishing portions and similarity metrics, acting as intermediaries that enable efficient retrieval without storing complete key data, thus reducing storage footprint while maintaining retrieval efficiency
2Quantity of substance
If database keys are compressed to reduce storage size, then storage efficiency improves, but decompression and verification time increases
Solution Approach 1:
The system applies partial decompression by only retrieving and verifying the necessary portions of reference keys and similarity metrics rather than fully decompressing entire key structures. This partial action approach reduces decompression time while maintaining storage efficiency benefits
Solution Approach 2:
Similarity metrics and reference key portions are pre-computed and stored during the compression phase, eliminating the need for complex real-time computations during decompression and verification. This preliminary preparation significantly reduces decompression time while preserving storage efficiency
3Quantity of substance
If reference keys are stored to enable key compression, then storage footprint is reduced, but memory usage increases
Solution Approach 1:
The patent segments the database key storage into multiple components: compressed key portions stored in the main database, and reference key data stored separately in memory or cache. This segmentation allows the system to reduce overall storage footprint while managing memory usage by placing frequently accessed reference data in faster memory and less frequently accessed data in slower storage
Data Source
AI summary
Techniques are disclosed relating to compressing database keys. A computer system may receive a request to write a database record to a storage medium. The database record may include a database key and a corresponding data value. The computer system may compress the database key by replacing a portion of the database key with particular data that identifies a location of a reference database key and an amount of similarity determined between the database key and the reference database key. The computer system may write the database record to the storage medium. The database record may include the compressed database key and the corresponding data value.