Database Index Compression via Key Prefix Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database management systems face inefficiencies in storage space utilization and query performance due to repeated storage of similar key values in index structures, especially in large datasets, leading to increased memory usage and decreased performance.
Innovation Solution
The method involves storing a lower and upper fence key for each page of records, extracting a common prefix from key values, and storing only the remaining values, allowing for efficient compression and reduced storage needs, while enabling dynamic compression adjustments based on load conditions and omitting the need for additional metadata recording.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If all key values are stored completely in index records, then query accuracy is maintained, but storage space is wasted due to repeated similar values
Solution Approach 1:
The patent merges common prefix portions of key values into a shared storage structure, while only storing the unique suffix portions in individual index records. This combining approach eliminates redundant storage of identical prefix data across multiple records, directly reducing storage space while preserving complete key value information through the combination of stored suffixes and their associated prefixes.
Solution Approach 2:
The patent extracts the common prefix portion from each key value and separates it from the unique suffix portion. By taking out the redundant prefix data and storing it separately or implicitly, the system reduces storage requirements in individual records while maintaining the ability to reconstruct complete key values when needed for queries.
2Quantity of substance
If compression is always applied to reduce storage space, then storage efficiency improves, but insert/delete operations become slower due to compression overhead
Solution Approach 1:
The patent implements dynamic compression adjustment based on workload characteristics. When insert/delete operations are frequent, the system reduces or disables compression to improve operation speed. When query operations dominate, compression is enhanced to maximize storage efficiency. This dynamic adaptation allows the system to optimize for the current operational context, balancing storage efficiency against operation performance.
3Adaptability or versatility
If compression metadata is recorded for each record, then compression can be managed individually, but the complexity of data management increases
Solution Approach 1:
The patent applies compression uniformly at the page level rather than individually to each record. By treating all records within a page as a single compression unit with shared metadata, the system maintains the flexibility to manage compression while dramatically reducing metadata complexity. The page-level approach allows one set of compression parameters to govern multiple records, eliminating the need for separate metadata per record.
Data Source
AI summary
Provided are a database (DB) managing method and system, wherein, while forming an index of a DB, a lower value and an upper value of key values of a plurality of records included in one page are stored as separators and an overlapping part of the key values is deleted from the plurality of records by using the separators to save a storage space where pages of the index is stored, and thus the performance of the DB is improved.