Inline Database Index Compression via Range Marking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing index compression techniques require separate phases of compression and decompression, making them computationally intensive and inefficient, especially for improving cache utilization and disk-to-memory transfer speeds.
Innovation Solution
A system and method that determines consistent changes in index key and row identifier values during index population, allowing for inline compression by marking the start and end of ranges without intermediate values, reducing computational intensity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If separate compression and decompression phases are used, then storage space is reduced, but computational intensity increases and data transfer speed decreases
Solution Approach 1:
The patent combines compression and data loading operations into a single inline compression phase. During index population, the system compresses index entries and loads them into cache simultaneously, eliminating the separate decompression phase. This merging of operations reduces computational overhead and improves data transfer speed while maintaining storage space efficiency.
Solution Approach 2:
The system performs compression in advance during the index population phase, before the data needs to be accessed. By pre-compressing the index entries and loading them directly into cache in compressed form, the system eliminates the need for decompression at query time, thereby improving data retrieval speed.
2Quantity of substance
If separate compression and decompression phases are used, then storage space is reduced, but computational intensity increases
Solution Approach 1:
The patent merges compression with the index population operation, performing both tasks in a single pass. This eliminates the need for a separate decompression phase, thereby reducing the total computational intensity and energy consumption while achieving the same storage space savings.
Solution Approach 2:
The patent extracts the decompression operation from the overall process by eliminating it entirely. Instead of compressing, storing, then decompressing, the system compresses once during population and retains data in compressed form in cache, removing the harmful decompression step that consumes computational resources.
3Quantity of substance
If conventional index compression techniques are used, then storage space is conserved, but cache utilization is reduced
Solution Approach 1:
The patent combines compression with direct cache loading in an inline operation. Compressed index entries are loaded directly into cache during the population phase, maximizing cache utilization with compressed data. This eliminates the intermediate storage step that would otherwise require decompression, allowing more data to reside in cache simultaneously.
4Loss of information
If intermediate values are entered during index population, then index completeness is maintained, but compression efficiency is reduced
Solution Approach 1:
The patent segments the index population process into distinct phases: compression phase where ranges are identified and marked, and completion phase where intermediate values are filled in. This segmentation allows compression to be performed on identified ranges first, achieving compression efficiency, while still maintaining index completeness by populating intermediate values in a subsequent pass.
Solution Approach 2:
The system performs preliminary compression on index ranges during the population phase by marking start and end points, achieving compression before all intermediate values are known. The intermediate values are then filled in afterward, allowing compression to be established in advance while maintaining completeness.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
Systems, methods and computer program products for compression of database indexes are described herein. A system embodiment includes a sequence determiner to scan a database index and to determine a start of a range and end of a range of consistently changing values in one or more index pages of said database index and an index updater to update said database index based on a sequence determined by said sequence determiner, while suspending writing of one or more values that lie within start of said range and end of said range of values. A method embodiment includes scanning an index, determining a pattern of changing values in one or more index pages of said index and selectively updating said index based on said determining step to minimize index insertions. The method embodiment further includes determining a start of a range of values and an end of said range of values in an index page, setting appropriate bits to identify said start of range of values and end of range of values, determining if an entry to be inserted can appended to at the end of said range of values, and compressing said index by suspending writing of one or more values that occur between said start of range of values and said end of range of values.