Database Dictionary Decompression Using Block Re-Pair Front Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database compression methods, such as Re-Pair Front Coding (RPFC), face challenges with high compression times and memory consumption, especially when dealing with large string dictionaries, making them impractical for multi-terabyte in-memory database systems.
Innovation Solution
The proposed solution is a block re-pair front coding (BRPFC) method that splits the dictionary into blocks, applies front coding, and then uses Re-Pair compression on each block individually, optimizing compression times and memory usage by storing offsets and grammar within each block.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If Re-Pair compression is applied to the entire string dictionary, then compression ratio is improved, but compression time and memory consumption increase significantly
Solution Approach 1:
The patent divides the string dictionary into multiple blocks, where each block is independently compressed using Re-Pair compression. This segmentation allows the compression algorithm to process smaller data units in parallel, significantly reducing total compression time and memory requirements while maintaining the overall compression ratio of the complete dictionary
2Quantity of substance
If Re-Pair compression is applied to the entire string dictionary, then compression ratio is improved, but memory consumption increases
Solution Approach 1:
By segmenting the dictionary into blocks that fit within available memory constraints, the patent enables Re-Pair compression to be applied effectively. Each block can be compressed and stored independently, allowing the system to handle large dictionaries that would otherwise exceed memory capacity while achieving significant compression ratios
3Volume of stationary object
If the dictionary is stored in compressed form, then storage space is reduced, but access time increases due to decompression requirements
Solution Approach 1:
The patent organizes compressed dictionary data into block structures with embedded metadata that enables selective decompression. When a query requires dictionary access, only the specific block containing the relevant string needs to be decompressed rather than the entire dictionary, significantly reducing access time while maintaining storage compression benefits
Data Source
AI summary
A method, a system, and a computer program product for decompressing data. One or more compressed blocks in a set of stored compressed blocks responsive to a request to access data in the set of stored compressed blocks are identified. String prefixes inside the identified compressed blocks are decompressed using front coding. String suffixes inside the identified compressed blocks are decompressed using a re-pair decompression. Uncompressed data is generated.