Database Dictionary Decompression Using Block Re-Pair Front Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database compression methods, such as Re-Pair Front Coding (RPFC), face challenges with high compression times and memory consumption, especially when dealing with large string dictionaries, making them impractical for datasets exceeding a few hundred megabytes.
Innovation Solution
The proposed solution is a block re-pair front coding (BRPFC) method that splits the dictionary into blocks containing a fixed number of front-coded buckets, applies Re-Pair compression to each block individually, and uses a termination character to facilitate efficient compression and decompression, reducing the size of the text to be compressed and optimizing compression times.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If Re-Pair compression is applied to the entire string dictionary, then compression ratio is improved, but compression time and memory consumption increase significantly
Solution Approach 1:
The patent divides the string dictionary into multiple blocks, where each block contains a subset of the dictionary data. Re-Pair compression is applied independently to each block rather than the entire dictionary at once. This segmentation reduces the computational burden and memory requirements for each compression operation while maintaining overall compression effectiveness across the complete dictionary.
2Loss of substance
If Re-Pair compression is applied to the entire string dictionary, then compression ratio is improved, but memory consumption increases
Solution Approach 1:
The string dictionary is segmented into multiple smaller blocks, allowing Re-Pair compression to process each block independently with reduced memory footprint. This enables compression of large dictionaries that would otherwise exceed available memory resources.
Solution Approach 2:
The patent applies compression to partial portions (blocks) of the dictionary rather than requiring the entire dictionary to be loaded into memory simultaneously. This partial action approach allows compression to proceed with manageable memory requirements.
3Loss of substance
If the dictionary is processed as a single unit, then compression completeness is maintained, but access time increases
Solution Approach 1:
The dictionary is divided into blocks that can be independently compressed and stored. During access operations, only the relevant blocks need to be decompressed rather than the entire dictionary, significantly reducing access time while maintaining complete compression coverage.
4Productivity
If block-based compression is used, then compression time is reduced, but compression complexity increases
Solution Approach 1:
The complex task of compressing a large dictionary is segmented into multiple simpler sub-tasks, each handling a smaller block. This reduces the instantaneous complexity of each compression operation while the overall system manages multiple blocks through standardized interfaces.
Data Source
AI summary
A method, a system, and a computer program product for decompressing data. One or more compressed blocks in a set of stored compressed blocks responsive to a request to access data in the set of stored compressed blocks are identified. String prefixes inside the identified compressed blocks are decompressed using front coding. String suffixes inside the identified compressed blocks are decompressed using a re-pair decompression. Uncompressed data is generated.