Row Prefix Compression for Database Query Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data compression techniques often achieve high compression ratios but at the cost of increased complexity and resource-intensive decompression, making it difficult to efficiently query and analyze data, especially in enterprise environments where rapid data growth is a challenge.
Innovation Solution
The proposed solution involves a method for compressing rows of a table by identifying and storing redundant information as a separate prefix, using a prefix compression technique that groups fixed-length and variable-length columns together, and generating prefix entries based on common strings, which allows for efficient compression and decompression while minimizing resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If conventional compression algorithms (Huffman, null suppression) are used, then compression ratio is improved (50-85%), but device complexity and decompression time increase
Solution Approach 1:
The patent extracts and removes redundant prefix data from each row, storing only the unique suffix portion. This extraction approach achieves compression by eliminating repeated information without requiring complex compression algorithms, thus reducing both data size and computational complexity.
Solution Approach 2:
The patent applies compression selectively to the prefix portion of rows that contain redundant data, while leaving the suffix portion uncompressed for efficient access. This local application of compression maintains high compression ratios for repetitive data while keeping decompression simple and fast.
2Loss of substance
If conventional compression algorithms are used, then compression ratio is improved, but decompression time increases
Solution Approach 1:
The patent performs preliminary compression during data insertion by removing the prefix and storing only the suffix. This preliminary action ensures that compression is already done when data is stored, eliminating the need for time-consuming decompression operations during query execution.
Solution Approach 2:
By extracting and removing the redundant prefix portion during data insertion, the system achieves compression without requiring complex decompression algorithms, thus reducing both storage requirements and decompression time.
3Loss of substance
If column-based compression is used, then compression ratio and I/O performance are improved, but query processing complexity increases
Solution Approach 1:
The patent segments each row into two distinct parts: a prefix portion containing redundant data and a suffix portion containing unique data. This segmentation allows for simple compression of the prefix while maintaining easy access to the suffix, reducing both data size and query processing complexity compared to full column-based compression.
Data Source
AI summary
Methods, systems, and computer program products for decompressing data are described. An ordinal column number of columnar data to be accessed is obtained, the ordinal column number identifying a location of the columnar data in a corresponding uncompressed row, the columnar data being stored in a first data structure. A breakpoint value in a breakpoint field of the at least partially compressed row is determined, the breakpoint value indicating a location of an end of a common prefix in the corresponding uncompressed row, the common prefix being stored in a second data structure. The ordinal column number of the columnar data to be accessed and a column number indicated by the breakpoint value are compared, the comparison identifying one or more locations of the columnar data to be accessed.