Multi-Dimensional Key Compression for Search Without Decompression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face inefficiencies in storing and retrieving large volumes of multi-dimensional data, particularly in online analytical processing (OLAP) systems, where complex queries require significant processing and storage resources, and existing compression methods do not effectively address redundancy in multi-dimensional keys.
Innovation Solution
A method for compressing multi-dimensional keys by identifying common and varying bit slots, forming masks and patterns, and storing compressed keys, allowing for efficient storage and search operations without decompression, using dictionary compression and bit translation techniques within a multi-dimensional online analytical processing (MOLAP) system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If multi-dimensional keys are stored in uncompressed format, then search operations are simple, but storage space is wasted and compression ratio is low
Solution Approach 1:
The multi-dimensional key is segmented into multiple bit slots, where common bits across multiple keys are extracted and stored separately in a compression structure, while varying bits are stored in the original key structure. This segmentation allows efficient storage by eliminating redundancy while maintaining fast search capability through the use of masks and patterns.
Solution Approach 2:
Common bits that are identical across multiple multi-dimensional keys are extracted and stored separately in a compression structure. This extraction process removes redundant information from each individual key, significantly reducing storage requirements while preserving the ability to reconstruct full keys when needed for search operations.
2Quantity of substance
If compression is applied to multi-dimensional keys, then storage efficiency improves, but search performance may degrade due to decompression requirements
Solution Approach 1:
Masks and patterns are pre-computed and stored alongside the compressed keys. These masks indicate which bit positions are common versus varying, and patterns store the actual common bit values. During search operations, these pre-computed structures enable direct comparison without requiring full decompression of the compressed keys, thus maintaining high search speed while benefiting from compression.
3Quantity of substance
If all bit slots are compressed, then compression ratio increases, but ability to perform fast searches without decompression is lost
Solution Approach 1:
Different bit slots within the multi-dimensional keys are treated differently based on their characteristics. Common bit slots (identical across multiple keys) are compressed and stored in the compression structure, while varying bit slots (different across keys) are retained in the original key structure. This local differentiation optimizes both compression ratio and search efficiency by applying compression only where beneficial.
Data Source
AI summary
A method includes comparing a search key that includes bits having respective values and bit positions to a mask to identify masked and unmasked portions of the search key. The mask corresponds to multi-dimensional keys and has first and second values in bit positions corresponding to and not corresponding to, respectively, common bits. Each common bit has a respective same value and occurs in a respective same position in the multi-dimensional keys. The masked and unmasked portions are bits at bit positions corresponding to bit positions of bits of the mask having first and second values, respectively. The method includes determining, based on determining that values in bit positions of the masked portion match values in corresponding bit positions of a pattern, that the unmasked portion matches a compressed key without decompressing the compressed key, and based thereon, identifying a successful match between the search and compressed keys.