Relational Table Compression Using Field-Value Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data compression methods for relational tables are inefficient, particularly when table characteristics are inconsistent, and often result in poor compression performance due to the complexity of column grouping and information redundancy among columns, with existing solutions either being non-lossless or not effectively exploiting column dependencies.
Innovation Solution
Mapping field values in relational tables to smaller sets and compressing these sets using suitable techniques, allowing for the use of flat table compression methods, while also compressing side information to reduce data encoding redundancy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If column grouping is performed using the Pzip algorithm with dynamic programming, then compression effectiveness is improved by exploiting column dependencies, but the computational complexity and processing time increase significantly (O(n³) steps)
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing column dependency information in a dependency graph structure before the actual compression process. This allows the compression algorithm to quickly query pre-analyzed relationships between columns without performing complex O(n³) dynamic programming calculations during compression, thus resolving the contradiction between achieving effective compression and maintaining fast processing speed
Solution Approach 2:
The patent substitutes the mechanical dynamic programming approach with a graph-theoretic representation of column dependencies. By modeling column relationships as a graph structure where nodes represent columns and edges represent dependencies, the system replaces the computationally intensive iterative optimization process with more efficient graph traversal and analysis operations, reducing time complexity while preserving compression effectiveness
2Productivity
If training data is restricted to a small amount for column grouping, then processing speed is improved, but compression performance deteriorates when table characteristics are inconsistent
Solution Approach 1:
The patent applies dynamics by making the column grouping strategy adaptive rather than static. The system dynamically adjusts the grouping approach based on the actual characteristics of the input table, using the pre-computed dependency graph to identify optimal groupings for each specific table structure. This allows the system to maintain fast processing while achieving high compression performance even when table characteristics vary, resolving the contradiction between speed and adaptability
Solution Approach 2:
The patent changes the parameter of column grouping from a fixed pre-trained configuration to a dynamically determined configuration based on table-specific dependency patterns. By using the dependency graph to guide grouping decisions rather than relying on generic training data, the system adapts to different table characteristics without requiring extensive retraining, thus maintaining both processing speed and compression effectiveness across diverse table types
Data Source
AI summary
Compression of relational table data is enhanced with a pre-compression step of mapping the values of each field. Illustratively, the data in each cell of the relational table is replaced with a number a smaller number of characters. Additionally side information that is necessary for reversing the mapping is also compressed.