Database Table Reordering for Higher Dictionary Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional database table compression methods do not optimize data distribution for achieving the best compression ratio, as some columns contain text with relatively small cardinality, leading to suboptimal compression ratios and increased storage costs.
Innovation Solution
Reorder database table columns based on field data cardinality, placing columns with lower cardinality adjacent to each other to facilitate dictionary-based compression, which identifies longer duplicate strings and improves compression efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If dictionary compression is applied to database tables without reordering columns, then compression can be performed on existing data, but the compression ratio is suboptimal due to poor data distribution
Solution Approach 1:
The patent applies preliminary action by reordering database table columns based on field data cardinality before applying dictionary compression. The system identifies fields with lower cardinality values and positions them in optimal locations (leading fields) to create better data distribution patterns that enable the compression algorithm to achieve higher compression ratios. This preprocessing step of column reordering based on statistical analysis of field data prepares the data in advance for more effective compression.
2Productivity
If columns are reordered based on field data cardinality, then compression efficiency is improved, but additional processing steps and computational overhead are introduced
Solution Approach 1:
The patent applies parameter changes by utilizing field data cardinality values as the key parameter for determining column reordering. The system calculates cardinality for each field (the number of distinct values) and uses these parameter values to reorder columns optimally. This parameter-based approach provides a systematic and efficient method for improving compression efficiency without requiring complex heuristics or extensive processing, as cardinality calculation is a straightforward statistical operation.
3Productivity
If fields with lower cardinality are placed adjacent to each other, then longer duplicate strings are identified for compression, but the table structure must be modified
Solution Approach 1:
The patent applies local quality by creating localized groups of fields with similar characteristics (low cardinality) adjacent to each other in the table structure. Rather than requiring global restructuring of the entire table, the system locally optimizes specific regions by placing fields with lower cardinality values next to each other. This localized approach allows the dictionary compression algorithm to efficiently identify longer duplicate strings within these grouped fields while maintaining the overall table structure and minimizing the complexity of modifications required.
Data Source
AI summary
Selecting tables for compression by threshold statistical values. Identified tables are reordered according to fields having the lowest cardinality to increase the size of character strings replaced by keys during compression. Field locations are mapped between the original table and the reordered table. Dictionary-based compression is performed on reordered tables.