Relational Data Compression via Topological Field Sorting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for compressing relational data files face challenges in preserving the order of records and lacking metadata, which are crucial for effective compression, especially in unordered and typeless data environments.
Innovation Solution
The method involves computing an optimum relational dependency transform by sorting fields topologically based on dependency relations, transforming fields using a defined permutation, and then compressing the transformed data using a tailored compressor, ensuring invertibility and efficient compression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If standard compressors (Gzip, Bzip2) are used on relational data files, then general compression is achieved, but compression ratios are insufficient due to lack of exploitation of data structure
Solution Approach 1:
The patent applies preliminary action by performing topological sorting on fields based on dependency relations before compression. This reordering of fields according to their dependency graph enables subsequent compressors to exploit the structured relationships in the data, achieving better compression ratios without requiring complex compression algorithms.
Solution Approach 2:
The patent changes the parameter ordering of fields based on their dependency relationships. By transforming the data representation through topological sorting and exploiting type-limited field values, similar values, and field correlations, the patent enables standard compressors to achieve superior compression ratios without increasing algorithmic complexity.
2Productivity
If field reordering is performed to improve compressibility, then compression efficiency increases, but record order may be altered which is critical for relational data files
Solution Approach 1:
The patent segments the compression process into two independent phases: (1) topological sorting of fields based on dependency relations to improve compressibility, and (2) application of standard compressors. This segmentation allows field reordering for compression while maintaining the ability to preserve record order during decompression through inverse transformation.
Solution Approach 2:
The patent introduces an intermediary transformation step (topological sorting based on dependency graph) that mediates between the original data structure and the compression process. This intermediary representation enables efficient compression while maintaining invertibility, ensuring that record order can be restored during decompression.
3Reliability
If metadata is added to preserve record order and structure information, then data integrity is improved, but storage overhead increases
Solution Approach 1:
The patent extracts and exploits existing structural information (field dependencies, type limitations, value similarities) that is already present in the relational data file without adding external metadata. By taking out and utilizing the inherent structure in the data, the patent achieves both data integrity and efficient storage without storage overhead.
Data Source
AI summary
A relational dependency transform is introduced as a way to exploit information redundancy in conditioning data in a relational database for better compressibility. An optimum relational dependency transform of the relational database is first computed. Fields of the relational database are then sorted topologically based on a weighted, directed graph having nodes representing predictor and predictee fields. For each predictee field in the topological order, a transformed field is then computed via the relationship between predictor and predictee in the optimum relational dependency transform.