Relational Database Compression Filters for Faster Query Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Relational database management systems face challenges in efficiently managing and querying large volumes of diverse data, requiring improved methods for data compression and query optimization to enhance performance and reduce data access.
Innovation Solution
The implementation of a relational database management system that utilizes analytical information and rough set analysis techniques for query planning, along with adaptive data compression methods involving a filter cascade and suffix-prediction algorithms, to minimize data access and optimize query execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is stored in uncompressed form, then query execution speed is improved, but data storage space is consumed
Solution Approach 1:
The patent segments data into distinct columns and applies compression filters independently to each column based on its specific characteristics. This allows different compression techniques to be applied to different data types (numeric, character, binary) while maintaining query performance on uncompressed or lightly compressed data
Solution Approach 2:
The system dynamically changes compression parameters by evaluating compression ratios and selecting appropriate filter stages. The query optimizer can choose to access compressed data directly or decompress based on query requirements, balancing storage efficiency with execution speed
2Quantity of substance
If compression filters are applied to data, then data storage space is reduced, but query processing complexity increases
Solution Approach 1:
The patent introduces an intermediary layer between compressed data and query processing. Compression filters act as intermediaries that transform data into a compressed representation, while the query optimizer serves as another intermediary that determines whether to use compressed data directly or decompress it, simplifying the overall processing complexity
Solution Approach 2:
Data is pre-compressed using various filter stages before storage, with compression ratios evaluated in advance. This preliminary compression action reduces the amount of data that needs to be processed during query execution, offsetting the added complexity through reduced data volume
3Loss of time
If analytical information is used for query planning, then data access is reduced, but system complexity increases
Solution Approach 1:
Analytical information such as compression ratios, data distributions, and query patterns are pre-calculated and stored in system catalogs. This preliminary analysis allows the query optimizer to make informed decisions about data access strategies without performing complex analysis during query execution
Solution Approach 2:
The system uses feedback from query execution patterns and compression ratio measurements to continuously refine query plans. The query optimizer incorporates analytical information about data characteristics and access patterns to improve data access efficiency while managing system complexity through learned patterns
Data Source
AI summary
A method for applying adaptive data compression in a relational database system using a filter cascade having at least one compression filter stage in the filter cascade. The method comprises applying a data filter associated with the compression filter stage to the data input to produce reconstruction information and filtered data, then compressing the reconstruction information to be included in a filter stream. The filtered data is provided as a compression filter stage output. The method may comprise evaluating whether the compression filter stage provides improved compression compared to the data input. The filter stage output may be used as the input of a subsequent compression filter stage.