Predicate Result Cache for Compressed Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for performing scan-and-filter operations on compressed tabular data incur significant compression overhead, as all rows are decompressed to evaluate filtering criteria, leading to inefficient retrieval and storage times.
Innovation Solution
The implementation of early predicate evaluation with conjunction result caching, where a cache records the results of predicate conjunction evaluations for each token, allowing for the bypass of unnecessary decompression and evaluation by checking the cache for FALSE results, thereby reducing compression overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If compressed storage is used for tabular data, then storage space is reduced, but computational overhead increases during scan-and-filter operations
Solution Approach 1:
The patent applies preliminary action by pre-computing and caching predicate evaluation results for compressed rows before actual query execution. The system evaluates predicates on compressed data formats in advance, stores these evaluation results in a cache, and retrieves them during query operations. This eliminates the need to decompress and re-evaluate predicates during scan-and-filter operations, thus reducing computational overhead while maintaining compressed storage benefits
Solution Approach 2:
The patent implements self-service by enabling compressed data structures to carry their own predicate evaluation results within the compressed format itself. The compressed rows include embedded cache data that contains pre-evaluated predicate results, allowing the data to serve its own query evaluation needs without external decompression processing. This self-contained approach reduces I/O and computational overhead during query operations
2Reliability
If all rows are decompressed to evaluate filtering criteria, then accurate filtering is achieved, but retrieval time increases
Solution Approach 1:
The patent applies partial action by evaluating and caching only the necessary predicate results on compressed rows that are likely to satisfy the query conditions, rather than fully decompressing all rows. The system performs partial predicate evaluation on compressed data, caches these partial results, and uses them to quickly filter out non-matching rows before any potential decompression occurs. This partial evaluation approach maintains filtering accuracy for the rows that matter while avoiding unnecessary processing of non-matching rows
Solution Approach 2:
The patent introduces an intermediary cache layer between the compressed storage and the filtering operation. This cache stores pre-evaluated predicate results that act as intermediaries, allowing the system to determine row eligibility without direct decompression. The cache mediates between the compressed data format and the filtering requirements, enabling accurate filtering decisions to be made on compressed representations, thus reducing retrieval time while maintaining filtering accuracy
3Quantity of substance
If compression is applied to reduce storage space, then storage efficiency improves, but scan-and-filter operation performance deteriorates
Solution Approach 1:
The patent applies preliminary action by pre-computing predicate evaluation results and caching them in the compressed data structure before query operations. This advance preparation allows scan-and-filter operations to retrieve and use pre-evaluated results directly from the compressed format without performing expensive decompression and predicate evaluation during actual query execution, thus maintaining both storage efficiency and operation performance
Solution Approach 2:
The patent implements self-service by embedding predicate evaluation results directly within the compressed data structure. The compressed rows become self-sufficient by containing their own evaluation cache data, enabling them to participate in scan-and-filter operations without requiring external decompression processing. This self-contained compressed format maintains storage efficiency while improving operation performance by eliminating redundant decompression steps
Data Source
AI summary
In token compression, a column value is represented by a encoded token. During early predicate evaluation, the result that a decoded token in a column does not satisfy a predicate condition in a predicate conjunction is recorded in a cache as a “FALSE”. Such a result not only means that when the column contains that token that the predicate condition is not satisfied, but also that a predicate conjunction containing the predicate condition is not satisfied. When performing early predicate evaluation on a subsequent row on a token in a column, the conjunction result cache is referred to. If the cache records a FALSE for that token in that column, then it is known the predicate conjunction cannot be satisfied by the decoded token.