Count Sketch Convolution for Fast Multi-Join Cardinality Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques struggle to efficiently estimate cardinality of multi-join queries across relations in a database while ensuring fast update times, as combining the strengths of methods like AMS sketch and Count sketch is a non-trivial task.
Innovation Solution
The method employs circular convolution and cross-correlation to create sketches that preserve information, enabling fast updates and accurate cardinality estimation of complex multi-join queries, using a novel sketching approach that includes determining sign values and bin indices for joined attributes and combining them to form tuple and relation sketches.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing cardinality estimation methods (AMS sketch, Count sketch) are combined to handle multi-join queries, then estimation accuracy is improved, but update time complexity increases significantly
Solution Approach 1:
The patent segments the sketch update operation into independent hash bucket updates. Each incoming tuple is hashed to specific buckets using hash functions h1 and h2, and only those specific buckets are updated, rather than updating the entire sketch. This segmentation enables O(1) update time while maintaining accurate multi-join cardinality estimation through the structured combination of hashed bucket counts.
Solution Approach 2:
The patent introduces hash functions h1 and h2 as intermediary mechanisms that map tuples to specific hash buckets. These hash functions act as mediators between the incoming data and the sketch structure, enabling efficient routing of updates to the correct buckets without scanning the entire sketch, thus achieving fast updates while preserving estimation accuracy.
2Quantity of substance
If traditional sketch methods are used for multi-join queries, then memory usage is reduced, but estimation error increases for complex joins
Solution Approach 1:
The patent transitions from single-hash sketching to a two-dimensional hashing approach by introducing a second hash function h2 that operates on the output of h1. This dimensional extension creates a hierarchical hashing structure where tuples are first hashed by h1 to produce intermediate values, then further hashed by h2 to final bucket indices. This multi-dimensional hashing preserves more information about tuple relationships, reducing estimation error for complex multi-join queries while maintaining compact sketch memory usage.
3Productivity
If fast update methods are implemented, then throughput is improved, but information loss in sketches increases
Solution Approach 1:
The patent performs preliminary hashing actions on incoming tuples before updating the sketch. By pre-computing the hash values using functions h1 and h2, the system determines the exact bucket locations in advance. This preliminary action enables direct updates to the correct buckets without scanning or searching, achieving O(1) update time while preserving complete information about tuple frequencies and relationships, thus preventing information loss during the fast update process.
Data Source
AI summary
Systems and methods to estimate cardinality of multi-join queries are disclosed. The method includes determining sign values and bin indices for joined attributes of a particular relation using corresponding sign functions and bin functions initialized for the joined attributes of a particular relation. The method includes combining the determined sign values to obtain a combined determined sign value and combining the determined bin indices to obtain a combined determined bin index for a particular tuple in the particular relation stream. The method includes creating a tuple sketch based on the combined determined sign value and the combined determined bin index. The method includes creating relation sketches for corresponding relations. The relation sketch is created by accumulating, on a relation basis, tuple sketches of corresponding tuples in the particular relation stream. The method includes, combining, at inference, the relation sketches to estimate a cardinality of the multi-join query.