Learned Join Cardinality Estimation via Adjacency Matrix Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing join cardinality estimation techniques in relational database management systems (RDBMS) face challenges in accurately estimating the number of rows resulting from joins due to unknown or changing data distributions, correlations between joining columns, and filters/predicates in queries, leading to ineffective cardinality estimates and long planning times.
Innovation Solution
The proposed solution involves building a learned join cardinality estimation model using a join graph representation. This is achieved by generating a training query with known join cardinality, creating an adjacency matrix encoding the join graph, encoding one side of the diagonal axis, and training the model using this encoded information. The model can then perform inference to predict join cardinality for new queries, enabling efficient join cardinality estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If classical or previous learning-based join cardinality estimation techniques are used, then the system can perform cardinality estimation, but the accuracy is insufficient and planning times are long
Solution Approach 1:
The patent transforms the join graph into an encoded representation by processing adjacency matrices and extracting structural parameters. This parameter transformation enables the model to capture essential join characteristics in a compressed format, improving estimation accuracy while reducing the computational time required for planning.
Solution Approach 2:
The patent creates a simplified encoded representation of the join graph that captures the essential structural information without requiring the full complex graph data. This copying approach allows the model to work with a condensed version of the problem, maintaining accuracy while significantly reducing processing time.
2Measurement precision
If complex models are built to understand data distributions and correlations, then estimation accuracy may improve, but training time increases and generalizability decreases
Solution Approach 1:
The patent extracts the essential structural information from the complex join graph by processing the adjacency matrix and identifying key patterns. This extraction creates a simplified representation that retains the critical characteristics needed for accurate cardinality estimation while removing unnecessary complexity, resulting in a more efficient and generalizable model.
Solution Approach 2:
The patent segments the join graph analysis into distinct components: adjacency matrix processing, pattern identification, and cardinality prediction. This segmentation allows each component to be optimized independently, reducing the overall model complexity while maintaining high estimation accuracy through specialized processing of each aspect.
3Measurement precision
If detailed data distribution and correlation information is collected, then estimation accuracy improves, but the system becomes less scalable with the number of tables
Solution Approach 1:
The patent creates a universal encoding mechanism that can handle join graphs of varying sizes and complexities through the same processing framework. The adjacency matrix-based approach provides a standardized representation that scales consistently, allowing the model to maintain accurate cardinality estimation across queries with different numbers of tables without requiring proportional increases in model complexity.
Data Source
AI summary
Aspects of the invention include techniques for providing a learned join cardinality estimation using a join graph representation. A non-limiting example method includes building a join cardinality estimation model. The model can be built by generating a training query having a known join cardinality, generating an adjacency matrix encoding a join graph of the training query, encoding one side of a diagonal axis of the adjacency matrix, and training the join cardinality estimation model using the encoded adjacency matrix and the known join cardinality. The method includes performing an inference using the join cardinality estimation model. The inference includes a predicted join cardinality for a query. The method includes executing a query execution plan for the query using the predicted join cardinality.