Query Optimization Using Scrambled Sample Tables
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database management systems face challenges in generating accurate query plans for complex predicates due to limitations in histogram-based selectivity estimation, particularly when data distribution is not uniform or when columns involved in predicates are not independent, leading to poor query execution performance.
Innovation Solution
The implementation of a method that uses compile time statistics and persistent sample tables to estimate selectivity, where a scrambled sample table is used to quickly determine the sample row count, reducing processing time and improving accuracy by allowing for efficient scanning and estimation of cardinality for complex queries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If histogram-based selectivity estimation is used for query optimization, then the query plan generation is fast and simple, but the selectivity estimation accuracy deteriorates for complex predicates
Solution Approach 1:
The patent pre-generates sample tables with stored statistical information about data distributions before query execution. These sample tables contain pre-computed selectivity data for various predicates, allowing the optimizer to quickly retrieve accurate estimates without performing complex calculations at query time. This preliminary preparation resolves the contradiction by providing both speed (pre-computed results) and accuracy (real data samples).
Solution Approach 2:
The patent creates simplified copy representations of the actual data through sample tables that contain representative subsets of the full dataset. These copies preserve the statistical properties and data distribution characteristics needed for accurate selectivity estimation, while being small enough to store and process efficiently. The copy approach maintains accuracy without requiring access to the entire large dataset during query optimization.
2Measurement precision
If dynamic sampling is performed at query time to improve selectivity estimation accuracy, then the estimation precision improves, but the processing time increases
Solution Approach 1:
Instead of performing sampling at query time, the patent performs the sampling action in advance and stores the results in sample tables. The sample tables capture data distribution characteristics and statistical information that can be directly reused for multiple queries. This eliminates the time-consuming sampling operation from query execution while maintaining the accuracy benefits of using actual data samples.
Solution Approach 2:
The sample tables serve themselves by containing pre-computed statistical information that automatically answers selectivity estimation queries without requiring additional processing. The stored sample data acts as a self-contained reference that provides accurate estimates instantly, eliminating the need for repeated sampling operations and reducing processing time for each query.
Data Source
AI summary
There is provided a computer-implemented method of optimizing a query. An exemplary method retrieves a subset of rows from a sample table based on a query for a table associated with the sample table. The query specifies the table. The sample table is generated in a previous optimization of the query. The sample table includes a scrambled, random selection of rows from the table. A cardinality of the table is determined based on the subset. A query plan is generated based on the query and the cardinality.


