Object Cardinality Estimation Using Global Statistics for Query Planning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional database systems struggle with estimating cardinality in object store database systems, particularly in cloud platforms like Teradata VantageCloud Lake, where data is saved in objects, leading to inefficient query execution plans due to the nontrivial cost of accessing objects.
Innovation Solution
Implementing object cardinality estimation methods such as General Estimation, Stratified Estimation, and Object-Range-Histogram-Based Estimation using global statistics to determine the number of objects that will be accessed, optimizing query execution plans by considering minimum and maximum values of columns in objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional cardinality estimation methods are used in object store database systems, then the query execution plan may be generated quickly, but the estimation accuracy of object cardinality is poor leading to inefficient query execution
Solution Approach 1:
The patent segments the object store data into distinct groups or partitions based on object metadata characteristics. By dividing the large object store into smaller manageable segments, the system can estimate cardinality for each segment independently and combine results, improving overall estimation accuracy while maintaining computational efficiency for query execution plan generation
Solution Approach 2:
The patent performs preliminary analysis of object metadata and maintains pre-computed statistics about object distributions, sizes, and access patterns. This preliminary action enables faster and more accurate cardinality estimation during query optimization without requiring extensive computation at query execution time, thus improving both accuracy and efficiency
2Productivity
If object cardinality estimation is not performed accurately, then query execution plans can be generated faster, but the cost of accessing objects increases due to inefficient access patterns
Solution Approach 1:
The patent implements feedback mechanisms where actual object access patterns and cardinality results from executed queries are fed back into the system. This feedback refines the cardinality estimation models over time, enabling the query optimizer to generate more efficient execution plans that minimize object access costs while maintaining fast query execution speeds
3Measurement precision
If detailed object statistics are collected and maintained for accurate cardinality estimation, then estimation accuracy improves, but the system complexity increases
Solution Approach 1:
The patent extracts only the most critical and useful object metadata fields and statistics needed for cardinality estimation, rather than collecting and maintaining all possible object attributes. By selecting and maintaining only essential statistics such as object size distributions, access frequencies, and key metadata patterns, the system achieves accurate estimation without excessive system complexity
Data Source
AI summary
In a database system, wherein data is stored as objects within an object storage system, a system and method for estimating object cardinality, determining query execution plan costs, and selecting a query plan for execution by the database system. Multiple object cardinality estimation approaches for estimating the number of objects to be accessed for a given query condition on a column of a relation composed of a set of objects, where each object maintains the minimum value and the maximum value of individual columns are presented. A set of global statistics is also maintained, consisting of the total number of objects and the minimum and maximum values of individual columns. The object cardinality estimation is determined based on the global statistics without retrieving individual object-level statistics.