Query Execution Classifier for Dynamic Engine Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
There is no single optimal combination of query engines and runtimes that suits all queries due to varying query characteristics and system configurations, making it difficult to select the best system for query processing.
Innovation Solution
A computer-implemented method generates a classification model by extracting feature vectors from Directed Acyclic Graphs using Cost-Based Optimizers and labels based on execution times, training the model to predict the optimal execution combination for query processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single combination of query engine and runtime is selected for all queries, then system configuration is simplified, but query performance cannot be optimized for different query characteristics
Solution Approach 1:
The system dynamically selects different query engine and runtime combinations based on query characteristics rather than using a fixed configuration. The classifier model adapts the system configuration in real-time according to the specific query being executed, transforming the static system into a dynamic one that optimizes performance for each query type.
Solution Approach 2:
Different parts of the system (query engine and runtime combinations) are assigned to different query types based on their specific strengths. Each query engine-runtime pair is optimized for particular query characteristics, allowing the system to apply the most suitable configuration locally to each query rather than using a uniform configuration everywhere.
2Productivity
If multiple execution combinations are tested to find the optimal one, then query performance is maximized, but execution time and resource consumption increase
Solution Approach 1:
The system performs preliminary testing and evaluation of multiple execution combinations during the training phase to build a classifier model. This preliminary action captures the performance characteristics of different query engine-runtime pairs, so that during actual query execution, the model can quickly predict the optimal combination without requiring real-time testing, thus avoiding the time loss that would occur from testing multiple combinations for each query.
Solution Approach 2:
Instead of directly testing multiple execution combinations for each query, the system creates a classifier model that copies the knowledge gained from preliminary testing. The model serves as a surrogate that can predict the optimal execution combination based on query characteristics, eliminating the need to actually execute multiple combinations for each query while still achieving performance optimization.
3Adaptability or versatility
If a classification model is trained with multiple execution combinations, then optimal system selection is achieved, but training complexity and computational overhead increase
Solution Approach 1:
The training process is segmented into distinct phases: extracting features from query execution plans, labeling execution combinations based on actual performance, and training the classifier model separately. This segmentation allows each component to be developed and optimized independently, reducing the overall training complexity while maintaining the ability to handle multiple execution combinations.
Data Source
AI summary
A method is provided for generating a classification model configured to select an optimal execution combination for query processing. The method provides, to a processor, training queries and different execution combinations for executing the training queries. Each different execution combination involves a respective different query engine and a respective different runtime. The method extracts, from a set of Directed Acyclic Graphs (DAGs) using a set of Cost-Based Optimizers (CBOs), a set of feature vectors for each of the plurality of training queries. The method adds, by the processor to each of merged feature vectors a respective label indicative of the optimal execution combination based on actual respective execution times of the plurality of different execution combinations, to obtain a set of labels. The method trains, by the processor, the classification model by learning the set of merged feature vectors with the set of labels.