ML-Based Query Execution Time Estimator for RDBMS
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Estimating the execution time of queries in relational database management systems (RDBMS) is challenging due to various factors, making it difficult to meet service level goals (SLGs) and preventing timely resource allocation.
Innovation Solution
A method that uses machine learning (ML) models trained with past execution data to predict execution times for query execution plan steps, and a boosting module that adjusts resource allocation based on predicted execution times to meet SLGs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional execution time estimation methods are used, then the system can process queries with simpler factors, but the accuracy of execution time prediction deteriorates due to multiple impacting factors
Solution Approach 1:
The patent introduces a machine learning model as an intermediary component between the query execution plan and the resource allocation system. This ML model acts as a mediator that processes multiple complex factors (query complexity, system load, resource availability, lock conditions) and transforms them into accurate execution time predictions, resolving the contradiction by handling complexity internally while providing precise predictions to the user.
Solution Approach 2:
The patent replaces traditional mechanical/mathematical estimation methods with a machine learning-based predictive system. Instead of using deterministic formulas that struggle with multiple variables, the system uses trained ML models that can capture complex non-linear relationships between various factors and execution time, achieving higher prediction accuracy without increasing apparent system complexity.
2Reliability
If resource allocation is adjusted dynamically to meet SLG, then query execution can meet service level goals, but the complexity of resource management increases
Solution Approach 1:
The patent implements a feedback mechanism where execution time predictions from the ML model are continuously fed back to the resource allocation system. This feedback loop enables dynamic adjustment of resource allocation (CPU shares, I/O shares, memory) to ensure SLG achievement. The boosting module uses prediction results to automatically adjust resources, maintaining reliability while managing complexity through automated closed-loop control.
Solution Approach 2:
The patent performs preliminary resource allocation adjustments based on predicted execution times before queries actually execute. By using ML predictions to pre-determine required resource levels, the system can proactively allocate resources to meet SLG requirements, avoiding the need for complex real-time reactive adjustments during query execution.
3Duration of action of moving object
If more resources are allocated to ensure SLG, then query execution time can be reduced, but resource utilization efficiency deteriorates
Solution Approach 1:
The patent dynamically changes resource allocation parameters (CPU shares, I/O shares, memory allocation) based on ML-predicted execution times and actual query performance. Instead of static over-provisioning, the system adjusts parameters in real-time to match actual needs, reducing execution time for critical queries while maintaining resource utilization efficiency by not over-allocating to queries that don't need additional resources.
Data Source
AI summary
A method, apparatus and computer program product for estimating resource consumption for steps in a query execution plan for a query performed by a relational database management system (RDBMS) in a computer system. Past execution data for the steps are used to train a machine learning (ML) model and its model parameters to predict execution times for the steps. A prediction module comprised of the ML model configured by the model parameters predicts an execution time for a current step of the query execution plan for the query, based on current step information and current system load. A boosting module boosts the current step either up or down for processing by the RDBMS to meet a service level goal (SLG) for the query, based on the predicted execution time for the current step, as well as an elapsed query time, a query SLG time, and/or a query CPU time.


