Machine Learning Database Cost Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database management systems face challenges in accurately modeling and optimizing query execution times and resource usage, particularly in selecting the best query plan to meet Service Level Agreements (SLAs), due to difficulties in estimating the cost and resource needs of query execution plans.
Innovation Solution
A machine-learning driven database management system that uses a trained machine-learning algorithm to predict database operations, modifying database constructs and resource allocations to improve query processing efficiency, identify problematic queries, and meet SLGs by providing accurate resource usage and cost predictions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manually derived formulas are used to estimate query cost and cardinality, then the query optimizer can select a least cost plan, but the accuracy of cost estimation is insufficient to meet Service Level Agreements
Solution Approach 1:
The patent replaces manual cost estimation formulas with a machine learning-based prediction system. The ML model is trained on historical query execution data to learn complex patterns and relationships that manual formulas cannot capture, thereby improving cost estimation accuracy and enabling better SLA adherence.
Solution Approach 2:
The system implements feedback by using actual query execution results to continuously train and refine the machine learning model. This feedback loop allows the system to improve its cost estimation accuracy over time by learning from real-world performance data, directly addressing the insufficiency of static manual formulas.
2Productivity
If the database system uses complex performance optimization techniques, then query execution efficiency can be improved, but the system complexity and difficulty of management increase
Solution Approach 1:
The patent implements self-service by enabling the database system to automatically optimize query execution using machine learning predictions. The system autonomously selects optimal execution plans and adjusts resource allocation based on ML-generated cost estimates, eliminating the need for manual intervention and reducing operational complexity while maintaining high execution efficiency.
Solution Approach 2:
The system dynamically adjusts execution parameters based on machine learning predictions. Instead of using fixed manual formulas, the system changes cost estimation parameters adaptively by incorporating ML model outputs, allowing efficient query execution while simplifying the optimization process through automated parameter adjustment.
3Adaptability or versatility
If resource allocation is adjusted manually based on ad hoc needs, then specific workload requirements can be met, but the process becomes challenging when parameters are numerous or conflicting workload requirements exist
Solution Approach 1:
The patent introduces dynamics by replacing static manual parameter adjustment with dynamic machine learning-driven resource allocation. The ML model continuously adapts resource allocation based on changing workload characteristics and performance patterns, enabling the system to handle numerous parameters and conflicting requirements automatically without manual intervention.
Solution Approach 2:
The system automatically changes resource allocation parameters based on ML predictions of workload requirements. Instead of manual adjustment of numerous parameters, the system dynamically modifies parameters such as query execution resources and scheduling priorities based on learned patterns, making the system adaptable while easy to operate.
Data Source
AI summary
A machine-learning driven Database Management System (DBMS) is provided. One or more machine-learning algorithms are trained on the database constructs and execution plans produced by a database optimizer for queries. The trained machine-learning algorithms provide predictors when supplied the constructs and plans for a given query. The predictors are processed by the DBMS to make resource, scheduling, and Service Level Agreement (SLA) compliance decisions with respect to the given query.