Database Workload Prediction Using Machine Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database workload forecasts often fail to accurately predict peak workloads despite normal resource utilization levels, leading to inefficiencies and potential database performance issues.
Innovation Solution
Utilize machine learning-based predictions that consider active and waiting sessions of database workloads, employing a ratio of total time spent on these sessions, and apply historical data to forecast future workloads, with dynamic threshold detection for anomalies and breaking points.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional database workload forecasting methods are used, then resource utilization appears normal, but peak workloads cannot be accurately predicted
Solution Approach 1:
The system performs preliminary actions by continuously collecting and storing historical workload data, session metrics, and database performance information before peak workloads occur. This historical data is then used to train machine learning models that can predict future peak workloads, allowing the database system to take preventive actions before congestion occurs.
Solution Approach 2:
The system implements feedback mechanisms by monitoring actual database performance metrics (response time, throughput, resource utilization) and comparing them against predicted values from machine learning models. This feedback loop allows the system to refine its predictions and adjust resource allocation dynamically to maintain performance during peak workloads.
2Reliability
If automated actions are implemented based on workload forecasts, then database performance is improved, but system complexity increases
Solution Approach 1:
The database system performs self-service by automatically monitoring its own workload, generating forecasts using integrated machine learning models, and executing corrective actions without external intervention. The system self-adjusts resource allocation, identifies congestion patterns, and optimizes performance based on its own historical data and predicted future state.
Solution Approach 2:
The machine learning model serves multiple functions: it predicts peak workloads, identifies anomalies, optimizes resource allocation, and provides recommendations for capacity planning. This multi-functionality reduces the need for separate specialized systems, thereby managing complexity while improving performance.
3Measurement precision
If historical workload data is collected and analyzed, then prediction accuracy improves, but data processing requirements increase
Solution Approach 1:
The system extracts and isolates the most critical features from vast amounts of historical data, such as session duration, query types, resource utilization patterns, and temporal patterns. By focusing on these key features rather than processing all raw data, the system reduces processing volume while maintaining high prediction accuracy.
Solution Approach 2:
The system transforms raw historical data into meaningful parameters and features through aggregation, normalization, and transformation processes. Historical raw data is converted into standardized metrics like workload intensity, session completion rates, and resource consumption patterns, which are then used for training models and reducing data processing requirements.
Data Source
AI summary
Techniques are provided for database control using machine learning based prediction. One method includes obtaining first and second sets of designated database metrics characterizing active sessions and waiting sessions, respectively, of a database workload for one or more designated time intervals; evaluating an amount of work performed by the active and waiting sessions of the database workload within a designated time interval; applying historical workload data of the database workload to a machine learning model to obtain a forecasted amount of work performed by the database workload within a subsequent designated time period; and initiating an automated action using the forecasted amount of work. The active sessions may utilize one or more database resources and the waiting sessions may execute in response to an occurrence of a designated event.