Query Workload Forecasting with Template-Based Neural Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database management systems struggle to accurately forecast evolving query workloads due to their dynamic and time-evolving nature, leading to ineffective recommendations for physical design features and resource allocation, as conventional methods fail to predict exact future queries and their parameters.
Innovation Solution
A template-based featurization approach combined with an encoder-decoder deep learning framework is employed to forecast future query workloads, utilizing a cut and pack technique to optimize model capacity and efficiency, enabling precise prediction of entire query statements and arrival times.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If historical query traces are used as input workloads for physical design tools, then the tools can automatically recommend physical design features like indexes and materialized views, but the recommendations are not always effective for future workloads because real workloads are dynamic and exhibit time-evolving behavior
Solution Approach 1:
The patent applies dynamics by transforming the static historical workload into a dynamic forecasted workload that captures time-evolving patterns. The system uses machine learning models to predict future query workloads, enabling physical design recommendations to adapt to changing workload characteristics rather than relying on static historical data.
Solution Approach 2:
The patent applies preliminary action by forecasting future query workloads before actual queries arrive. The system predicts upcoming query patterns and uses these forecasts as input to physical design tools, allowing recommendations to be prepared in advance for future workloads rather than reacting to past data.
2Productivity
If views are recommended based on queries with old predicate parameter values, then view selection can be optimized for historical data, but the views may not be useful for future queries if the predicate parameters change over time
Solution Approach 1:
The patent applies dynamics by using forecasted query workloads with predicted parameter values instead of static historical parameter values. The machine learning model predicts future predicate parameters, allowing view recommendations to be dynamically adjusted to match upcoming query patterns rather than being fixed based on past data.
Solution Approach 2:
The patent applies parameter changes by transforming old predicate parameter values from historical queries into forecasted parameter values that reflect future workload characteristics. The system uses these changed parameter values to generate view recommendations that are adapted to future query conditions rather than historical ones.
3Device complexity
If database optimization techniques assume static workloads, then the implementation is simpler, but the optimization techniques are ineffective for evolving workloads in modern data-driven applications
Solution Approach 1:
The patent applies the intermediary principle by introducing a workload forecasting system as a mediator between historical query traces and physical design tools. This intermediary component predicts future workloads and transforms them into appropriate input formats for optimization tools, bridging the gap between static optimization methods and dynamic workload requirements without fundamentally redesigning the optimization pipeline.
Data Source
AI summary
Systems and methods for characterizing and forecasting evolving query workloads. The method includes receiving a query, the received query including a parameter value and an arrival time; identifying the query as a recurrent query; extracting a query template from the received query by parsing the received query; based at least on the identifying, generating a feature vector for the received query, the feature vector generated based on the extracted template and the parameter value; and forecasting a future query based on the generated feature vector by applying a neural network.