Query Database Selection Under Fluctuating System Load
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems fail to achieve faster query execution due to fluctuating computing resource availability, despite queries being executed on specialized databases, as they do not account for relative system load when selecting the optimal database for query execution.
Innovation Solution
Implement a machine learning model, such as a decision tree, to predict the optimal database for query execution based on system load and query characteristics, using features like query complexity and current load information, enabling low-latency selection without full optimization of each database.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a query is executed on the database specialized for processing it (row-major for transactional, column-major for analytical), then query execution speed is improved, but system load fluctuations cause suboptimal performance when the specialized database is overloaded
Solution Approach 1:
The system dynamically selects between row-major and column-major databases based on real-time system load conditions rather than using a fixed specialization rule. The query router monitors current system state and adapts the database selection decision, allowing the system to switch from the statically optimized path to an alternative path when load conditions deteriorate, thus maintaining performance consistency
Solution Approach 2:
A machine learning-based query router acts as an intermediary between the query submission and database execution. This intermediary evaluates system load, query characteristics, and historical performance data to predict the optimal database selection, mediating between the query's ideal execution path and the actual system capacity to ensure both speed and reliability
2Measurement precision
If full optimization is performed for each database to ensure accurate query execution selection, then selection accuracy is improved, but computational overhead and latency increase
Solution Approach 1:
Instead of performing complete optimization analysis for each database candidate, the system applies partial optimization by using a machine learning model that captures the most critical selection factors (system load, query type, database specialization). This partial approach achieves sufficient accuracy for practical decision-making while avoiding the excessive computational cost of exhaustive optimization, thus reducing selection latency
Solution Approach 2:
The system performs preliminary analysis by pre-training machine learning models on historical query performance data and system load patterns. This preliminary action creates a predictive framework that can quickly evaluate new queries without performing full optimization analysis in real-time, thereby maintaining high selection accuracy while minimizing selection latency during actual query execution
Data Source
AI summary
Disclosed herein are techniques related to load-aware selection of query execution databases. Techniques may include receiving a query to be executed on a database selected from a plurality of databases. Each database of the plurality of databases may have a different data layout. The techniques may also include generating, for the received query, one or more query features and one or more current load features. The one or more current load features may indicate availability of one or more computing resources for query execution on least one database of the plurality of databases. Additionally, the techniques may include generating an output based on applying a model to the one or more generated query features. The techniques may further include selecting, from the plurality of databases and based on the output, a database for executing the query. Thus, the query may be executed on the selected database.