Robust Cost-Based Scheduling for Cloud Database Query Prioritization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cost-aware query scheduling algorithms in database systems assume exact query execution times, which is unrealistic, leading to suboptimal performance due to imprecision in execution time estimation, especially in dynamic and concurrent workloads.
Innovation Solution
The robust cost-based scheduling (rCBS) system determines priority scores in real-time under imprecise query execution times using dynamic convex hull algorithms and probabilistic models, evaluating all possible execution times and their costs, allowing for efficient handling of multiple queries with different SLAs without relying on exact execution times.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If exact query execution time is assumed to be known beforehand, then scheduling decisions can be optimized, but this assumption is unrealistic and leads to poor performance when execution time estimation is imprecise
Solution Approach 1:
The patent changes the parameter representation from a single deterministic execution time value to a probability distribution function. This allows the scheduling algorithm to work with ranges of possible execution times and their probabilities, rather than relying on an imprecise single-point estimate, thereby resolving the contradiction between measurement precision and scheduling performance
Solution Approach 2:
The patent introduces dynamic convex hull algorithms that can efficiently update and query priority scores as queries are added or removed from the system. The use of probabilistic models and dynamic data structures allows the system to adapt to changing workload conditions while maintaining optimal scheduling decisions, addressing the performance issue when exact execution times are unknown
2Ease of operation
If mean or max execution time estimation is used, then scheduling decisions can be made, but these estimations mislead scheduling decisions and result in bad performance
Solution Approach 1:
The patent transforms the execution time parameter from simple scalar estimates (mean or max) to a full probability distribution function. This change enables the scheduling algorithm to consider the complete range of possible execution times and their probabilities, leading to more accurate priority calculations and better overall performance while maintaining ease of operation through automated probabilistic modeling
3Reliability
If robust cost-based scheduling considering all possible execution times is implemented, then scheduling quality improves, but computational complexity increases
Solution Approach 1:
The patent employs dynamic convex hull algorithms that maintain an optimized data structure of query priority scores. When queries are added or removed, the algorithm efficiently updates the convex hull rather than recalculating from scratch, achieving near-optimal scheduling robustness with manageable computational complexity through incremental updates
Solution Approach 2:
The patent uses probabilistic models and histogram-based representations to approximate the full probability distribution of query execution times. This copying approach creates a simplified yet sufficiently accurate representation that captures the essential characteristics of execution time variability without requiring exhaustive analysis of all possible scenarios, thereby reducing computational complexity while maintaining scheduling quality
Data Source
AI summary
Systems and methods for prioritizing queries in a cloud based database by determining a priority score in real time under an imprecise query execution time; and incrementally maintaining query priority scores.