Database Query Scheduling via Elapsed Time Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional scheduling techniques for online systems, such as database systems, often allocate disproportionate resources to consumers based on priority, leading to inaccurate allocation and inefficient resource utilization.
Innovation Solution
The system schedules queries by maintaining consumer groups and using accumulated elapsed time and resource allocation ratios to determine resource distribution, adjusting processing rates based on these metrics to ensure fair resource usage across consumer groups.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional priority-based scheduling is used, then high-priority consumers receive more resources, but resource allocation becomes disproportionate and inaccurate
Solution Approach 1:
The system implements feedback by continuously monitoring the elapsed time of query execution for each consumer group and using this information to dynamically adjust resource allocation. The scheduler compares the actual resource consumption (measured by elapsed time) against the allocated resources and modifies scheduling decisions to correct any discrepancies, ensuring accurate and proportional resource distribution over time.
Solution Approach 2:
The scheduling system transitions from static priority-based allocation to dynamic allocation based on real-time elapsed time measurements. The resource allocation ratios are adjusted dynamically according to the monitored execution times, allowing the system to adapt to changing workload conditions and consumer group requirements, thereby achieving both accuracy and efficiency.
2Speed
If disproportionate resources are allocated to some consumers, then those consumers get faster processing, but other consumers experience resource starvation
Solution Approach 1:
The system applies equipotentiality by ensuring that all consumer groups receive fair and proportional access to resources over time. By monitoring elapsed time and adjusting allocation ratios, the system creates a balanced state where no consumer group is systematically favored or disadvantaged, eliminating resource starvation while maintaining overall system throughput and processing speed.
3Reliability
If accumulated elapsed time is monitored and used for scheduling, then fair resource distribution is achieved, but scheduling complexity increases
Solution Approach 1:
The scheduling system implements self-service by automatically monitoring its own resource allocation effectiveness through elapsed time measurements and autonomously adjusting scheduling decisions to maintain fairness. This self-regulating mechanism eliminates the need for external intervention or complex manual configuration, achieving fair resource distribution while keeping the system relatively simple and self-managing.
Data Source
AI summary
A system, for example, an online system or a database system schedules requests received for execution. The system maintains consumer groups and receives a measure of amount of resources allocated for each consumer group. The system maintains a measure of accumulated elapsed time of execution of requests received from each consumer group. If the system determines that resources are available for executing a request, the system selects a request for execution from a consumer group based on the measure of accumulated elapsed time for processing queries for the consumer group and the measure of amount of resources allocated for the consumer group. The system maintains relative cumulative elapsed times of queries across consumer groups in the same ratio as the allocated share for the consumer groups. Accordingly, the system schedules requests such that each consumer group approaches its allocation of resources compared to other groups smoothly over time.