Workload Request Rate Control via Priority-Based Throttling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data store systems face challenges in managing workload prioritization and concurrency limits, leading to delays and skewed efficiency metrics due to the buildup of delay queues during surges in incoming requests, which affects the timely execution of critical workloads and accuracy of performance metrics.
Innovation Solution
A data store system that determines the workload type for each request and adjusts execution rates dynamically, releasing requests based on priority levels to manage concurrency and prevent queue overflow, ensuring timely execution and accurate metrics by controlling the flow of requests through rate-based scheduling and throttling mechanisms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If workload requests are throttled to control execution rate, then concurrency limits are enforced and queue size is managed, but request latency increases and timely execution of critical workloads is delayed
Solution Approach 1:
The patent applies different throttle rates based on workload priority levels. Critical workloads with higher priority receive higher throttle rates (less throttling) while non-critical workloads receive lower throttle rates (more throttling). This local differentiation allows the system to manage queue size for non-critical workloads while ensuring timely execution for critical workloads, resolving the contradiction between queue management and latency prevention.
Solution Approach 2:
The system dynamically adjusts throttle rates based on real-time queue depth monitoring. When queue depth exceeds thresholds, the system increases throttling for non-critical workloads and reduces throttling for critical workloads. This dynamic adaptation allows the system to respond to changing workload conditions, maintaining both queue size control and timely execution for critical requests.
2Quantity of substance
If delay queue becomes overly long during request surge, then incoming requests are buffered, but efficiency metrics become skewed and difficult to obtain accurate performance data
Solution Approach 1:
The patent extracts critical workloads from the general delay queue by implementing separate queue management for different priority levels. Critical workloads are processed separately with higher priority, ensuring they are not diluted by non-critical workloads in the same queue. This extraction allows accurate efficiency metrics to be calculated for critical workloads without being skewed by the presence of non-critical workloads.
Solution Approach 2:
The system segments the delay queue into multiple priority-based queues rather than using a single unified queue. This segmentation allows the system to track and measure performance metrics for each priority level separately, preventing skewing of overall efficiency metrics. Each queue can be monitored independently, providing accurate performance data for critical workloads while still buffering non-critical requests.
3Reliability
If priority levels are assigned to workload types and sources, then critical workloads are identified for preferential execution, but system complexity increases due to additional rules and management overhead
Solution Approach 1:
The patent changes the parameter of throttle rate based on workload priority levels. Instead of complex routing logic, the system uses a simplified parameter adjustment approach where throttle rates are pre-configured for different priority levels and dynamically adjusted based on queue depth. This parameter-based approach maintains reliable critical workload execution while reducing management complexity compared to multi-rule priority systems.
Data Source
AI summary
A data store system may include a storage device configured to store a plurality of data store tables and may include a processor in communication with the storage device. The processor may receive a plurality of requests. For each request, the processor may: (1) determine an associated workload type for the request; (2) determine a first respective rate at which the request is to be released for scheduling of execution; and (3) release the request for scheduling of execution based on the first respective rate. For each released request, the processor may: (1) determine a second respective rate based on the associated workload type at which each released request is scheduled to be executed; and (2) in response to execution being scheduled for a released request, execute the released request. A method and computer-readable medium are also disclosed.