Token Bucket Admission Control for Burst-Mode Workload Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems face challenges in managing workload variations, particularly in multi-tier e-commerce systems, as they struggle to handle differing types and rates of service requests effectively, leading to potential overloads and reduced customer satisfaction.
Innovation Solution
Implementing a token-based admission control mechanism that uses provisioned-capacity and burst-mode token buckets to manage work requests, allowing for dynamic adjustment of throughput capacity and prioritization of requests based on available tokens, thereby preventing overloads and optimizing resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a fixed or varying global threshold is used to compare request rate and selectively refuse service, then the system can prevent overload, but it cannot account for differences in work amount for different service request types and instances
Solution Approach 1:
The patent segments the workload management system into multiple independent components: token buckets for different service request types, individual counters for each request instance, and separate threshold evaluations. This segmentation allows the system to track and manage workload for each request type independently, enabling differentiated handling of various service requests while maintaining overall overload prevention through aggregate threshold monitoring.
2Ease of operation
If a single global threshold is defined for request rate limiting, then the system can simplify control logic, but it becomes difficult or impossible to define a meaningful threshold in systems with varying request types and unpredictable work amounts
Solution Approach 1:
The patent applies preliminary action by pre-defining multiple service-specific thresholds for different request types before the system operates. These thresholds are established in advance based on expected workload characteristics for each service type. During operation, the system automatically selects and applies the appropriate pre-defined threshold based on the incoming request type, eliminating the need for complex real-time threshold calculation while maintaining meaningful workload limits for each service category.
3Productivity
If services are designed to work best with uniformly distributed client requests over time, then optimal performance can be achieved, but in practice workloads are rarely uniformly distributed and may be non-uniform with respect to time and data set
Solution Approach 1:
The patent implements dynamics by making the request admission decision process adaptive to changing workload conditions. The system dynamically evaluates each incoming request against service-specific thresholds and current workload state, allowing the throughput capacity to flexibly adjust to actual demand patterns. This dynamic approach enables the system to maintain optimal performance during uniform workloads while automatically adapting to handle non-uniform distributions in both time and data set access patterns.
4Reliability
If the system accepts all service requests without selective refusal, then customer satisfaction can be maximized, but the system may become overloaded and unable to provide expected quality of service
Solution Approach 1:
The patent applies local quality by implementing service-specific threshold evaluations rather than a blanket system-wide limit. Each service type has its own threshold and admission criteria, allowing the system to selectively accept or refuse requests based on the specific service being accessed and current workload conditions for that service. This enables the system to maintain high acceptance rates for services with available capacity while protecting overloaded services, thereby maintaining quality of service without unnecessarily rejecting requests.
Data Source
AI summary
Methods and apparatus for token-sharing mechanisms for burst-mode operations are disclosed. A first and a second token bucket are respectively configured for admission control at a first and a second work target. A number of tokens to be transferred between the first bucket and the second bucket, as well as the direction of the transfer, are determined, for example based on messages exchanged between the work targets. The token transfer is initiated, and admission control decisions at the work targets are made based on the token population resulting from the transfer.