Probabilistic Throttling for Cold Storage Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cold storage systems face limitations in data access due to bandwidth constraints, which restrict the usage of non-storage resources and hinder efficient data processing, leading to suboptimal performance in handling multiple requests from clients.
Innovation Solution
A global throttler system is implemented to manage bandwidth assignments across multiple endpoints, allowing endpoints to probabilistically determine the admission of requests based on assigned bandwidth, accrued tokens, and maximum admission latency, ensuring fair distribution and minimizing periods of underutilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If bandwidth limits are imposed on cold storage systems to reduce costs, then storage provider costs are reduced, but data access performance and resource utilization deteriorate
Solution Approach 1:
The system dynamically adjusts bandwidth allocation and admission probabilities based on current system state, accrued tokens, and request characteristics. Instead of static bandwidth limits, the throttler adapts allocation in real-time to balance cost constraints with access performance, allowing flexible resource management that responds to changing conditions.
Solution Approach 2:
The system changes key parameters including admission probability, bandwidth allocation, and token accrual rates based on system state. By adjusting these parameters dynamically, the system optimizes the trade-off between maintaining cost-effective bandwidth limits and preserving data access performance under varying load conditions.
2Loss of energy
If maximum bandwidth limits are enforced globally, then resource cost control is improved, but request handling efficiency and user experience deteriorate
Solution Approach 1:
The system segments bandwidth allocation by assigning specific bandwidth quotas and token buckets to individual users or user groups. This segmentation allows differentiated service levels where different users receive different bandwidth allocations, improving overall request handling efficiency while maintaining global cost control through aggregated bandwidth management.
Solution Approach 2:
The system performs preliminary actions by pre-allocating token buckets and bandwidth quotas to users before they make requests. This advance preparation enables faster request processing decisions at the endpoint without requiring real-time bandwidth calculations, reducing request handling time while maintaining cost control.
3Ease of operation
If bandwidth assignments are made for each endpoint, then fair distribution among endpoints is improved, but system complexity increases
Solution Approach 1:
The global throttler acts as an intermediary between endpoints and bandwidth management. It centralizes the complex logic of bandwidth allocation, token management, and admission control decisions, while endpoints receive simple bandwidth assignments and admission probabilities. This mediator approach distributes fairness efficiently while containing system complexity in a centralized component.
4Productivity
If probabilistic admission control is implemented, then bandwidth utilization efficiency is improved, but decision-making complexity increases
Solution Approach 1:
The system implements self-service through automated probabilistic admission control where the throttler independently makes admission decisions based on pre-configured policies, accrued tokens, and current bandwidth assignments. This eliminates the need for complex manual decision-making or sophisticated algorithms at each endpoint, achieving efficient bandwidth utilization through rule-based automated decisions.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for throttling data probabilistically. One of the methods includes receiving, from a client device for a particular entity, a request to process data, determining a size of data to be processed, providing, to a throttler system, a bandwidth assignment request indicating the particular entity and the size of data to be processed, receiving, from the throttler system, a bandwidth assignment for the particular entity to use when serving the request, and probabilistically determining whether to currently serve the request based on the bandwidth assignment, the size of the data to be processed, and an accrued quantity of tokens for the particular entity.