Distributed Database Compute Allocation With Dynamic Token Buckets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In cloud computing systems with distributed data analysis engines, resource allocation can lead to latency issues when one user consumes most of the shared computing resources, affecting other users' queries.
Innovation Solution
Implement a token bucket algorithm to allocate a certain number of tokens per request, adjusting limits based on actual usage, and evict requests if insufficient tokens are available to maintain fair resource access across users.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If multiple users share computing resources in a distributed database system, then resource utilization efficiency is improved, but query latency increases when one user consumes most of the shared resources
Solution Approach 1:
The patent segments the shared computing resources into discrete token units that can be individually allocated and tracked. Each user's resource consumption is segmented into token buckets that are independently managed, allowing the system to control and limit individual user consumption while maintaining overall high resource utilization across multiple users.
Solution Approach 2:
The patent implements dynamic resource allocation through token bucket algorithms that adjust resource limits based on actual usage patterns. The system dynamically modifies token refill rates and bucket capacities in real-time, allowing resource allocation to adapt to changing workload conditions and prevent any single user from monopolizing resources, thereby maintaining both high utilization and low latency.
2Productivity
If a user runs multiple concurrent queries, then productivity is improved, but other users experience increased latency due to resource exhaustion
Solution Approach 1:
The patent implements feedback mechanisms through token bucket algorithms that continuously monitor each user's resource consumption and provide real-time feedback by adjusting token availability. When a user's concurrent queries approach resource limits, the system dynamically reduces token refill rates or evicts requests, providing immediate feedback that prevents resource exhaustion and protects other users from latency while still allowing high productivity within allocated limits.
3Reliability
If resource allocation limits are set strictly to prevent latency, then fairness is improved, but overall system productivity decreases
Solution Approach 1:
The patent resolves this contradiction by making resource allocation dynamic rather than static. Token bucket parameters such as refill rates and bucket capacities are adjusted in real-time based on actual system conditions, user behavior patterns, and available resources. This allows the system to enforce fairness and prevent latency when necessary while maximizing throughput when resources are available, achieving both reliability and productivity simultaneously.
Solution Approach 2:
The patent changes allocation parameters dynamically based on system state. Instead of fixed resource limits, the system modifies token bucket parameters (refill rates, bucket sizes, eviction thresholds) in response to changing conditions such as total system load, individual user patterns, and available capacity. This allows the system to adapt between enforcing strict fairness limits and allowing high productivity based on real-time conditions.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present disclosure provides a method for managing allocation of compute resources in a distributed database. Each request on every task is allocated a certain number of tokens when it arrives, such as by using a token bucket algorithm. This initial allocation may be the per-request maximum usage permitted in the system. This request is made to a central quota server in the region to coordinate allocations between tasks. As the task runs, the system evaluates how much of the limit the request is using, and adjusts its limit up or down to keep the quota allocation tight on the actual usage of the request. If there aren't enough tokens to keep the request active, the request can be evicted.