Multi-Dimensional Token Bucket for Cloud Resource Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current resource limitation techniques in cloud computing environments, such as the token bucket algorithm, are not suitable for distributed, multi-tenant scenarios as they fail to provide fine-grained control over resource allocation at the user or tenant level, leading to potential denial of service issues due to inadequate resource management.
Innovation Solution
Implementing a multi-dimensional token bucket mechanism with a hierarchy of token buckets (global, tenant, and user levels) that allows for dynamic resource allocation and rebalancing, enabling interactive communication between token management components and applications to manage resources more effectively across multiple buckets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a simple token bucket algorithm is used for resource limitation, then resource allocation is simple to implement, but fine-grained control at user or tenant level is not achieved
Solution Approach 1:
The patent segments the single token bucket into multiple hierarchical levels: global bucket, tenant buckets, and user buckets. This segmentation enables fine-grained control at different levels while maintaining manageable complexity through the hierarchical structure. Each level manages tokens independently, allowing precise control over resource allocation to specific users and tenants.
Solution Approach 2:
The patent introduces a hierarchical dimension to the token bucket structure, transforming a flat single-level system into a multi-level hierarchy. This dimensional change allows the system to provide both coarse-grained global control and fine-grained user-level control simultaneously, resolving the contradiction between simplicity and adaptability.
2Device complexity
If a single token bucket is used for all servers, then resource management is centralized and simple, but distributed multi-tenant awareness is not achieved
Solution Approach 1:
The patent divides the single centralized token bucket into multiple segmented buckets organized hierarchically: one global bucket for overall resource management, multiple tenant-level buckets for isolating tenant resources, and multiple user-level buckets for controlling individual user allocations. This segmentation ensures that one tenant's resource exhaustion cannot affect other tenants, providing multi-tenant awareness and protection.
Solution Approach 2:
The patent introduces intermediate tenant-level buckets that act as mediators between the global resource pool and individual user buckets. These intermediate structures enable the system to provide both centralized global control and distributed tenant-level protection, resolving the contradiction between simplicity and reliability in multi-tenant environments.
3Adaptability or versatility
If dedicated token buckets are assigned to each server instance, then resource control at server level is achieved, but user-level or tenant-level limitation is not possible
Solution Approach 1:
The patent merges multiple independent token buckets into a unified hierarchical structure where global, tenant, and user buckets work together through defined relationships. Instead of managing completely separate buckets at each level, the system combines them into an integrated hierarchy where tokens flow from global to tenant to user levels, achieving user-level control without proportionally increasing management complexity.
Solution Approach 2:
The hierarchical token bucket structure serves multiple functions simultaneously: it provides global resource management, tenant isolation, user-level control, and automatic resource allocation. This multi-functionality allows the system to achieve fine-grained control at various levels without requiring separate independent systems for each function, thereby managing complexity efficiently.
4Reliability
If resource limits are enforced strictly at all times, then resource exhaustion is prevented, but service flexibility and scalability are reduced
Solution Approach 1:
The patent implements dynamic resource limitation through the hierarchical token bucket system where token allocation and limits can be adjusted at different levels (global, tenant, user) based on current system conditions and service requirements. This dynamic approach allows the system to prevent resource exhaustion while maintaining flexibility to scale services by adjusting token rates and allocations without rigid fixed limits.
Data Source
AI summary
Certain example embodiments relate to techniques for dynamic resource use limitations in a cloud computing environment. A service request from a user is received, in connection with a first application process of the application processes executing in the environment. A resource strategy based on the received service request is generated in connection with the first application process. The resource strategy specifies at least one resource shared by the application processes and an amount of the at least one resource for use by the first application process to subsequently perform a service requested. In connection with a resource controller process different from the first application process, a determination is made regarding whether the generated resource strategy is feasible. Either the service is performed (e.g., when the resource strategy is feasible), or the resource strategy is revised and re-submitted to the resource controller process (e.g., when the resource strategy is infeasible).