Cloud Throttle Control via Enqueue and Dequeue Counters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multi-tenancy cloud systems, without proper throttle control, unintended preferential treatment of service requests from certain customers or categories can lead to dissatisfaction and performance imbalances, causing underserved customers and potential service outages.
Innovation Solution
Implementing an enqueue counter to monitor unfinished requests and a dequeue counter to monitor executing requests per customer, ensuring that service requests are managed to prevent exceeding these counters, thereby balancing computing load and preventing abuse.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a cloud system continuously allocates computing resources to service requests from certain customers, then resource allocation efficiency is improved, but fairness among customers deteriorates causing underserved customers and potential service outages
Solution Approach 1:
The patent segments service requests into different queues based on customer identity and request category. Each customer is assigned a dedicated queue with independent throttle control, allowing the system to manage resources for different customers separately. This segmentation enables fair resource distribution while maintaining efficient processing for each customer's requests.
Solution Approach 2:
The patent implements feedback mechanisms through enqueue counters and dequeue counters that continuously monitor the number of pending and executing requests for each customer. When counters approach thresholds, the system adjusts resource allocation dynamically, providing feedback to prevent any single customer from monopolizing resources while ensuring fair treatment across all customers.
2Reliability
If a cloud system continuously allocates computing resources to larger scale data write requests, then important tasks are completed, but smaller scale data read requests become timed out due to resource contention
Solution Approach 1:
The patent segments service requests by category (e.g., data read, data write) and assigns different throttle thresholds to each category. This allows the system to independently control resource allocation for different request types, ensuring that critical write requests receive adequate resources while preventing read requests from being indefinitely delayed.
Solution Approach 2:
The patent implements dynamic throttle control where the system continuously monitors enqueue and dequeue counters and adjusts resource allocation in real-time. When write requests are processed, the system dynamically modifies available resources for read requests, preventing resource exhaustion while maintaining fair access for all request categories.
3Stability of the object's composition
If the total number of service requests in the waiting queue is controlled, then system stability is improved, but the ability to handle peak demand deteriorates
Solution Approach 1:
The patent implements dynamic throttle control where enqueue thresholds and dequeue thresholds can be adjusted based on system load and demand patterns. During peak demand periods, the system can temporarily increase thresholds to accommodate higher request volumes, while maintaining stability during normal operation. This dynamic adjustment enables the system to adapt to varying demand conditions.
Solution Approach 2:
The patent establishes predefined enqueue thresholds and dequeue thresholds that are set in advance based on system capacity and service level agreements. These preliminary thresholds provide a framework for stable queue management while allowing for configured flexibility to handle peak demand through threshold adjustment mechanisms.
Data Source
AI summary
Systems and methods for throttle control on cloud-based computing tasks are provided. An example method includes, obtaining a service request from a first user, in a plurality of users, of the computer system; in accordance with a first determination that placing the service request in a service queue associated with the first user would not cause an enqueue counter associated with the first user to be exceeded, causing the service request to be placed in the service quest to await execution. The method also includes, after the service request is placed in the service queue, in accordance with a second determination that executing the service request would not cause a dequeue counter associated with the first user to be exceeded, causing the service request to be executed.