Capacity-Based Scaling for Queue Services
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional cloud computing scaling systems oscillate wildly in scaling service processors, leading to inefficient resource utilization and higher costs due to their reliance on message queue size alone, causing unnecessary tie-up of resources to maintain quality of service.
Innovation Solution
A capacity-based scaling system that measures both the number of messages in a queue and the input rate, along with processing capacity, to determine when to scale up or down service processors, ensuring efficient resource allocation and minimizing unnecessary resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional scaling systems scale service processors based on queue size alone, then quality of service is maintained, but resource utilization becomes inefficient and costs increase
Solution Approach 1:
The patent changes the scaling parameters from queue size alone to a composite metric incorporating both queue size and input rate. This allows the system to maintain quality of service while avoiding unnecessary scaling actions, thereby improving resource utilization efficiency and reducing costs.
Solution Approach 2:
The system implements feedback mechanisms that continuously monitor both queue size and input rate, using this information to dynamically adjust the number of service processors. This feedback loop prevents wild oscillations and enables more stable, efficient resource allocation while maintaining service quality.
2Reliability
If conventional scaling systems keep a minimum number of service processors instantiated, then quality of service is maintained during load spikes, but computing resources are unnecessarily tied up and costs increase
Solution Approach 1:
The patent makes the service processor instantiation dynamic by introducing the input rate as an additional factor. Instead of maintaining a static minimum number of processors, the system dynamically adjusts processor count based on real-time conditions (queue size and input rate), allowing resources to be released when not needed while still handling load spikes effectively.
3Ease of manufacture
If scaling systems use simple rules based on queue size, then implementation is straightforward, but the number of service processors oscillates wildly
Solution Approach 1:
The patent adds the input rate parameter to the scaling decision process. This additional parameter provides more nuanced information about system state, allowing the scaling algorithm to make more informed decisions that reduce wild oscillations while maintaining reasonable implementation complexity.
Data Source
AI summary
Capacity-based scaling of queue-based resources is described. Initially, a scaling system measures capacity of service processors that are instantiated at a cloud computing service system to provide a service on behalf of a service provider, and also measures a load on these processors. In contrast to conventional scaling systems—which base scalings on a number of queued messages which the instantiated service processors process to provide the service—the scaling system measures the load in terms of not only the number of messages held in a queue but also an input rate of the messages to the queue. The described scaling system then determines whether and by how much to scale the instantiated processors based on this number of messages and input rate. Given this, the scaling system instructs the cloud computing service system how to scale the instantiated service processors to provide the service.