Dynamic Application Scaling via Queue Depth Monitoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing dynamic application scaling systems fail to accurately assess workload processing requirements, leading to inefficient scaling of application instances based on CPU load, memory usage, and front-end requests, which can result in suboptimal performance.
Innovation Solution
A method for dynamically scaling application instances based on queue depth, where the number of instances is adjusted according to observed queue depth values and scaling configuration parameters, including thresholds, scale-up and scale-down intervals, and minimum/maximum instance limits, to ensure optimal processing capacity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If dynamic application scaling is performed based on CPU load, memory usage, and front-end requests, then the system can respond to workload changes, but the scaling accuracy is insufficient leading to suboptimal performance
Solution Approach 1:
The patent changes the scaling parameter from composite metrics (CPU load, memory usage, front-end requests) to a direct queue depth metric. This parameter change enables more accurate workload assessment because queue depth directly reflects the actual processing backlog that applications must handle, providing a precise measure of scaling needs without the noise and indirectness of other metrics.
2Productivity
If the number of application instances is increased to handle higher workload, then processing capacity improves, but resource waste occurs when scaling is not accurately matched to actual needs
Solution Approach 1:
The patent implements a feedback mechanism where the scaling system continuously monitors queue depth values and uses this information to dynamically adjust the number of application instances. This closed-loop feedback ensures that scaling decisions are based on actual processing needs reflected in the queue, preventing both over-provisioning (resource waste) and under-provisioning (insufficient capacity) by constantly adapting to the current workload state.
3Adaptability or versatility
If scaling decisions are made frequently to respond to workload changes, then adaptability improves, but system instability occurs due to oscillating scaling actions
Solution Approach 1:
The patent applies preliminary action by requiring the queue depth to exceed the scaling threshold for a predetermined time period before triggering a scale-up event. This advance condition prevents premature or reactive scaling decisions that could lead to oscillations. By requiring the threshold to be sustained for a specific duration, the system ensures that scaling actions are taken only when the workload increase is persistent, thereby maintaining stability while still responding to genuine workload changes.
Data Source
AI summary
Techniques for performing application scaling may include: receiving a plurality of observed queue depth values for a queue, wherein the queue includes data packets to be processed by a first set of one or more applications of a first application type; determining, in accordance with the observed queue depth values and scaling configuration parameters for the first application type, whether to modify the number of applications currently in the first set, wherein the scaling configuration parameters includes a scaling threshold; and responsive to determining to modify the number of application currently in the first set, performing first processing that modifies the number of applications currently in the first set. Modifying the number of applications in the first set may include scaling up or scaling down the number of instances in the first set.