Cloud Cluster Scaling via Message Threshold Monitoring

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current cloud-based multitenant computing systems face challenges in scaling virtual computer clusters to achieve optimal performance and reduce costs, as existing methods do not account for variations in workload, leading to underutilization or overprovisioning of resources.

Innovation Solution

Implement dynamic scaling of virtual computer clusters based on real-time processing of steady-state and non-steady-state jobs, automatically adjusting the number of nodes to match message volumes, using cloud computing services like AWS to minimize costs and ensure high performance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If virtual computer clusters are statically provisioned to handle peak workload, then service reliability is improved, but resource utilization deteriorates during low-demand periods

Engineering Contradiction:

Improveservice reliabilityVSAvoidresource utilization

Core Design Contradiction:

ReliabilityVSLoss of energy

Solution Approach 1:

The patent implements dynamic scaling of virtual computer clusters by continuously monitoring workload metrics (CPU utilization, memory usage, message queue depth) and automatically adjusting the number of active nodes. This transforms the static provisioning model into a dynamic one where cluster size adapts in real-time to match actual demand, resolving the contradiction between maintaining peak-capability and avoiding idle resource waste

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system establishes a feedback loop where workload metrics are continuously measured and fed back to the scaling controller, which then adjusts cluster size accordingly. This closed-loop control ensures that the system automatically responds to changing conditions, maintaining service reliability during peaks while reducing resource consumption during low-demand periods

Inventive Principle:

Principle #23Feedback

2Loss of energy

If virtual computer clusters are dynamically scaled to match workload, then resource utilization is improved, but system complexity deteriorates

Engineering Contradiction:

Improveresource utilizationVSAvoidsystem complexity

Core Design Contradiction:

Loss of energyVSDevice complexity

Solution Approach 1:

The scaling system operates autonomously by automatically monitoring workload metrics, determining scaling decisions, and executing node provisioning or termination without manual intervention. This self-service approach manages the inherent complexity of dynamic scaling through automation, allowing resource utilization to be optimized while the complexity is handled by the self-managing system rather than requiring complex manual coordination

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent introduces a scaling controller as an intermediary component that mediates between workload monitoring and cluster provisioning. This intermediary abstracts the complexity of dynamic scaling logic, coordinating the interaction between monitoring agents, cloud provisioning services, and cluster nodes, thereby managing system complexity while enabling effective resource utilization

Inventive Principle:

Principle #24Intermediary (Mediator)

3Loss of energy

If cloud computing services are used for virtual clusters, then cost efficiency is improved, but performance consistency deteriorates during scaling operations

Engineering Contradiction:

Improvecost efficiencyVSAvoidperformance consistency

Core Design Contradiction:

Loss of energyVSReliability

Solution Approach 1:

The system performs preliminary actions by pre-configuring scaling policies, thresholds, and node templates before scaling events occur. When scaling is needed, pre-defined configurations and approved node images are rapidly deployed, reducing the time and performance impact of scaling operations. This preliminary preparation maintains performance consistency while utilizing cost-effective cloud services

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent implements cushioning mechanisms by maintaining buffer capacity in the cluster and using gradual scaling approaches. During scaling operations, the system cushions performance impacts by staggering node additions/removals and maintaining minimum operational capacity, ensuring that performance consistency is preserved even while utilizing variable-cost cloud computing resources

Inventive Principle:

Principle #11Beforehand cushioning (Prior cushioning)

Data Source

PatentUS11321139B2Streaming traffic pattern for public cloud auto scaling

Publication Date: 2022.05.03 SALESFORCE INC

AI summary

A total number of messages in a set of messages that are processed by a computing node in a virtual computer cluster during a time interval is determined. The virtual computer cluster is deployed with a cloud computing service and includes the computing node and other computing nodes at an end time of the time interval. It is determined whether the total number of messages in the set of messages processed by the computing node is no less than a maximum per-interval message number threshold. In response to determining that the total number of messages is no less than a maximum per-interval message number threshold, the cloud computing service is caused to start an additional computing node in the virtual computer cluster after the time interval, wherein the additional computing node is used to process one or more subsequent messages to be processed after the time interval.