Dynamic Cluster Scaling for Low-Delay Cloud Task Queues

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing cloud computing architectures face inefficiencies due to the need to continuously monitor and provision new clusters for processing tasks, leading to slow processing speeds and increased costs when clusters are underutilized or idle.

Innovation Solution

A method for dynamically scaling cloud resources by identifying idle or underutilized processing resources and assigning them to tasks without provisioning new resources, while continuously monitoring and scaling down resources when not needed, thereby optimizing resource utilization.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If new clusters are provisioned to handle new processing tasks, then processing capacity is increased, but provisioning time causes task queue delays

Engineering Contradiction:

Improveprocessing capacityVSAvoidtask queue delay

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The system pre-provisions multiple processing clusters in advance and maintains them in a ready state before tasks arrive. When tasks are submitted, the scheduler immediately assigns them to pre-available clusters without waiting for provisioning, thereby eliminating task queue delays while maintaining high processing capacity

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system dynamically adjusts cluster allocation based on real-time task queue conditions and cluster utilization metrics. Clusters are activated or deactivated dynamically according to workload demands, allowing the system to maintain optimal processing capacity while minimizing idle resource time

Inventive Principle:

Principle #15Dynamics

2Loss of energy

If clusters are terminated when idle to reduce costs, then resource utilization is optimized, but termination time causes delays for subsequent tasks

Engineering Contradiction:

Improveresource costVSAvoidcluster activation delay

Core Design Contradiction:

Loss of energyVSLoss of time

Solution Approach 1:

The system pre-emptively maintains clusters in a warm standby state with minimal resources allocated, allowing them to be quickly activated when needed. This preliminary preparation ensures that clusters can be brought online in minutes rather than the several minutes required for full provisioning, reducing both cost and activation delay

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system changes operational parameters of clusters by adjusting resource allocation levels rather than fully terminating them. Clusters transition between different operational states (fully active, partially active, standby) based on workload demands, enabling quick reactivation while optimizing cost efficiency

Inventive Principle:

Principle #35Parameter changes

3Reliability

If minimum number of nodes are maintained in clusters, then cluster availability is ensured, but costs increase when clusters are underutilized

Engineering Contradiction:

Improvecluster availabilityVSAvoidresource cost

Core Design Contradiction:

ReliabilityVSLoss of energy

Solution Approach 1:

The system creates a pool of shared processing clusters that can be dynamically allocated to multiple different tasks and clients. Instead of each client maintaining dedicated minimum-node clusters, the shared pool serves universal processing needs, reducing total resource requirements while maintaining availability through efficient resource sharing and allocation

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The system allows clusters to be temporarily deactivated or scaled down when not in use, then quickly recovered and activated when processing demands arise. This discarding and recovering approach replaces the need to continuously maintain minimum node counts, reducing costs during low-utilization periods while ensuring availability when needed

Inventive Principle:

Principle #34Discarding and recovering

Data Source

PatentUS12468574B2Systems and methods for dynamically scaling remote resources

Publication Date: 2025.11.11 CAPITAL ONE SERVICES LLC

AI summary

Systems and methods for dynamically selecting idle or underutilized resources to complete tasks in a queue are disclosed. The systems and methods include maintaining a plurality of processing resources operable to process one or more tasks. Each resource is scalable to increase or decrease a number of nodes available to perform the one or more tasks. The systems and methods include maintaining a queue for tasks to be processed, receiving a first task requiring a processing resource, and accessing at least a portion of the plurality of processing resources. A first processing resource of the plurality of processing resources is identified that is operating below a predetermined processing threshold. The first processing resource is assigned to the first task and scaled up according according to a processing requirement of the first task.