Container Load Detection via Service Metrics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Container systems relying on traditional CPU and RAM monitoring for load balancing and scaling are inadequate for detecting high loads in services dependent on external API calls, where bottlenecks may occur even with stable resource usage, leading to inefficiencies and difficulties in tracing and debugging.
Innovation Solution
A system and method that programmatically determines a software container's load using a load probe handler and probe function definitions, allowing for scaling based on custom criteria such as queue size or unit of work characteristics, independent of traditional resource consumption, to dynamically add or remove replicas in response to changing demands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional CPU and RAM monitoring is used for load detection, then resource usage can be tracked, but service load independent bottlenecks cannot be detected
Solution Approach 1:
The patent changes the monitoring parameter from traditional resource-based metrics (CPU, RAM) to service-specific metrics such as queue size, number of active connections, and custom business logic indicators. This allows the system to detect load conditions that are independent of resource consumption, enabling accurate load detection for services where bottlenecks occur despite stable resource usage.
2Productivity
If scaling is based on resource thresholds, then resource allocation can be optimized, but service responsiveness cannot be improved during resource-stable bottlenecks
Solution Approach 1:
The patent implements a feedback mechanism where the load probe handler continuously monitors service-specific load indicators and provides real-time feedback to the orchestration monitor. This feedback loop enables the system to detect when a service is approaching its processing capacity independent of resource usage, triggering proactive scaling actions that maintain service responsiveness while optimizing resource allocation.
3Ease of operation
If load balancing is based on traditional metrics, then resource distribution can be managed, but bottlenecks in API-dependent services cannot be traced
Solution Approach 1:
The patent introduces a load probe handler as an intermediary component that sits between the service and the orchestration monitor. This intermediary captures service-specific load indicators such as queue size and active connections, translating them into metrics that can be monitored and used for load balancing decisions. This intermediary layer enables bottleneck tracing in API-dependent services without complicating resource management operations.
Data Source
AI summary
A computer implemented method and related system determine a current load result of a software container executing on a compute node in a container system. In response to determining that the current load result exceeds a predetermined scale-up threshold for the software container, the method adds a first plurality of replicas of the software container to the compute node, where a quantity of the first plurality of replicas is related to the current load result. In response to determining that the current load result is less than a predetermined scale-down threshold for the software container, the method deletes a second plurality of replicas of the software container from the compute node, where a quantity of the second plurality of replicas is related to the current load result.