Intelligent Microservice Scaling via Predictive Request Metrics

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In microservice-based distributed systems, downstream services often fail to scale timely, leading to cascading failures and bottlenecks due to the reliance on observed metrics for autoscaling, which results in delayed responses or service crashes, especially for stateful services where reconfiguration takes significant time.

Innovation Solution

Implementing an intelligent scaling method that analyzes request metrics and attribute templates to pre-scale target services before incoming requests, ensuring they are ready to handle load by the time requests arrive, thereby avoiding cascading failures and bottlenecks.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If downstream services scale based on observed metrics, then the system can respond to actual load, but the scaling is delayed and causes bottlenecks and cascading failures

Engineering Contradiction:

Improveservice stabilityVSAvoidscaling response time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent applies preliminary action by analyzing service call chains and request metrics to predict future load on downstream services, then triggering scaling operations in advance before the load actually arrives. This predictive approach eliminates the delay inherent in reactive metric-based scaling, ensuring services are ready to handle incoming requests immediately.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements feedback by continuously monitoring request metrics and service call chains, using this information to dynamically adjust scaling decisions. The feedback loop enables the system to learn from actual traffic patterns and improve prediction accuracy, resolving the contradiction between timely response and reliable scaling.

Inventive Principle:

Principle #23Feedback

2Productivity

If stateful services are scaled out, then service capacity increases, but reconfiguration takes significant time causing delays

Engineering Contradiction:

Improveservice capacityVSAvoidreconfiguration time

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

For stateful services, the patent triggers scaling operations in advance of predicted load increases, allowing the reconfiguration process to complete before requests arrive. This preliminary action ensures that even though reconfiguration takes time, the service is already configured and ready to immediately handle incoming traffic at the required capacity.

Inventive Principle:

Principle #10Preliminary action

3Adaptability or versatility

If microservices are highly decoupled and independent, then flexibility and language choice improve, but coordination for scaling becomes complex

Engineering Contradiction:

Improveprogramming language flexibilityVSAvoidscaling coordination complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent introduces an intermediary scaling coordination mechanism that analyzes service call chains and automatically determines scaling requirements for downstream services. This intermediary layer handles the complexity of coordination between independent microservices, allowing each service to remain decoupled and flexible while ensuring proper scaling synchronization through automated metric analysis and prediction.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS11281510B2Intelligent scaling in microservice-based distributed systems

Publication Date: 2022.03.22 KYNDRYL INC

AI summary

In an approach to intelligent scaling in a cloud platform, an attribute template is stored for one or more target services based on one or more system data. One or more request metrics for each target service is stored, wherein the request metrics are based on an analysis of one or more incoming requests of one or more service call chains. Responsive to receiving a request for a target service in a service call chain, the target service is scaled based on the attribute template of the target service and the request metrics of the target service.