Dynamic Microservice Admission with Continuously Trained Demand Models

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing microservice management techniques in clustered computing systems face inefficiencies due to inaccurate or static predictions about future demands, leading to excessive resource consumption or latency, particularly in dynamic and stochastic environments.

Innovation Solution

Implementing a continuously trained microservices lifecycle predictive model that distinguishes between short-term and long-term demand predictions, using unsupervised learning to dynamically prioritize microservice admissions and resource allocation based on actual in-situ scenarios.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If microservices are pre-loaded and readied for potential future calls, then service availability is improved, but resource consumption increases

Engineering Contradiction:

Improveservice availabilityVSAvoidresource consumption

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The system performs preliminary actions by pre-loading microservices only when demand is predicted to be high in the near future. The demand prediction model analyzes patterns to determine when pre-loading will be beneficial, triggering preliminary service preparation only under specific predicted conditions rather than universally, thus balancing availability improvement with resource conservation.

Inventive Principle:

Principle #10Preliminary action

2Quantity of substance

If microservices are loaded lazily until actual call occurs, then resource consumption is reduced, but service latency increases

Engineering Contradiction:

Improveresource consumptionVSAvoidservice latency

Core Design Contradiction:

Quantity of substanceVSLoss of time

Solution Approach 1:

Inventive Principle:

Principle #10Preliminary action

3Device complexity

If demand predictions are made without continuous model training, then system complexity is reduced, but prediction accuracy deteriorates

Engineering Contradiction:

Improvesystem complexityVSAvoidprediction accuracy

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

The demand prediction model performs self-service through continuous automated training using actual observed demand data from the microservice platform. The system automatically collects data, retrains the model, and updates predictions without manual intervention, maintaining high prediction accuracy while minimizing operational complexity through self-managing automation.

Inventive Principle:

Principle #25Self-service

4Measurement precision

If short-term and long-term demand predictions are handled separately, then prediction accuracy is improved, but computational overhead increases

Engineering Contradiction:

Improveprediction accuracyVSAvoidcomputational overhead

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The system segments demand prediction into distinct short-term and long-term models, each optimized for its specific time horizon. The short-term model captures immediate demand fluctuations while the long-term model identifies broader trends, allowing each segment to be trained and executed independently with appropriate computational resources, improving overall prediction accuracy without requiring a single computationally intensive unified model.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS12445529B2Microservice admission control based on learned demand predictions

Publication Date: 2025.10.14 NUTANIX INC

AI summary

A dynamically-managed microservices platform. The microservices platform is configured to selectively accept admission of microservices and to selectively throttle microservices based on a continually-trained machine learning model. The system is configured to implement multiple microservice lifecycle strategies, where individual ones of the multiple microservice lifecycle strategies use a prediction model having long-term and short term demand predictions. One or another of the multiple microservice lifecycle strategies are invoked at different times under different conditions. Under a first set of conditions, a prediction model is used to select a first strategy to admit or throttle a microservice based upon long-term or short-term demand predictions. Under a second set of conditions, the prediction model is used to select a different strategy to admit or throttle a microservice. The prediction model is continually trained such that new sets of observations and new sets of predicted outputs are codified into the machine learning model.