Trace-Level Microservice Autoscaling for Coordinated SLO Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current autoscaling methods in microservices-based applications set service level objectives (SLOs) at the microservice level, leading to inefficient resource allocation and potential violations, as users experience performance at the aggregated trace level for end-to-end user transactions.
Innovation Solution
A trace-driven, dependency-set-aware proactive coordinated autoscaling system that generates performance-resource elasticity models at the trace level, using machine learning to predict workload levels and recommend microservice replica scaling to meet predefined trace-level SLOs, leveraging column-generation based optimization for resource allocation across microservices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If autoscaling is performed at the microservice level with individual SLOs, then resource scaling can be controlled per service, but resource allocation becomes inefficient and SLO violations occur due to lack of coordination
Solution Approach 1:
The patent merges individual microservice SLOs into a unified trace-level SLO framework. By combining the scaling decisions of multiple microservices into a coordinated trace-level optimization problem, the system achieves efficient resource allocation while maintaining SLO compliance across the entire distributed transaction flow.
Solution Approach 2:
The patent creates a universal trace-level autoscaling mechanism that simultaneously manages multiple microservices. The trace-level SLO framework serves multiple functions: it coordinates scaling across services, optimizes resource allocation holistically, and ensures SLO compliance for the entire distributed transaction, rather than handling each service independently.
2Productivity
If trace-level SLOs are implemented with coordinated autoscaling, then resource allocation efficiency improves, but system complexity increases due to multi-service coordination
Solution Approach 1:
The patent segments the complex trace-level autoscaling problem into manageable components: trace identification, dependency graph construction, elasticity model generation, and coordinated optimization. This segmentation allows the system to handle complexity through structured decomposition while maintaining trace-level coordination.
Solution Approach 2:
The patent introduces an intermediary trace-level coordination layer that mediates between individual microservices and the autoscaling decision-making process. This intermediary framework translates complex multi-service coordination requirements into coordinated scaling actions, reducing the perceived complexity while maintaining efficient resource allocation.
3Reliability
If proactive coordinated autoscaling is implemented, then performance violations are minimized, but computational overhead increases due to elasticity modeling and optimization
Solution Approach 1:
The patent performs preliminary actions by generating elasticity models in advance and using them for proactive scaling decisions. The system pre-computes the relationship between resource allocation and performance metrics, allowing it to make informed scaling decisions before performance violations occur, thereby reducing reactive computational overhead.
Solution Approach 2:
The patent implements feedback mechanisms where actual performance metrics are continuously monitored and fed back into the elasticity models. This feedback loop allows the system to refine its predictions and optimize scaling decisions over time, improving reliability while managing computational overhead through learned patterns rather than exhaustive recalculation.
Data Source
AI summary
A computer-implemented method for trace-driven dependency-set-aware proactive coordinated autoscaling of component microservices in an application includes generating performance-resource elasticity models at a trace-level for traces of the application using dependency set of microservices for each trace. The method predicts workload levels of each of the traces, and also predicts a trace-level performance of the application for different microservice replica scaling based on the dependency set of microservices for each trace, performance-resource elasticity models and the predicted workload levels. The method uses distributed computing to recommend a microservice replica scaling for each of the component microservices to meet one or more predefined trace-level user service level objectives.