Micro-service Scaling via Dependency Graphs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing service mesh systems face challenges in optimizing latency and resource efficiency due to delays in scaling up and down services, which affect overall system performance.
Innovation Solution
A computer-implemented method and distributed multi-services system that uses a directed dependency graph to dynamically scale services by tracing request flows, determining dependent services, and instantiating them together with the selected service, thereby reducing latency and improving resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If services are dynamically scaled up only when requested, then resource efficiency is improved, but latency increases due to instantiation delays
Solution Approach 1:
The system performs preliminary actions by proactively instantiating dependent services before they are actually needed to process requests. When a service is scaled up, the system automatically starts its dependent services in advance based on the dependency graph, so that when requests arrive, all required services are already available, eliminating instantiation delays and reducing latency.
2Productivity
If multiple micro-services are scaled up simultaneously, then request processing capability is improved, but overall latency increases due to multiple scale-up operations
Solution Approach 1:
The system merges the scale-up operations of multiple dependent services into a single coordinated action. When a service needs to be scaled up, the system identifies all its dependent services through the dependency graph and scales them up together in one operation rather than sequentially, reducing the total time required and improving request processing capability.
3Loss of energy
If services are scaled down promptly, then resource efficiency is improved, but system reliability may worsen if scaling down too aggressively
Solution Approach 1:
The system dynamically adjusts service scaling based on real-time monitoring of request patterns and service dependencies. When scaling down, it checks the dependency graph to ensure that scaling a service won't affect the availability of dependent services that still need to handle requests, thus maintaining system reliability while improving resource efficiency.
Data Source
AI summary
A method for reduced latency between software services operating in a service mesh, where the software services are instantiated when fulfilling requests is disclosed. The method comprises providing a plurality of services which fulfill a request, where communication between the services is based on support components, The method also comprises creating a directed dependency graph of the plurality of services by tracing request flows between the plurality of services, thereby nodes of the directed dependency graph represent services and edges of the directed dependency graph represent used communication paths between selected ones of the services, determining a dependent service for an incoming request to a selected one of the plurality of services based on the directed dependency graph, and starting an instance of the dependent service together with the selected one of the plurality of services.