Service Mesh Latency Prediction Routing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing service mesh solutions fail to account for variations in network performance, leading to performance bottlenecks and inefficient routing of service requests across distributed microservices, especially in dynamic networks with wireless or lower bandwidth connections.

Innovation Solution

A method and system for monitoring and predicting network latency of each network link in a service mesh, allowing service requests to be routed to the service instance with the lowest predicted total latency, which includes both network and processing latency, and dynamically selecting the optimal network link based on performance characteristics such as capacity and activity.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If service requests are routed based on overall latency statistics without distinguishing network and processing latency, then routing decisions are simple to implement, but network performance variations cause bottlenecks and inefficient routing

Engineering Contradiction:

Improverouting efficiencyVSAvoidmonitoring complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent segments the overall latency measurement into two distinct components: network latency (measured by time-stamping requests at the proxy) and processing latency (measured by the service instance). This segmentation allows the system to identify and optimize for network performance variations without increasing overall system complexity, as each component measures only what it is responsible for.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The service proxy acts as an intermediary that introduces time-stamps into service requests to measure network latency. This intermediary mechanism enables the separation of network and processing latency measurements without requiring complex instrumentation throughout the entire system, thereby improving routing efficiency while maintaining manageable monitoring complexity.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If service meshes are deployed on hybrid cloud/edge networks with dynamic network performance, then cost-effective scalability and higher bandwidth are achieved, but network performance variations cause bottlenecks

Engineering Contradiction:

Improvenetwork flexibilityVSAvoidrouting reliability

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The patent implements dynamic routing by continuously monitoring network latency measurements from multiple service instances and using this information to update routing policies. The control plane distributes updated routing policies to proxies based on current network conditions, allowing the service mesh to adapt to dynamic network performance variations in hybrid cloud/edge environments while maintaining reliable service delivery.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system establishes a feedback loop where network latency measurements from service proxies are collected by the control plane, which then uses this feedback information to generate updated routing policies. This feedback mechanism ensures that routing decisions are continuously optimized based on actual network performance, improving both adaptability and reliability in dynamic hybrid networks.

Inventive Principle:

Principle #23Feedback

3Productivity

If routing policies are distributed to service proxies to enable dynamic routing, then load balancing is improved, but the system assumes homogenous service mesh with high bandwidth connections

Engineering Contradiction:

Improveload balancing efficiencyVSAvoidnetwork heterogeneity handling

Core Design Contradiction:

ProductivityVSAdaptability or versatility

Solution Approach 1:

The patent applies local quality by having each service proxy measure and report its specific network latency to the control plane, rather than assuming uniform network conditions across the entire service mesh. The control plane then uses these localized measurements to generate routing policies that are tailored to the specific network characteristics of different regions, enabling effective load balancing in heterogeneous network environments.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The system changes the routing parameter from static, assumption-based routing to dynamic, measurement-based routing. By continuously monitoring network latency and using this information to adjust routing decisions, the system adapts to heterogeneous network conditions without requiring changes to the underlying service mesh architecture, thereby improving load balancing efficiency across diverse network environments.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS11240146B2Service request routing

Publication Date: 2022.02.01 KK TOSHIBA

AI summary

A computer implemented method of routing service requests to service instances in a service mesh. The method comprises monitoring one or more performance characteristics of each of a plurality of network links to service instances. The method further comprises making a prediction of the network latency of transmitting a service request via each of at least two of the plurality of network links and selecting one of the plurality of service instances and routing the service request thereto based on the predictions of the network latencies.