Routing Service Concurrency Limits for Fault Isolation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing routing services face inefficiencies due to static call limits that do not adapt to changing machine capabilities and traffic patterns, leading to potential system unhealthiness and slow request processing.

Innovation Solution

A dynamic limit model is generated to adjust call limits based on machine capabilities and traffic patterns, ensuring efficient operation and isolating unhealthy dependency services.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If static call limits are used to control dependency service calls, then system simplicity is maintained, but the routing service cannot adapt to changing machine capabilities and traffic patterns, leading to reduced productivity and potential system unhealthiness

Engineering Contradiction:

Improveadaptability to machine capabilities and traffic patternsVSAvoidcomplexity of call limit management

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent implements dynamic call limits that automatically adjust based on real-time machine capabilities and traffic patterns. Instead of static thresholds, the system continuously monitors dependency service response times, error rates, and machine resource utilization to dynamically modify call limits for each dependency service, enabling adaptation to changing conditions without manual intervention

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system establishes a feedback loop where the routing service continuously monitors the health and performance of dependency services, using this information to adjust call limits. The feedback mechanism tracks metrics such as response time, error rates, and throughput, and automatically modifies calling behavior to maintain system health while optimizing performance based on current conditions

Inventive Principle:

Principle #23Feedback

2Productivity

If call limits are increased to handle higher traffic, then productivity improves, but unhealthy dependency services may be overwhelmed, worsening system reliability

Engineering Contradiction:

Improverequest processing throughputVSAvoiddependency service health

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent applies different call limits to different dependency services based on their individual health status, capacity, and performance characteristics. Each dependency service receives a customized call limit rather than a uniform threshold, allowing the system to maximize throughput to healthy services while protecting vulnerable services from overload, thereby simultaneously improving productivity and maintaining reliability

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The system proactively monitors dependency service health metrics and adjusts call limits before services become overwhelmed. By detecting early signs of stress such as increasing response times or error rates, the system pre-reduces call limits to prevent service failure, cushioning against potential reliability issues before they impact overall system productivity

Inventive Principle:

Principle #11Beforehand cushioning (Prior cushioning)

3Measurement precision

If manual adjustment of call limits is performed, then control precision is improved, but the system loses adaptability to automatic changes in machine capabilities and traffic patterns

Engineering Contradiction:

Improvecontrol precision of call limitsVSAvoidautomatic adaptation capability

Core Design Contradiction:

Measurement precisionVSExtent of automation

Solution Approach 1:

The routing service automatically monitors its own performance metrics, machine capabilities, and dependency service health status to self-adjust call limits without external intervention. The system uses built-in monitoring agents that collect data on resource utilization, traffic patterns, and service performance, then automatically recalculate and apply optimized call limits, achieving both precision and automation

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS20260081868A1Normalized concurrency limits for throttling and fault isolation in a routing service

Publication Date: 2026.03.19 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

The processing capabilities of the machines in a routing service are evaluated and traffic patterns indicative of how calls are made to different dependency services are identified. The capabilities of the routing service and the traffic patterns are used to generate a dynamic limit model that dynamically limits the number of calls made to each dependency service. When the capabilities of the routing service change, the dynamic limit model automatically adjusts the limit corresponding to each dependency service.