Network Call Management Framework for Mitigating API Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In large enterprise environments, shared servers hosting multiple applications experience response latency issues due to downstream dependent applications experiencing delays, which adversely affect critical applications and overall network call response times, as non-critical applications' delays cause queuing and slow down critical network calls.
Innovation Solution
A network call management framework that continuously monitors response times of downstream dependent applications, automatically detecting slow APIs and temporarily 'short-circuiting' non-critical network calls to prevent queuing and ensure critical calls are not delayed, while providing a user interface to visualize and manage the status of dependent applications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If servers scale-up with multiple network call threads to accommodate high volume of users, then the capacity to handle user requests is improved, but the response time deteriorates when downstream applications slow down
Solution Approach 1:
The patent introduces a circuit breaker component as an intermediary layer between the server application and downstream dependent applications. This circuit breaker monitors response times and automatically opens the circuit when downstream applications become unresponsive, preventing request accumulation and protecting the server from cascading failures. The circuit breaker acts as a mediator that isolates the server from the problematic downstream applications while maintaining system stability.
2Adaptability or versatility
If non-critical applications are allowed to process network calls without restrictions, then their operational flexibility is improved, but critical applications experience delays due to queuing
Solution Approach 1:
The patent implements differential treatment of network calls based on their criticality. Critical network calls are given priority processing with guaranteed response times, while non-critical calls are subject to circuit breaker restrictions. The system dynamically adjusts the behavior of the circuit breaker based on the criticality level of each call, ensuring that critical applications maintain operational flexibility while non-critical applications bear the burden of circuit breaker limitations when downstream applications are unhealthy.
3Reliability
If servers continuously monitor and attempt to process all network calls, then thoroughness of processing is improved, but system resource consumption increases and response time deteriorates
Solution Approach 1:
The circuit breaker performs preliminary actions by monitoring downstream application health and proactively opening the circuit before critical failures occur. Instead of continuously attempting to process all network calls and dealing with failures afterward, the system preemptively stops sending requests to unhealthy downstream applications based on response time thresholds. This preliminary action prevents resource waste on doomed requests and maintains system responsiveness.
Data Source
AI summary
A resilient network call management framework that automatically detects slow Application Programming Interfaces (APIs) at downstream dependent applications based on constantly measuring and monitoring the response time coming from such applications. Once a slow API has been identified, the framework provides the capability to temporarily shut down (otherwise referred to herein as, “short-circuit”) the network call path associated therewith until the monitoring determines that the response time is back to an acceptable level (i.e., returns to a healthy status). Implemented in conjunction with a shared services server hosting different service applications, processing by the framework may be limited to only those service applications or network calls deemed non-critical, such that, critical service applications or network calls are not subjected to being shut down regardless of response time.