Dynamic Proxy Timeout Adjustment for Microservice Reliability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing proxy servers face challenges in setting optimal timeout values, as fixed values can lead to either premature cancellation of requests due to short timeouts or excessive delays due to long timeouts, especially in complex microservice environments.
Innovation Solution
A computer-implemented method dynamically adjusts timeout values in a proxy server by identifying and mapping microservices, defining REST API calling relationships, building regression models for response times, and predicting response times for incoming API calls to update the timeout values accordingly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a short timeout value is set, then errors can be reported quickly when they occur, but complex requests may be cancelled prematurely before completion
Solution Approach 1:
The patent applies dynamics by transitioning from a static timeout value to a dynamic timeout value that adapts based on request characteristics. The system analyzes historical response time data and request patterns to determine appropriate timeout values for different request types, allowing complex requests to receive longer timeouts while simple requests maintain shorter timeouts for quick error detection.
Solution Approach 2:
The patent implements parameter changes by modifying the timeout parameter based on multiple factors including request complexity, historical performance data, and current system conditions. Different timeout parameters are established for different microservice paths and request types, enabling optimized timeout settings rather than a single universal value.
2Productivity
If a long timeout value is set, then complex requests can complete successfully, but error reporting is delayed substantially
Solution Approach 1:
The patent applies segmentation by dividing the timeout configuration into multiple segments corresponding to different request types, microservice paths, and complexity levels. Each segment has its own optimized timeout value based on historical data and request characteristics, allowing the system to avoid both premature cancellations and excessive delays.
Solution Approach 2:
The patent implements preliminary action by pre-calculating and storing timeout values for different request scenarios based on historical performance data. When a request arrives, the system quickly retrieves the pre-determined timeout value matching the request pattern, avoiding both overly conservative and overly aggressive timeout settings.
3Ease of operation
If fixed timeout values are used, then system operation is simple to manage, but the system cannot adapt to varying request complexities
Solution Approach 1:
The patent applies self-service by enabling the system to automatically determine and adjust timeout values without manual intervention. The system monitors request patterns, analyzes historical response times, and autonomously optimizes timeout configurations for different request types and microservice paths, eliminating the need for manual tuning while maintaining adaptability.
Solution Approach 2:
The patent implements feedback by continuously monitoring request outcomes and response times, then using this information to refine and adjust timeout values. The system learns from actual performance data and adapts timeout configurations to match real-world request patterns and system conditions.
Data Source
AI summary
A method for dynamically adjusting a proxy server timeout includes identifying each microservice within a proxy server and mapping a topology of microservices, assigning each microservice to one of an application layer, a middleware layer and an infrastructure layer. Defining each representational state transfer (REST) application programming interface (API) calling relationship between each microservice and each other microservice. Determining a corresponding regression model defining a response time of each microservice based at least in part on a set of available response time predictors. Building a sequence model for at least one microservice in the application layer. Predicting an incoming REST API call and identifying a probable sequence model corresponding to the predicted incoming REST API call. Updating a timeout value of the predicted REST API call within the proxy server based on the sequentially predicted response times.