Microservice API Rate Limiting Based on Aggregate Service Load
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems fail to dynamically adjust rate limits for API requests based on the load of micro-services, leading to inefficiencies and potential service disruptions for power users who exceed predefined thresholds.
Innovation Solution
A system that adjusts rate limits for API requests based on the overall load of micro-services, allowing for dynamic increases or decreases in response to current service demands, thereby optimizing resource allocation and preventing overload.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a fixed rate limit is imposed on API requests, then service overload is prevented, but system efficiency and resource utilization deteriorate when load is low
Solution Approach 1:
The patent implements dynamic rate limiting by continuously monitoring the load states of multiple micro-services and adjusting the rate limit parameters in real-time. The system transitions from a static, pre-defined rate limit to a dynamic mechanism that adapts to current system conditions, allowing the rate limit to increase when services are underutilized and decrease when overload risks are detected.
Solution Approach 2:
The system establishes a feedback loop where the load states of micro-services are continuously monitored and fed back to the rate limiting mechanism. This feedback enables the system to automatically adjust rate limits based on actual service performance and demand patterns, creating a closed-loop control system that optimizes both reliability and productivity.
2Productivity
If rate limits are increased to improve productivity, then resource utilization improves, but service disruption risk increases during high demand
Solution Approach 1:
The system dynamically adjusts rate limits based on real-time monitoring of micro-service load states. When services are operating below capacity, rate limits are increased to improve resource utilization. When load approaches critical thresholds, rate limits are automatically reduced to prevent service disruption, creating a balanced adaptive control mechanism.
Solution Approach 2:
The patent changes the rate limit parameter dynamically based on system conditions. Instead of using a fixed parameter value, the system adjusts the rate limit parameter in response to changing load states, enabling optimization of both productivity and reliability through parameter adaptation.
3Productivity
If dynamic rate limit adjustment is implemented, then system efficiency improves, but system complexity increases
Solution Approach 1:
The system segments the rate limiting function into distinct components: load monitoring, state evaluation, and rate limit adjustment. Each micro-service's load state is monitored independently, and rate limits are adjusted based on aggregated system state. This segmentation makes the complex dynamic control more manageable and maintainable.
Solution Approach 2:
The patent introduces an intermediary rate limiting layer that sits between API requests and micro-services. This intermediary handles the complexity of dynamic adjustment logic, isolating it from both the request handlers and service implementations. The intermediary translates complex load state information into simple rate limit decisions.
Data Source
AI summary
A service providing system executes connection to a micro-service according to a specific request in a case where the service providing system has received the specific request, which is a specific application programming interface (API) request assigned specific identification information, and where the number of times of reception of the API request assigned the specific identification information has not exceeded a specific rate limit. The specific identification information is identification information of a specific client. The specific rate limit is a rate limit associated with the specific identification information. The service providing system updates the specific rate limit in accordance with a whole load of all the micro-services that operate in response to the specific request in a case where the service providing system has received the specific request.