Flow Controller Throttling Web API Service Rate
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing solutions for throttling the rate of service in Web APIs often require consumers to implement error-handling code, penalize overactive consumers, or incur unpredictable costs, and are not scalable to accommodate changing provider models.
Innovation Solution
A method and system that utilize a flow controller to queue incoming requests from consumers at a fixed rate determined by their licensed service rate, using a first-in-first-out model, independent of other consumers, to manage the rate of service without rejecting requests or penalizing consumers, thereby maintaining predictable and controlled usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing throttling solutions are implemented, then service rate control is achieved, but consumers must implement error-handling code and face unpredictable costs
Solution Approach 1:
The patent introduces a flow controller as an intermediary component between consumers and the Web API service. This flow controller automatically manages request throttling by maintaining queues for each consumer and regulating request flow according to licensed rates, eliminating the need for consumers to implement their own error-handling code for rate limiting while ensuring predictable service rate control.
2Productivity
If existing throttling solutions are implemented, then service rate control is achieved, but consumers are penalized for overactive usage
Solution Approach 1:
The flow controller performs preliminary actions by pre-allocating queues to each consumer and proactively regulating request flow before rate violations occur. By monitoring and controlling the rate of requests sent to the Web API in advance, the system prevents overactive usage before it happens, eliminating the need for punitive measures against consumers.
3Productivity
If manual throttling approaches are used, then service rate can be controlled, but the system is not scalable to accommodate changing provider models
Solution Approach 1:
The flow controller implements segmentation by maintaining separate queues for each consumer, allowing independent rate control for each consumer based on their licensed rate. This segmented approach enables the system to scale to accommodate multiple consumers with different service models and rate limits, while the modular queue-based architecture allows easy adaptation to changing provider requirements.
Data Source
AI summary
A mechanism is provided in a data processing system for automatically throttling the rate of service provided by a Web application programming interface (API) for a software service. A flow controller executing on the data processing system assigns a queue to each consumer of the software service. Responsive to receiving a current request for the software service from a given consumer of the software service, a flow controller executing on the data processing system adds the current request to a given queue assigned to the given consumer. The flow controller sends a next request from the given queue to the Web API based on a licensed rate of service of the given consumer.