API Concurrency Management via Segmented Rate Limiting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing amount of concurrent requests from users/developers in cloud-based telephony platforms can impede data traffic and communications between servers and applications, leading to a degradation of the user/developer experience.
Innovation Solution
A system and method for managing concurrent events in telephony platforms, which includes a controller, an account database, and an API cluster. This system monitors, partitions, allocates, distributes, and manages dedicated API resources to ensure efficient use and prevent excessive requests from one account or resource type from adversely impacting others.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If large amounts of data are transmitted between servers and applications to support cloud-based telephony services, then user access to data and communication capabilities is improved, but data traffic congestion and system performance degradation occur
Solution Approach 1:
The patent segments API resources into different types (asynchronous, synchronous, long-running, short-running) and implements separate rate limiting for each type. This segmentation allows the system to manage different data traffic patterns independently, preventing congestion in one resource type from affecting others, thus maintaining overall system productivity while providing versatile user access.
Solution Approach 2:
The patent applies different rate limiting strategies to different API resource types based on their specific characteristics. Asynchronous resources have different rate limits compared to synchronous resources, and long-running resources have different limits compared to short-running resources. This local quality approach optimizes data traffic efficiency for each resource type while maintaining overall system adaptability.
2Adaptability or versatility
If concurrent requests from multiple users/developers are allowed to access the system, then system versatility and user experience are improved, but excessive requests from one account can impede communications and slow down the system
Solution Approach 1:
The patent implements preliminary rate limiting checks before processing API requests. By monitoring and limiting the number of concurrent requests from each account before they fully execute, the system prevents any single account from overwhelming the system. This preliminary action ensures that concurrent request handling remains versatile while maintaining communication efficiency and system reliability.
Solution Approach 2:
The patent implements monitoring and feedback mechanisms that track concurrent request rates from each account. When rate limits are approached or exceeded, the system provides feedback by rejecting additional requests or throttling them. This feedback loop allows the system to maintain high versatility for concurrent requests while ensuring reliability by preventing any single account from impeding overall communication efficiency.
3Reliability
If dedicated API resources are allocated to specific accounts to prevent interference, then system reliability is improved, but device complexity and resource management overhead increase
Solution Approach 1:
The patent implements a universal rate limiting framework that handles multiple API resource types through a single, unified system. Rather than creating separate complex management systems for each account or resource type, the universal framework applies consistent rate limiting principles across all resources. This approach maintains account isolation and reliability while minimizing the complexity overhead of the resource management system.
Data Source
AI summary
A system and method that includes receiving an API request to a type of API resource; retrieving an API concurrency value for the API request; determining a comparison status associated with a comparison of the API concurrency value to a concurrency threshold; if the comparison status is within the concurrency threshold, transmitting the API request to an API processing resource; if the comparison status indicates the concurrency threshold is not satisfied, impeding processing of the API request; accounting for an increase in the API concurrency value if the API request is transmitted to an API processing resource; and accounting for a decrease in the API concurrency value at a time associated with the API processing resource completing processing of the API request.