Statistical Distributed Rate Limiting for Low-Latency API Gateways
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing API Gateway clusters face high computational overhead and increased latency due to the need for continuous communication among nodes to implement rate limiting policies, which is inefficient and inaccurate.
Innovation Solution
Implement a statistical distributed rate limiting algorithm using physics-based formulas for average velocity, acceleration, and distance to estimate the total number of API requests across the cluster, allowing nodes to make decisions locally without frequent synchronization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If API Gateways continuously communicate status among cluster nodes to implement rate limiting policies, then rate limiting accuracy is improved, but computational overhead increases and latency is greatly increased
Solution Approach 1:
The patent divides the cluster into regions and assigns region identifiers to requests. Each API Gateway node independently tracks rate limits for its own region rather than continuously communicating with all other nodes. This segmentation allows nodes to make rate limiting decisions locally without frequent synchronization, reducing latency while maintaining accuracy through periodic status exchanges at predetermined intervals.
Solution Approach 2:
Instead of continuous communication, the patent implements periodic status exchange among cluster nodes at predetermined time intervals. Each node shares its current status (request counts, region identifiers) with other nodes at these scheduled intervals. This periodic action reduces the frequency of computational overhead and network communication while still maintaining accurate rate limiting information across the cluster.
2Measurement precision
If API Gateways continuously communicate status among cluster nodes to implement rate limiting policies, then rate limiting accuracy is improved, but computational overhead increases
Solution Approach 1:
The patent segments the rate limiting problem by region identifier, allowing each API Gateway node to independently track and enforce rate limits for specific regions. This eliminates the need for continuous inter-node communication and complex coordination, significantly reducing computational overhead while maintaining accuracy through local decision-making combined with periodic status updates.
Solution Approach 2:
Each API Gateway node autonomously makes rate limiting decisions based on its own local status and the region identifier of incoming requests. Nodes serve themselves by independently tracking their own request counts and enforcing rate limits without requiring constant validation or coordination with other nodes, thereby reducing computational overhead while maintaining system-wide accuracy.
3Reliability
If API Gateways use traditional rate limiting policies with continuous synchronization, then cluster-wide rate limit enforcement is improved, but system performance deteriorates
Solution Approach 1:
The patent enforces cluster-wide rate limits by segmenting requests with region identifiers and distributing rate limit tracking across multiple nodes. Each node independently enforces rate limits for its assigned regions, achieving cluster-wide enforcement without requiring continuous synchronization. This segmentation approach maintains reliability while preserving system performance through reduced communication overhead.
Solution Approach 2:
The patent implements periodic status exchange among cluster nodes at predetermined intervals to maintain accurate cluster-wide rate limit information. This periodic synchronization ensures reliable rate limit enforcement across the entire cluster while minimizing the impact on system performance by reducing the frequency of coordination activities compared to continuous synchronization approaches.
Data Source
AI summary
Disclosed herein are system, method, and computer program product embodiments for implementing statistical distributed rate limiting in an Application Programming Interfaces (API) Gateway cluster. An API Gateway cluster may comprise a plurality of gateway nodes and manage API traffic to ensure proper function and protect the health of an API. Each gateway node may use a distributed rate limiting algorithm based on the physics formulas for average velocity, average acceleration and distance based on time determine the total number of API requests accepted by all the nodes in the cluster. Implementation of statistical distributed rate limiting allows for accurate estimations of the total requests accepted by cluster without requiring each node to share its status with the other nodes in the cluster upon receiving each API request. This approach allows for minimum computational overhead while prioritizing the health of the API.