API Gateway Segmentation for Low Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional API gateways introduce latency due to their focus on real-time metering and throttling, which becomes inefficient in environments with high volumes of requests for large payloads, limiting their ability to provide adequate throughput.
Innovation Solution
An API gateway design with an on-line component for real-time authentication and access control, and an off-line component for periodic metering and throttling, allowing the on-line component to service high volumes of requests with low latency by separating authentication and access control from metering and throttling decisions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional API gateways perform real-time metering and throttling for each API request, then authentication and access control are ensured, but latency increases and throughput decreases for high volume requests
Solution Approach 1:
The patent segments the API gateway functionality into two distinct components: an online component that handles real-time authentication, access control, and request routing; and an offline component that performs periodic metering and throttling decisions. This segmentation allows the online component to process requests with low latency while the offline component handles resource-intensive metering operations separately, resolving the contradiction between reliable authentication and low latency.
Solution Approach 2:
The offline component performs metering and determines throttling decisions in advance on a periodic basis, storing these decisions in a data structure that the online component can quickly reference. This preliminary action eliminates the need for real-time metering calculations during request processing, significantly reducing latency while maintaining authentication reliability.
2Measurement precision
If conventional API gateways perform real-time metering for each request, then usage tracking is accurate, but throughput is limited for high volume requests with large payloads
Solution Approach 1:
The patent implements periodic action by having the offline component perform metering operations at scheduled intervals rather than for every single request. The offline component aggregates usage data and performs metering calculations periodically, while the online component handles high-volume request routing. This periodic approach maintains sufficient usage tracking accuracy while dramatically improving throughput for high-volume scenarios.
Solution Approach 2:
By segmenting the gateway into online and offline components, the patent separates the high-throughput request routing function from the accuracy-critical metering function. The online component focuses on fast request handling with reference to pre-computed throttling decisions, while the offline component ensures accurate usage tracking through periodic metering, thus resolving the contradiction between measurement precision and productivity.
3Reliability
If conventional API gateways process metering and throttling in real-time, then usage quotas are enforced accurately, but system complexity increases
Solution Approach 1:
The patent reduces system complexity by segmenting the gateway into specialized online and offline components with clearly defined responsibilities. The online component handles authentication and request routing with simple reference to pre-computed decisions, while the offline component handles complex metering and quota enforcement logic periodically. This segmentation simplifies the real-time processing path while maintaining reliable quota enforcement through the offline component.
Solution Approach 2:
The offline component performs complex metering calculations and quota enforcement decisions in advance, storing results in lookup tables that the online component can quickly reference. This preliminary action moves computational complexity away from the real-time request path, reducing the complexity of the online component while maintaining accurate quota enforcement through the pre-computed decisions.
Data Source
Figure 1~3
Figure 2
Figure 4
AI summary
An API gateway designed to process a high volume of API requests for data and services where relatively large payloads are returned to the requester. The API gateway includes an "on]line" component that, for each API request, examines information in the request to increment an API usage count by API ID and product key. The usage information is periodically pushed to an outbound message queue, where it is later populated in a database. An "off]line" component determines if a customer has exceeded its predetermined quota based on the information in the database. This determination is pushed to another message queue that is periodically read by the on]line component to update the in]memory API ID, product key and API validity flag values. As such the on]line component is able to service a high volume of API requests for data with a low latency.