Dynamic API Traffic Shaping Engine for Multiclient Network Protection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing traffic shaping solutions for APIs fail to dynamically adjust to current conditions, leading to unnecessary slowdowns for well-behaved users and inability to separate Transactions Per Second (TPS) protection from concurrent transaction protection, resulting in a negative impact on customer experience.
Innovation Solution
A Traffic Rules Engine (TRE) that dynamically measures and adjusts traffic shaping based on overall traffic, segregating bad actors and applying shaping only when bandwidth is reached, allowing customization of TPS and concurrent transaction protection on a per-API basis, and using machine learning for real-time optimizations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional traffic shaping solutions are applied to limit API requests, then system protection from overload is improved, but customer experience deteriorates due to unnecessary slowdowns for well-behaved users
Solution Approach 1:
The patent implements dynamic traffic shaping where the system continuously monitors actual API usage patterns and adjusts rate limits in real-time. Well-behaved users experiencing temporary slowdowns due to infrastructure issues are automatically granted higher rate limits, while bad actors continue to be restricted. This dynamic adjustment resolves the contradiction by making the system both protective and user-friendly simultaneously.
Solution Approach 2:
The system incorporates feedback loops that monitor API response times, error rates, and usage patterns to automatically adjust traffic shaping parameters. When the system detects that well-behaved users are experiencing degradation, it feedback-adjusts their rate limits upward, thereby maintaining good customer experience while preserving system protection mechanisms.
2Reliability
If static TPS limits are enforced for all users, then infrastructure protection is improved, but system adaptability to different usage patterns deteriorates
Solution Approach 1:
The patent applies different traffic shaping parameters to different users based on their individual usage patterns and behavior characteristics. Each user receives a customized rate limit profile that reflects their specific needs and historical behavior, rather than a universal static limit. This local differentiation resolves the contradiction by maintaining infrastructure protection while adapting to diverse usage patterns.
Solution Approach 2:
The system dynamically changes traffic shaping parameters (rate limits, burst allowances, time windows) based on monitored usage patterns, user reputation, and system conditions. This parameter adaptation allows the system to maintain protective limits for bad actors while providing flexible, higher limits for well-behaved users with varying需求的, thereby resolving the contradiction between protection and adaptability.
3Productivity
If gateway instances independently track requests, then system scalability is improved, but measurement precision of total traffic deteriorates
Solution Approach 1:
The patent implements a centralized traffic aggregation mechanism that combines request count data from all distributed gateway instances. This centralization enables precise measurement of total API traffic across the entire system while preserving the scalability benefits of having multiple independent gateway instances processing requests in parallel, thereby resolving the contradiction between scalability and measurement precision.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A real-time dynamic API traffic shaping and infrastructure resource protection in a multiclient network environment is provided. A traffic rules engine (TRE) applies traffic shaping only to customers that are utilizing "more than their fair share" of the currently available bandwidth without allowing them to negatively impact the user experience of other users. The present invention takes current API traffic into consideration, allowing one or a few high volume users to utilize most of all available bandwidth as long as other users do not need that bandwidth. This includes dynamically measuring and adjusting which users had traffic shaping applied to them based on the overall traffic during any given second. The solution of the present invention avoids any slowdown of customer API requests unless the maximum allowable TPS limit is near to being reached.