Dynamic API Throttling for SaaS SLA Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
SaaS vendors impose throughput penalties on user groups due to excessive API events, leading to performance issues and delayed processing for other users, as existing approaches fail to effectively manage Service Level Agreement (SLA) guarantees and efficiently utilize API resources.
Innovation Solution
The system monitors API event rates for individual users and identifies 'power users' exceeding limits, taking actions such as throttling their notifications to maintain SLA guarantees, by rationing API event submissions and potentially assigning them to separate queues, ensuring resource utilization for other users.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the system processes all API events from all users without differentiation, then complete service coverage is provided, but throughput penalties are imposed due to excessive API events from power users
Solution Approach 1:
The patent segments users into different categories (power users vs. regular users) based on their API event submission patterns. Power users who exceed threshold rates are identified and handled differently through separate queues, while regular users continue in standard processing queues. This segmentation allows the system to guarantee SLA for the majority of users while managing the resource consumption of power users separately, thus maintaining overall system throughput.
2Ease of operation
If API events from power users are processed at the same rate as other users, then fair resource allocation is maintained, but head-of-line blocking occurs causing delayed processing for other users
Solution Approach 1:
The patent implements segmentation by creating separate processing queues for power users and regular users. Power users who exceed API event thresholds are routed to dedicated queues that are processed independently, preventing their high-volume events from blocking the processing of regular users' events. This eliminates head-of-line blocking while maintaining fair resource allocation through structured queue management.
Solution Approach 2:
The patent applies local quality by providing differentiated service quality to different user groups. Regular users receive standard processing service, while power users who exceed thresholds receive a different service level through separate queues. This allows the system to optimize processing for the majority of users while still accommodating power users' needs, preventing universal service degradation.
3Reliability
If the system monitors and manages individual user API event rates, then SLA guarantees can be maintained, but system complexity increases due to user identification and classification mechanisms
Solution Approach 1:
The patent implements feedback mechanisms where the system continuously monitors API event submission rates from each user, compares them against defined thresholds, and dynamically adjusts routing decisions. When a power user exceeds the threshold, the system provides feedback by routing their events to separate queues. This automated feedback loop maintains SLA guarantees through real-time monitoring without requiring complex manual user identification or classification processes.
Data Source
AI summary
A method and system disclosed dynamically throttling a rate or volume in time of a power user for avoiding throughput penalties imposed by SaaS vendors on a user group due to excessive Application Programming Interface (API) events from users in the group, monitoring API event rate for requests from the group, collectively, and from individual users of the user group to a SaaS vendor is disclosed. Also, identifying a power user as submitting API events in excess of a limit, and on behalf of the user, throttling the power user's rate of API events submissions, based on a configurable policy specific to the SaaS vendor managed by a proxy, to reduce the user's impact on the API event rate of the group at least when the group's API rate, overall, exceeds or approaches a SaaS imposed trigger of a throughput penalty on the group, thereby avoiding triggering of the throughput penalty by the SaaS.