API Gateway Data Plane Auto-Scaling with IIR Capacity Filters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud computing environments face challenges in efficiently allocating computing resources for API gateways, leading to underutilization or poor user experience due to inadequate scaling, which can be exacerbated by spikes in user demand.
Innovation Solution
Implementing infinite impulse response (IIR) filters to estimate capacity parameters for auto-scaling API gateway data planes, ensuring sufficient resources are allocated without rapid release to handle demand spikes efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If more computing resources are allocated to API gateways, then user experience is improved, but resource utilization deteriorates
Solution Approach 1:
The patent implements dynamic resource allocation by continuously monitoring API gateway traffic patterns and automatically adjusting the number of computing resources allocated to each gateway. This allows the system to allocate more resources during high-demand periods to maintain user experience while releasing resources during low-demand periods to improve utilization efficiency.
Solution Approach 2:
The system changes the allocation parameter (number of computing resources) based on observed traffic characteristics and demand patterns. By analyzing historical data and adjusting allocation thresholds, the system optimizes the balance between ensuring sufficient resources for good user experience and avoiding over-provisioning that would waste resources.
2Productivity
If computing resources are released quickly to improve utilization, then resource allocation efficiency is improved, but response time to demand spikes deteriorates
Solution Approach 1:
The patent maintains a pool of pre-provisioned computing resources that can be quickly deployed when demand spikes occur. Instead of waiting for resources to be allocated from scratch after a spike is detected, the system has resources ready in advance that can be activated immediately, reducing response time while still allowing efficient allocation during normal conditions.
Solution Approach 2:
The system builds a cushion of available computing capacity that can be rapidly deployed when needed. This cushion allows the system to absorb sudden demand increases without delay, as resources are already in place and can be activated immediately rather than requiring time for provisioning and deployment.
3Measurement precision
If complex monitoring and forecasting solutions are used to determine scaling, then measurement precision is improved, but device complexity deteriorates
Solution Approach 1:
The patent employs self-service mechanisms where the API gateway data plane automatically monitors its own traffic patterns and triggers scaling actions based on predefined thresholds and policies. This eliminates the need for complex external monitoring and forecasting systems, as the system uses its own operational data to make scaling decisions autonomously.
Solution Approach 2:
The system implements feedback loops where traffic metrics are continuously measured and compared against target thresholds. When conditions change, the system adjusts resource allocation based on this feedback, creating a simple but effective closed-loop control mechanism that doesn't require complex predictive models or external monitoring systems.
Data Source
AI summary
Techniques are described for auto-scaling an API gateway. Instead of using complicated monitoring, forecasting, and other compute intensive solutions to determine how to perform the auto-scaling, a much quicker, less compute intensive solution is performed. In some examples, infinite impulse response (IIR) filters can be used to estimate different parameters (e.g., available capacity) used to scale the API gateway data plane.