Multi-Core Intermediary Rate Limiting via Shared Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Managing access and rate limiting for web resources in multi-core systems is challenging due to varying user requests and object sizes, which can impact network resource allocation and prioritize certain content over others.
Innovation Solution
An intermediary device with multiple cores is deployed between clients and servers, where one core acts as the owner for a requested resource, monitoring request rates and blocking or delaying requests when limits are exceeded, using packet processing engines and shared memory to manage request rates across multiple cores.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple packet processing engines are used to handle requests from multiple clients, then the system can serve more clients simultaneously, but the complexity of managing request rates across multiple cores increases
Solution Approach 1:
The system divides the request rate management task among multiple packet processing engines, each responsible for monitoring requests from specific clients. Each engine independently manages rate limiting for its assigned clients, reducing the coordination complexity that would arise from a centralized approach while maintaining the ability to serve multiple clients simultaneously.
Solution Approach 2:
A shared memory structure acts as an intermediary between the multiple packet processing engines, providing a common data space for storing rate limit counters and configuration information. This eliminates the need for complex inter-process communication while enabling coordinated rate limiting across multiple cores.
2Reliability
If rate limiting is enforced for each client, then network resources are protected from overload, but legitimate high-priority requests may be delayed
Solution Approach 1:
The system implements differentiated rate limiting policies for different clients based on their priority levels. High-priority clients receive more permissive rate limits allowing faster request delivery, while low-priority clients are subject to stricter rate limiting. This local differentiation of rate limit parameters enables both resource protection and prioritized service delivery.
Data Source
AI summary
The present invention is directed towards systems and methods for managing a rate of request for an object transmitted between a server and one or more clients via a multi-core intermediary device. A first core of the intermediary device can receive a request for an object and assume ownership of the object. The first core can store the object in shared memory along with a rate-related counter for the object and generate a hash to the object and counter. Other cores can obtain the hash from the first core and access the object and counter in shared memory. Policy engines and throttlers in operation on each core can control the rate of access to the stored object.