Multi-Core Server Surge Protection via Local Global Counters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multi-core systems, providing connection surge protection to servers is complicated due to lack of sufficient information sharing between cores, leading to computation delays and bandwidth consumption, especially during request surges from numerous clients.
Innovation Solution
A multi-core intermediary device maintains local and global counters to track pending requests, allowing packet processing engines to determine the rate of change and apply surge protection policies, thereby delaying requests to manage connection rates effectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If each packet processing engine maintains local counters and communicates with shared memory to track pending requests, then surge protection capability is improved, but computation delay and system bandwidth consumption increase
Solution Approach 1:
The patent divides the surge protection function into segments: each packet processing engine maintains its own local counter in private memory, and only periodic summaries are communicated to shared memory. This segmentation reduces the frequency and volume of inter-core communication, thereby reducing computation delay while maintaining surge protection capability.
Solution Approach 2:
The patent implements periodic action by having packet processing engines update shared memory counters at predetermined intervals rather than continuously. Each engine adds its local counter value to the global counter periodically, and retrieves the global counter value at these intervals to calculate pending requests for other engines. This periodic update mechanism reduces system bandwidth consumption and computation delay compared to continuous synchronization.
2Loss of information
If continuous core-to-core communication is implemented for surge protection, then accurate request tracking is improved, but system bandwidth consumption increases
Solution Approach 1:
The patent reduces system bandwidth consumption by implementing periodic updates of counter values in shared memory at predetermined intervals, rather than continuous communication between cores. Each packet processing engine accumulates its local counter and updates the global counter periodically, significantly reducing the frequency of memory access and inter-core communication while maintaining accurate request tracking.
Solution Approach 2:
The patent applies preliminary action by having each packet processing engine maintain its own local counter in private memory before communicating with shared memory. This local tracking is performed continuously without requiring system bandwidth, and only the summarized results are communicated periodically to the shared memory, reducing overall bandwidth consumption while maintaining tracking accuracy.
3Productivity
If packet processing engines delay requests to manage connection rates, then server throughput is improved, but request response time increases
Solution Approach 1:
The patent applies preliminary action by having packet processing engines calculate and determine the number of pending requests for other engines in advance, before processing incoming requests. By using the global counter value and local counter value to compute pending requests and applying surge protection policies proactively, the system can manage connection rates without causing excessive delays, improving server throughput while minimizing response time impact.
Data Source
AI summary
The present application is directed towards systems and methods for providing connection surge protection to one or more servers by an intermediary multi-core system. A packet processing engine of a multi-core device deployed as an intermediary between a plurality of clients and one or more servers determines an estimated number of total pending requests received by all packet processing engines based on a value of a local counter of received requests, the total number of pending requests received by all other packet processing engines at a last predetermined interval, and a rate of change of the total number of pending requests received by all other packet processing engines multiplied by the time since the last predetermined interval. The packet processing engine applies a surge protection policy to received pending requests responsive to the determined estimated number of total pending requests.