Client-Side Completion Throttling for Server Overload Protection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In client-server networks, traditional request-side throttling mechanisms lead to inefficiencies due to lack of coordination among clients, resulting in unnecessary request collisions and idle server resources, with additional overhead from permission-based request systems.
Innovation Solution
Implementing completion-side throttling by estimating and managing the completion time of server requests, allowing clients to send requests that will be completed within throttle limits, using a throttling application in the kernel of the client's operating system to prioritize server work and optimize resource usage through Remote Direct Memory Access (RDMA).
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If request-side throttling is used to limit client performance, then the server is protected from overload, but request collisions occur and overall network efficiency is reduced
Solution Approach 1:
The patent inverts the traditional throttling approach by moving from request-side throttling (preventing requests before they are sent) to completion-side throttling (enforcing limits after requests are processed). The throttling decision is made based on completion times rather than request timing, allowing multiple requests to be sent simultaneously while ensuring they complete within acceptable timeframes. This resolves the contradiction by enabling higher request throughput (improving productivity) while maintaining server protection through completion-time-based throttling (preserving reliability).
Solution Approach 2:
The patent applies preliminary action by having clients estimate and advertise completion times for requests before the server processes them. The server uses these advance completion time estimates to make throttling decisions and prioritize requests. This allows the server to prepare for resource allocation and conflict resolution in advance, reducing request collisions and improving overall network efficiency while maintaining server protection.
2Reliability
If clients hold back requests to enforce throttles, then individual client limits are maintained, but server resources remain idle
Solution Approach 1:
The patent implements feedback mechanisms where clients continuously report their current completion time estimates to the server. The server uses this feedback to dynamically adjust throttling decisions and resource allocation. This feedback loop allows the server to match request completion timing with available server resources, reducing idle time while ensuring client throttle limits are enforced through completion-time-based control rather than request suppression.
Solution Approach 2:
The patent introduces dynamic throttling where the throttling threshold and request acceptance decisions change based on real-time server state and completion time estimates. Rather than static request blocking, the system dynamically adjusts which requests are accepted and their priority based on current server resource availability and predicted completion times. This dynamic approach increases server resource utilization while maintaining reliable throttle enforcement.
3Reliability
If permission-based request systems are used, then server control is improved, but additional overhead is introduced
Solution Approach 1:
The patent extracts the throttling control function from the request processing path and places it in the completion handling path. Instead of the server controlling each request individually (which introduces overhead), the server controls the completion timing of requests. This extraction separates the control function from the data path, reducing protocol overhead while maintaining server control through completion-time-based throttling decisions.
Solution Approach 2:
The patent introduces completion time estimation as an intermediary mechanism between client request and server processing. Rather than direct server control of each request (which creates overhead), the client estimates completion times and advertises them to the server as an intermediary control signal. The server then uses these intermediary completion time estimates to make throttling decisions, reducing direct control overhead while maintaining effective server control.
Data Source
AI summary
A computing device includes a processor and memory. A throttling application is stored in the memory and is executed by the processor. The throttling application receives a server request from a first application. The throttling application is located in a non-accessible location relative to the first application. The throttling application is configured, in response to the server request, to estimate a completion time when the server request will complete; send the server request with the completion time to a server; receive a response to the server request from the server prior to the completion time; when the completion time for the server request is in the future, wait until the completion time is now; and when the completion time is now, indicate completion to the first application.