Completion-side client throttling for server overload protection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing client-server networks face inefficiencies due to request-side throttling, which lacks coordination among clients, leading to unnecessary collisions and idle server resources, and requires additional overhead for permission management.
Innovation Solution
Implementing completion-side throttling by a throttling application that estimates completion times for server requests, inserts them into a queue, and sends a time-to-live field, allowing the server to prioritize work and manage resources more efficiently, while avoiding unnecessary delays and overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If request-side throttling is used to limit client requests, then the server is protected from overload, but clients lack coordination leading to unnecessary collisions and reduced network efficiency
Solution Approach 1:
The patent inverts the traditional request-side throttling approach by implementing completion-side throttling. Instead of preventing requests from being sent, the system allows requests to be sent but throttles based on completion status. The client tracks completed requests and only sends new requests when completion slots are available, eliminating collisions while maintaining server protection.
Solution Approach 2:
The system implements feedback by having the client monitor server response status and use this information to control request timing. The client waits for completion feedback from the server before sending subsequent requests, creating a coordinated request pattern that improves network efficiency while preventing server overload.
2Reliability
If permission-based request protocol is used to coordinate clients, then server overload is prevented, but additional overhead is introduced for request and permission management
Solution Approach 1:
The patent extracts the throttling logic from the server side and places it on the client side. The client independently determines when to send requests based on completion tracking, eliminating the need for server-based permission granting and reducing protocol overhead while maintaining effective load control.
Solution Approach 2:
The client performs self-service throttling by autonomously tracking its own request completion status and making independent decisions about when to send new requests. This eliminates the need for server-mediated permission management and reduces communication overhead.
3Reliability
If clients hold back requests to enforce throttles, then individual client limits are respected, but the server remains idle and overall throughput is reduced
Solution Approach 1:
The client performs preliminary action by pre-tracking completion slots before sending requests. Instead of holding back requests arbitrarily, the client proactively monitors and utilizes available completion capacity, ensuring that server resources are fully utilized while maintaining throttle compliance.
Solution Approach 2:
The system maintains continuity of useful action by ensuring the server continuously processes requests without idle periods. The client continuously monitors completion status and immediately sends new requests when capacity becomes available, maximizing server utilization while respecting throttle limits.
Data Source
AI summary
A computing device comprises a processor and memory. A throttling application stored in the memory that is executed by the processor is configured, in response to a new server request from a first application, to determine whether performing the new server request exceeds a predetermined number of requests in progress; if performing the new server request does not exceed a predetermined number of requests in progress, estimate a completion time for the new server request; insert the new server request and the completion time in a request queue; and send the new server request and a time to live field to a server, wherein the time to live field is equal to the completion time minus a current time that the new server request is sent to the server.


