Load Control Device for Server Request Buffering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Web servers experience performance degradation due to extremely high traffic volumes, leading to increased input/output and context switching overhead, which degrades request processing performance and results in longer response times, especially during peak usage periods such as actively-traded stocks or natural disasters.
Innovation Solution
A load control device is introduced between clients and servers to limit the number of response-waiting requests by storing them in a buffer when the threshold is reached, ensuring that requests are only sent once the previous response has been processed, thereby reducing server overhead and maintaining performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the server accepts all incoming requests without limitation, then the service availability is improved, but the request processing performance degrades due to increased input/output overhead and context switching
Solution Approach 1:
The patent introduces a load control device as an intermediary component positioned between the client and server. This mediator monitors the number of response-waiting requests and selectively buffers incoming requests when the threshold is exceeded, thereby protecting the server from overload while maintaining service availability. The intermediary resolves the contradiction by filtering traffic based on server capacity rather than accepting or rejecting all requests uniformly.
Solution Approach 2:
The load control device performs preliminary action by buffering requests before they reach the server when the response-waiting request count exceeds the threshold. This proactive buffering prevents the server from being overwhelmed by sudden traffic spikes, allowing the server to process requests at its optimal pace while still maintaining service availability for legitimate users.
2Productivity
If the server processes requests in parallel with multiple threads, then the throughput is improved, but the context switching overhead increases and degrades performance
Solution Approach 1:
The load control device implements feedback control by continuously monitoring the number of response-waiting requests and adjusting its buffering behavior accordingly. When the count is below the threshold, requests are forwarded to the server for parallel processing to maximize throughput. When the threshold is exceeded, incoming requests are buffered to prevent excessive context switching, thereby reducing overhead while maintaining optimal throughput.
3Loss of time
If the server responds to all requests immediately, then the response time is improved, but the server becomes congested and processing performance degrades
Solution Approach 1:
The load control device dynamically adjusts its operation based on real-time server conditions. It monitors the number of response-waiting requests and adaptively buffers or forwards requests accordingly. This dynamic behavior allows the system to maintain short response times during low-load conditions while preventing server congestion during high-load periods, thereby resolving the contradiction between response time and processing performance.
Data Source
Figure 1~2
Figure 3(a)~3(b)
Figure 4(a)~4(b)
AI summary
The number of response-waiting requests which are already sent to a server (4) but to which a response is not yet returned from the server (4) is limited. To limit this number, received requests are temporarily accumulated in a buffer if the number of response-waiting requests has reached a threshold and, until the number of response-waiting requests falls below the threshold, requests are not sent from the buffer. The execution status of the server (4) is monitored, and the threshold is increased when the response time from the server (4) to a request is within an allowable range, and the threshold is decreased when the response time exceeds the allowable range. In addition, TCP connections between a load control device (3) and clients (1-1, ..., 1-n) are aggregated so that the number of simultaneous connections of TCP connections between the server (4) and the load control device (3) becomes equal to or smaller than the threshold of the number of response-waiting requests.