Server Queue Management via Dynamic Request Discarding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
During busy periods, server queues can become large, leading to client timeouts before requests are processed, causing servers to appear unresponsive to new requests despite peak throughput.
Innovation Solution
A computer-implemented method that analyzes each request to determine a client wait time limit and an indicator of the request's intent by analyzing the associated application, allowing for dynamic queue actions such as removing or reordering requests to optimize resource allocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the server queue accepts all incoming requests during busy periods, then the server processes requests at peak throughput, but clients timeout before their requests are processed and the server appears unresponsive
Solution Approach 1:
The system dynamically changes the queue action parameter based on queue depth and client wait time limits. When the queue exceeds a threshold, the server transitions from processing all requests to selectively discarding or deprioritizing requests that would timeout, thereby maintaining both high throughput and reliable client responses
Solution Approach 2:
The system implements feedback by monitoring queue depth and comparing it against thresholds, then adjusting queue actions accordingly. The server analyzes client wait time limits and request intent indicators to determine whether to process, discard, or deprioritize requests, creating a closed-loop control system that adapts to changing load conditions
2Productivity
If the server processes all requests in the queue, then maximum throughput is achieved, but server resources are wasted on requests for clients that are no longer waiting
Solution Approach 1:
The system applies discarding by removing requests from the queue when clients are no longer waiting (determined by comparing current time against client wait time limits). This prevents waste of server resources on requests that would timeout, while maintaining the ability to process valuable requests
Solution Approach 2:
Instead of processing all requests or none, the system applies partial action by selectively processing only those requests where the client is still waiting. This optimized subset of requests represents the exact amount of processing needed to maintain client satisfaction without wasting resources
3Loss of energy
If the server discards requests from the queue when clients timeout, then resource efficiency improves, but requests that should be processed may be incorrectly discarded
Solution Approach 1:
The system performs preliminary analysis of each request before adding it to the queue, determining the client wait time limit and request intent indicator in advance. This pre-processing information is stored and used later to make accurate decisions about whether to discard or process the request
Solution Approach 2:
The system introduces an intermediary analysis layer that examines request characteristics (intent indicators) and queue conditions before determining the final queue action. This intermediary step prevents premature discarding by evaluating whether requests are worth processing even if clients might timeout
Data Source
AI summary
Proposed embodiments may thus provide one or more concepts for automatically managing the requests in a server queue, based on information describing the client wait time limit and an indicator of the intent of the request. In particular, embodiments provide a mechanism for managing requests in a server queue.