Dynamic Request Routing via Queue-Aware Traffic Manager
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current load balancing systems face performance degradation when a large number of simultaneous requests exceed the concurrency threshold, leading to request queuing and reduced server performance.
Innovation Solution
A traffic manager system with multiple child processes and a shared memory mechanism that dynamically de-queues requests from servers based on connection limits and queue lengths, optimizing the distribution of requests across available connections to prevent queue buildup.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a large number of simultaneous requests are routed to servers, then the number of served requests increases, but request queuing occurs and performance is reduced
Solution Approach 1:
The system dynamically adjusts the routing of requests based on real-time server queue lengths and connection availability. The traffic management system continuously monitors server states and adapts routing decisions to prevent queue buildup, transforming a static load balancing approach into a dynamic one that responds to changing conditions.
Solution Approach 2:
The system implements feedback mechanisms by monitoring queue lengths and connection states on servers, then using this information to adjust routing decisions. The traffic management system receives feedback about server capacity and modifies request distribution accordingly, creating a closed-loop control system that prevents performance degradation.
2Productivity
If load balancing maximizes the number of requests across servers, then request distribution is optimized, but concurrency threshold is breached and queues form
Solution Approach 1:
The system performs preliminary actions by pre-establishing connection pools and setting connection limits before traffic arrives. The traffic management system proactively configures server capacities and connection parameters in advance, allowing it to quickly route requests without needing to make complex decisions during high-volume traffic periods.
3Productivity
If servers handle more concurrent connections, then more requests can be served simultaneously, but queue management overhead increases
Solution Approach 1:
The traffic management system acts as an intermediary between clients and servers, absorbing the complexity of queue management and connection tracking. Rather than having servers directly manage complex routing decisions and queue states, the intermediary traffic management system handles these functions centrally, simplifying server operations.
Data Source
AI summary
Apparatus for routing requests from a plurality of connected clients to a plurality of connected servers comprises a processor, memory and a network interface. The processor is configured to run a plurality of identical processes, each being for receiving requests and connecting each received request to a server. For each process, the processor is configured to maintain a queue of requests in memory, determine a number of queued requests that may be connected to a server, and attempt to connect this number of queued requests. The processor then accepts further requests, and if the queue is not empty, places the further requests in the queue, and if the queue is empty, attempts to connect the further requests. The processor determines the number of queued requests that may be connected to a server in dependence upon the length of the queues of all the processes and the number of available connections.