Queue Balancer Mechanism for Load Distribution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In scenarios with multiple consumers accessing a shared queue, there is an imbalance in resource usage, leading to some consumers having disproportionate access while others have inadequate access, resulting in inefficient server and consumer resource utilization.
Innovation Solution
Implementing a queue balancer mechanism where each consumer calculates and adjusts its polling hit rate to achieve a target polling hit rate, allowing for balanced access by tuning its read rate from the queue, thereby balancing the load among consumers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If multiple consumers use polling to access the queue, then consumers can asynchronously remove items from the queue, but significant server resources are consumed handling polls and consumer resources are wasted in competition
Solution Approach 1:
The patent implements a feedback mechanism where consumers report their polling hit rates to the server, and the server uses this feedback to dynamically adjust consumer read rates. This allows the system to optimize resource usage based on actual queue conditions rather than relying on constant polling by all consumers.
Solution Approach 2:
The patent makes the consumer read rate dynamic by allowing consumers to adjust their polling frequency based on their observed hit rates and target hit rates. This dynamic adjustment reduces unnecessary polling when the queue is empty and maintains adequate polling when items are available, optimizing server resource consumption.
2Speed
If multiple consumers poll the queue frequently, then consumers can quickly access available items, but imbalance occurs where some consumers get disproportionate access while others receive inadequate access
Solution Approach 1:
The patent uses feedback from polling hit rates to balance load distribution. Consumers with higher hit rates are given lower target hit rates (reducing their polling frequency), while consumers with lower hit rates are given higher target hit rates (increasing their polling frequency), achieving balanced access without manual intervention.
Solution Approach 2:
The patent changes the parameter of consumer read rates dynamically based on observed performance. By adjusting each consumer's target polling hit rate parameter according to their actual performance and the queue's current state, the system achieves balanced load distribution while maintaining quick access to available items.
3Productivity
If consumers increase polling frequency to improve access, then queue items can be removed faster, but server resource consumption increases
Solution Approach 1:
The patent makes the polling frequency dynamic rather than static. Consumers adjust their read rates based on their observed hit rates and target hit rates, increasing polling when items are available and decreasing polling when the queue is empty, thus maintaining productivity while reducing energy loss.
Solution Approach 2:
The patent optimizes the balance between productivity and energy consumption by dynamically changing the polling frequency parameter. The system finds the optimal polling rate that maintains high queue item removal rates while minimizing server resource consumption, avoiding both excessive polling and insufficient access.
Data Source
AI summary
Disclosed are various embodiments for balancing a load on a queue among multiple consumers. A target polling hit rate is derived for at least one queue from a consumer load. The consumer load on the at least one queue is adjusted responsive to a change in an observed polling hit rate for the at least one queue.