Shuffle Sharding for Shared Queue Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large data centers face challenges in managing shared queues efficiently, leading to traffic spikes that affect other users, due to the overwhelming capacity and difficulty in managing numerous queues for multiple users, resulting in queue backups and unavailability.
Innovation Solution
Implementing shuffle sharding with a centralized or distributed poller manager to select the queue with the smallest load for event routing, ensuring that events are directed to the least-busy queue, even when all available queues are full, thereby minimizing the impact of heavy usage by one user on others.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If multiple users share the same queues in large data centers, then resource utilization improves, but traffic spikes from one user can significantly affect other users
Solution Approach 1:
The patent divides the queue management system into multiple independent queue groups, where each group handles traffic from specific users or user sets. This segmentation isolates traffic spikes to individual groups, preventing them from affecting the entire system while maintaining high resource utilization through shared access to multiple groups.
Solution Approach 2:
The patent implements user-specific queue group assignments where each user's traffic is routed to particular queue groups based on their characteristics and historical patterns. This local quality approach ensures that each user experiences optimized service while their traffic spikes remain contained to their assigned groups, protecting other users from interference.
2Device complexity
If a single queue is used for all users, then management complexity is reduced, but queue backups and unavailability occur when one user generates excessive traffic
Solution Approach 1:
The system segments queues into multiple groups organized by user or user sets, creating a hierarchical structure that balances management simplicity with reliability. Each segment operates semi-independently, so failures or overloads in one segment do not propagate to others, maintaining overall system availability while keeping management complexity manageable through standardized group operations.
Solution Approach 2:
The patent introduces queue group assignments as an intermediary layer between users and individual queues. This intermediary structure absorbs and distributes traffic patterns, preventing any single queue from becoming a bottleneck while maintaining organized management through group-based control policies.
3Object-affected harmful factors
If queues are isolated for each user, then traffic isolation is improved, but resource utilization decreases due to empty queues during low-traffic periods
Solution Approach 1:
The patent creates a multi-level segmentation structure where queues are grouped by user but multiple user groups share the same physical queue infrastructure. This allows traffic isolation at the logical level (each user has dedicated queue groups) while enabling resource sharing at the physical level (queues can be reused across user groups during low-utilization periods), optimizing both isolation and utilization.
Solution Approach 2:
The queue groups serve multiple functions: they provide user-specific traffic isolation when needed, enable resource sharing across users during low-demand periods, and offer scalable expansion capabilities. This multi-functionality resolves the contradiction by allowing the same queue infrastructure to operate in different modes based on traffic patterns.
Data Source
AI summary
Techniques for performing shared queue management utilizing shuffle sharding are described. For an event, one shared queue can be selected by first identifying a shard of a pool of queues, selecting two or more queues from the shard, and selecting the one queue that has a minimum queue load from those queues in the selected two or more queues. The selection significantly reduces or eliminates negative impacts upon a user or user function from activity of other users that utilize the shared queues.