Scalable Concurrent Queue with Temporary Batching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional queues in distributed data grids experience contention issues due to multiple threads competing to insert messages or data, leading to bottlenecks and reduced performance, especially when handling large numbers of short-lived transactions.
Innovation Solution
A scalable concurrent queue is implemented, utilizing a central queue with multiple temporary queues for batch insertion, reducing contention by allowing producers to insert nodes in batches rather than individually, and using a CAS operation on a next reference instead of the tail reference to minimize contention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a conventional queue is used with multiple producers inserting nodes individually, then the queue structure is simple, but contention between producers increases and performance deteriorates
Solution Approach 1:
The queue structure is segmented into multiple temporary queues (one per producer) that buffer nodes before insertion into the central queue. This segmentation allows producers to insert nodes into their own temporary queue without contention, while only the consumer thread contends for the central queue, thereby resolving the contradiction between structural simplicity and insertion performance.
Solution Approach 2:
Temporary queues act as intermediary buffers between producers and the central queue. Producers deposit nodes into their temporary queue (no contention), and the consumer retrieves from temporary queues and inserts into the central queue (controlled contention). This intermediary mechanism eliminates producer-producer contention while maintaining centralized ordering.
2Stability of the object's composition
If multiple producers insert nodes one at a time into the central queue, then ordering is maintained, but the number of CAS operations increases and performance deteriorates
Solution Approach 1:
Multiple individual insertion operations are merged into a single batch insertion. The consumer thread retrieves nodes from temporary queues and inserts them into the central queue in batches rather than one-at-a-time. This merging reduces the number of CAS operations from O(n) to O(1) per batch while maintaining ordering through the consumer's sequential processing.
Solution Approach 2:
Nodes are preliminarily organized in temporary queues by producers before the actual insertion into the central queue. This preliminary organization allows the consumer to perform bulk insertion operations, reducing CAS overhead. The ordering is established during the preliminary organization phase and preserved during batch insertion.
3Device complexity
If a single tail reference is used for queue insertion, then the queue structure is simple, but contention on the tail reference increases with multiple producers
Solution Approach 1:
The single tail reference is segmented into multiple temporary queue structures, each with its own tail reference. Producers insert into their own temporary queue tail without contention. A separate consumer thread manages the central queue tail reference, isolating contention to a single point while enabling parallel producer operations.
Solution Approach 2:
Temporary queues with their own tail references serve as intermediaries that eliminate producer contention on the central queue tail. The consumer thread acts as an intermediary that transfers nodes from temporary queues to the central queue, centralizing tail reference contention to a single thread while enabling multiple producers to operate concurrently.
Data Source
AI summary
A scalable concurrent queue includes a central queue associated with multiple temporary queues for holding batches of nodes from multiple producers. When a producer thread or service performs an insertion operation on the scalable concurrent queue, the producer inserts one or more nodes into a batch in one of the multiple temporary queues associated with the central queue. Subsequently, the producer (or another producer) inserts the batch held in the temporary queue into the central queue. Contention between the multiple producers is reduced by providing multiple temporary queues into which the producers may insert nodes, and also by inserting nodes in the central queue in batches rather than one node at a time. The scalable concurrent queue scales to serve large number of producers with reduced contention thereby improving performance in a distributed data grid.