Concurrent Queue Per-Cell Locking Reduces Thread Contention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional concurrent queues exhibit poor scaling behavior and high synchronization overhead due to contention over a single lock, leading to performance degradation as the number of producers and consumers increases, resulting in significant thread contention and poor system scalability.
Innovation Solution
An efficient concurrent queue is implemented using an element array with per-cell concurrency control, employing insert and consume sequencers to allow multiple producers and consumers to access the queue concurrently while maintaining strict FIFO ordering, reducing contention through per-cell level concurrency control.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single lock is used to coordinate access between producers and consumers, then thread safety is ensured, but synchronization overhead increases and scalability deteriorates
Solution Approach 1:
The patent divides the single lock into multiple per-cell locks, where each cell in the array has its own lock. This segmentation allows multiple producers and consumers to operate concurrently on different cells without interfering with each other, thereby reducing synchronization overhead while maintaining thread safety. The queue array is segmented into multiple independently accessible cells, each with its own concurrency control mechanism.
2Stability of the object's composition
If a single lock is used to control queue access, then message ordering is guaranteed, but thread contention increases and performance degrades
Solution Approach 1:
The patent segments the queue into multiple cells, each with its own lock and sequencer. This allows different threads to access different cells simultaneously without contention, while still maintaining FIFO ordering within each cell through per-cell sequencers that track insertion and consumption order independently.
Solution Approach 2:
Each cell in the queue array has its own concurrency control mechanism with a dedicated sequencer that maintains ordering properties locally. The sequencer for each cell tracks the order of insertions and consumptions for that specific cell, ensuring FIFO ordering is maintained at the local level without requiring global coordination.
3Productivity
If per-cell concurrency control is implemented, then thread contention is reduced and scalability improves, but device complexity increases
Solution Approach 1:
The patent implements a universal sequencer structure that is reused for each cell in the queue array. Each cell's sequencer follows the same pattern of tracking insertion and consumption order, allowing the same code logic to be applied across all cells. This multi-functionality reduces overall system complexity despite the increased number of concurrency control mechanisms.
Data Source
AI summary
A method, system, and medium are disclosed for facilitating communication between multiple concurrent threads of execution using an efficient concurrent queue. The efficient concurrent queue provides an insert function usable by producer threads to insert messages concurrently. The queue also includes a consume function usable by consumer threads to read the messages from the queue concurrently. The consume function is configured to guarantee a per-producer ordering, such that, for any producer, messages inserted by the producer are read only once and in the order in which the producer inserted those messages.