Parallel Queue Segmentation for Thread Data Exchange
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional queue systems in parallel computer systems face bottlenecks due to sequential access and synchronization requirements, limiting scalability and performance, especially in Non-Uniform Memory Access (NUMA) systems.
Innovation Solution
Implementing a method using a global logical queue with associated physical queues for each processor core, where each producer thread manages its data elements in local queues, and consumer threads access these queues locally or randomly across cores, eliminating the need for global synchronization and minimizing remote memory access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional queue systems use global synchronization mechanisms (mutexes or locks) to ensure data consistency, then data consistency is maintained, but system scalability and throughput deteriorate due to sequential access bottlenecks
Solution Approach 1:
The patent divides the traditional single global queue into multiple independent physical queues, each associated with a specific processor core. Each queue maintains its own data consistency independently, eliminating the need for global synchronization. Producer threads enqueue data to local physical queues, and consumer threads dequeue from local or remote physical queues based on availability, enabling parallel access without synchronization bottlenecks.
2Reliability
If global locks are used for queue access, then data consistency is ensured, but access speed deteriorates due to sequentialization of all operations
Solution Approach 1:
The queue system is segmented into multiple independent physical queues, each with its own head pointer and data structure. Threads access only their local physical queue or randomly selected remote physical queues, eliminating global lock requirements. Each physical queue operates independently with its own consistency guarantees, enabling high-speed parallel access without sequentialization.
3Productivity
If separate locks are used for enqueue and dequeue operations, then some parallelism is achieved, but scalability deteriorates because multiple producer or consumer threads cannot operate simultaneously
Solution Approach 1:
The system segments the queue functionality across multiple physical queues, each independent and accessible by multiple threads. Producer threads can enqueue to their local physical queue simultaneously with other producers, and consumer threads can dequeue from local or remote physical queues simultaneously. This segmentation removes the limitation of single-lock approaches and enables full scalability.
Solution Approach 2:
The patent introduces a new dimension of parallelism by allowing consumer threads to access multiple physical queues (both local and remote) simultaneously. Instead of a single queue dimension with sequential access, the system creates a multi-dimensional access space where threads can operate across multiple queues in parallel, dramatically improving scalability.
4Productivity
If atomic operations are used without locks, then synchronization overhead is minimized, but cache coherency bottlenecks occur when threads access nearby memory areas
Solution Approach 1:
The queue data structures are segmented into separate physical queues located in different memory regions, each accessible by specific processor cores. This spatial segmentation reduces cache coherency traffic because threads primarily access their local physical queue in nearby memory, minimizing remote memory accesses and cache invalidation events compared to single global queue approaches.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
Computer system having a plurality of processors (P) that each have a plurality of processor cores (Ci), wherein each processor (P) is provided with a memory (M) that stores, for each processor core (Ci) of the respective processor (P), an associated physical queue (Qi) that comprises a plurality of sequentially linked memory elements, wherein each memory element respectively stores a data element, transmitted between two threads executed on processor cores (Ci), and an associated thread index that indicates a source thread that is the origin of the transmitted data element, wherein the physical queues of all the processor cores (Ci) together form a global logical queue of the parallel computer system, which queue has a data-element management table that stores, for each source thread executed on a processor core (Ci), a count (Count), which indicates the total number of data elements queued by the respective source thread, and located, in one of the physical queues of the logical queue, and a processor core index (CORE), which indicates the processor core (Ci) whose physical queue contains the data elements queued therein by the respective source thread.