Lockless Circular Buffer for Multi-Threaded Data Parallelization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computer systems face challenges in processing and distributing high-volume data streams efficiently due to the limitations of traditional locking mechanisms in multi-core processors, which hinder scalability and interactivity in big data applications.
Innovation Solution
The implementation of a multi-threaded application using lockless data structures with circular buffers and queues allows concurrent execution of send and receive threads, enabling atomic access and reducing resource utilization by requiring only a single copy operation for data blocks, thus facilitating wire-speed processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional locking mechanisms are used in multi-core processors, then data access safety is ensured, but scalability and processing speed are hindered
Solution Approach 1:
The patent removes locking mechanisms entirely from the data structure access operations. Send threads and receive threads access the same data structure without any synchronization primitives, achieving lockless operation that eliminates the performance bottleneck while maintaining data integrity through careful design of the data structure itself
Solution Approach 2:
The patent introduces copy-on-write semantics as an intermediary mechanism. When a receive thread needs data, it creates a private copy rather than directly accessing the shared data structure, allowing send threads to continue modifying the original data without conflict. This intermediary copying approach enables concurrent access without locks
2Reliability
If multiple copies of data blocks are made for concurrent access, then thread safety is achieved, but resource utilization increases
Solution Approach 1:
The patent performs preliminary validation and preparation in the send path by organizing data into the shared data structure with proper metadata and state tracking. This preliminary organization eliminates the need for multiple defensive copies, as the structured data can be safely shared and selectively copied only when actually needed by receive threads
Solution Approach 2:
The patent implements selective copying where data blocks are copied only when a receive thread actually needs them, rather than pre-copying for all possible concurrent accesses. The copy operation is triggered by actual data consumption needs, minimizing redundant memory usage while maintaining thread safety
3Reliability
If synchronization primitives are used for concurrent access, then data consistency is maintained, but processing throughput decreases
Solution Approach 1:
The patent extracts and removes all synchronization primitives (mutexes, semaphores, atomic operations) from the concurrent access pattern. Data consistency is achieved not through synchronization but through the inherent design of the data structure that allows lockless send and receive operations to proceed simultaneously without interference
Solution Approach 2:
The data structure is designed to be self-sufficient for concurrent access without external synchronization control. Each thread can independently determine whether data is available and safely access or modify it based on the state information maintained within the data structure itself, eliminating the need for synchronization services
4Productivity
If lockless data structures are implemented, then processing speed increases, but implementation complexity increases
Solution Approach 1:
The patent segments the data structure into distinct components with clear responsibilities: a shared buffer for data blocks, a separate metadata structure for tracking state, and organized indexing mechanisms. This segmentation makes the lockless implementation more manageable by dividing complexity into isolated, well-defined segments rather than a monolithic complex structure
Data Source
AI summary
In general, techniques are described for parallelizing a high-volume data stream using a data structure that enables lockless access by a multi-threaded application. In some examples, a multi-core computing system includes an application that concurrently executes multiple threads on cores of the system. The multiple threads include one or more send threads each associated with a different lockless data structure that each includes both a circular buffer and a queue. One or more receive threads serially retrieve incoming data from a data stream or input buffer, copy data blocks to one of the circular buffers, and push metadata for the copied data blocks to the queue. Each of the various send threads, concurrent to the operation of the receive threads, dequeues the next metadata from its associated queue, reads respective blocks of data from its associated circular buffers based on metadata information, and offloads the block to a server.