Thread Messaging via Per-Thread Arenas and Batch Buckets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional locking mechanisms for memory allocation and deallocation in multi-threaded systems result in significant latency due to contention and the need for threads to wait for locks, leading to performance bottlenecks.
Innovation Solution
Assigning a separate arena to each thread with a batch messaging method that allows threads to request memory deallocation from other threads, using a data structure of buckets to accumulate and transmit messages in batches, reducing the need for frequent lock acquisitions and improving resource access efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional locking mechanisms are used for memory allocation and deallocation in multi-threaded systems, then thread safety is ensured, but significant latency occurs due to contention and threads waiting for locks
Solution Approach 1:
The shared arena metadata is segmented into per-thread arenas, with each thread having its own dedicated arena instance. This segmentation eliminates contention for arena metadata access between threads, as each thread operates on its own arena without needing to acquire locks on shared resources. The segmentation transforms a single shared resource into multiple independent resources, resolving the latency issue while maintaining thread safety.
2Stability of the object's composition
If locks are used to prevent multiple threads from manipulating arena metadata simultaneously, then data consistency is maintained, but the locking and unlocking mechanism adds latency
Solution Approach 1:
Each thread serves itself by having its own dedicated arena instance, eliminating the need for lock acquisition and release operations. Threads independently manage their own arena metadata without interfering with other threads, thus maintaining data consistency without the overhead of locking mechanisms. This self-service approach removes the latency associated with lock operations while ensuring thread safety through isolation.
3Productivity
If each thread has its own instance of memory allocator, then resource contention is reduced, but a mechanism is needed for threads to request memory deallocation from other threads' arenas
Solution Approach 1:
A message passing mechanism serves as an intermediary between threads with separate arenas. When one thread needs to request memory deallocation from another thread's arena, it sends a message through this intermediary mechanism. This allows threads to maintain their own arenas while still enabling cross-thread memory management operations, balancing resource access efficiency with the necessary complexity for inter-thread communication.
Data Source
AI summary
A method of communicating messages between threads. For each thread there are defined M buckets. When a transmitting thread has a message to send, it assigns the message to the bucket numbered m=Tid_r mod M to await transmission, where Tid_r is the ID of the receiving thread. The bucket m=Tid_t mod M is the home bucket, where Tid_t is the ID of the transmitting thread. After accumulating multiple messages, a batch transmission is performed, comprising a plurality of successive phases p. Each phase comprises: i) from each bucket other than the home bucket, transmitting some or all of the messages in the bucket as a batch to one of the receiving threads of the bucket, and ii) except in the last phase, incrementing the phase p by 1, and redistributing the messages remaining in the home bucket according to m=(Tid_r/M{circumflex over ( )}p) mod M.