Concurrent Thread Pool Queuing for Parallel Message Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional group communication protocols process messages sequentially in a First In First Out (FIFO) order, leading to bottlenecks as messages from different senders must wait for their turn to be processed, which limits efficiency in multicasting scenarios.
Innovation Solution
Implementing a concurrent stack with thread pools that allow messages from different senders to be processed in parallel by forming separate channels for each sender, based on user-defined criteria such as queue size or message priority, enabling simultaneous processing of messages from multiple senders.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If messages are processed sequentially in a single queue using FIFO order, then processing simplicity is maintained, but processing speed and throughput deteriorate due to bottlenecks
Solution Approach 1:
The single message queue is segmented into multiple concurrent queues, one for each sender. Each queue handles messages from a specific sender independently, allowing parallel processing across multiple threads. This segmentation eliminates the sequential bottleneck while maintaining organized message flow through dedicated queues per sender.
Solution Approach 2:
The system transitions from a single-dimension sequential processing model to a multi-dimensional parallel processing model by introducing thread pools. Messages are no longer processed in a single linear sequence but across multiple concurrent threads, adding a temporal parallelism dimension that dramatically improves throughput.
2Productivity
If a single thread processes all incoming messages, then resource consumption is minimized, but processing capacity is limited by the single thread's throughput
Solution Approach 1:
The system implements dynamic thread pool management where the number of active threads adjusts based on message load and queue states. Thread pools are created dynamically for each sender based on actual messaging needs, allowing the system to scale resource consumption proportionally with workload rather than maintaining a fixed large number of threads.
Solution Approach 2:
The system changes the parameter of thread quantity from a fixed single value to a dynamic range based on sender count and message volume. By allowing the thread pool size to vary with system state, the system optimizes between resource efficiency and processing capacity, creating threads only when needed for specific senders.
Data Source
AI summary
A method and apparatus for processing message is described. In one embodiment, an application programming interface is configured for receiving and sending messages. A building block layer is coupled to the application programming interface. A channel layer is coupled to the building block layer. A transport protocol stack is coupled to the channel layer for implementing properties specified by the channel layer. The transport protocol stack has a concurrent stack consisting of an out of band thread pool and a regular thread pool. The transport protocol layer is to process messages from each sender in parallel with the corresponding channel for each sender.