Blocking Queue Mediator for Non-Blocking Concurrency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In large organizations with complex IT infrastructures, interconnecting and sharing resources across different computer hardware and software components is challenging, with few all-in-one platforms effectively supporting middleware machine environments.
Innovation Solution
The use of continuation-passing to transform a non-blocking queue into a blocking queue in a middleware machine environment, allowing for the management of idle workers in a thread pool and eliminating serialization points, enabling efficient resource utilization and concurrency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a non-blocking queue is used to maintain idle workers in a thread pool, then concurrency and performance are improved, but the queue cannot be accessed from outside and serialization points remain
Solution Approach 1:
The patent introduces a blocking queue as an intermediary component that wraps the non-blocking queue. This blocking queue provides external accessibility while maintaining the performance benefits of the non-blocking queue internally. The blocking queue acts as a mediator between external callers and the internal non-blocking queue, allowing serialization points to be eliminated while maintaining controlled access.
2Productivity
If continuation-passing is used to transform the queue, then serialization points are eliminated and thread management is enabled from outside, but the system complexity increases
Solution Approach 1:
The patent segments the queue system into distinct components: a non-blocking queue for internal high-performance operations and a blocking queue for external accessibility. This segmentation allows each component to optimize for its specific function while reducing overall system complexity through modular design.
Solution Approach 2:
The blocking queue serves as an intermediary layer that simplifies external access to the non-blocking queue without exposing its complex internal structure. This mediator handles the complexity of continuation-passing and thread management, shielding external callers from the underlying complexity.
3Ease of operation
If a blocking queue is used to allow external access, then ease of operation is improved, but throughput and cache efficiency may be reduced
Solution Approach 1:
By segmenting the queue into non-blocking and blocking components, the patent allows the non-blocking queue to handle high-throughput internal operations while the blocking queue manages external access. This segmentation preserves cache efficiency in the hot path (non-blocking queue) while providing ease of operation through the blocking queue interface.
Data Source
AI summary
A system and method can use continuation-passing to transform a queue from non-blocking to blocking. The non-blocking queue can maintain one or more idle workers in a thread pool that is not accessible from outside of the non-blocking queue. The continuation-passing can eliminate one or more serialization points in the non-blocking queue, and allows a caller to manage the one or more idle workers in the thread pool from outside of the non-blocking queue.