Segmented Thread Pool Message Broker for Asynchronous Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In PaaS environments, shared worker threads lead to high backlogs and delays when one client fails, affecting all clients due to latency and failed messaging, especially with high volume clients impacting low volume ones, and persistence mechanisms fail to meet real-time expectations.
Innovation Solution
Implementing a circuit breaker-based message broker with multiple thread pools and a message dealer service that assigns messages to threads based on client volume and responsiveness, using a REST interface and message broker to manage and process requests efficiently, with a circuit breaker design pattern to handle failures and prioritize high volume clients.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If shared worker threads are used to update clients with statuses, then resource utilization is improved, but system reliability deteriorates when one client fails due to latency or messaging failures
Solution Approach 1:
The patent segments the worker thread pool into multiple independent thread pools, each dedicated to serving specific clients or client groups. This segmentation prevents a single client's failure from impacting the entire thread pool, as each thread pool operates independently. When one client fails, only the threads serving that specific client are affected, while other thread pools continue to function normally, thus maintaining system reliability while still achieving resource utilization through dedicated pools.
2Reliability
If all worker threads are utilized for a failing destination, then message delivery attempts are maintained, but throughput for all clients deteriorates due to high backlogs
Solution Approach 1:
By segmenting the thread pool into multiple independent pools, the patent ensures that when one destination fails, only the threads in that specific pool are occupied with retry attempts. Other thread pools remain available to process messages for successful destinations, thereby maintaining overall system throughput while still making persistent delivery attempts for the failing destination through its dedicated threads.
3Reliability
If persistence is implemented at the service provider end with message broker, then message delivery reliability is improved, but real-time response expectations deteriorate due to high backlog
Solution Approach 1:
The patent implements persistence through message brokers in a segmented manner, where each thread pool has its own dedicated message broker connection or queue. This segmentation allows high-volume clients to utilize persistence mechanisms without creating backlogs that affect low-volume clients. Each segment processes messages independently, ensuring that persistence operations for one client do not delay real-time responses for other clients, thus maintaining both reliability and real-time performance.
Data Source
AI summary
One example method of operation may include one or more of receiving one or more messages from one or more clients, transmitting receipt notice messages acknowledging the one or more messages were received, assigning the one or more messages to a first thread of a various threads in a thread pool, and determining whether the messages are processed in a threshold amount of time assigned to the first thread.