Paged Topics Messaging in Distributed Data Grids
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing messaging systems in distributed computing environments face challenges in maintaining message order and scalability, particularly in distributed data grids, where contention among publishers and subscribers degrades performance and limits the use of distributed processing power and memory.
Innovation Solution
The implementation of a paged topics messaging system, which uses a hash function to distribute messages across multiple partitions, reducing contention and maintaining order through a linked list-like structure within each partition, allowing for efficient publishing and subscribing while leveraging the distributed data grid's capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If messages are stored in a linked list structure to maintain ordering, then message order is preserved, but contention among publishers and subscribers increases and performance degrades
Solution Approach 1:
The patent segments the single linked list into multiple linked lists, each maintained by a different node in the distributed data grid. Publishers can choose which list to publish to based on a hash of the message key, and subscribers can choose which list to read from. This segmentation reduces contention because multiple publishers and subscribers can operate on different lists simultaneously without interfering with each other, while each individual list maintains its ordering property.
2Stability of the object's composition
If the linked list is limited to a single node to solve ordering, then message order is maintained, but distributed processing power and memory are not utilized
Solution Approach 1:
The patent divides the messaging system into multiple segments distributed across different nodes in the data grid. Each node maintains one or more linked lists for specific message keys or key ranges. This allows the system to utilize the distributed processing power and memory of multiple nodes while preserving order within each node's local lists.
Solution Approach 2:
The patent introduces a new dimension of distribution by organizing linked lists across multiple nodes rather than confining them to a single node. Messages are routed to specific nodes based on a hash function applied to the message key, creating a distributed structure that scales with the number of nodes while maintaining local ordering guarantees.
3Adaptability or versatility
If multiple publishers and subscribers contend for the same linked list, then messaging functionality is maintained, but performance degrades due to increased contention
Solution Approach 1:
The patent segments the messaging workload by creating multiple independent linked lists that can be assigned to different nodes. Publishers select which list to use based on the message key, and subscribers select which list to consume from. This segmentation allows multiple publishers and subscribers to operate concurrently with minimal contention, as they are working with different segments of the overall messaging system.
Solution Approach 2:
The patent applies local quality by allowing different nodes to specialize in handling specific message keys or key ranges. Each node optimizes its local linked lists for the messages it handles, and the system as a whole benefits from the distributed specialization. This reduces contention because each node becomes an expert in handling its assigned messages independently.
Data Source
AI summary
A system and method for providing store and forward messaging on a partitioned cache provided by a distributed data grid. The store and forward message system implements publish and subscribe semantics in a way that distributed messages over the partitions of the distributed cache, reduces contention and makes efficient use of processor and network resources.