RDMA Data Publisher Memory Allocation for Subscriber Pull
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Multicast data transport in communication systems faces issues such as significant overhead, unnecessary data delivery, latency, and dependency on the publisher for data delivery, leading to inefficiencies and reliability challenges.
Innovation Solution
Implementing a data publisher with a memory allocator for Remote Direct Memory Access (RDMA) operations, allowing subscribers to remotely pull data from message stores independently of the publisher's timing, using RDMA-capable hardware like InfiniBand to bypass the Operating System, enabling decoupling and efficient data distribution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multicast data transport is used to deliver data to multiple subscribers, then data distribution capability is improved, but system overhead and complexity increase significantly
Solution Approach 1:
The patent extracts the data delivery mechanism from the publisher's control domain by enabling subscribers to directly access and pull data from the publisher's memory space via RDMA. This eliminates the need for publisher-initiated multicast transport, removing the complex multicast protocol handling, group management, and delivery tracking overhead from the publisher while maintaining efficient one-to-many data distribution.
Solution Approach 2:
Subscribers independently retrieve data by performing RDMA read operations directly on the publisher's memory addresses. Each subscriber autonomously manages its own data retrieval without requiring publisher intervention, coordination, or push mechanisms. This self-service model eliminates the publisher's burden of managing multiple subscriber connections and data push operations.
2Productivity
If the publisher actively pushes data to all subscribers, then data delivery coverage is improved, but data relevance and efficiency deteriorate due to unnecessary data delivery
Solution Approach 1:
The patent inverts the traditional push model by implementing a pull model where subscribers initiate data retrieval. Instead of the publisher pushing data to all subscribers regardless of need, subscribers actively pull only the data they require from the publisher's memory space. This inversion ensures that data transmission occurs only when and where it is needed, eliminating wasted bandwidth on unnecessary data delivery.
3Reliability
If re-transmit mechanisms are implemented to improve data delivery reliability, then reliability is improved, but system performance deteriorates due to multicast storm
Solution Approach 1:
The patent removes the re-transmit mechanism from the publisher's responsibilities by enabling direct subscriber access to data in the publisher's memory space. Since subscribers can directly read data via RDMA, there is no need for publisher-side re-transmission protocols, acknowledgments, or retry logic. This extraction of the reliability mechanism eliminates the potential for multicast storms while maintaining data delivery reliability through the inherent reliability of RDMA operations.
4Stability of the object's composition
If synchronization scheme is implemented between publisher and subscribers, then data consistency is improved, but system complexity and latency increase
Solution Approach 1:
Subscribers independently manage their data retrieval timing and pace without requiring synchronization with the publisher or other subscribers. Each subscriber autonomously determines when to pull data based on its own processing needs, eliminating the need for complex synchronization protocols, timing coordination, or state management between publisher and subscribers while maintaining data consistency through direct memory access.
5Reliability
If publisher maintains active delivery to all subscribers, then data availability is improved, but publisher resource consumption and dependency increase
Solution Approach 1:
Subscribers independently retrieve data directly from the publisher's memory space using RDMA operations, eliminating the need for the publisher to actively push data to each subscriber. The publisher simply maintains data in its memory space, and subscribers autonomously pull data as needed. This dramatically reduces publisher CPU overhead, network stack processing, and resource consumption while maintaining continuous data availability.
Solution Approach 2:
The active data delivery function is extracted from the publisher and transferred to the subscribers. The publisher's role is reduced to data storage and passive availability, while subscribers assume responsibility for data retrieval. This extraction eliminates the publisher's burden of managing active delivery sessions, connection maintenance, and push operations to multiple subscribers.
Data Source
AI summary
Device, system, and method of publishing information to multiple subscribers. For example, a data publisher capable of communication with one or more subscribers includes: a memory allocator to allocate a memory area of a local memory unit of the data publisher for storing data to be accessible for Remote Direct Memory Access (RDMA) read operations by one or more of said subscribers.