Federated Publish-Subscribe Message Broker Delta Data Transfer

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current federated publish/subscribe message broker technologies are inefficient in data transfer, particularly in scenarios where in-flight message processing occurs, leading to unnecessary duplication and increased network traffic, especially in resource-constrained edge devices with slow and unreliable networks.

Innovation Solution

Implement a method where a message broker performs in-flight processing on messages, compiling original and processed messages into delta data sets, allowing only the original message and delta data sets to be sent between brokers, enabling efficient re-construction of messages at the receiving broker, thereby reducing network traffic.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If multiple copies of the same message are sent between brokers when there are many subscribers, then all subscribers receive the message, but network bandwidth is wasted due to unnecessary duplication

Engineering Contradiction:

Improvemessage delivery completenessVSAvoidnetwork bandwidth usage

Core Design Contradiction:

ReliabilityVSLoss of energy

Solution Approach 1:

The patent applies copying by sending only one message between brokers and using virtual copying at the receiving broker. The receiving broker creates multiple copies of the message locally to distribute to multiple subscribers, rather than sending multiple copies over the network. This maintains complete message delivery to all subscribers while eliminating unnecessary network bandwidth consumption.

Inventive Principle:

Principle #26Copying

2Loss of energy

If only one message is sent between brokers even with many subscribers, then network bandwidth is saved, but the complexity of managing message distribution increases

Engineering Contradiction:

Improvenetwork bandwidth usageVSAvoidmessage distribution management

Core Design Contradiction:

Loss of energyVSDevice complexity

Solution Approach 1:

The receiving broker acts as an intermediary that receives a single message from the sending broker and then distributes it to multiple subscribers. This intermediary approach simplifies the network communication (only one message needs to be sent) while the receiving broker handles the complexity of managing multiple subscriptions and creating local copies for distribution.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Reliability

If multiple copies of messages are sent to account for in-flight processing, then processed messages are delivered to subscribers, but network traffic increases significantly

Engineering Contradiction:

Improveprocessed message deliveryVSAvoidnetwork traffic volume

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

Inventive Principle:

Principle #26Copying

4Adaptability or versatility

If the hub and spoke model is extended to provide bus or mesh models, then federation across brokers is achieved, but data transfer efficiency decreases

Engineering Contradiction:

Improvefederation communication modelVSAvoiddata transfer efficiency

Core Design Contradiction:

Adaptability or versatilityVSLoss of energy

Solution Approach 1:

The patent segments the message distribution function between the network transport layer and the application layer. The network layer (broker-to-broker communication) sends only one message, while the application layer (broker-to-subscriber distribution) handles multiple copies locally. This segmentation allows the system to support various federation models (bus, mesh) while maintaining data transfer efficiency by keeping network traffic minimal.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS10542108B2Data transfer in a federated publish/subscribe system

Publication Date: 2020.01.21 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

A method and federated publish/subscribe system for data transfer in the federated publish/subscribe system. The federated publish/subscribe system includes a processor, a first message broker, and a second message broker. The second message broker transmits, via the processor, a message A to a first subscribing application and a message (A+Δ) to a second subscribing application. The message (A+Δ) differs from the message A by changes Δ. The first subscribing application and the second subscribing application are different subscribing applications. Prior to the transmitting by the second message broker, the second message broker receives, from the first message broker via the processor, either (i) a message (A, Δ) comprising the message A and the changes Δ or (ii) the message A and a message Δ comprising the changes Δ.