MQTT Broker Cell Matrix Architecture for Scalable Resilience
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing MQTT implementations face limitations in network performance, resiliency, and scalability due to single-node broker designs, which constrain application resiliency and scale, and introduce bottlenecks and single points of failure.
Innovation Solution
The implementation of an MQTT broker cell comprising multiple back-end and front-end brokers arranged in a matrix structure, with vertical and horizontal chains for state update replication, enabling scalable and resilient network architecture.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single-node broker design is used, then the system is simple to implement, but network performance, resiliency, and scalability are limited
Solution Approach 1:
The broker is segmented into multiple independent nodes organized in a matrix structure with m vertical chains and k horizontal chains. Each node maintains a portion of the system state, distributing the broker functionality across multiple components rather than relying on a single node. This segmentation enables the system to tolerate node failures while maintaining operational capacity.
Solution Approach 2:
The patent implements a nested replication structure where state updates propagate through vertical chains from head to tail nodes, and horizontal chains replicate across all nodes. This nested chain structure allows each node to be part of multiple replication paths, providing redundant state copies while maintaining organized state management within the distributed architecture.
2Productivity
If a single-node broker is used, then the implementation is straightforward, but the system cannot scale to large networks
Solution Approach 1:
The broker system is divided into m vertical chains with k nodes each, allowing the network to scale by adjusting m and k independently. This segmentation enables horizontal scaling (adding more chains) and vertical scaling (adding more nodes per chain) without requiring complete system redesign, thus supporting large network deployments.
Solution Approach 2:
The patent transitions from a single-dimension (single-node or simple cluster) architecture to a two-dimensional matrix structure with vertical and horizontal chains. This dimensional expansion allows the system to scale along multiple independent axes, providing flexible capacity expansion and load distribution that supports large-scale networks while maintaining manageable complexity through structured organization.
3Reliability
If replication is implemented across multiple nodes, then fault tolerance improves, but replication overhead increases
Solution Approach 1:
The nested chain structure allows state updates to be replicated efficiently through organized propagation paths. Vertical chains replicate state from head to tail nodes, while horizontal chains provide additional replication across the matrix. This nested organization reduces redundant transmissions by structuring replication along defined chains rather than requiring all-to-all communication, thus lowering replication overhead while maintaining fault tolerance.
Solution Approach 2:
The system uses selective copying of state updates along chain structures rather than full state synchronization between all nodes. Each node copies only the necessary state portions relevant to its position in the chains, reducing the amount of data that needs to be replicated and processed compared to full state copying approaches.
4Reliability
If a single-node broker is used, then the system requires no third-party storage, but the broker becomes a single point of failure
Solution Approach 1:
The broker functionality is segmented across multiple nodes in the matrix structure, eliminating the single point of failure inherent in single-node designs. Each node in the vertical and horizontal chains can continue operating independently if other nodes fail, providing fault tolerance without requiring external third-party storage systems.
Solution Approach 2:
The patent combines the storage and computation functions within the broker nodes themselves, merging the role of message broker with distributed state storage. This integration eliminates the need for separate third-party storage systems while distributing the storage burden across multiple broker nodes, thereby achieving both fault tolerance and architectural simplicity.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
Examples are disclosed that relate to message queuing telemetry transport (MQTT) broker cells. One example provides a computing system configured to implement an MQTT broker cell comprising instructions executable to operate two or more back-end brokers arranged in a matrix, the matrix comprising m vertical chains of back-end brokers and k back-end brokers in each vertical chain, each vertical chain comprising at least a head back-end broker and a tail back-end broker, each vertical chain configured to replicate a state update received at the head back-end broker through the vertical chain to the tail back-end broker, and operate n front-end brokers, each front-end broker configured to output a control message to a selected vertical chain of the m vertical chains and to output an application message for publication to subscribers and to one or more other MQTT broker cells. The instructions are further executable to operate r networking devices.