Distributed Storage Replication Using Underlying Network Multicast
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data replication in distributed storage clusters is inefficient and resource-intensive due to the reliance on a primary node for unicast communications with each node, leading to slow replication speeds and excessive resource utilization.
Innovation Solution
Implementing data replication through an underlying network using multicast messaging, where data is initially sent from a primary node to a slave node, which then replicates it to other nodes within the network, eliminating the need for the primary node to send multiple copies and reducing resource consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If source-based replication with unicast communication is used, then data can be replicated across nodes, but replication speed is slow and network resources are excessively consumed
Solution Approach 1:
The patent merges multiple unicast communication streams into a single multicast stream. Instead of the primary node sending separate data copies to each slave node via unicast, the system uses a single multicast transmission that delivers data to all slave nodes simultaneously, thereby improving replication speed and reducing network resource consumption
Solution Approach 2:
The patent introduces an intermediary approach where slave nodes that receive data via multicast act as secondary sources to replicate data to other nodes. This intermediary replication mechanism reduces the burden on the primary node and improves overall replication efficiency by distributing the replication task across multiple nodes
2Ease of operation
If a primary node coordinates with each node individually, then data replication can be controlled, but the system becomes inefficient and resource-intensive
Solution Approach 1:
The patent combines multiple individual coordination interactions into a single multicast communication event. The primary node coordinates data replication with all slave nodes simultaneously through one multicast transmission rather than through multiple separate unicast interactions, improving coordination efficiency and reducing resource utilization
Solution Approach 2:
The patent implements partial action by allowing slave nodes to independently replicate data to other slave nodes after receiving the initial multicast transmission. This distributes the replication workload across multiple nodes rather than requiring the primary node to coordinate with every node individually, reducing overall system resource utilization
Data Source
AI summary
Systems, methods, and computer-readable media for replicating data in a distributed storage cluster using an underlying network. In some examples, a primary node of a placement group in a network overlay of a distributed storage cluster can receive data for replication in the placement group. The primary node can provide the data to a first slave node of a plurality of slave nodes within the placement group in an underlying network of the distributed storage cluster. The data can subsequently be replicated using the underlying network by providing the data to at least one other slave node of the plurality of slave nodes within the placement group in the underlying network directly from the first slave node in the underlying network.