Multicast Replicast Transport Layer for Storage Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage systems face challenges in efficiently replicating and distributing data across multiple storage servers due to congestion issues and the limitations of conventional point-to-point protocols, which hinder the reliability and efficiency of data transfer in cloud storage environments.
Innovation Solution
The implementation of a multicast-based Replicast transport layer that optimizes data replication by using unreliable datagrams and edge-based congestion control, allowing for efficient distribution of data across multiple servers while minimizing network congestion and packet drops.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If point-to-point protocols are used for data replication, then data can be transferred between individual servers, but network congestion increases and transfer efficiency decreases
Solution Approach 1:
The patent merges multiple unicast data transfer operations into a single multicast operation. Instead of establishing separate point-to-point connections from the source server to each destination server, the system uses a single multicast group to simultaneously deliver data to multiple recipients, thereby reducing network congestion and improving transfer efficiency.
Solution Approach 2:
The multicast mechanism serves multiple functions simultaneously: it replicates data to multiple destination servers, provides load balancing across the network, and enables efficient data distribution to negotiating groups. This multi-functional approach replaces multiple specialized point-to-point transfer operations with a single universal multicast operation.
2Reliability
If conventional replication protocols are used, then data can be distributed across storage servers, but reliability is hindered due to congestion and packet drops
Solution Approach 1:
The patent implements feedback mechanisms where destination servers send acknowledgments back to the source server to confirm successful data receipt. The source server tracks which destination servers have received the data and can retransmit to those that haven't, ensuring reliable delivery while maintaining efficient multicast-based distribution.
Solution Approach 2:
The system establishes multicast groups and configures network routing in advance before data transfer begins. This preliminary setup optimizes the network path and reduces congestion during the actual data transfer, thereby improving both reliability and efficiency simultaneously.
3Quantity of substance
If data is stored in chunks across multiple servers, then storage capacity is increased, but network congestion increases during replication
Solution Approach 1:
The patent segments data into chunks and distributes them across multiple storage servers using multicast. Each chunk can be independently replicated to multiple destination servers through separate multicast operations, allowing parallel data distribution that reduces network congestion while increasing total storage capacity.
Data Source
AI summary
The present disclosure provides methods and systems for multicast collaborative erasure encoding and methods and systems for distributed parity protection. One embodiment relates to a method of multicast collaborative erasure encoding of a chunk stored in a distributed object storage cluster. A roll-call request is multicast to every storage server in a negotiating group for the chunk. Roll-call inventory responses are generated and multicast by every storage server in the negotiating group. The roll-call inventory responses are collected by every storage server in the negotiating group from other storage servers in the negotiating group to form a set of roll-call inventory responses. A logical evaluation of the set of roll-call inventory responses may then be performed by every storage server in the negotiating group. Other embodiments, aspects and features are also disclosed.