Storage Node Bandwidth Allocation for Redirected Snapshot Replication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge of efficiently allocating bandwidth to individual nodes in a cluster while avoiding overloading or underutilization due to unknown node-to-port mapping and unmapped nodes in data replication requests across clusters, particularly when network throttling is applied.
Innovation Solution
Implementing techniques to allocate bandwidth based on node mapping, where nodes directly mapped to a NAT gateway receive allocation, while unmapped nodes have their requests redirected and bandwidth allocated to the redirecting nodes, with tracking to avoid overloading and ensure efficient distribution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If bandwidth is allocated to individual nodes in a cluster, then network utilization efficiency is improved, but complexity increases due to unknown node-to-port mapping and unmapped nodes
Solution Approach 1:
The patent introduces a gateway as an intermediary component that sits between external networks and the cluster of storage nodes. The gateway maintains the complex node-to-port mapping information and handles the complexity of bandwidth allocation, while individual storage nodes simply communicate with the gateway without needing to know about other nodes or ports. This intermediary absorbs the complexity while enabling efficient bandwidth management.
2Loss of energy
If network throttling is applied to regulate bandwidth, then available bandwidth for other services is maintained, but risk of overloading individual nodes increases due to unmapped nodes
Solution Approach 1:
The patent implements a feedback mechanism where the gateway monitors the load status of each storage node and dynamically adjusts bandwidth allocation accordingly. When a node approaches its capacity limits, the gateway receives feedback about the node's current load and reduces or redirects its bandwidth allocation to prevent overload. This closed-loop control enables network throttling while protecting individual nodes from being overloaded.
3Loss of energy
If replication requests are distributed across multiple nodes, then cluster-wide bandwidth limits are maintained, but efficiency decreases due to unmapped nodes requiring redirection
Solution Approach 1:
The patent applies preliminary action by having the gateway pre-establish and cache the node-to-port mapping information before replication requests arrive. The gateway proactively manages the mapping relationships and prepares bandwidth allocation plans in advance. When replication requests come in, the gateway can immediately route them to appropriate mapped nodes without needing to perform real-time lookups or redirections, thus maintaining both bandwidth limits and replication efficiency.
Data Source
AI summary
Methods, systems, and devices for data management are described. A first computing node within a first set of computing nodes may receive, from a network gateway, a first replication request that is associated with a first replication job for the first cluster of computing nodes to provide copies of a first set of one or more computing snapshots to a second cluster of computing nodes. The first set of computing nodes may determine that a second computing node different from the first computing node will service the data replication request. A network bandwidth allocation may be increased for the first computing node that received the replication request directly from the network gateway while the network bandwidth allocation for the second computing node is maintained for the second computing node based on the second computing node receiving the request via an internal redirection of the request.