Snapshot Distribution for Load Balancing Data Node Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In clusters of data nodes providing network-attached storage, the accumulation of snapshots for data recovery leads to imbalanced loading, resulting in poor read/write access due to the concentration of IO operations on a single data node, which can cause overloading and affect overall system performance.
Innovation Solution
Implementing a method that involves creating snapshots (snaps) on a primary SAN node, instantiating virtual NAS (VNAS) servers on multiple data nodes, and distributing these snaps across the cluster or between clusters to balance the load, using snap creation, instantiation, and selection instructions to manage filesystems and configure service level obligations (SLOs) for efficient access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If snapshots are accumulated on a single data node for data recovery, then data recovery capability is improved, but system performance deteriorates due to imbalanced loading and concentration of IO operations
Solution Approach 1:
The patent segments the snapshot storage function from the original data node and distributes it across multiple data nodes in the cluster. Instead of accumulating all snapshots on a single node, the system divides snapshot storage responsibilities among multiple nodes, thereby maintaining data recovery capability while preventing IO operation concentration and improving overall system performance.
2Productivity
If snapshots are distributed across multiple data nodes, then load balancing is improved, but system complexity increases due to additional management overhead
Solution Approach 1:
The patent implements a universal snapshot management mechanism that operates across all data nodes in the cluster. The system uses standardized protocols and interfaces for snapshot creation, distribution, and management that work consistently regardless of which data node is involved, thereby simplifying the management overhead despite the distributed architecture.
Solution Approach 2:
The system incorporates feedback mechanisms that monitor the distribution and status of snapshots across data nodes. This feedback enables automatic load balancing decisions and simplifies management by providing real-time visibility into snapshot locations and system state, reducing the complexity of managing distributed snapshots.
Data Source
AI summary
Snaps are distributed among data nodes for load balancing and overload avoidance. A snap is initially associated with a first VNAS server on a first data node that hosts the primary volume being snapped. A second data node is selected based on loading. The second data node may be in the same or a different cluster. A second VNAS server is instantiated on the second data node. The snap is then mounted on the second VNAS server. The second VNAS server may be configured as the target for all reads to the snap. If the second data node is in a different cluster than the snap may be copied or moved from a first SAN node to a second SAN node. Multiple copies of snaps may be distributed, and individual snaps and copies may be associated with different SLOs.