Storage Cluster Redundancy via Static Zone Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current storage systems face inefficiencies in expanding capacity, particularly when adding new nodes, as they require extensive data migration, leading to prolonged downtime and performance degradation, and do not effectively utilize new nodes' resources until the migration is complete.
Innovation Solution
The method involves defining static maps for different node configurations to minimize data movement during capacity expansion by mirroring data across nodes, allowing new nodes to contribute immediately and maintaining zone ownership to simplify locking and handle failures, thus enabling efficient horizontal capacity expansion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If data is re-striped across all nodes when a new node is added, then data is evenly distributed across the cluster, but the migration process takes several days and drastically decreases cluster performance
Solution Approach 1:
The patent segments the data migration process by identifying and migrating only the specific storage zones that need to be moved to the new node, rather than re-striping all data across the entire cluster. This selective approach divides the migration task into manageable portions, reducing the overall migration time and performance impact.
Solution Approach 2:
The patent uses pre-computed maps that identify exactly which zones need to be migrated before the expansion operation begins. This preliminary identification allows the system to prepare the migration plan in advance, avoiding the need for comprehensive data movement and reducing the time the cluster spends in a degraded performance state.
2Adaptability or versatility
If comprehensive data migration is performed to expand cluster capacity, then new nodes are fully integrated, but the process is time-consuming and disrupts ongoing operations
Solution Approach 1:
The migration process is segmented to transfer only the specific zones designated for the new node, rather than migrating all data. This selective segmentation enables faster integration of new nodes while maintaining data distribution integrity, reducing the time loss associated with expansion operations.
Solution Approach 2:
The system performs preliminary identification of zones to be migrated using pre-computed maps before the actual expansion begins. This preliminary action allows the system to know exactly what data needs to move, enabling a more efficient and faster migration process that minimizes operational disruption.
3Productivity
If data is striped across multiple nodes to maximize resource utilization, then I/O performance is improved, but the complexity of managing data placement and handling failures increases
Solution Approach 1:
The patent introduces zone maps as an intermediary data structure that simplifies the management of striped data across multiple nodes. These maps provide a straightforward representation of data placement, making it easier to track which zones reside on which nodes and simplifying failure handling and data recovery operations.
Solution Approach 2:
The system pre-computes and stores zone maps that define data placement before operations begin. This preliminary action creates a reference guide for data management, reducing the complexity of real-time data placement decisions and failure recovery operations by having all necessary information readily available.
4Reliability
If existing capacity is depleted before migration completes, then the system lacks redundancy, but waiting for migration to complete delays capacity expansion
Solution Approach 1:
The patent segments the data migration into selective zone transfers rather than comprehensive re-striping, allowing the expansion process to complete much faster. This segmentation enables the system to achieve capacity expansion while maintaining redundancy, as the process completes before existing capacity is depleted.
Solution Approach 2:
The system uses pre-computed maps to identify and migrate only the necessary zones before existing capacity is depleted. This preliminary identification and selective migration allows the expansion to complete in time, ensuring redundancy is maintained while avoiding the delays associated with comprehensive data migration.
Data Source
AI summary
A system, method, apparatus, and computer-readable medium are described for providing redundancy in a storage system. According to one method, maps are generated and stored that define stripe patterns for storing data on the storage nodes of a storage cluster. The maps are defined such that when a new storage node is added to the cluster, no movement of data occurs between two storage nodes that existed in the cluster prior to the addition of the new storage node during re-striping, and such that the data stored on each storage node is mirrored on another storage node. Storage nodes may also be designated as an owner or a peer for each storage zone. Input/output operations received at an owner node are fielded directly and mirrored to the peer node, while input/output operations received at a peer node are redirected to the owner node for processing.