Cluster-Level Redundancy for Distributed Storage Node Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing distributed storage systems face inefficiencies in addressing node-level errors, often resorting to generalized recovery processes that are resource-intensive and may lead to unnecessary resource usage or double failures due to traditional RAID recovery/reconstruction techniques.
Innovation Solution
Implementing cluster-level redundancy mechanisms to specifically address identified node-level error scenarios, such as single block loss, RAID stripe errors, LSM tree branch missing, and missing block IDs, by reading redundant copies from unaffected nodes and writing them to storage areas within the RAID group.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional RAID recovery/reconstruction techniques are used to address node-level errors, then data integrity can be restored, but resource consumption increases and cascading failures may occur
Solution Approach 1:
The patent segments the recovery process by identifying specific error types (single block loss, RAID stripe errors, LSM tree branch missing, missing block IDs) and applying targeted recovery actions for each scenario rather than performing full RAID reconstruction. This segmentation allows recovery to focus only on affected data blocks, significantly reducing resource consumption while maintaining data integrity.
Solution Approach 2:
The patent applies local quality by reading redundant copies from unaffected nodes and writing them to specific storage areas within the RAID group that are localized to the failure. Instead of global reconstruction, the recovery process operates locally at the block level, using checksums to identify and restore only the corrupted blocks, thereby minimizing resource usage while ensuring data integrity.
2Ease of operation
If generalized recovery processes are used for all node-level errors, then simplicity is maintained, but efficiency decreases due to unnecessary resource usage
Solution Approach 1:
The patent changes the parameter of recovery granularity from system-level (full RAID reconstruction) to block-level (individual data block restoration). By using checksums to identify specific corrupted blocks and restoring only those blocks from redundant copies, the system maintains operational simplicity while dramatically improving recovery efficiency and reducing unnecessary resource consumption.
3Reliability
If full RAID reconstruction is performed, then data integrity is ensured, but recovery time increases and system availability decreases
Solution Approach 1:
The patent applies preliminary action by maintaining redundant copies of data blocks across multiple nodes before failures occur. When errors are detected, the system can immediately restore corrupted blocks from these pre-positioned redundant copies without waiting for complex reconstruction algorithms, significantly reducing recovery time while ensuring data integrity through the use of checksums for verification.
Data Source
AI summary
Systems and methods that make use of cluster-level redundancy within a distributed storage system to address various node-level error scenarios are provided. Rather than making use of a generalized one-size-fits-all approach in an effort to reduce complexity, an approach tailored to the node-level error scenario at issue may be performed to avoid doing more than necessary. According to one embodiment, responsive to identification of a failed RAID stripe by a node of a cluster of a distributed storage management system, for each block ID of multiple block IDs associated with the failed RAID stripe, a data block is restored corresponding to the block ID by reading the data block from another node of the cluster having a redundant copy of the data block; and writing the redundant copy of the data block to a storage area of the node that is unaffected by the failed RAID stripe.