Partial Node Rebuild for Distributed Storage Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data recovery techniques in distributed storage systems often require rebuilding an entire storage node, which can be resource-intensive and lead to network burdens, delayed transactions, and increased vulnerability to further data loss.
Innovation Solution
The system implements a partial node rebuild method, where only the necessary data chunks are retrieved and rebuilt, reducing the network impact and allowing the storage node to continue servicing transactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If entire node rebuild is performed, then data recovery completeness is improved, but network burden increases and transaction processing is delayed
Solution Approach 1:
The patent segments the node rebuild process by identifying and prioritizing critical data chunks that must be recovered for system functionality. Instead of rebuilding the entire node uniformly, the system divides the recovery process into essential segments (critical chunks) and non-essential segments, recovering only what is necessary to restore system operation and data integrity.
Solution Approach 2:
The patent applies partial action by performing only the necessary portion of the rebuild process. The system determines the minimum set of data chunks required for node functionality and recovers only those, rather than performing a complete node rebuild. This partial recovery approach maintains data protection while reducing network burden and allowing continued transaction processing.
2Reliability
If entire node rebuild is performed, then data protection is improved, but network data transfer volume increases
Solution Approach 1:
The patent extracts only the necessary data chunks from the full node data set for recovery purposes. By analyzing which chunks are critical for maintaining data protection and system functionality, the system separates these essential chunks from the rest of the node data, transferring only the extracted critical portions over the network rather than the entire node data set.
Solution Approach 2:
The system performs partial recovery action by transferring and rebuilding only the minimum necessary data chunks required for data protection, rather than transferring the complete node data. This reduces network data transfer volume while maintaining adequate protection levels.
3Reliability
If entire node rebuild is performed, then recovery completeness is improved, but processing burden on rebuilding node increases
Solution Approach 1:
The patent segments the recovery workload into critical and non-critical portions. The rebuilding node processes only the essential data chunks that are necessary for system recovery and data protection, dividing the overall processing burden into manageable segments rather than handling the complete node rebuild simultaneously.
Solution Approach 2:
The system applies partial action by limiting the processing burden to only the necessary recovery operations. The rebuilding node performs processing only on the minimum required data chunks rather than the entire node data set, reducing energy consumption and processing load while achieving sufficient recovery completeness.
4Reliability
If entire node rebuild is performed, then data recovery reliability is improved, but recovery time increases
Solution Approach 1:
The patent implements partial recovery action by recovering only the critical data chunks necessary for system functionality rather than waiting for complete node rebuild. This allows the system to achieve adequate recovery reliability in less time, enabling faster return to service while maintaining data protection.
Solution Approach 2:
The system skips non-essential recovery steps by prioritizing and completing only the critical recovery operations first. By rushing through the essential recovery process and deferring or omitting non-critical rebuild steps, the system reduces overall recovery time while maintaining sufficient data protection and system functionality.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system, method, and machine-readable storage medium for recovering data in a distributed storage system are provided. In some embodiments, the method includes identifying a failing storage device of a first storage node having an inaccessible data segment. When it is determined that the inaccessible data segment cannot be recovered using a first data protection scheme, a first chunk of data associated with the inaccessible data segment is identified and a group associated with the first chunk of data is identified. A second chunk of data associated with the group is selectively retrieved from a second storage node such that data associated with an accessible data segment of the first storage node is not retrieved. The inaccessible data segment is recovered by recovering the first chunk of data using a second data protection scheme and the second chunk of data.