Local Data Copying After Drive Failure in Distributed Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed storage systems face challenges in maintaining data durability and availability while minimizing storage overhead, especially when individual storage devices or nodes fail, leading to potential data loss and increased costs.
Innovation Solution
A method that involves determining a failed drive in a node, broadcasting failure information, and creating local copies of data components on other nodes to protect against further failures, utilizing a +2 error correction policy to ensure data recovery without requiring excessive storage resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is distributed across multiple nodes with traditional replication, then data availability is improved, but storage overhead increases
Solution Approach 1:
The system performs preliminary actions by creating local copies of data components on other nodes within the same data center before failures occur. When a drive fails, the system can rapidly restore data using these pre-positioned copies, avoiding the need for extensive cross-datacenter replication and reducing overall storage overhead while maintaining high availability.
Solution Approach 2:
The patent applies local quality by implementing different replication strategies for different locations. Within the same data center, nodes store local copies of data components for fast recovery. Cross-datacenter replication is minimized, creating a hierarchical approach where local recovery is prioritized over remote redundancy, thereby reducing storage overhead while maintaining availability.
2Reliability
If cross-datacenter replication is used for data protection, then data durability is improved, but storage costs increase
Solution Approach 1:
The system pre-positions data copies locally within data centers before failures occur. This preliminary action enables rapid restoration without requiring extensive cross-datacenter data transfer, reducing the need for expensive remote replication infrastructure and lowering storage costs while maintaining durability through local redundancy.
Solution Approach 2:
The patent uses copying to create local replicas of data components on nodes within the same data center. These copies serve as immediate backups that can be used for restoration without expensive cross-datacenter operations, providing durability at lower cost by replacing some cross-datacenter replication with local copying.
3Reliability
If extensive replication is implemented for data protection, then data loss tolerance is improved, but storage efficiency deteriorates
Solution Approach 1:
The system implements local quality by concentrating replication efforts within data centers rather than distributing copies across all locations. Nodes within the same data center store local copies of data components, providing loss tolerance for drive failures while maintaining high storage efficiency by avoiding redundant cross-datacenter replication.
Solution Approach 2:
The system performs preliminary replication actions locally within data centers, creating sufficient redundancy for drive failure recovery without implementing extensive global replication. This preliminary local replication achieves the needed data loss tolerance while preserving storage efficiency by limiting the scope of replication to local boundaries.
Data Source
AI summary
The technology disclosed herein enhances data protection in a distributed storage system. In a particular example, a method includes determining a drive in a subject node of the distributed storage system has failed while storing first data of a data set distributed across nodes of the distributed storage system by a data protection mechanism. The method further includes broadcasting failure information indicating the data set from the subject node to other nodes of the distributed storage system. At the other nodes, in response to receiving the failure information, the method includes identifying a subset of the other nodes that also store a portion of the data set. In each identified node of the subset, the method includes identifying second data of the data set stored on a local drive and copying the second data to a different local drive to protect the data set from further drive failure.