Deferred Slice Rebuilding Across Alternate Storage Locations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current dispersed storage networks face challenges in maintaining data integrity and availability due to storage unit failures, requiring redundant copies and complex error correction mechanisms, which can lead to increased costs and complexity.
Innovation Solution
A dispersed storage network architecture that uses Cauchy Reed-Solomon error encoding to divide data into encoded data slices, stored across multiple geographically distributed storage units, allowing for efficient error correction and data reconstruction without the need for redundant copies, using a managing unit to coordinate storage and integrity processing for data encoding, decoding, and rebuilding of corrupted slices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If redundant copies of data are stored to maintain data availability during storage unit failures, then data reliability is improved, but storage cost and system complexity increase
Solution Approach 1:
The patent segments data into multiple encoded slices using Cauchy Reed-Solomon error encoding. Instead of storing complete redundant copies of data, the system divides data into fragments and distributes them across multiple storage units. This segmentation allows the system to tolerate failures of up to k-1 storage units while maintaining data availability, reducing the overhead compared to traditional redundant copying methods.
Solution Approach 2:
The patent changes the storage parameter from storing complete data copies to storing encoded data slices. By applying Cauchy Reed-Solomon encoding with configurable parameters (total slices n, failure tolerance k), the system dynamically adjusts the balance between storage efficiency and reliability. This parameter-based approach allows flexible optimization based on specific reliability requirements without proportionally increasing complexity.
2Reliability
If error correction mechanisms are implemented to maintain data integrity, then data reliability is improved, but processing complexity and computational overhead increase
Solution Approach 1:
The patent applies error correction encoding to data before storage, creating encoded slices that inherently contain error correction capabilities. This preliminary action ensures that data integrity is maintained without requiring complex real-time error correction during retrieval. The Cauchy Reed-Solomon encoding is performed once during data ingestion, and the encoded slices are then stored as-is, simplifying the overall error correction mechanism.
Solution Approach 2:
The patent creates multiple copies of encoded data slices and distributes them across different storage units. Instead of copying complete data and then applying error correction, the system encodes data into slices first, then creates and distributes copies of these encoded slices. This approach reduces the computational overhead of error correction while maintaining data integrity, as the encoding is performed once on the original data.
3Reliability
If data is divided into encoded slices and distributed across multiple storage units, then data availability during failures is improved, but storage and retrieval time increase
Solution Approach 1:
The patent segments data into n encoded slices that are distributed across multiple storage units. During retrieval, the system only needs to gather k slices (where k < n) to reconstruct the original data, rather than retrieving all n slices or waiting for all storage units. This segmentation strategy reduces retrieval time while maintaining the ability to tolerate up to n-k failures, effectively balancing availability with time efficiency.
Solution Approach 2:
The patent implements partial action by requiring only k slices out of n total slices for data reconstruction. This means the system performs less work during retrieval than traditional methods would require, as it doesn't need to gather all distributed copies or perform exhaustive searches. The partial retrieval of k slices is sufficient to restore data availability, reducing the time loss associated with distributed storage operations.
Data Source
AI summary
A method includes, when a first batch rebuild threshold is met, a computing device of a dispersed storage network (DSN) determining whether target storage units of a set of storage units of the DSN are available, where a data segment of data is dispersed storage error encoded into a set of encoded data slices, a number of encoded data slices that need rebuilding corresponds to the first batch rebuild threshold, and the number of encoded data slices is to be stored in the target storage units. When a target storage unit is unavailable, the method continues by delaying a batch rebuild process until the target storage unit is available or a second batch rebuild threshold is met. When the target storage unit becomes available before the second batch rebuild threshold is met, the method continues by executing the batch rebuild process for the number of encoded data slices.


