Distributed Data Rebuild Using Performance-Based Node Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current dispersed storage networks face challenges in maintaining data integrity and security, particularly in the event of storage unit failures, as they often require redundant copies and are vulnerable to data corruption and unauthorized access.
Innovation Solution
The implementation of a dispersed storage network (DSN) with error encoding and decoding capabilities using Cauchy Reed-Solomon encoding, where data is segmented and encoded into multiple slices, stored across geographically diverse units, and decoded using a threshold-based approach, ensuring data recovery and security without the need for redundant copies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If redundant copies are used for data storage, then data reliability is improved, but storage efficiency deteriorates due to wasted space
Solution Approach 1:
The patent segments data into multiple slices and disperses them across different storage units. Instead of storing complete redundant copies, the system divides data into fragments (slices) and distributes them geographically, allowing data recovery with fewer stored copies while maintaining reliability.
Solution Approach 2:
The patent applies error encoding transformations to data slices, changing their mathematical representation through encoding functions. This allows the system to store encoded slices that can be decoded to recover original data, improving reliability without requiring simple redundant copies.
2Object-affected harmful factors
If data is dispersed across geographically diverse units, then security against unauthorized access is improved, but system complexity increases
Solution Approach 1:
The patent divides data into multiple slices stored at different geographic locations. This segmentation provides security because an attacker would need to compromise multiple distributed storage units to access complete data, while the system manages this complexity through automated encoding and decoding processes.
Solution Approach 2:
The patent introduces encoding functions as intermediaries between data and storage units. These encoding/decoding mechanisms manage the complexity of geographic distribution by automatically handling the mathematical transformations needed to reconstruct data from dispersed slices, shielding users from the underlying system complexity.
3Reliability
If error encoding is applied to data slices, then data integrity is improved, but processing time increases
Solution Approach 1:
The patent applies error encoding to data slices during the initial storage process rather than during retrieval or verification. This preliminary encoding action ensures data integrity is built-in from the start, and the encoded slices can be quickly decoded when needed, reducing processing time delays.
Data Source
AI summary
A distributed storage network method begins by determining an encoded data slice to rebuild, determining a storage set of distributed storage units associated with the encoded data slice to rebuild, determining an historical performance level of each distributed storage unit of the storage set of distributed storage units, determining an estimated performance level of each distributed storage unit of the storage set of distributed storage units, selecting at least a decode threshold number of distributed storage units of the storage set of distributed storage units to produce selected distributed storage units, determining an encoded data slice partial aggregation scheme for the selected distributed storage units, sending encoded data slice partial request messages to the selected distributed storage units, and receiving encoded data slice partial response messages and extracting encoded data slice partials from the encoded data slice partial response messages to rebuild or verify the encoded data slice.