Encoded Data Rebuilding Rate Selection in Distributed Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage and processing systems face challenges in efficiently and securely storing and retrieving large volumes of data across distributed networks while maintaining data integrity and resilience against failures without the need for redundant copies.
Innovation Solution
A distributed computing system that employs dispersed storage and task processing units, utilizing error correction schemes to encode data slices and distribute them across geographically diverse locations, enabling secure, resilient, and efficient data storage and retrieval with built-in error detection and reconstruction capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored using traditional redundant copying methods, then data reliability is improved, but storage efficiency deteriorates due to wasted space on duplicate copies
Solution Approach 1:
The patent divides data into multiple encoded slices distributed across different storage locations. Instead of storing complete redundant copies, the data is segmented and encoded such that any subset of slices can reconstruct the original data, eliminating the need for full duplicate storage while maintaining reliability
Solution Approach 2:
The patent transforms data from its original form into encoded representations using error correction coding. This parameter transformation allows the data to be stored in a form that is more resilient to loss and enables efficient reconstruction without requiring traditional redundant copying
2Reliability
If data is encoded and distributed across geographically diverse locations, then resilience against failures is improved, but system complexity increases
Solution Approach 1:
The system performs self-verification through built-in error detection and correction codes. The encoded slices contain redundant information that enables automatic detection and correction of errors without requiring complex external verification systems, allowing the data to serve its own integrity checks
Solution Approach 2:
The encoded data slices serve multiple functions simultaneously: they store the actual data, provide error detection capability, enable error correction, and allow for efficient reconstruction. This multi-functionality reduces the need for separate systems for each purpose, managing complexity
3Reliability
If error correction schemes are used to encode data slices, then data integrity is improved, but processing time increases
Solution Approach 1:
Error correction codes are applied during the initial data encoding and storage phase rather than during retrieval. This preliminary application of error correction prepares the data in advance, so that during retrieval the system can quickly decode and reconstruct data without time-consuming error correction processing
Solution Approach 2:
The system uses partial error correction by requiring only a subset of encoded slices for reconstruction rather than all slices. This partial action approach allows faster retrieval by not requiring the complete set of encoded data, reducing processing time while maintaining integrity
Data Source
AI summary
A storage unit of a storage network is operable to determine rebuilding performance parameter values, where data is dispersed storage error encoded into a set of encoded data slices in accordance with error encoding parameters for storage via a set of storage units that includes the storage unit. A rate of internal rebuilding of encoded data slices is selected based on the rebuilding performance parameter values. At least one encoded data slice of the set of encoded data slices is internally rebuilt in accordance with the rate of internal rebuilding.