Adaptive Error Encoding for Distributed Data Slice Writes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current dispersed storage networks face challenges in securely and reliably storing and retrieving large amounts of data across geographically diverse locations, particularly in maintaining data integrity and handling failures without redundant copies, while ensuring security against hacking attempts.
Innovation Solution
A distributed computing system that employs dispersed error encoding and decoding techniques, where data is segmented, encoded, and distributed across multiple storage units, allowing for secure storage and retrieval with error correction capabilities, and includes a mechanism for rebuilding corrupted or missing slices to maintain data integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is distributed across multiple storage units using error encoding, then data reliability and fault tolerance are improved, but system complexity increases
Solution Approach 1:
The patent segments data into multiple slices that are distributed across different storage units. Each slice is encoded using error correction codes, allowing the system to reconstruct original data even if some slices are lost or corrupted. This segmentation approach improves reliability by eliminating single points of failure while managing complexity through standardized encoding procedures.
Solution Approach 2:
The system dynamically adjusts error encoding parameters such as the number of data slices, redundancy level, and error correction code strength based on storage conditions and data criticality. By changing these parameters, the system optimizes the balance between reliability and complexity for different operational scenarios.
2Reliability
If error correction encoding is applied to data slices, then data security against hacking attempts is improved, but processing time increases
Solution Approach 1:
Error correction codes are applied to data slices during the initial storage process rather than during retrieval or verification. This preliminary encoding ensures security and integrity are established upfront, reducing processing time during data access operations while maintaining strong security protections.
3Reliability
If data is encoded and distributed across geographically diverse locations, then fault tolerance is improved, but network bandwidth requirements increase
Solution Approach 1:
The system stores more data slices than the minimum required for reconstruction, allowing any subset of slices to be used for data retrieval. This excessive redundancy enables fault tolerance across geographically diverse locations while reducing network bandwidth requirements, as data can be reconstructed from any available slices without requiring specific high-bandwidth connections.
Data Source
AI summary
A method includes writing sets of encoded data slices to storage units of a storage network in accordance with error encoding parameters, where for a set of encoded data slices, the error encoding parameters include an error coding number and a decode threshold number, the error coding number indicates a number of encoded data slices that results when a data segment is encoded using an error encoding function and the decode threshold number indicates a minimum number needed to recover the data segment. The method further includes monitoring processing of the writing the sets of encoded data slices to produce write processing performance information. When the write processing performance information compares unfavorably to a desired write performance range, the method further includes adjusting at least one of the error coding number and the decode threshold number to produce adjusted error encoding parameters for writing subsequent encoded data slices.