Cooperative Data Forwarding With Encoded Slices for Failure-Tolerant Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current cloud data storage systems face challenges in efficiently managing and storing large volumes of data across distributed networks, particularly in maintaining data integrity and security while being resilient to storage unit failures without the need for redundant copies.
Innovation Solution
A dispersed storage network (DSN) is implemented, utilizing computing devices with a managing unit and integrity processing unit, which employs error encoding and decoding techniques like Cauchy Reed-Solomon encoding to distribute data into encoded slices stored across multiple geographically dispersed storage units, ensuring data integrity and security through error correction and secure storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored in a centralized cloud storage system, then data access and management are simplified, but the system becomes vulnerable to single points of failure and cannot tolerate storage unit failures without redundant copies
Solution Approach 1:
The patent segments data into multiple encoded slices and disperses them across different storage units in the network. Each slice is independently stored, so that no single storage unit holds the complete data. This segmentation enables the system to tolerate failures of individual storage units while maintaining data accessibility through the remaining slices.
Solution Approach 2:
The patent introduces an intermediary encoding mechanism (Cauchy Reed-Solomon encoding) that transforms original data into encoded slices before distribution. This intermediary process adds redundancy and error correction capabilities, allowing the system to retrieve and reconstruct original data even when some storage units fail, without requiring simple direct storage copies.
2Reliability
If redundant copies of data are stored to ensure data integrity and availability, then data loss is prevented, but storage space is wasted and system efficiency decreases
Solution Approach 1:
The patent changes the parameter of data representation by applying Cauchy Reed-Solomon encoding, which transforms original data into a different form (encoded slices) that contains redundant information in a mathematically optimized way. This encoding allows the system to achieve data integrity and availability with less total storage space compared to simple redundant copying, because the redundancy is embedded efficiently within the encoded structure rather than requiring separate duplicate copies.
3Reliability
If data is dispersed across multiple geographically distributed storage units, then system scalability and reliability are improved, but data retrieval and processing efficiency may be reduced due to distributed access
Solution Approach 1:
The patent performs preliminary encoding of data into multiple slices before distribution. This preliminary action organizes the data in advance so that during retrieval operations, the system can efficiently access only the necessary slices from distributed storage units and reconstruct the original data through the decoding process, rather than needing to retrieve and process all data from all storage units.
4Reliability
If error correction encoding is applied to ensure data integrity without redundant copies, then data security and reliability are enhanced, but processing complexity and computational overhead increase
Solution Approach 1:
The patent applies Cauchy Reed-Solomon encoding, which is a well-established error correction algorithm with optimized mathematical properties. This encoding scheme changes the data representation into a form that provides robust error correction capabilities while maintaining reasonable processing complexity, as the mathematical operations are well-understood and can be efficiently implemented. The encoding adds structured redundancy that enables reliable data recovery without requiring excessive computational resources compared to less optimized error correction methods.
Data Source
AI summary
A method begins with a first group of computing devices of a plurality of computing devices of a storage network receiving data objects having a first data type identifier of a plurality of data type identifiers, where the plurality of data type identifiers correspond to a plurality of data types associated with the data objects. The method continues with the first group of computing devices interpreting the data objects having the first data type identifier to sort, based on sorting criteria the data objects into a first processing category and a second processing category. The method continues with the first group of computing devices error encoding the data objects in the second processing category based on short term storage error encoding parameters to produce pluralities of sets of encoded data slices and sending the slices to storage and execution units.