Dispersed Storage Network Data Retrieval via Erasure Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current distributed storage and task processing systems face challenges in efficiently managing and retrieving large amounts of data across multiple geographically dispersed storage units, particularly in ensuring data integrity and security while handling complex tasks, and in optimizing storage and processing operations.
Innovation Solution
A distributed computing system that employs dispersed error encoding and decoding schemes, allowing data to be segmented, encoded, and stored across multiple geographically dispersed storage units, with a network of execution units capable of executing tasks on encoded data slices, ensuring data integrity and security through error correction and secure encoding techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored using traditional RAID systems or simple cloud storage, then storage capacity and accessibility are improved, but data integrity and security are compromised in the face of failures or attacks
Solution Approach 1:
The patent segments data into multiple slices and disperses them across different storage units in a distributed network. Each slice is independently stored, and the system uses erasure coding to reconstruct the original data from any sufficient subset of slices, thereby improving reliability without requiring complex redundant copying mechanisms
Solution Approach 2:
The patent introduces an intermediary encoding layer that transforms original data into encoded slices before storage. This intermediary representation provides built-in error correction and security features, allowing the system to tolerate failures and attacks without compromising data integrity
2Reliability
If data is dispersed across multiple geographically distributed storage units, then system reliability and security are improved, but data retrieval complexity and latency increase
Solution Approach 1:
By segmenting data into smaller slices distributed across multiple storage units, the system enables parallel retrieval operations. Multiple slices can be fetched simultaneously from different locations and reassembled, reducing overall retrieval time while maintaining the reliability benefits of geographic distribution
Solution Approach 2:
The patent allows data retrieval with fewer than the total number of stored slices by using erasure coding. The system can reconstruct the original data from a sufficient subset of slices, enabling partial action that reduces retrieval time while maintaining data integrity
3Reliability
If redundant copies of data are stored for backup and recovery, then data security and recovery capability are improved, but storage space consumption and cost increase
Solution Approach 1:
The patent extracts the redundancy function from traditional backup approaches and integrates it into the encoding process itself. Through erasure coding, the system embeds error correction and recovery information within the encoded slices, eliminating the need for separate redundant storage while maintaining recovery capability
Solution Approach 2:
The patent changes the fundamental parameter of how redundancy is achieved - moving from storing multiple complete copies to storing encoded slices where redundancy is mathematically embedded. This parameter change enables the system to achieve the same or better reliability with significantly reduced storage space
Data Source
AI summary
A method begins with a group of storage units of a dispersed storage network (DSN) receiving a common access request. The method continues with each storage unit of the group of storage units interpreting the access request to determine whether the storage unit is an intended recipient of the access request, where storage units of a sub-set of storage units of the group of storage units are intended recipients. The method continues with each storage unit of the sub-set of storage units determining a type of the access request and a level of the access request. When the access request is a read request, the method continues with each storage unit of the sub-set of storage units retrieving each encoded data slice having an identifier of its slice address substantially matching the a given identifier of the level to produce a retrieved set of encoded data slices.