Encoded Data Slice Redundancy Sharing for Fault-Tolerant Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Dispersed storage systems face challenges in efficiently managing and recovering data across multiple storage units without data loss, especially when dealing with high-volume data access applications like video on demand, where concurrent read requests and storage of multiple data copies are required, and existing error correction schemes may not adequately address storage unit failures or data corruption.
Innovation Solution
A dispersed storage network (DSN) with a managing unit, integrity processing unit, and computing devices that use error encoding techniques like Cauchy Reed-Solomon encoding to split data into encoded data slices, which are then redundantly stored across multiple storage units, allowing for efficient recovery and decoding of data segments even with failures, and managing unit coordinates storage and retrieval processes to ensure data integrity and security.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple copies of encoded data slices are stored across different storage units, then data reliability and fault tolerance are improved, but storage space consumption and system complexity increase
Solution Approach 1:
The patent segments data into multiple encoded data slices using error correction encoding (e.g., Reed-Solomon or Cauchy Reed-Solomon codes). Each slice is stored on a different storage unit, allowing the system to tolerate failures of multiple storage units while maintaining data reliability. This segmentation approach resolves the contradiction by distributing data across multiple units without requiring complete duplication of entire data sets.
Solution Approach 2:
The patent changes the parameter of data representation from complete data copies to encoded data slices with specific redundancy properties. By using error correction codes with configurable redundancy levels, the system can adjust the balance between reliability and storage consumption. The encoding transforms the original data into a form where fewer slices are needed to reconstruct the original data, reducing overall storage requirements while maintaining fault tolerance.
2Productivity
If multiple concurrent read requests are handled by different storage units, then response time and throughput are improved, but coordination overhead and management complexity increase
Solution Approach 1:
The patent segments both data and request handling across multiple storage units. Each storage unit independently handles portions of read requests for different data slices, enabling parallel processing of concurrent requests. This segmentation allows throughput to scale with the number of storage units while keeping individual unit management relatively simple.
Solution Approach 2:
The patent creates a universal interface and protocol layer that standardizes how storage units handle read requests. The dispersed storage network interface defines common operations for data slice retrieval, allowing different storage units to be managed through a unified mechanism. This universality reduces management complexity by providing consistent interfaces despite the distributed nature of the system.
3Reliability
If error correction encoding is applied to data before storage, then data integrity and recovery capability are improved, but processing time and computational overhead increase
Solution Approach 1:
The patent applies error correction encoding in advance during the data storage phase, creating encoded data slices that inherently contain redundancy information. This preliminary action ensures that data integrity is protected before any read operations occur, eliminating the need for time-consuming integrity checks during data retrieval. The encoding is performed once during storage, and the benefits persist across multiple read operations.
Solution Approach 2:
The patent changes the processing approach from verifying data integrity during each read operation to pre-encoding data with error correction capabilities during storage. By transforming the data into encoded slices with built-in redundancy, the system trades initial processing time for significantly faster and more reliable read operations, where integrity is guaranteed by the encoding structure rather than requiring verification.
Data Source
AI summary
A method includes dispersed storage error encoding, by a device of the dispersed storage network (DSN), a data segment of a data object into a set of encoded data slices. The method further includes sending, by the device, a set of write fan out with redundancy sharing requests to a set of storage units of the DSN. The method further includes, in response to the set of write fan out with redundancy sharing requests, storing, by the set of storage units, a number of copies of a decode threshold number of encoded data slices of the set of encoded data slices. The method further includes storing, by the set of storage units, a single copy of a redundancy number of encoded data slices of the set of encoded data slices.