Dispersed Storage Address Mapping Beyond IDA Width
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current dispersed storage networks face challenges in maintaining data integrity and availability due to storage unit failures, as they often require redundant copies and are vulnerable to data loss and unauthorized access.
Innovation Solution
A dispersed storage network (DSN) that employs error encoding using Cauchy Reed-Solomon encoding, distributing data across multiple storage units with a decode threshold, allowing for data recovery even if some units fail, and secure storage with encryption and authentication mechanisms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is distributed across multiple storage units using error encoding, then data availability and reliability are improved, but storage space utilization deteriorates due to the need for additional encoded slices
Solution Approach 1:
The patent segments data into multiple slices and distributes them across different storage units. Each data object is divided into K slices, which are then encoded into N slices (where N > K) and stored across N storage units. This segmentation allows the system to tolerate up to N-K storage unit failures while maintaining data availability, resolving the contradiction between reliability and storage utilization by efficiently using encoding to create redundant information that is distributed rather than duplicated.
Solution Approach 2:
The patent changes the parameter of storage location capacity from fixed IDA width to variable capacity greater than IDA width. By allowing storage units to have capacity greater than the IDA width, the system can store multiple sets of encoded slices across the same storage units. This parameter change enables better storage utilization while maintaining the reliability benefits of error encoding, as the same storage units can serve multiple data objects through their increased capacity.
2Quantity of substance
If storage locations greater than IDA width are used, then storage utilization is improved, but system complexity increases due to address mapping and slice distribution management
Solution Approach 1:
The patent introduces an address mapping function as an intermediary between the logical view of data storage and the physical storage locations. This mapping function translates data object identifiers and slice indices into specific storage unit addresses, abstracting the complexity of managing storage locations greater than IDA width. The intermediary layer handles the complexity of slice distribution and retrieval, allowing the rest of the system to work with simplified logical addresses while achieving improved storage utilization.
3Reliability
If redundant copies are used for data protection, then data security is improved, but storage efficiency deteriorates due to duplicate data storage
Solution Approach 1:
The patent uses error encoding to create encoded copies of data slices rather than simple redundant copies. Instead of duplicating entire data objects, the system encodes K data slices into N encoded slices, where each encoded slice contains distributed information from multiple original slices. This copying approach provides data protection through mathematical encoding rather than simple duplication, improving storage efficiency while maintaining data security and availability.
Data Source
AI summary
A method comprises encoding, by a DS processing unit of a dispersed storage network (DSN), a data segment using an information dispersal algorithm with a first pillar width number to produce a set of encoded data slices. The method continues by generating a set of DSN addresses for the set of encoded data slices based on the first pillar width number, a second pillar width number and a DSN address mapping function. The method continues by identifying, based on the DSN addresses, a first group of storage units of a set of storage units that includes the second width pillar number of storage units and where the first group of storage units includes the first pillar width number of storage units. The method continues by sending the set of encoded data slices to the first group of storage units in accordance with the set of DSN addresses.