Storage Slot Mapping for Batch Rebuilding of Encoded Data Slices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current dispersed storage networks face challenges in efficiently rebuilding encoded data slices, particularly when storage units are unavailable or corrupted, leading to increased processing and storage costs due to the need for separate rebuilding of individual slices.
Innovation Solution
Implementing a batch rebuilding process that defers rebuilding until a threshold number of encoded data slices are available or until storage units become available again, using a rebuilding mapping approach to assign rebuilt slices to available storage slots, thereby reducing processing and storage costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If individual encoded data slices are rebuilt separately when storage units are unavailable or corrupted, then data reliability is maintained, but processing costs and storage costs increase
Solution Approach 1:
The patent combines multiple individual slice rebuilding operations into a single batch rebuilding process. When multiple encoded data slices are identified as needing reconstruction, the system groups them into batches and processes them together, sharing computational resources and reducing overall processing costs while maintaining data reliability through systematic reconstruction of all required slices.
Solution Approach 2:
The system performs preliminary identification and grouping of slices that need rebuilding before executing the actual reconstruction. By pre-processing the identification of corrupted slices and organizing them into batches, the system prepares the rebuilding workload in advance, enabling more efficient resource allocation and cost-effective processing when the batches are executed.
2Reliability
If individual encoded data slices are rebuilt separately, then data recovery is achieved, but storage costs increase due to temporary storage requirements
Solution Approach 1:
The batch rebuilding process merges multiple slice reconstruction operations, allowing the system to manage temporary storage resources more efficiently. By processing slices in batches rather than individually, the system can reuse temporary storage space across multiple reconstructions within the same batch, reducing the total storage capacity required and lowering storage costs.
3Reliability
If rebuilding is performed immediately when slices are corrupted, then data integrity is maintained, but processing efficiency decreases
Solution Approach 1:
The system performs preliminary identification of corrupted slices and groups them into batches before executing reconstruction. This pre-processing step allows the system to maintain data integrity by identifying all required reconstructions while improving efficiency by consolidating the actual rebuilding operations into optimized batches that can be processed more effectively.
Solution Approach 2:
The batch rebuilding process implements periodic action by grouping reconstruction operations into discrete batches that are processed at intervals rather than continuously individually. This periodic batching allows the system to maintain data integrity through systematic reconstruction while improving processing efficiency by reducing the overhead of frequent individual operation management.
Data Source
AI summary
A method includes determining an information dispersal algorithm width number and determining a number of primary storage slots from a number of storage slots associated with a set of storage units deployed across multiple sites, where the number of primary storage slots is equal to or greater than the information dispersal algorithm width number. The method further includes determining a mapping of primary storage slots to storage units. The method further includes sending configuration information to the set of storage units that includes the mapping. The method further includes storing a set of encoded data slices in the primary storage slots in accordance with the configuration information, where a data segment is error encoded into the set of encoded data slices in accordance with the information dispersal algorithm width number and a decode threshold number, which is a number of encoded data slices are needed to reconstruct the data segment.