Deferred Slice Rebuilding Across Alternate Storage Locations

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current dispersed storage networks face challenges in maintaining data integrity and availability due to storage unit failures, requiring redundant copies and complex error correction mechanisms, which can lead to increased costs and complexity.

Innovation Solution

A dispersed storage network architecture that uses Cauchy Reed-Solomon error encoding to divide data into encoded data slices, stored across multiple geographically distributed storage units, allowing for efficient error correction and data reconstruction without the need for redundant copies, using a managing unit to coordinate storage and integrity processing for data encoding, decoding, and rebuilding of corrupted slices.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If redundant copies of data are stored to maintain data availability during storage unit failures, then data reliability is improved, but storage cost and system complexity increase

Engineering Contradiction:
Improvedata availabilityVSAvoidstorage system complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent segments data into multiple encoded slices using Cauchy Reed-Solomon error encoding. Instead of storing complete redundant copies of data, the system divides data into fragments and distributes them across multiple storage units. This segmentation allows the system to tolerate failures of up to k-1 storage units while maintaining data availability, reducing the overhead compared to traditional redundant copying methods.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent changes the storage parameter from storing complete data copies to storing encoded data slices. By applying Cauchy Reed-Solomon encoding with configurable parameters (total slices n, failure tolerance k), the system dynamically adjusts the balance between storage efficiency and reliability. This parameter-based approach allows flexible optimization based on specific reliability requirements without proportionally increasing complexity.

Inventive Principle:
Principle #35Parameter changes

2Reliability

If error correction mechanisms are implemented to maintain data integrity, then data reliability is improved, but processing complexity and computational overhead increase

Engineering Contradiction:
Improvedata integrityVSAvoiderror correction complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent applies error correction encoding to data before storage, creating encoded slices that inherently contain error correction capabilities. This preliminary action ensures that data integrity is maintained without requiring complex real-time error correction during retrieval. The Cauchy Reed-Solomon encoding is performed once during data ingestion, and the encoded slices are then stored as-is, simplifying the overall error correction mechanism.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent creates multiple copies of encoded data slices and distributes them across different storage units. Instead of copying complete data and then applying error correction, the system encodes data into slices first, then creates and distributes copies of these encoded slices. This approach reduces the computational overhead of error correction while maintaining data integrity, as the encoding is performed once on the original data.

Inventive Principle:
Principle #26Copying

3Reliability

If data is divided into encoded slices and distributed across multiple storage units, then data availability during failures is improved, but storage and retrieval time increase

Engineering Contradiction:
Improvedata availability during failuresVSAvoidstorage and retrieval time
Core Design Contradiction:
ReliabilityVSLoss of time

Solution Approach 1:

The patent segments data into n encoded slices that are distributed across multiple storage units. During retrieval, the system only needs to gather k slices (where k < n) to reconstruct the original data, rather than retrieving all n slices or waiting for all storage units. This segmentation strategy reduces retrieval time while maintaining the ability to tolerate up to n-k failures, effectively balancing availability with time efficiency.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent implements partial action by requiring only k slices out of n total slices for data reconstruction. This means the system performs less work during retrieval than traditional methods would require, as it doesn't need to gather all distributed copies or perform exhaustive searches. The partial retrieval of k slices is sufficient to restore data availability, reducing the time loss associated with distributed storage operations.

Inventive Principle:
Principle #16Partial or excessive action

Data Source

PatentUS11340993B2Deferred rebuilding with alternate storage locations
Publication Date: 2022.05.24 PURE STORAGE INC
  • US11340993B2 patent drawing
  • US11340993B2 patent drawing
  • US11340993B2 patent drawing

AI summary

A method includes, when a first batch rebuild threshold is met, a computing device of a dispersed storage network (DSN) determining whether target storage units of a set of storage units of the DSN are available, where a data segment of data is dispersed storage error encoded into a set of encoded data slices, a number of encoded data slices that need rebuilding corresponds to the first batch rebuild threshold, and the number of encoded data slices is to be stored in the target storage units. When a target storage unit is unavailable, the method continues by delaying a batch rebuild process until the target storage unit is available or a second batch rebuild threshold is met. When the target storage unit becomes available before the second batch rebuild threshold is met, the method continues by executing the batch rebuild process for the number of encoded data slices.