DSN Migration Tracking for Distributed Slice Transfer Completion

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing data storage systems face challenges in efficiently managing data distribution and storage across multiple storage units in a dispersed storage network, particularly when storage devices fail or new units are added, requiring effective error encoding and decoding mechanisms to ensure data integrity and availability.

Innovation Solution

A dispersed storage network (DSN) utilizing error encoding techniques like Cauchy Reed-Solomon encoding, where data is segmented into encoded data slices stored across multiple storage units, with a decentralized agreement protocol to determine optimal storage locations and manage data migration during changes in the storage configuration.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If data is segmented and distributed across multiple storage units using error encoding, then data reliability and availability are improved, but system complexity increases

Engineering Contradiction:

Improvedata integrityVSAvoidsystem complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent segments data into multiple encoded slices using error correction codes (e.g., Cauchy Reed-Solomon encoding). Each slice is stored in a different storage unit, allowing the system to tolerate failures while maintaining data integrity. This segmentation directly improves reliability by distributing risk across multiple storage units.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a decentralized agreement protocol as an intermediary mechanism that coordinates data migration and storage operations across distributed units. This protocol manages the complexity of distributed operations by providing a standardized interface for slice transfers, authentication, and completion verification, thereby handling system complexity systematically.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If data migration is performed during storage configuration changes, then system adaptability is improved, but migration completion tracking becomes more difficult

Engineering Contradiction:

Improvestorage configuration adaptabilityVSAvoidmigration completion detection

Core Design Contradiction:

Adaptability or versatilityVSDifficulty of detecting and measuring

Solution Approach 1:

The patent implements a feedback mechanism where storage units report migration status to the decentralized agreement protocol. The protocol tracks which encoded slices have been successfully transferred from source to destination units, and uses this feedback to determine when migration is complete. This systematic tracking resolves the difficulty of detecting migration completion in distributed systems.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The decentralized agreement protocol serves multiple functions: it coordinates data migration, authenticates storage units, tracks migration completion, and manages slice transfers. This multi-functional approach improves adaptability by providing a universal coordination mechanism that handles various storage configuration changes through a single standardized protocol.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Adaptability or versatility

If encoded data slices are transferred between storage units, then storage flexibility is improved, but data loss risk during transfer increases

Engineering Contradiction:

Improvestorage flexibilityVSAvoiddata loss prevention

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The patent uses error correction encoding to create redundant copies of data across multiple storage units before any migration occurs. This beforehand cushioning ensures that even if data loss occurs during transfer, the original data can be recovered from remaining encoded slices, thereby preventing actual data loss while maintaining storage flexibility.

Inventive Principle:

Principle #11Beforehand cushioning (Prior cushioning)

Solution Approach 2:

The patent creates encoded copies of data slices and distributes them across multiple storage units. During migration, these copies are transferred from source to destination units. The redundant nature of the encoding ensures that data can be reconstructed even if some copies are lost during transfer, thus protecting against data loss while enabling flexible storage reconfiguration.

Inventive Principle:

Principle #26Copying

Data Source

PatentUS10133632B2Determining completion of migration in a dispersed storage network

Publication Date: 2018.11.20 PURE STORAGE INC

AI summary

A method for determining completion of a data migration that results from a distributed agreement protocol (DAP) change within a distributed storage network (DSN). The method begins by transferring, in accordance with the DAP change, encoded data slices to one or more other storage units within the DSN. The method continues by maintaining a storage unit migration tracking repository that tracks migration of the encoded data slices. The method continues by maintaining a storage pool migration tracking repository based on the storage unit migration tracking repositories of the plurality of storage units. The method continues by maintaining a DSN migration tracking repository based on the storage pool migration tracking repositories of the plurality of storage pools. The method continues by indicating completion of the data migration as a result of the DAP change based on information within the DSN tracking repository.