Distributed Storage Fragment Indexing for Accurate Relocation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Distributed storage systems face challenges in tracking object fragments and maintaining data durability and availability due to topology changes, such as server outages and reconfigurations, which can lead to fragment mislocation and reconstruction errors.

Innovation Solution

Assigning unique indices to fragments to identify their location and ensure correct relocation, using error correction coding to reconstruct objects even after topology changes.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If fragments are distributed across storage nodes without unique indices, then storage flexibility is improved, but fragment tracking and relocation accuracy deteriorate

Engineering Contradiction:

Improvestorage flexibilityVSAvoidfragment tracking accuracy

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The system segments the identification of fragments by assigning unique indices to each fragment. This allows the storage system to track individual fragments precisely while maintaining the distributed nature of storage, resolving the contradiction between storage flexibility and fragment tracking accuracy.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Unique fragment indices act as intermediaries between the distributed storage nodes and the object manager. These indices enable precise tracking and relocation of fragments without compromising the flexibility of distributed storage architecture.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If topology changes occur in distributed storage systems, then system reconfigurability is improved, but data reconstruction reliability deteriorates

Engineering Contradiction:

Improvesystem reconfigurabilityVSAvoiddata reconstruction reliability

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The system performs preliminary actions by assigning unique indices to fragments before topology changes occur. This pre-establishment of fragment identities enables reliable reconstruction even after system reconfiguration, as the indices remain valid references regardless of storage node changes.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The object manager uses fragment indices to continuously track fragment locations and verify reconstruction integrity. This feedback mechanism ensures that even when topology changes occur, the system can reliably reconstruct objects by referencing the stable fragment indices.

Inventive Principle:

Principle #23Feedback

3Adaptability or versatility

If fragments are relocated without unique indices, then storage adaptability is improved, but reconstruction error rate increases

Engineering Contradiction:

Improvestorage adaptabilityVSAvoidreconstruction accuracy

Core Design Contradiction:

Adaptability or versatilityVSManufacturing precision

Solution Approach 1:

By segmenting the identification space into unique fragment indices, the system enables precise tracking of each fragment during relocation. This segmentation allows storage adaptability through flexible relocation while maintaining reconstruction accuracy through precise fragment identification.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentEP3433765B1Methods and apparatus to assign indices and relocate object fragments in distributed storage systems

Publication Date: 2025.07.16 INTEL CORP

Figure 1
Figure 2A
Figure 2B

AI summary

Methods and apparatus to dynamically assign and relocate object fragments in distributed storage systems are disclosed. In some examples, the methods and apparatus encode an object with error correction coding to separate the object into fragments, create a first index indicative of storage nodes where the fragments of the object are to be stored, encode a second index into identifiers of the fragments of the object, the second index based on the first index, and store the fragments of the object and the corresponding second index encoded identifiers in the storage nodes based on the first index.