Coded Fragment Updates for Distributed Storage Version Control

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing distributed storage systems face inefficiencies in updating coded fragments, requiring full recoding and network transmission, especially when files are modified, leading to high network strain and storage usage, and lack seamless update capabilities.

Innovation Solution

The use of linear erasure codes to efficiently update coded fragments by calculating and transmitting only the coded difference between original and modified data, allowing for partial recovery without full reupload, and enabling efficient version control in distributed and cloud storage systems.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If the entire file is recoded and all coded fragments are transmitted through the network when any portion of a file is changed, then the distributed storage system maintains data reliability and consistency, but the network strain and storage usage increase significantly

Engineering Contradiction:
Improvedata reliabilityVSAvoidnetwork strain
Core Design Contradiction:
ReliabilityVSLoss of energy

Solution Approach 1:

The patent segments the file update process by dividing the file into fragments and applying erasure coding to generate coded fragments. When updates are needed, only the affected coded fragments are identified and transmitted, rather than recoding the entire file. This segmentation allows selective updates that reduce network strain while maintaining data reliability through the distributed coded fragment storage.

Inventive Principle:
Principle #1Segmentation

2Reliability

If the entire file is recoded and all coded fragments are transmitted through the network when any portion of a file is changed, then the distributed storage system maintains data reliability and consistency, but the storage space requirements increase

Engineering Contradiction:
Improvedata reliabilityVSAvoidstorage space
Core Design Contradiction:
ReliabilityVSQuantity of substance

Solution Approach 1:

The patent changes the parameter representation by storing coded fragments instead of complete file copies. When updates occur, the system modifies only the necessary coded fragments using erasure code mathematics, rather than storing entire new file versions. This parameter change in the storage representation reduces storage space requirements while preserving data reliability through the mathematical properties of erasure codes.

Inventive Principle:
Principle #35Parameter changes

3Quantity of substance

If differences between original and modified data are stored in unencoded form, then storage space is reduced, but the system loses the benefits of coding and cannot update files seamlessly

Engineering Contradiction:
Improvestorage spaceVSAvoidfile update capability
Core Design Contradiction:
Quantity of substanceVSEase of operation

Solution Approach 1:

The patent creates a composite storage system that combines encoded and unencoded representations strategically. Coded fragments are stored in encoded form for efficient retrieval and update operations, while difference information is maintained in unencoded form for space efficiency. This composite approach allows the system to leverage both the space-saving benefits of unencoded differences and the operational benefits of encoded storage for seamless file updates.

Inventive Principle:
Principle #40Composite materials

4Adaptability or versatility

If multiple versions of a file are maintained, then version control capability is improved, but the network use and storage use of the system increase

Engineering Contradiction:
Improveversion control capabilityVSAvoidstorage use
Core Design Contradiction:
Adaptability or versatilityVSQuantity of substance

Solution Approach 1:

The patent implements version control by nesting difference information within the coded fragment structure. Instead of storing complete separate versions of files, the system stores coded fragments that can represent multiple versions through layered difference encoding. This nested structure allows version control capability while reducing storage use, as each coded fragment can serve multiple version requirements through mathematical transformations rather than requiring separate full-file storage for each version.

Inventive Principle:
Principle #7Nested doll (Nesting)

Data Source

PatentUS10270468B2Method for file updating and version control for linear erasure coded and network coded storage
Publication Date: 2019.04.23 AALBORG UNIV
  • US10270468B2 patent drawing
  • US10270468B2 patent drawing
  • US10270468B2 patent drawing

AI summary

A method for use in a distributed storage system having a plurality of nodes, the method including receiving, at a source node, original data, encoding the original data into plurality of coded fragments using a linear code, transmitting at least a first one of the coded fragments from the source node to a first sink node. The method further includes receiving, at the source node, modified data, calculating, at the source node, a coded difference between the original data and the modified data, transmitting the coded difference from the source node to the first sink node; and recovering, at the first sink node, at least a first portion of the modified data using the coded difference and the at least a first one of the coded fragments.