Hierarchical Erasure Coding for Segment-Level Data Recovery

Solution Overview

Problem

Conventional data recovery methods in geographically diverse storage systems are resource-intensive and inefficient, particularly when dealing with large numbers of compromised chunks, as they often require accessing and processing entire chunks rather than specific segments, leading to high computational burdens and storage overheads.

Innovation Solution

The implementation of hierarchical erasure coding, which combines zone-level and chunk-level erasure coding schemes to encode data into fewer coding chunks, allowing for the recovery of data segments rather than entire chunks, thereby reducing the amount of data that needs to be accessed and processed, and enabling efficient recovery of compromised data without needing to restore entire chunks.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If conventional data recovery methods are used to recover compromised chunks in geographically diverse storage systems, then data recovery reliability is achieved, but computational resources and processing overhead increase significantly

Engineering Contradiction:
Core Design Contradiction:
ReliabilityVSUse of energy by moving object

Solution Approach 1:

The patent divides data chunks into smaller segments and applies erasure coding at both zone-level and chunk-level. This segmentation allows the system to recover only specific compromised segments rather than entire chunks, significantly reducing the computational resources required for recovery operations while maintaining data reliability.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent implements partial recovery by allowing the system to recover only the necessary portions of compromised data (specific segments) rather than requiring complete chunk recovery. This partial action approach reduces processing overhead while achieving the minimum necessary recovery reliability.

Inventive Principle:
Principle #16Partial or excessive action

2Reliability

If conventional data recovery methods access entire chunks for segment recovery, then complete data restoration is achieved, but data transfer volume between zones increases

Engineering Contradiction:
Core Design Contradiction:
ReliabilityVSQuantity of substance

Solution Approach 1:

By segmenting chunks into smaller units and applying hierarchical erasure coding, the patent enables targeted retrieval of only the specific segments needed for recovery. This eliminates the need to transfer entire chunks between zones, significantly reducing data transfer volume while ensuring complete restoration of compromised data.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent extracts and recovers only the specific compromised segments from the distributed storage system rather than retrieving entire chunks. This extraction approach minimizes unnecessary data transfer between geographically distributed zones while achieving complete data restoration.

Inventive Principle:
Principle #2Taking out (Extraction)

3Productivity

If hierarchical erasure coding is implemented to enable segment-level recovery, then computational overhead is reduced, but storage system complexity increases

Engineering Contradiction:
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

The patent applies segmentation at multiple levels (zone-level and chunk-level erasure coding) to enable efficient segment recovery. While this multi-level segmentation increases structural complexity, it dramatically improves recovery productivity by allowing targeted recovery operations.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent implements nested erasure coding where zone-level coding contains chunk-level coding. This nested structure, while increasing system complexity, enables hierarchical recovery operations that significantly improve productivity by allowing recovery at different granularities.

Inventive Principle:
Principle #7Nested doll (Nesting)

4Reliability

If entire chunks are recovered instead of specific segments, then data integrity is ensured, but processing time and resource consumption increase

Engineering Contradiction:
Core Design Contradiction:
ReliabilityVSLoss of time

Solution Approach 1:

The patent segments chunks into recoverable units and uses hierarchical erasure coding to ensure that recovering specific segments maintains data integrity. This segmentation approach reduces processing time by avoiding unnecessary recovery of intact portions of chunks while preserving complete data integrity.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent recovers only the partial amount of data necessary (specific compromised segments) rather than requiring complete chunk recovery. This partial action maintains data integrity for the recovered portions while significantly reducing processing time and resource consumption.

Inventive Principle:
Principle #16Partial or excessive action

Data Source

PatentUS11288139B2Two-step recovery employing erasure coding in a geographically diverse data storage system
Publication Date: 2022.03.29
  • US11288139B2 patent drawing
  • US11288139B2 patent drawing
  • US11288139B2 patent drawing

AI summary

Recovery of chunk segments stored via hierarchical erasure coding in a geographically diverse data storage system is disclosed. Chunks can be stored according to a first-level erasure coding scheme in zones of a geographically diverse data storage system. The chunks can then be further protected via one or more second-level erasure coding schema within a corresponding zone of the geographically diverse data storage system. In response to determining a segment of a chunk has become less accessible, recovering at least the segment to enable intra-zone recovery of the compromised chunk can be performed according to the hierarchical erasure coding scheme of relevant chunks at relevant zones of the geographically diverse data storage system.