Single-Region Erasure Coding for Cloud Object Loss Recovery

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Cloud object storage platforms face a significant risk of object loss as the number of objects increases, despite high durability guarantees for individual objects, leading to substantial chances of data loss for customers with large volumes of data.

Innovation Solution

Implementing erasure coding by grouping data objects into virtual segments and computing parity objects within a single region of a cloud object storage platform, which allows for the reconstruction of lost data objects and enhances durability beyond provider guarantees, along with modified garbage collection workflows to maintain data protection.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If cloud object storage platforms store a large number of objects to meet customer backup needs, then the volume of stored data increases, but the likelihood of losing at least one object increases significantly

Engineering Contradiction:

Improvenumber of objects storedVSAvoidlikelihood of object loss

Core Design Contradiction:

Quantity of substanceVSReliability

Solution Approach 1:

The patent groups objects into buckets and further segments buckets into virtual segments, creating a hierarchical structure. This segmentation allows erasure coding to be applied at the virtual segment level, enabling recovery of individual objects even when many objects are stored, thereby resolving the contradiction between storing large quantities of objects and maintaining reliability

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent pre-computes and stores parity information for virtual segments before any object loss occurs. This preliminary action enables reconstruction of lost objects without requiring real-time computation, ensuring that reliability is maintained regardless of the number of objects stored

Inventive Principle:

Principle #10Preliminary action

2Reliability

If erasure coding is implemented across multiple regions to reduce object loss, then data protection improves, but system complexity and cost increase

Engineering Contradiction:

Improveobject durabilityVSAvoidmulti-region architecture complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent introduces a hierarchical segmentation strategy where objects are grouped into buckets, and buckets into virtual segments. This allows erasure coding to be applied at the virtual segment level within a single region, achieving multi-region-like protection without the actual complexity of distributed multi-region architecture

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces virtual segments as an intermediary layer between objects and regions. This virtual segmentation layer enables erasure coding protection without requiring physical distribution across multiple regions, thereby reducing system complexity while maintaining reliability

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS11556423B2Using erasure coding in a single region to reduce the likelihood of losing objects maintained in cloud object storage

Publication Date: 2023.01.17 VMWARE INC

AI summary

Techniques for using erasure coding in a single region to reduce the likelihood of losing objects in a cloud object storage platform are provided. In one set of embodiments, a computer system can upload a plurality of data objects to a region of a cloud object storage platform, where the plurality of data objects including modifications to a data set. The computer system can further compute a parity object based on the plurality of data objects, where the parity object encodes parity information for the plurality of data objects. The computer system can then upload the parity object to the same region where the plurality of data objects was uploaded.