Replica Segment Pair Placement for Data Availability

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Modern data storage systems face challenges in balancing storage overhead and data availability, as existing solutions either require high storage space with multiple copies or suffer from long data recovery times due to erasure codes, leading to potential long-term data unavailability issues.

Innovation Solution

The method involves dividing data into segments, generating coding segments, and creating replication segments, which are stored across multiple devices, allowing for immediate data availability without the need for recovery operations when storage devices fail, thereby reducing overhead while ensuring high data availability.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If multiple copies of data are stored in different storage devices, then data availability is improved, but storage overhead increases

Engineering Contradiction:

Improvedata availabilityVSAvoidstorage overhead

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The data is divided into multiple segments, and only critical segments are replicated across storage devices. Non-critical segments use erasure coding instead of full replication, reducing overall storage overhead while maintaining data availability for essential information.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Different storage devices are assigned different qualities of data protection based on local requirements. Some devices store fully replicated critical data segments for immediate availability, while others store encoded non-critical segments that require recovery operations, optimizing the balance between availability and storage efficiency.

Inventive Principle:

Principle #3Local quality

2Quantity of substance

If erasure codes are generated and stored with data, then storage overhead is reduced, but data recovery time increases

Engineering Contradiction:

Improvestorage overheadVSAvoiddata recovery time

Core Design Contradiction:

Quantity of substanceVSLoss of time

Solution Approach 1:

Critical data segments are pre-replicated across multiple storage devices before any failure occurs. This preliminary replication ensures that when failures happen, the replicated critical segments are immediately available without requiring time-consuming erasure decoding operations.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

Instead of applying erasure coding uniformly to all data segments, the system applies full replication selectively to critical segments and uses erasure coding only for non-critical segments. This partial application of replication reduces recovery time for essential data while maintaining storage efficiency for less important data.

Inventive Principle:

Principle #16Partial or excessive action

3Quantity of substance

If data is divided into segments with coding segments, then storage efficiency is improved, but data availability during recovery deteriorates

Engineering Contradiction:

Improvestorage efficiencyVSAvoiddata availability during recovery

Core Design Contradiction:

Quantity of substanceVSReliability

Solution Approach 1:

Data is segmented into critical and non-critical portions, with different protection strategies applied to each. Critical segments are replicated for immediate availability, while non-critical segments use erasure coding for storage efficiency, achieving both goals simultaneously across the distributed system.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system dynamically adjusts the protection level (replication factor) for different data segments based on their criticality parameters. Critical segments have higher replication factors ensuring immediate availability, while non-critical segments have lower replication factors optimizing storage efficiency, with recovery priority also adjusted according to segment criticality.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS11023158B2Constraining placement of replica segment pairs among device pairs based on coding segment count

Publication Date: 2021.06.01 EMC IP HLDG CO LLC

AI summary

Embodiments of the present disclosure provide a method, apparatus, and computer program product for storing data. A method for storing data comprises: dividing data to be stored into a first number of data segments; generating a second number of coding segments based on the first number of data segments, such that at least a part of data segments from the first number of data segments can be derived from the second number of coding segments and remaining data segments in the first number of data segments; generating, for each of the first number of data segments, a replication data segment identical to the data segment; and storing the first number of data segments, the first number of replication data segments and the second number of coding segments into a plurality of storage devices. Embodiments of the present disclosure can reduce extra overhead for protecting data while ensuring high data availability.