Object Storage Resiliency Adjustment Using Dynamic Erasure Coding

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing enterprise object storage systems face challenges in achieving data loss resiliency while maintaining cost-effectiveness and compliance with copyright laws, as traditional methods like data mirroring are resource-intensive and violate fair use restrictions, and erasure coding increases costs with higher resiliency overhead.

Innovation Solution

The system dynamically adjusts the realized resiliency factor of an object by redistributing data and parity information across storage entities based on a target resiliency factor, optimizing the number of data and parity storage entities to provide fault-tolerance within a threshold range, thereby reducing resource usage and avoiding mirroring violations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If data mirroring is used to provide fault tolerance, then data loss resiliency is improved, but resource consumption increases significantly

Engineering Contradiction:

Improvedata loss resiliencyVSAvoiddisk space

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The patent uses erasure coding to create fractional copies (parity blocks) of data blocks, where only a portion of the total stored data consists of actual copies. This allows the system to achieve fault tolerance with less storage capacity compared to traditional full mirroring, directly resolving the contradiction between reliability and quantity of substance.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The system dynamically adjusts the resiliency factor parameter to change the degree of redundancy. By modifying this parameter, the system can optimize the balance between data loss resiliency and resource consumption, allowing flexible adaptation to different storage capacity and reliability requirement scenarios.

Inventive Principle:

Principle #35Parameter changes

2Reliability

If data mirroring is used to ensure fault tolerance, then data loss resiliency is improved, but cost increases due to resource intensity

Engineering Contradiction:

Improvefault toleranceVSAvoidresource consumption

Core Design Contradiction:

ReliabilityVSLoss of energy

Solution Approach 1:

By implementing erasure coding that creates parity blocks rather than full mirrors, the system reduces the amount of redundant data storage required. This decreases the computational resources needed for data replication and the physical storage capacity required, thereby reducing overall resource consumption while maintaining fault tolerance.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The dynamic adjustment of the resiliency factor allows the system to optimize resource allocation based on actual needs, reducing unnecessary resource consumption while maintaining adequate fault tolerance levels.

Inventive Principle:

Principle #35Parameter changes

3Reliability

If data mirroring is implemented for fault tolerance, then data loss resiliency is improved, but policy conflicts arise with copyright laws

Engineering Contradiction:

Improvefault toleranceVSAvoidcompliance with copyright laws

Core Design Contradiction:

ReliabilityVSAdaptability or versatility

Solution Approach 1:

The erasure coding mechanism creates mathematical parity blocks that are not exact copies of the original data, but rather derived values that enable reconstruction. This approach provides fault tolerance without creating multiple unique instances of copyrighted material, thus avoiding DMCA violations while maintaining data resiliency.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

By adjusting the resiliency factor parameter, the system can control the level of redundancy to comply with legal requirements while maintaining adequate fault tolerance, demonstrating adaptability to legal constraints.

Inventive Principle:

Principle #35Parameter changes

4Reliability

If higher resiliency overhead is used to increase fault tolerance, then data loss resiliency is improved, but storage cost increases

Engineering Contradiction:

Improvedata loss resiliencyVSAvoidstorage capacity

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The system dynamically adjusts the resiliency factor parameter to optimize the balance between fault tolerance and storage capacity. By changing this parameter, the system can adapt to different storage availability scenarios and maintain cost-effectiveness while providing adequate data loss resiliency.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS10983730B2Adapting resiliency of enterprise object storage systems

Publication Date: 2021.04.20 SYNAMEDIA LTD

AI summary

Various implementations disclosed herein enable managing a resiliency factor of an object stored in an enterprise object storage system. For example, in various implementations, a method of adjusting a realized resiliency factor of an object based on a target resiliency factor for the object is performed by an ingest entity of a storage system that includes a cluster of storage entities. The ingest entity includes a non-transitory computer readable storage medium, and one or more processors. In various implementations, the method includes obtaining a target resiliency factor for an object. In various implementations, the method includes determining whether or not to adjust a realized resiliency factor of the object based on the target resiliency factor. In various implementations, the method includes adjusting the realized resiliency factor of the object to an adjusted resiliency factor in response to determining to adjust the realized resiliency factor.