Dynamic Erasure Code Placement for Balanced Object Storage

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional object storage systems use static configuration settings to determine erasure code distribution, failing to dynamically adapt to changing conditions such as load imbalances, capacity issues, and usage patterns, leading to sub-optimal performance and inefficiencies.

Innovation Solution

Implementing a dynamic distribution controller that monitors operating conditions and adjusts erasure code distribution based on real-time feedback from activity and capacity monitors, using user-defined or automated rules to optimize load balancing, capacity balancing, and read performance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If static, pre-defined configuration settings are used to determine erasure code distribution, then the system configuration is simple and easy to manage, but the system cannot dynamically adapt to changing conditions such as load imbalances and capacity issues, leading to sub-optimal performance

Engineering Contradiction:

Improvedynamic adaptation to changing conditionsVSAvoidsystem configuration complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent implements dynamic erasure code distribution by allowing the system to automatically adjust code placement based on real-time monitoring of load conditions, capacity availability, and usage patterns. The distribution controller continuously evaluates system state and reallocates erasure codes to optimize performance metrics without requiring manual reconfiguration.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system incorporates feedback mechanisms through activity monitors and capacity monitors that continuously report system state to the distribution controller. This feedback loop enables the system to detect hot spots, load imbalances, and capacity constraints, then automatically adjust erasure code distribution in response to these conditions.

Inventive Principle:

Principle #23Feedback

2Productivity

If static configuration is used for erasure code distribution, then the system is easier to operate and maintain, but load balancing is sub-optimal as hot spots of high loads on certain disks, servers, or sites go unnoticed

Engineering Contradiction:

Improveload balancing efficiencyVSAvoidsystem operation simplicity

Core Design Contradiction:

ProductivityVSEase of operation

Solution Approach 1:

The system performs self-service by automatically detecting and correcting load imbalances without human intervention. The distribution controller monitors system conditions and autonomously redistributes erasure codes to achieve optimal load balancing, eliminating the need for manual intervention while maintaining operational simplicity.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system takes preliminary action by proactively redistributing erasure codes before severe performance degradation occurs. By continuously monitoring system state and preemptively adjusting code placement in response to emerging hot spots or load imbalances, the system prevents performance issues rather than merely reacting to them.

Inventive Principle:

Principle #10Preliminary action

3Productivity

If static, pre-defined settings are consulted only at store time, then the system operation is straightforward, but relationships between erasure codes, usage patterns, capacity considerations, and other factors affecting efficiency are not considered

Engineering Contradiction:

Improvesystem efficiencyVSAvoidcontrol mechanism complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The distribution controller serves multiple functions: it monitors system conditions, analyzes usage patterns, evaluates capacity availability, determines optimal code placement, and executes redistribution. This multi-functional approach consolidates complex control operations into a single system component that comprehensively manages erasure code distribution based on multiple factors.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The system implements comprehensive feedback by continuously monitoring not only basic system state but also usage patterns, capacity trends, and performance metrics. This multi-dimensional feedback enables the distribution controller to make informed decisions that consider relationships between erasure codes, usage patterns, and capacity considerations simultaneously.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS10459821B2Dynamically controlling erasure code distribution in an object store

Publication Date: 2019.10.29 ALTER DOMUS (US) LLC AS AGENT FOR THE SECURED PARTIES

AI summary

Example apparatus and methods monitor conditions in an object storage system. The conditions monitored may include a load balance measure in the system, a capacity balance measure in the system, a fault tolerance measure in the system, or a usage pattern measure in the system. A distribution plan or redistribution plan for storing or moving erasure codes in the object storage system may be determined based on the conditions. The distribution plan or the redistribution plan for the erasure codes may be updated dynamically in response to changing conditions in the object storage system. The distribution or redistribution may depend on a weighted combination of the load balance measure, the capacity balance measure, the fault tolerance measure, or the usage pattern measure so that responding to one sub-optimal condition (e.g., load imbalance) does not create a different sub-optimal condition (e.g., unacceptable fault tolerance).