Erasure Coding Rebuild Load Balancing with Selective Member Skips

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Unbalanced rebuild-related reads in multiple parity erasure coding systems lead to increased time required for recovering from drive failures, as only a subset of remaining members are needed to rebuild a failed drive, leading to bottlenecks in the recovery process.

Innovation Solution

Implementing a method to compute and selectively skip protection group members during rebuild operations by using skip counts to balance read operations across drives, ensuring predictable distribution of protection group members and spares, thereby optimizing the rebuild process.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If multiple parity erasure coding is implemented to enable recovery from multiple drive failures, then reliability is improved, but rebuild time increases due to unbalanced read operations

Engineering Contradiction:

ImprovereliabilityVSAvoidrebuild time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system dynamically adjusts the rebuild process by selectively skipping protection group members based on real-time skip count calculations. The rebuild controller monitors skip counts for each non-failed drive and dynamically determines which members to skip during rebuild operations, optimizing the rebuild process adaptively rather than using a static approach

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system changes the parameter of skip count for each drive during the rebuild process. By calculating and applying different skip counts to different drives based on their current state and the distribution of protection group members, the system optimizes read operation balance and reduces overall rebuild time while maintaining reliability

Inventive Principle:

Principle #35Parameter changes

2Reliability

If all remaining members are read during rebuild operations, then complete data recovery is ensured, but read operation load becomes unbalanced and rebuild time increases

Engineering Contradiction:

Improvedata recoveryVSAvoidrebuild speed

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The system extracts and removes unnecessary read operations from the rebuild process by calculating skip counts that identify protection group members that do not need to be read. The rebuild controller selectively skips these members, extracting only the essential read operations needed for successful rebuild while eliminating redundant reads that cause load imbalance

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

Instead of reading all remaining members (excessive action), the system performs partial read operations by skipping unnecessary members based on skip count calculations. This partial action approach reads only the minimum necessary data required for recovery, optimizing rebuild speed while ensuring complete data recovery is achieved

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS20250348381A1Load balancing for erasure coding with multiple parities

Publication Date: 2025.11.13 DELL PROD LP

AI summary

In a drive cluster in which multiple parity EC(N+P) is implemented, rebuild-related reads are balanced across drives for recovery from drive failure. N members per protection group are read and (P−1) members are skipped, where skipping a member means omission from member rebuild calculations. Per-disk skip counts are calculated, and members that are eligible to be skipped are selected such that per-disk read counts are balanced.