Erasure-Coded Volume Segmentation for Incremental Storage Expansion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing fault-tolerant storage systems require unnecessary resource consumption and increased costs when expanding storage capacity, as adding a new node necessitates adding multiple new nodes to maintain the erasure coding ratio, leading to inefficient data relocation and bandwidth usage.
Innovation Solution
The method involves dividing distributed storage devices into data and parity portions, creating erasure-coded volumes, and combining these volumes to form a file system, allowing a new storage device to be added and utilized efficiently without requiring multiple new nodes, thereby enabling incremental capacity expansion and optimizing resource management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If a new storage device is added to expand storage capacity, then storage capacity is improved, but the system requires adding (m+n) new nodes to maintain erasure coding ratio, which increases costs and wastes resources
Solution Approach 1:
The patent segments the new storage device into multiple portions (first portion, second portion, third portion) and integrates them differently into the erasure coding structure. Specifically, the first portion is integrated into an existing volume, the second portion creates a new volume, and the third portion expands an existing volume. This segmentation allows incremental capacity expansion without requiring addition of complete node sets, resolving the contradiction between storage capacity improvement and resource consumption.
2Quantity of substance
If complete relayout of data is performed to use a single new node, then storage capacity is improved, but huge amounts of data churn occur, consuming costly storage and network bandwidth
Solution Approach 1:
The patent performs preliminary actions by pre-establishing the portioning structure of the new storage device before data migration. The new device is divided into portions that can be seamlessly integrated into existing volumes or create new volumes without requiring complete data relayout. This preliminary structuring enables incremental data placement and avoids the massive data churn associated with complete relayout operations.
Solution Approach 2:
The patent introduces dynamic flexibility in volume configuration by allowing volumes to span across multiple storage devices with different portion assignments. The system can dynamically assign portions of new storage devices to different volumes based on capacity needs, enabling flexible capacity expansion without rigid data redistribution requirements.
3Quantity of substance
If (m+n) new nodes are added to expand storage, then storage capacity is improved, but device complexity and cost increase
Solution Approach 1:
The patent makes the new storage device universal by having it serve multiple functions simultaneously - its first portion expands an existing volume, its second portion creates a new volume, and its third portion can be allocated flexibly. This multi-functional use of a single storage device replaces the traditional requirement of adding multiple specialized nodes, reducing system complexity while achieving the same storage capacity expansion.
Data Source
AI summary
The disclosed computer-implemented method for distributing information across failure domains in servers may include (1) dividing, at a computing device, each of a quantity of “K” failure domains (FDs) in a plurality of FDs into a quantity of “P” portions, where the “K” FDs in the plurality of FDs are constituent parts of respective servers in a plurality of servers, “P” is less than “K,” and “P” is a sum of a quantity of “M” data portions and a quantity of “N” parity portions, (2) creating a quantity of “K” erasure-coded volumes in the “K” FDs, where each erasure-coded volume includes “M” data portions and “N” parity portions, and each portion in each erasure-coded volume is stored in a different FD and (3) combining the “K” volumes to create a file system. Various other methods, systems, and computer-readable media are also disclosed.