Distributed Storage Redundancy Adjustment for Capacity Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems do not dynamically adjust data storing schemes when the number of nodes increases, leading to unnecessary redundancy and reduced storage capacity despite exceeding the desired reliability threshold.
Innovation Solution
A method and system that dynamically adjust the data storing scheme by reducing redundancy, such as parity chunks, to maintain desired reliability levels while increasing usable storage capacity as the number of nodes changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the number of nodes increases in the storage system, then reliability increases, but storage capacity is reduced due to unnecessary redundancy
Solution Approach 1:
The patent implements dynamic adjustment of the data storing scheme based on the current number of nodes. The system automatically transitions from static redundancy configuration to dynamic configuration, where the erasure coding parameters (k, m) are adjusted according to the number of nodes n. This allows the system to optimize the balance between reliability and storage capacity as nodes are added or removed, preventing unnecessary redundancy while maintaining the desired reliability threshold.
Solution Approach 2:
The patent changes the parameters of the erasure coding scheme (specifically the ratio of data chunks to parity chunks) based on the number of nodes. By adjusting the redundancy parameter m in the (k, m) erasure coding scheme, the system optimizes storage efficiency while maintaining reliability above the threshold. This parameter adjustment allows the system to reclaim storage space as nodes are added, converting redundant capacity into usable storage capacity.
2Quantity of substance
If redundancy is reduced to increase storage capacity, then storage efficiency improves, but reliability may fall below the desired threshold
Solution Approach 1:
The patent implements a feedback mechanism that continuously monitors the number of nodes in the storage system and automatically adjusts the redundancy level accordingly. The system calculates the current reliability based on the number of nodes and the erasure coding parameters, then compares it against the desired threshold. This feedback loop ensures that redundancy is reduced only when the reliability threshold is still met, preventing reliability from falling below the desired level while maximizing storage capacity.
Solution Approach 2:
The system automatically manages its own redundancy configuration without requiring manual intervention. The data storing scheme is self-adjusting, where the system independently determines the optimal redundancy level based on the current number of nodes and automatically reconfigures the erasure coding parameters. This self-service capability ensures that the system continuously operates at optimal efficiency while maintaining required reliability levels.
Data Source
AI summary
A storage system distributes data across multiple nodes, where the data is stored with partial redundancy to meet a predetermined reliability level. The system monitors a number of nodes in the storage system to determine whether a number of nodes changes. Responsive to determining that the number of nodes changes, the system determines whether reliability level of the storage system exceeds the predetermined reliability level by a threshold. Responsive to determining that the reliability level of the storage system exceeds the predetermined reliability level by the threshold, the system adjusts a data storing scheme of the storage system to reduce redundancy and increase available storage capacity in the storage system, while maintaining the predetermined reliability level.