Storage System Parity Redistribution for Node Scaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In storage systems, adding or removing nodes disrupts data locality and decreases storage performance due to the need to move data and change storage destinations, especially when using redundancy techniques like Erasure Coding, leading to inefficiencies and performance losses.
Innovation Solution
A storage system that manages inter-node parity movement and stripe reconfiguration without moving data, by calculating and rearranging parity based on node configuration changes, ensuring data locality is maintained and storage performance is preserved.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is moved from existing nodes to added nodes to utilize available capacity, then storage capacity utilization is improved, but data locality is lost and storage performance decreases
Solution Approach 1:
The invention segments the storage system into data blocks and parity blocks, allowing independent management of data and redundancy information. When nodes are added, only parity blocks need to be redistributed while data blocks remain in place, maintaining data locality and performance while utilizing new capacity.
Solution Approach 2:
Instead of moving data to new nodes as in conventional approaches, this invention inverts the approach by keeping data stationary and moving only parity information to new nodes. This reversal maintains data locality while achieving capacity utilization.
2Quantity of substance
If a stripe row is generated after node addition and data is moved according to the new stripe row, then the added node's capacity is utilized, but the storage destination changes and adversely affects the storage system
Solution Approach 1:
The invention performs preliminary actions by pre-calculating new parity values based on existing data and the new node configuration before actual node addition. This allows smooth transition without disrupting storage system stability or requiring data destination changes.
Solution Approach 2:
The invention changes parameters by adjusting only the parity distribution parameters when nodes are added, while keeping data placement parameters unchanged. This selective parameter modification achieves capacity efficiency without affecting storage system stability.
3Adaptability or versatility
If nodes are added or removed in the storage system, then system flexibility and adaptability are improved, but data locality is disrupted and storage performance decreases
Solution Approach 1:
The invention implements dynamic parity redistribution that adapts to node configuration changes while maintaining data locality. The system dynamically recalculates and moves only parity blocks when nodes are added or removed, preserving storage performance while achieving configuration flexibility.
Data Source
AI summary
Provided is a storage system that performs inter-node movement of parity and reconfiguration of a stripe when a node configuration is changed. The storage system includes a plurality of nodes and a management unit, in which the nodes are targets for data write and read requests, form a stripe by a plurality of data stored in different nodes and parity generated based on the plurality of data, and store the parity of the stripe to which the data under the write request belongs in a node different from the plurality of nodes that store the plurality of data so as to perform redundancy; and the management unit transmits, to the node, an arrangement change request to perform the inter-node movement of the parity and the reconfiguration of the stripe when the node configuration is changed.