Distributed Storage Node Data Rearrangement for Local Read Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional distributed storage systems face bottlenecks in network throughput and access latency due to the need to transfer data blocks between multiple storage nodes, which can lead to increased latency and reduced performance.
Innovation Solution
A distributed storage system that rearranges write data by generating redundant data blocks and distributing them across multiple storage nodes, allowing for local read operations without inter-node data transfer, thereby reducing network communication and enhancing performance and reliability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is distributed across multiple storage nodes in a conventional distributed storage system, then scalability and capacity are improved, but network throughput becomes a bottleneck and access latency increases
Solution Approach 1:
The system performs preliminary data rearrangement during write operations, proactively positioning data blocks and redundant blocks on the same storage node before read operations occur. This advance preparation eliminates the need for network transfers during reads, as both data and its redundancy are already co-located locally.
Solution Approach 2:
Each storage node autonomously performs data rearrangement and redundancy generation for its local data blocks without requiring coordination or data transfer from other nodes. The node serves its own read requests using locally stored data and generated redundant blocks, making the system self-sufficient at the node level.
2Adaptability or versatility
If data blocks are transferred between storage nodes for read operations, then data accessibility is maintained, but network throughput becomes a bottleneck
Solution Approach 1:
The system performs preliminary data rearrangement during write operations, proactively positioning data blocks and redundant blocks on the same storage node before read operations occur. This advance preparation eliminates the need for network transfers during reads, as both data and its redundancy are already co-located locally.
Solution Approach 2:
The invention extracts the network communication step from the read operation by pre-positioning redundant blocks locally. Read operations can then be performed using only local storage resources, completely removing network dependency from the read path and freeing network bandwidth for other operations.
3Productivity
If redundant data blocks are stored on the same storage node as data blocks, then read performance is improved, but reliability may be compromised
Solution Approach 1:
The system performs preliminary data rearrangement during write operations, proactively positioning data blocks and redundant blocks on the same storage node before read operations occur. This advance preparation eliminates the need for network transfers during reads, as both data and its redundancy are already co-located locally.
4Productivity
If data is rearranged across storage nodes, then local read operations become possible, but device complexity increases
Solution Approach 1:
Each storage node autonomously performs data rearrangement and redundancy generation for its local data blocks without requiring coordination or data transfer from other nodes. The node serves its own read requests using locally stored data and generated redundant blocks, making the system self-sufficient at the node level.
Data Source
AI summary
A first storage node generates a plurality of distributedly arranged write data blocks from write data and generates a first redundant data block from the plurality of distributedly arranged write data blocks. One distributedly arranged write data block is arranged in a second data block and the first redundant data block is arranged in a third storage node. A second storage node generates a second redundant data block from the plurality of distributedly arranged write data blocks selected from the distributedly arranged write data block held therein. The second storage node rearranges each of the plurality of selected distributedly arranged write data blocks in a rearrangement destination storage node and arranges the second redundant data block in a storage node other than the rearrangement destination storage node.