Distributed Storage Redundancy Coding for Low-Latency Writes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional ServerSAN storage systems face increased data access latency due to network throughput bottlenecks when distributing write data among server nodes, which affects data access performance and reliability.
Innovation Solution
A distributed storage system is implemented with a network of nodes that store data locally and transfer it to other nodes for redundancy, creating redundant codes and distributing them across different nodes to ensure data protection and availability while minimizing network traffic, thereby reducing latency and enhancing reliability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If write data is distributed to multiple server nodes for redundancy, then data reliability is improved, but network throughput becomes a bottleneck increasing data access latency
Solution Approach 1:
The system segments data into data blocks and distributes them across multiple nodes, with each node storing only a portion. This segmentation allows parallel processing and reduces the load on any single node, improving both reliability and access efficiency
Solution Approach 2:
Redundant codes are pre-calculated and stored alongside data blocks during the write operation. When data needs to be retrieved, the system can immediately use the pre-stored redundant codes without performing additional calculations, reducing access latency
2Quantity of substance
If redundant codes are created and stored on the same nodes holding data blocks, then storage capacity efficiency is improved, but reliability is reduced
Solution Approach 1:
The system applies different storage locations for data blocks and redundant codes - data blocks are stored on data nodes while redundant codes are stored on separate code nodes. This local quality differentiation ensures that even if data nodes fail, the codes remain intact for data recovery
Data Source
AI summary
A first node group including at least three nodes is predefined in a distributed storage system. Each node of the first node group is configured to send data blocks stored in storage devices managed by the node to other nodes belonging to the first node group. A first node is configured to receive data blocks from two or more other nodes in the first node group. The first node is configured to create a redundant code using a combination of data blocks received from the two or more other nodes and store the created redundant code to a storage device different from storage devices holding the data blocks used to create the redundant code. Combinations of data blocks used to create at least two redundant codes in redundant codes created by the first node are different in combination of logical addresses of constituent data blocks.