Distributed Storage Redundancy Coding for Lower Write Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional ServerSAN storage systems face increased data access latency due to network throughput bottlenecks when distributing write data among server nodes, which affects data access efficiency and reliability.
Innovation Solution
A distributed storage system is implemented with a network of nodes that create redundant codes from data blocks and store them on different nodes, optimizing data distribution to minimize network traffic and maintain high reliability while reducing overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If write data is distributed to multiple server nodes for redundancy, then data reliability is improved, but network throughput becomes a bottleneck increasing data access latency
Solution Approach 1:
The data is segmented into data blocks and distributed across multiple server nodes. Each node stores specific data blocks and their corresponding redundant codes, allowing parallel access and reducing the impact of network throughput bottlenecks on overall data access latency
Solution Approach 2:
A head node is introduced as an intermediary to coordinate redundant code creation and distribution. The head node receives data blocks from data nodes, creates redundant codes, and distributes them to appropriate nodes, optimizing the data distribution process and reducing network traffic bottlenecks
2Reliability
If redundant codes are created and distributed across the network, then data protection capability is improved, but network traffic increases
Solution Approach 1:
Each data node creates and stores redundant codes locally for its own data blocks, rather than all nodes participating in all redundant code creations. This localizes the redundant code generation process, reducing overall network traffic while maintaining data protection capability
Solution Approach 2:
Redundant codes are created and distributed in advance as part of the data writing process. When data is written to the storage system, the corresponding redundant codes are immediately generated and distributed to appropriate nodes, so that no additional network traffic is required during data retrieval operations
Data Source
AI summary
A first node group including at least three nodes is predefined in a distributed storage system. Each node of the first node group is configured to send data blocks stored in storage devices managed by the node to other nodes belonging to the first node group. A first node is configured to receive data blocks from two or more other nodes in the first node group. The first node is configured to create a redundant code using a combination of data blocks received from the two or more other nodes and store the created redundant code to a storage device different from storage devices holding the data blocks used to create the redundant code. Combinations of data blocks used to create at least two redundant codes in redundant codes created by the first node are different in combination of logical addresses of constituent data blocks.