Distributed Storage Redundancy Layout for Low-Latency Reads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional ServerSAN storage systems face increased data access latency due to network throughput bottlenecks when distributing write data among server nodes, which affects data access performance and reliability.
Innovation Solution
A distributed storage system is implemented with a network of nodes that store data locally and transfer it to other nodes for redundancy, creating redundant codes across different nodes without relying on network intervention during read operations, thus optimizing data access and maintaining high reliability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If write data is distributed to multiple server nodes for redundancy, then data reliability is improved, but data access latency increases due to network throughput bottlenecks
Solution Approach 1:
The system pre-calculates and stores mapping relationships between logical addresses and physical locations of data blocks and redundant codes across nodes. When read requests arrive, the system can directly determine which node holds the required data without network-wide queries, significantly reducing access latency while maintaining the distributed redundancy structure for reliability
Solution Approach 2:
The patent introduces a management node that acts as an intermediary between client nodes and storage nodes. The management node maintains the mapping information and redirects read requests directly to the appropriate nodes, avoiding network bottlenecks. This intermediary layer enables efficient data retrieval while preserving the distributed storage architecture's reliability benefits
2Reliability
If redundant codes are created and stored across multiple nodes, then data protection is improved, but network traffic increases
Solution Approach 1:
The system segments the redundant code creation process by having each node independently generate its own redundant code from its local data blocks using erasure coding. This eliminates the need for nodes to exchange data over the network for redundancy creation, reducing network traffic while maintaining comprehensive data protection across the distributed system
Solution Approach 2:
Each storage node autonomously generates and stores its own redundant codes locally without requiring network communication with other nodes for the code creation process. This self-service approach to redundancy generation minimizes network traffic while ensuring that each node contributes to and benefits from the distributed data protection mechanism
Data Source
AI summary
A first node group including at least three nodes is predefined in a distributed storage system. Each node of the first node group is configured to send data blocks stored in storage devices managed by the node to other nodes belonging to the first node group. A first node is configured to receive data blocks from two or more other nodes in the first node group. The first node is configured to create a redundant code using a combination of data blocks received from the two or more other nodes and store the created redundant code to a storage device different from storage devices holding the data blocks used to create the redundant code. Combinations of data blocks used to create at least two redundant codes in redundant codes created by the first node are different in combination of logical addresses of constituent data blocks.