Scale-Out Storage Active Failover for Legacy Network Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As storage platforms scale, communication bottlenecks and data throughput limitations arise when connecting newer storage system architectures to legacy networks, necessitating innovative solutions to enhance data storage and management.
Innovation Solution
The implementation of a storage system architecture that includes non-volatile solid state storage units, distributed storage nodes, and erasure coding schemes to ensure data redundancy and efficient data management across multiple storage nodes, with a focus on proactive data rebuilding and load balancing within a storage cluster.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If storage platforms are scaled to increase storage memory and network connections, then storage capacity and system capabilities are improved, but communication bottlenecks and data throughput limitations occur when connecting to legacy networks
Solution Approach 1:
The storage system is divided into multiple storage nodes that can be independently managed and connected to the network. Each node operates semi-autonomously, allowing data to be distributed across multiple connection paths rather than bottlenecking through a single controller, thereby improving throughput when connecting to legacy networks.
Solution Approach 2:
A network interface card (NIC) with a cache buffer is introduced as an intermediary between the storage nodes and the legacy network. The NIC buffer temporarily stores data packets, allowing the storage system to operate at higher speeds while legacy networks receive data at their native, slower speeds without creating bottlenecks.
2Adaptability or versatility
If storage platforms are scaled to increase the number of network connections, then system capabilities are improved, but communication bottlenecks arise
Solution Approach 1:
The storage nodes are designed with multi-functional capabilities, serving both as storage devices and network interface points. This eliminates the need for separate dedicated network switches and controllers, reducing overall network connection complexity while maintaining system versatility.
Solution Approach 2:
Multiple storage nodes are merged into a unified cluster that presents a single logical interface to clients. This consolidation reduces the number of individual network connections required while maintaining the scalability and adaptability benefits of having multiple physical nodes.
3Reliability
If erasure coding schemes are implemented across multiple storage nodes, then data redundancy and reliability are improved, but computational overhead and processing time increase
Solution Approach 1:
Erasure coding is performed proactively during data write operations rather than reactively during read operations or failure events. Data is pre-encoded and distributed across multiple nodes before any potential failure occurs, ensuring rapid recovery without adding processing delays during critical operations.
Solution Approach 2:
Each storage node independently performs erasure coding calculations on its local data segments without requiring centralized coordination. This distributed self-service approach reduces computational overhead and processing time compared to centralized encoding schemes.
Data Source
AI summary
A storage system that has blades and fabric modules connects to a customer legacy network that has a first, active switch and a second, passive switch. A first link aggregation group (LAG) is configured active and includes ports of the first, active switch that connect via links to the first and second fabric modules of the storage system. A second LAG is configured passive and includes ports of the second, passive switch that connect via links to the first and second fabric modules. A multi-chassis link aggregation group (MLAG, MCLAG or MC-LAG) is configured and includes ports of the first and second fabric modules that connect via links to the first and second switches.