Storage Cluster Pathways for Load Balancing and Data Rebuild
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional storage systems, particularly those using spinning media, are limited in flexibility and variation of connections and communication paths between storage units or nodes, making it difficult to leverage the unique characteristics of solid-state memory effectively.
Innovation Solution
A storage system architecture that includes multiple storage units and nodes with direct communication pathways between them, allowing for distributed data ownership and redundancy through erasure coding, enabling proactive data rebuilding and load balancing across multiple storage clusters within a chassis and between chassis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If storage units communicate through storage nodes (conventional architecture), then data ownership and control are centralized, but communication flexibility and system scalability are limited
Solution Approach 1:
The system segments communication paths into multiple independent pathways (first pathway for storage unit-to-storage unit communication, second pathway for storage node communication). This segmentation allows storage units to communicate directly without always requiring storage node mediation, increasing communication flexibility while maintaining manageable system complexity through structured path separation.
Solution Approach 2:
The patent introduces pathway selection logic as an intermediary that dynamically chooses between direct storage unit communication (first pathway) and storage node-mediated communication (second pathway). This intermediary enables flexible communication routing based on system state and requirements, resolving the contradiction between communication flexibility and architectural complexity.
2Reliability
If multiple communication pathways are implemented, then system redundancy and fault tolerance improve, but device complexity increases
Solution Approach 1:
The system implements dynamic pathway selection where the available communication paths and their usage are adjusted based on real-time system conditions such as node availability and load. This dynamic approach provides redundancy and fault tolerance while managing complexity through adaptive rather than static path management.
Solution Approach 2:
The pathway selection mechanism incorporates feedback from system state monitoring to determine optimal communication routes. When storage nodes are unavailable or overloaded, the system receives feedback and automatically routes communications through alternative pathways, maintaining reliability while simplifying management through rule-based feedback responses.
3Productivity
If storage nodes have centralized ownership of data portions, then data management is simplified, but system scalability and reconfiguration flexibility are reduced
Solution Approach 1:
Storage units are designed with multi-functionality, serving both as data storage entities and as independent communication nodes. They can directly communicate with each other through the first pathway without always requiring storage node involvement, enabling both efficient data management and flexible reconfiguration as needed.
Solution Approach 2:
Storage units are empowered to perform self-service communications through the first pathway, handling data transfer and coordination tasks directly without constant storage node intervention. This self-service capability maintains data management efficiency while enabling flexible system reconfiguration and scaling.
Data Source
AI summary
A storage system is provided. The storage system includes a first storage cluster, the first storage cluster having a first plurality of storage nodes coupled together and a second storage cluster, the second storage cluster having a second plurality of storage nodes coupled together. The system includes an interconnect coupling the first storage cluster and the second storage cluster and a first pathway coupling the interconnect to each storage cluster. The system includes a second pathway, the second pathway coupling at least one fabric module within a chassis to each blade within the chassis.