Accelerator Node Hybrid Federated Storage Namespace
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing distributed storage systems face challenges in providing a single, high-speed unstructured data namespace that spans multiple storage clusters, as referral-based federated storage solutions require client protocol support and introduce additional hops in the data path, while proxy-based solutions compromise performance by adding an extra hop.
Innovation Solution
Implementing a hybrid approach that combines referral-based and proxy-based federated storage systems using accelerator nodes, which participate in both clusters' storage networks, allowing clients to access data directly and scaling performance independently of storage capacity without compromising resilience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If referral-based federated storage solutions are used, then clients can access data across multiple clusters, but performance deteriorates due to additional hops in the data path and requirement for client protocol support
Solution Approach 1:
The patent introduces accelerator nodes as intermediaries between clients and storage clusters. These accelerator nodes participate in both clusters' storage networks and can serve data directly from their local caches, eliminating the need for clients to perform referral-based cluster discovery and reducing data access hops. The accelerator node acts as a intelligent mediator that can buffer data locally and serve it directly to clients, thereby improving access speed while maintaining multi-cluster functionality.
2Adaptability or versatility
If proxy-based federated storage solutions are used, then data access across clusters is enabled, but performance is compromised by adding an extra hop in the data path
Solution Approach 1:
The accelerator node serves as an intelligent intermediary that can operate in multiple modes. When a client requests data, the accelerator node can determine whether to serve it from its local cache (reducing hops) or forward the request to the appropriate storage cluster. This intermediary approach provides federated storage access without mandating an extra hop, as the accelerator node may have the data locally available, thereby reducing data path complexity while maintaining access capability.
Solution Approach 2:
The accelerator node performs partial caching of data from multiple clusters locally. Instead of requiring all data to be fetched from remote clusters, the accelerator node pre-loads or caches frequently accessed data segments locally. This partial local storage eliminates the need for clients to traverse the full data path through proxies for cached data, reducing data path complexity for those specific data access scenarios.
3Productivity
If accelerator nodes are used to combine referral and proxy approaches, then performance is improved by eliminating unnecessary hops, but system complexity increases
Solution Approach 1:
The accelerator node is designed as a multi-functional component that can perform multiple roles: it acts as a cache server, a referral provider, a data proxy, and a client protocol terminator. By consolidating these functions into a single node type, the system achieves improved data access efficiency without proportionally increasing overall system complexity. The universal accelerator node can adapt its behavior based on data availability and client requirements, eliminating unnecessary hops while maintaining a manageable architecture through functional consolidation.
Data Source
AI summary
Techniques are provided for providing data across multiple clusters in a single namespace. In an example, a first cluster can comprise an accelerator node that communicates with a client computer, and is configured both to refer the client computer to another cluster, and to proxy data from another cluster to the client. When the client computer accesses data at a junction in the namespace between the first cluster and a second cluster, the accelerator node can determine whether to refer the client computer or proxy the client computer. Then, the accelerator node can use this determined approach so that the client computer can access data from the second cluster.