Hierarchical Cache System with pNFS for Scalable Storage Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional Network File System (NFS) protocols face limitations in scalability and throughput due to bottlenecks at mount points, restricting the ability to utilize aggregated throughput from clusters of storage servers and hindering the performance and manageability of storage solutions.
Innovation Solution
Implementing a hierarchical cache system with parallel Network File System (pNFS) configuration that allows concurrent data retrieval by distributing data segments across multiple data servers, eliminating the concept of mount points and enabling scalable data distribution and retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional NFS protocol is used with mount points, then file system access is provided to clients, but throughput is bottlenecked at the mount point and cannot be scaled with additional servers
Solution Approach 1:
The patent segments the file system into multiple data segments that can be accessed concurrently through multiple data servers. Instead of a single mount point handling all access, the file system is divided and distributed across multiple servers, allowing parallel data retrieval and eliminating the single-point bottleneck.
Solution Approach 2:
The patent introduces a new dimensional approach by implementing parallel NFS (pNFS) that adds concurrency across multiple servers. Rather than scaling throughput through a single server's mount point, the system scales by adding multiple data servers that operate in parallel, transforming the throughput limitation from a one-dimensional bottleneck to a multi-dimensional scalable architecture.
2Productivity
If a cluster of storage servers is implemented, then aggregated throughput is increased, but conventional NFS protocol cannot utilize multiple servers simultaneously for a single file system
Solution Approach 1:
The patent implements a universal pNFS protocol that enables multiple storage servers to simultaneously serve a single file system. The protocol is designed to be multi-functional, allowing clients to access data segments distributed across any number of servers in the cluster, making the file system accessible through multiple servers rather than being restricted to a single server.
Solution Approach 2:
The patent introduces a metadata server as an intermediary that manages the distribution of data segments across multiple storage servers. The metadata server coordinates client requests and directs them to the appropriate data servers, enabling seamless multi-server access while maintaining file system integrity and allowing the cluster to function as a unified storage resource.
3Ease of manufacture
If NFS storage solutions are closely integrated with storage applications, then simple and cost-effective storage is provided, but scalability and bandwidth are limited
Solution Approach 1:
The patent implements a dynamic architecture where the storage system can scale from a single server to a cluster of servers without requiring fundamental redesign. The pNFS protocol dynamically routes requests to appropriate data servers based on data segment location, allowing the system to adapt its capacity and bandwidth by simply adding or removing servers from the cluster while maintaining simplicity of implementation.
Data Source
AI summary
A method implements a hierarchical cache system with a parallel Network File System (pNFS) configuration for a storage system. Upon receiving a request by the hierarchical cache system to access data stored in the storage system, the method divides the data into a plurality of data segments and distributes the plurality of data segments to a plurality of cache servers of the cache system. The method responds to the request a metadata layout for the plurality of data segments distributed among the plurality of cache servers. Based on the metadata layout, the plurality of data segments can be concurrently retrieved from the plurality of cache servers.