Super Cluster File System Availability via Dynamic Bundle Reassignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current network file servers face limitations in availability and efficiency due to node failures and overload, requiring clients to be configured for multiple mount points and leading to I/O bottlenecks, where one node's downtime affects all clients until it resumes service.
Innovation Solution
A super cluster architecture is introduced, where clusters of nodes are designated as active or standby for processing file system requests, with mirror file systems hosted on shared storage, allowing seamless failover and load balancing by routing requests through a sub-network, ensuring continuous service and efficient resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If two nodes each serve different file systems with multiple mount points configured, then availability is improved when one node goes down, but device complexity increases due to client configuration requirements
Solution Approach 1:
The system segments file system services across multiple nodes with each node serving specific file systems. When a node fails, only the affected file systems need to be reassigned, not the entire service. This segmentation allows availability improvement while reducing the complexity of client reconfiguration by isolating failures to specific segments.
Solution Approach 2:
The patent introduces an intermediary mechanism that automatically handles failover between nodes. Instead of clients directly managing multiple mount points and manually reconfiguring when nodes go down, the system provides automatic detection and redirection of file system requests to available nodes, eliminating client configuration complexity while maintaining availability.
2Device complexity
If a single node services all file system requests, then device complexity is reduced, but productivity decreases due to I/O bottlenecks and client starvation
Solution Approach 1:
The system divides the workload by assigning specific file systems to specific nodes. This segmentation prevents any single node from becoming a bottleneck for all I/O operations, as each node only handles requests for its assigned file systems. This maintains simplicity in node configuration while significantly improving overall I/O processing capacity and eliminating client starvation.
Solution Approach 2:
The system dynamically assigns file systems to nodes based on current operational status. When a node is operational, it services its assigned file systems; when it fails, the assignment is dynamically redistributed to other nodes. This dynamic approach maintains high productivity by preventing I/O bottlenecks while keeping the system simple to manage through automated rather than manual reconfiguration.
3Productivity
If all available network and storage bandwidth is used for I/O processing, then productivity increases, but reliability decreases due to system overload and potential failures
Solution Approach 1:
The system segments I/O processing across multiple nodes, each handling specific file systems. This segmentation prevents any single node from becoming overloaded with all I/O operations, as the workload is distributed. Nodes can operate at high throughput for their assigned file systems without compromising system stability, since failures or overload in one node do not affect the entire system.
Solution Approach 2:
The system changes the operational parameters by introducing selective bandwidth utilization. Instead of all nodes attempting to process all I/O requests simultaneously (maximizing throughput but risking overload), each node processes I/O for its assigned file systems with appropriate bandwidth allocation. This parameter change maintains high productivity for active file systems while improving reliability by preventing system-wide overload.
Data Source
AI summary
Techniques are provided for processing file system requests using a super cluster of clusters of nodes. Mirror file systems for processing the requests are exported through multiple clusters in the super cluster. A cluster may be assigned to an active or passive role for processing file system requests for a set of mirror file systems. A super cluster bundle, or mapping between a cluster in the super cluster and a file system resource on the set of mirror file systems, is created to process the file system requests. The super cluster bundle represents an amount of work assigned to the cluster. A super cluster bundle is reassigned from one cluster to another in response to a failover, or in response to a load balancing determination.