Controller Daemon Bandwidth Reservation for Shared File Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current Storage Area Networks (SANs) do not provide guaranteed rate access to shared file systems across a cluster of nodes, leading to inefficiencies in data transfer and increased hardware requirements, particularly in data-intensive applications like video production and media broadcasting.
Innovation Solution
A method is introduced that establishes a controller daemon to manage bandwidth reservations across a shared file system, determining a qualified bandwidth and ensuring that requests do not exceed this limit, thereby preventing oversubscription and maintaining quality of service by scheduling I/O operations at the file system level.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If network file systems (NFS) are used to transfer data between nodes, then data sharing between storage volumes is achieved, but data duplication occurs and LAN traffic increases
Solution Approach 1:
The invention segments the storage access path into two distinct routes: direct SAN access for bandwidth-intensive operations and network file system access for sharing requirements. This segmentation allows each path to serve its specific purpose optimally, preventing unnecessary data duplication while maintaining sharing capability.
Solution Approach 2:
The invention introduces a bandwidth scheduler as an intermediary component between the file system and the storage device. This scheduler mediates bandwidth allocation, ensuring that critical bandwidth-intensive operations receive sufficient resources while preventing LAN overload from unnecessary data transfers.
2Reliability
If bandwidth reservations are implemented at the client level (GRIO), then guaranteed rate access is achieved, but hardware requirements and complexity increase
Solution Approach 1:
The invention moves the bandwidth scheduling function from the client level to a central server-level scheduler that manages bandwidth allocation for the entire file system. This intermediary approach provides guaranteed rate access without requiring complex hardware configuration at each client, as the server centrally coordinates bandwidth distribution.
Solution Approach 2:
The server-based bandwidth scheduler serves multiple functions: it manages bandwidth allocation for GRIO operations, handles non-GRIO operations, and coordinates access for multiple clients simultaneously. This universal approach eliminates the need for separate hardware configurations at each client while maintaining reliable guaranteed rate access.
3Productivity
If conventional SANs are used, then storage consolidation is achieved, but data sharing between volumes and nodes is limited
Solution Approach 1:
The invention merges the storage consolidation benefits of SANs with the data sharing capabilities of network file systems by implementing a hybrid architecture. The bandwidth scheduler integrates both functions, allowing efficient storage consolidation while enabling flexible data sharing between nodes through the network file system interface.
Data Source
AI summary
A method for preventing oversubscription to a file storage by multiple processes, whether such processes are operating on one node with directly attached storage or on several nodes of a computing cluster sharing a storage area network. Processes or nodes issue requests for bandwidth reservations to a controller daemon. The controller daemon maintains records of all existing bandwidth reservations and ensures that new reservations are granted only if a qualified bandwidth of the file storage will not be exceeded. The qualified bandwidth is empirically determined to take into account installation specific hardware configurations, workloads, and quality of service requirements. In conjunction with suitable enabled client kernels the controller daemon serves to encapsulate all I/O activity including non-reserved I/O activity to the file storage by issuing non-guaranteed bandwidth leases for use by each node in servicing non-guaranteed process I/O activity, such leases being revokable by the controller daemon in order to service new guaranteed bandwidth reservation requests.