File System Redirector Routing for Deduplicated Cluster Caches
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing approaches to distributing network traffic across servers in a cluster with a distributed deduplicated file system do not account for the unique characteristics and challenges posed by such systems, leading to inefficiencies in handling namespace and file operations.
Innovation Solution
A file system redirector and proxy (FSRP) service is introduced to differentiate between namespace and file operations, using distinct algorithms to redirect requests to appropriate access object (AoB) services based on cached data, ensuring efficient resource utilization and caching strategies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single algorithm is used to distribute all requests across cluster nodes, then the system is simpler to implement, but performance is suboptimal because namespace and file operations have different characteristics
Solution Approach 1:
The patent segments the request routing process by creating separate algorithms for namespace operations and file operations. The FSRP service identifies the operation type and applies the appropriate algorithm, allowing each algorithm to be optimized for its specific operation type while maintaining overall system performance.
Solution Approach 2:
The system dynamically selects which algorithm to apply based on the operation type. Rather than using a static single algorithm for all requests, the FSRP service adapts the routing strategy by choosing different algorithms depending on whether the request is a namespace operation or a file operation.
2Reliability
If requests are distributed evenly across all nodes, then load balancing is simplified, but cache utilization is inefficient causing more cache invalidations
Solution Approach 1:
The system performs preliminary actions by caching namespace data at each node before file operations occur. When a namespace operation is received, the FSRP redirects to a node that has the relevant namespace data cached, ensuring that subsequent file operations can utilize this pre-cached data and avoid cache invalidations.
Solution Approach 2:
The FSRP service maintains awareness of which nodes have cached which namespace data, using this feedback information to make intelligent routing decisions. The system continuously updates its understanding of cache states and uses this information to redirect requests to nodes with relevant cached data.
3Productivity
If the FSRP service redirects all requests through a single entry point, then request tracking is simplified, but the system cannot scale to handle thousands of clients efficiently
Solution Approach 1:
The FSRP service acts as an intermediary between clients and the distributed AoB services. It maintains a list of available nodes and redirects client requests appropriately, allowing the system to scale by adding more nodes to the list without changing the client interface or requiring complex client-side load balancing logic.
Solution Approach 2:
The FSRP service provides multiple functions: it serves as an entry point for clients, maintains the node list, performs operation type identification, selects appropriate routing algorithms, and redirects requests. This multi-functional approach consolidates complexity in a single service that can scale by adding nodes to its managed list.
Data Source
AI summary
A file system redirector and proxy (FSRP) service is provided to clients as an entry point to a cluster having a deduplicated file system managing files backed up from the clients. The cluster includes nodes hosting access object (AoB) services as microservices and the file system includes a namespace organizing the files. Data is cached at each node hosting an AoB service. FSRP maintains a list of nodes currently available in the cluster. Upon FSRP receiving a request from a client, a determination is made as to whether the request includes a namespace operation or a file operation. When the request includes the namespace operation, a first algorithm is used to redirect the request to an AoB on the list based on the cached data. When the request includes the file operation, a second algorithm, different from the first algorithm, is used to redirect the request to an AoB on the list based on the cached data.