In-Memory vSAN Object Replication for File Handle Availability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing enterprise-grade storage systems face performance overhead due to the need for persistent file handles to be stored in highly available stable storage, which can disrupt service during server failures and upgrades, necessitating more efficient methods for continuous availability.
Innovation Solution
Implementing an in-memory 'virtual storage area network' (vSAN) object that replicates or uses erasure coding across host devices, storing persistent file handles in volatile memory instead of stable storage, thereby reducing overhead and ensuring continuous availability with low latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If persistent file handles are stored in highly available stable storage, then reliability is improved, but productivity deteriorates due to performance overhead
Solution Approach 1:
The patent segments the storage system into two distinct layers: volatile memory storage for active file handles and stable storage for persistence. This segmentation allows the system to enjoy the speed of volatile memory for frequent operations while maintaining the reliability of stable storage through selective use, thereby resolving the contradiction between performance and reliability
Solution Approach 2:
The patent introduces a file handle cache as an intermediary component between the file system and stable storage. This cache acts as a buffer that holds recently accessed file handles in volatile memory, reducing the frequency of stable storage accesses and thereby improving performance while maintaining reliability through the cache's backup mechanisms
2Productivity
If persistent file handles are stored in volatile memory, then productivity is improved, but reliability deteriorates during host failures
Solution Approach 1:
The patent implements preliminary actions by proactively replicating file handle data to stable storage before volatile memory failures occur, and by pre-establishing backup mechanisms that ensure data can be recovered or transferred to alternative hosts, thus maintaining reliability while using volatile memory for performance
Solution Approach 2:
The patent provides beforehand cushioning through multiple mechanisms: replicating file handle data across multiple hosts before failure occurs, maintaining backup copies in stable storage, and implementing failover clusters that are prepared in advance to take over if a host fails, thereby cushioning against reliability risks while enabling volatile memory usage
3Reliability
If file operations are performed with stable storage persistence, then reliability is improved, but loss of time increases
Solution Approach 1:
The patent applies local quality by making the file handle cache host-specific and location-aware. Each host maintains its own cache of file handles in local volatile memory, allowing rapid local access for file operations. Stable storage persistence is applied selectively only when needed for recovery or cross-host scenarios, thereby minimizing latency for common operations while maintaining data persistence reliability
Data Source
AI summary
Techniques are disclosed for using in-memory replicated object to support file services. Certain embodiments provide a method of storing persistent file handles in a storage system comprising a plurality of computing devices. The method may include requesting to write a persistent file handle corresponding to a file to a file system stored on the plurality of computing devices. The request may be translated to a block input/output (I/O) command to an in-memory object, the in-memory object representing at least a portion of the file system, a copy of the in-memory object being stored at each of the plurality of computing devices in volatile memory. The persistent file handle may then be written to the copy of the in-memory object stored in the volatile memory of each of the plurality of computing devices.