Asymmetric Distributed File System Snapshot Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current scalable and robust storage solutions for cloud-native applications with Unix-like operating environments and file systems are limited, particularly in supporting rapid deployment of thousands of application instances with security and performance isolation requirements, beyond proprietary services from vendors like Amazon and Google.
Innovation Solution
A distributed file system that stores file content across multiple storage nodes with time-limited leases, allowing clients direct access, and enables snapshot generation by revoking leases and creating snapshots after acknowledgment or lease expiration, ensuring efficient and secure data management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If time-limited leases are provided to clients for direct access to storage nodes, then client access efficiency and performance isolation are improved, but snapshot consistency and data safety deteriorate due to concurrent access
Solution Approach 1:
The system performs preliminary actions by revoking leases from all clients before creating a snapshot. The lease revocation is initiated by the file system manager, and clients are notified to release their leases. Only after all leases are successfully revoked or expired does the system proceed to create the snapshot, ensuring consistent and safe snapshotting despite the time-limited nature of the leases.
2Adaptability or versatility
If multiple storage nodes are used for distributed file storage, then scalability and robustness are improved, but system complexity and data management difficulty increase
Solution Approach 1:
The system segments storage into distributed storage nodes, with each node independently storing portions of the file system volume. The file system manager coordinates access to these segmented storage nodes through lease mechanisms, allowing clients to access specific segments without managing the entire distributed system complexity. This segmentation enables scalability while distributing the complexity across multiple independent units.
3Reliability
If leases are revoked to enable snapshotting, then snapshot safety is improved, but access time and productivity deteriorate due to lease revocation overhead
Solution Approach 1:
The system implements periodic or event-driven lease revocations rather than continuous monitoring. Lease revocations are triggered by specific events such as snapshot requests, and the system periodically checks for lease status. This approach reduces the overhead of lease management while ensuring that snapshots are created safely after all active leases are terminated or expired.
Data Source
AI summary
Processing in an asymmetrically distributed file system may include storing first data representative of the content of the files in a file system volume among a plurality of storage nodes. Second data representative of attributes of the files in the file system volume may be stored in only one of the storage nodes. Time-limited leases allow clients direct access to the plurality of storage node in order to access portions of the file system volume. The time-limited leases may be provided to client lessors. Snapshots of the file system volume may be generated after sending a revocation to the client lessors to revoke time-limited leases provided to the client lessors and having received the acknowledgements of the revocations or after the leases have expired for non-responding lessors, to ensure that changes are not made to the file system volume during snapshot processing.