POSIX File Semantics Over Cloud Object Storage Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional cloud-based storage systems, particularly object-based storage, face limitations such as restricted data access semantics, high latency, and incompatibility with local file system applications, making it difficult to integrate legacy applications without significant expense or complexity.
Innovation Solution
Layering file system functionality on cloud object interfaces, allowing for POSIX interfaces and semantics, while providing memory mapping to ensure consistency between memory changes and writes, effectively transforming disk-based storage systems like ZFS into cloud-based storage solutions that maintain file-based access and organization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If cloud object storage is used, then scalability and elasticity are improved, but latency increases and file system compatibility deteriorates
Solution Approach 1:
The patent segments the storage system into multiple components: a file system layer that handles file operations and a cloud object storage layer that provides scalable storage. This segmentation allows the system to maintain file system semantics locally while leveraging cloud scalability, thereby reducing latency for file operations while maintaining adaptability for storage capacity.
Solution Approach 2:
The patent introduces an intermediary file system layer between applications and cloud object storage. This intermediary translates file system operations into object storage operations, maintaining file system compatibility while enabling cloud scalability. The intermediary buffers and manages data transfers, helping to mitigate latency issues.
2Quantity of substance
If cloud object storage interfaces are used, then storage capacity is improved, but compatibility with legacy file system applications deteriorates
Solution Approach 1:
Instead of converting legacy applications to use object storage interfaces, the patent inverts the approach by implementing a file system layer on top of object storage. This allows legacy applications to continue using familiar file system interfaces while the underlying object storage provides enhanced capacity. The file system layer translates traditional file operations into object storage operations.
3Reliability
If encryption keys are stored in cloud object storage, then data security is improved, but vulnerability increases
Solution Approach 1:
The patent extracts encryption key management from the cloud object storage system and implements it locally within the file system layer. This extraction allows encryption keys to be managed securely on-premises, reducing vulnerability associated with cloud-based key storage while maintaining data security through proper encryption of stored objects.
Data Source
AI summary
Techniques described herein relate to systems and methods of data storage, and more particularly to providing layering of file system functionality on an object interface. In certain embodiments, file system functionality may be layered on cloud object interfaces to provide cloud-based storage while allowing for functionality expected from a legacy applications. For instance, POSIX interfaces and semantics may be layered on cloud-based storage, while providing access to data in a manner consistent with file-based access with data organization in name hierarchies. Various embodiments also may provide for memory mapping of data so that memory map changes are reflected in persistent storage while ensuring consistency between memory map changes and writes. For example, by transforming a ZFS file system disk-based storage into ZFS cloud-based storage, the ZFS file system gains the elastic nature of cloud storage.