Cloud File System Layering for Memory-Mapped POSIX Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud object storage systems have limited search capabilities and high latency, making it difficult to align with the needs of local file system applications, and converting legacy applications to use an object interface is costly and impractical.
Innovation Solution
Layering file system functionality on cloud object interfaces, allowing for POSIX interfaces and semantics, while providing memory mapping to ensure consistency between memory map changes and writes, thereby transforming disk-based storage systems like ZFS into cloud-based storage systems that maintain file-based access and organization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If cloud object storage is used directly, then storage elasticity and scale are improved, but search capability and latency are worsened
Solution Approach 1:
The patent introduces a file system layer as an intermediary between cloud object storage and applications. This file system layer provides caching, memory mapping, and file system semantics, thereby reducing latency and improving search capability while maintaining the storage elasticity of cloud object storage.
Solution Approach 2:
The patent segments the storage system into multiple layers: cloud object storage layer, file system layer, and application layer. This segmentation allows each layer to optimize for its specific function, with the file system layer handling latency-critical operations and the cloud layer providing scalable storage.
2Adaptability or versatility
If cloud object storage is used directly, then storage elasticity is improved, but interface compatibility with legacy applications is worsened
Solution Approach 1:
The file system layer acts as an intermediary that translates between cloud object storage interfaces and traditional file system interfaces (POSIX). This allows legacy applications to continue using familiar file system calls while the underlying storage is provided by cloud object storage.
Solution Approach 2:
The file system layer provides multiple interface types (POSIX, memory mapping, etc.) that can serve different application needs, making the system universally compatible with various legacy applications while maintaining cloud storage elasticity.
3Ease of operation
If file system functionality is layered on cloud object interfaces, then interface compatibility is improved, but system complexity is worsened
Solution Approach 1:
The file system layer is designed as a standardized intermediary component that can be implemented once and reused across multiple applications. While it adds complexity, this complexity is encapsulated in a manageable layer that provides significant benefits in interface compatibility and application performance.
Data Source
AI summary
Techniques described herein relate to systems and methods of data storage, and more particularly to providing layering of file system functionality on an object interface. In certain embodiments, file system functionality may be layered on cloud object interfaces to provide cloud-based storage while allowing for functionality expected from a legacy applications. For instance, POSIX interfaces and semantics may be layered on cloud-based storage, while providing access to data in a manner consistent with file-based access with data organization in name hierarchies. Various embodiments also may provide for memory mapping of data so that memory map changes are reflected in persistent storage while ensuring consistency between memory map changes and writes. For example, by transforming a ZFS file system disk-based storage into ZFS cloud-based storage, the ZFS file system gains the elastic nature of cloud storage.