POSIX File System Layering on Cloud Object Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional cloud-based storage systems face limitations such as high latency, limited search capabilities, and incompatibility with local file system applications, making it challenging to integrate legacy applications with cloud object storage.
Innovation Solution
The system layers file system functionality on top of cloud object interfaces, allowing for POSIX interfaces and semantics, memory mapping of data, and integration of ZFS file system services, thereby transforming disk-based ZFS storage into cloud-based storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If cloud object storage is used, then storage elasticity and scalability are improved, but latency increases and search capability is limited
Solution Approach 1:
The patent segments storage into two layers: cloud object storage for elastic, scalable capacity and local disk storage for low-latency access. The file system divides data between these layers, keeping frequently accessed data locally while maintaining full cloud connectivity for expanded storage needs.
Solution Approach 2:
The patent introduces a file system as an intermediary layer between applications and cloud object storage. This file system provides caching mechanisms and local storage buffers that mediate between application data access requests and cloud storage operations, reducing latency by serving data from local cache when possible.
2Quantity of substance
If cloud object storage is used, then storage capacity is improved, but compatibility with local file system applications deteriorates
Solution Approach 1:
The file system acts as an intermediary that translates between traditional file system interfaces (POSIX semantics, directory structures, file operations) and cloud object storage interfaces. This allows legacy applications to continue using familiar file system calls while data is stored in cloud objects, maintaining application compatibility without requiring conversion.
Solution Approach 2:
The file system provides multiple functions: it implements POSIX file system semantics for application compatibility, manages cloud object storage for capacity expansion, and handles data translation between file and object models. This multi-functionality resolves the contradiction by serving both compatibility and capacity needs.
3Adaptability or versatility
If cloud object storage is used, then storage scalability is improved, but search capability deteriorates
Solution Approach 1:
The patent segments search functionality between local and cloud layers. The local file system layer provides fast indexing and search for frequently accessed data, while cloud object storage provides scalable capacity. This segmentation maintains search capability by keeping search-relevant metadata and indexes locally while using cloud storage for bulk data capacity.
Data Source
Figure 1
Figure 2
Figure 3A~3D
AI summary
Techniques described herein relate to systems and methods of data storage, and more particularly to providing layering of file system functionality on an object interface. In certain embodiments, file system functionality may be layered on cloud object interfaces to provide cloud-based storage while allowing for functionality expected from a legacy applications. For instance, POSIX interfaces and semantics may be layered on cloud-based storage, while providing access to data in a manner consistent with file-based access with data organization in name hierarchies. Various embodiments also may provide for memory mapping of data so that memory map changes are reflected in persistent storage while ensuring consistency between memory map changes and writes. For example, by transforming a ZFS file system disk-based storage into ZFS cloud-based storage, the ZFS file system gains the elastic nature of cloud storage.