POSIX File Semantics Layer for Cloud Object Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud object storage systems have limitations such as restricted data/object access semantics, high latency, and non-POSIX compliance, making it difficult to integrate with legacy file system applications without expensive adaptation layers.
Innovation Solution
Layering file system functionality on cloud object interfaces, specifically using POSIX interfaces and semantics, to provide cloud-based storage while maintaining file-based access and organization, and enabling memory mapping for consistent data access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If cloud object storage interfaces are used directly, then elasticity and scale are improved, but POSIX compliance and file system functionality are lost
Solution Approach 1:
The patent introduces a file system layer as an intermediary between cloud object storage and legacy applications. This layer translates POSIX file operations into cloud object storage operations, allowing applications to interact with cloud storage using familiar file system interfaces while maintaining access to the elasticity and scale of object storage.
Solution Approach 2:
The system segments the storage architecture into distinct layers: the cloud object storage layer providing elasticity and scale, and a file system layer providing POSIX compliance. This segmentation allows each layer to independently optimize for its specific function while working together to solve the contradiction.
2Adaptability or versatility
If legacy applications are converted to use object interface, then cloud storage integration is improved, but cost and complexity increase
Solution Approach 1:
Rather than converting legacy applications to use object interfaces directly, the patent introduces a file system intermediary that translates object storage operations into POSIX-compliant file operations. This approach maintains cloud storage integration while avoiding the complexity of application conversion.
3Quantity of substance
If cloud object storage is used, then storage capacity is improved, but latency increases
Solution Approach 1:
The file system layer performs preliminary actions by caching frequently accessed data and pre-processing operations locally before interacting with cloud object storage. This reduces the frequency and impact of remote storage accesses, thereby reducing latency while maintaining large storage capacity.
Data Source
Figure 1
Figure 2
Figure 3A~3D
AI summary
Techniques described herein relate to systems and methods of data storage, and more particularly to providing layering of file system functionality on an object interface. In certain embodiments, file system functionality may be layered on cloud object interfaces to provide cloud-based storage while allowing for functionality expected from a legacy applications. For instance, POSIX interfaces and semantics may be layered on cloud-based storage, while providing access to data in a manner consistent with file-based access with data organization in name hierarchies. Various embodiments also may provide for memory mapping of data so that memory map changes are reflected in persistent storage while ensuring consistency between memory map changes and writes. For example, by transforming a ZFS file system disk-based storage into ZFS cloud-based storage, the ZFS file system gains the elastic nature of cloud storage.