POSIX File System Layering for Cloud Object Store Integrity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional cloud-based storage systems face limitations such as limited searchability, high latency, and security concerns due to their object-based interfaces, which do not align with the needs of local file system applications, and converting legacy applications to use object interfaces is costly and impractical.
Innovation Solution
Layering file system functionality on cloud object interfaces, specifically by providing POSIX interfaces and semantics, allowing for memory mapping of data to ensure consistency between memory map changes and writes, and integrating ZFS file system services like compression, encryption, deduplication, snapshots, and clones, while using a hybrid cloud storage system to bridge traditional local file systems with cloud object stores.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If cloud object storage interfaces are used, then scalability and elasticity are improved, but searchability and access efficiency deteriorate
Solution Approach 1:
The patent segments cloud storage access into two distinct interfaces: an object interface for bulk data operations (improving scalability) and a file system interface for detailed data access and search (improving productivity). This segmentation allows each interface to be optimized for its specific purpose without compromising the other.
Solution Approach 2:
The patent introduces a file system layer as an intermediary between applications and cloud object storage. This intermediary provides file system semantics (directories, files, paths) that enable efficient searching and access while the underlying object storage maintains scalability and elasticity.
2Adaptability or versatility
If cloud object storage interfaces are used, then scalability is improved, but latency increases
Solution Approach 1:
The patent implements preliminary actions by caching frequently accessed data and metadata in local storage before cloud access is needed. This reduces latency for subsequent operations while maintaining the scalability benefits of cloud storage for the overall system.
Solution Approach 2:
The file system layer acts as an intermediary that buffers and caches data locally, reducing the frequency and impact of cloud access operations. This intermediary layer absorbs latency variations and provides smoother access patterns.
3Quantity of substance
If cloud object storage is used, then storage capacity is improved, but data security and integrity deteriorate
Solution Approach 1:
The patent introduces multiple intermediary layers between applications and cloud storage: a file system layer that enforces access controls and a local cache layer that stores data copies. These intermediaries provide security boundaries while enabling access to large cloud storage capacity.
Solution Approach 2:
The patent implements beforehand cushioning through local caching and buffering mechanisms that protect against cloud storage failures or security issues. Data is cached locally before cloud operations, providing a safety buffer that maintains security and reliability.
4Adaptability or versatility
If legacy applications are converted to use object interfaces, then adaptability to cloud storage is improved, but implementation cost increases
Solution Approach 1:
Instead of converting legacy applications to use object interfaces (the conventional approach), the patent inverts the approach by making the cloud storage system conform to traditional file system interfaces. This allows legacy applications to continue using familiar interfaces while gaining access to cloud storage capacity.
Solution Approach 2:
The patent creates a universal access layer that supports both traditional file system interfaces and modern object storage interfaces. This multi-functional approach allows the same cloud storage infrastructure to serve both legacy and modern applications without requiring application conversion.
Data Source
Figure 1
Figure 2
Figure 3A~3D
AI summary
Techniques described herein relate to systems and methods of data storage, and more particularly to providing layering of file system functionality on an object interface. In certain embodiments, file system functionality may be layered on cloud object interfaces to provide cloud-based storage while allowing for functionality expected from a legacy applications. For instance, POSIX interfaces and semantics may be layered on cloud-based storage, while providing access to data in a manner consistent with file-based access with data organization in name hierarchies. Various embodiments also may provide for memory mapping of data so that memory map changes are reflected in persistent storage while ensuring consistency between memory map changes and writes. For example, by transforming a ZFS file system disk-based storage into ZFS cloud-based storage, the ZFS file system gains the elastic nature of cloud storage.