Cloud File Hierarchies With POSIX Semantics on Object Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud object storage systems have limited search capabilities and high latency, making it difficult to align with the needs of local file system applications, and converting legacy applications to use an object interface is costly and impractical.
Innovation Solution
Layering file system functionality on cloud object interfaces, allowing for POSIX interfaces and semantics, while providing memory mapping to ensure consistency between memory map changes and writes, effectively transforming disk-based storage into cloud-based storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If cloud object storage is used directly, then storage capacity and elasticity are improved, but access latency increases and search capability is limited
Solution Approach 1:
The patent introduces a file system layer as an intermediary between cloud object storage and applications. This layer translates file system operations into object storage operations, enabling applications to access cloud storage with file system semantics while maintaining the elasticity and capacity of object storage. The intermediary layer handles operations like read, write, and search requests, converting them into appropriate object storage API calls.
Solution Approach 2:
The patent segments the storage system into distinct layers: the cloud object storage layer for capacity and elasticity, and a file system layer for access patterns and semantics. This segmentation allows each layer to optimize for its specific function while working together to provide both high capacity and low-latency access.
2Adaptability or versatility
If cloud object storage interfaces are used, then storage elasticity is improved, but compatibility with legacy applications deteriorates
Solution Approach 1:
The file system layer acts as an intermediary that translates legacy application file system calls into cloud object storage operations. This allows legacy applications to continue using familiar POSIX interfaces while the underlying storage leverages the elasticity of cloud object storage. The intermediary handles namespace management, file operations, and data organization transparently.
Solution Approach 2:
The patent creates a universal interface layer that supports both traditional file system operations and cloud object storage operations. This multi-functional layer can handle various access patterns and operation types, making the system compatible with diverse applications while maintaining cloud storage benefits.
3Ease of operation
If file system functionality is layered on cloud object interfaces, then application compatibility is improved, but system complexity increases
Solution Approach 1:
While the intermediary layer does add architectural components, it consolidates complexity into a single manageable layer. The file system layer implements standard interfaces and operations, making it predictable and maintainable. This centralized approach to handling complexity is more manageable than distributed complexity across multiple components.
Solution Approach 2:
The file system layer implements self-managing capabilities for operations like namespace management, data organization, and operation translation. This automation reduces the operational burden and makes the added complexity self-contained rather than requiring extensive external management.
4Quantity of substance
If cloud object storage is used, then storage scale is improved, but search capability deteriorates
Solution Approach 1:
The file system layer restores search capability by implementing file system semantics on top of object storage. It can traverse the namespace hierarchy, search for files by name or path, and retrieve metadata, all while the actual data resides in cloud object storage. This intermediary layer bridges the gap between scalable object storage and traditional file system search operations.
Data Source
AI summary
Techniques described herein relate to systems and methods of data storage, and more particularly to providing layering of file system functionality on an object interface. In certain embodiments, file system functionality may be layered on cloud object interfaces to provide cloud-based storage while allowing for functionality expected from a legacy applications. For instance, POSIX interfaces and semantics may be layered on cloud-based storage, while providing access to data in a manner consistent with file-based access with data organization in name hierarchies. Various embodiments also may provide for memory mapping of data so that memory map changes are reflected in persistent storage while ensuring consistency between memory map changes and writes. For example, by transforming a ZFS file system disk-based storage into ZFS cloud-based storage, the ZFS file system gains the elastic nature of cloud storage.