POSIX Layering on Cloud Object Storage for Low-Latency Backup
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud object storage systems have limitations such as high latency, limited search capabilities, and non-compliance with local file system interfaces, making it difficult to integrate legacy applications and ensuring data security, which necessitates a solution for layering file system functionality on an object interface.
Innovation Solution
Layering POSIX interfaces and semantics on cloud-based storage systems, enabling file-based access and data organization in name hierarchies, while providing memory mapping and data services like compression, encryption, deduplication, snapshots, and clones, to transform disk-based storage systems like ZFS into cloud-based systems, ensuring consistency and security.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If cloud object storage is used directly, then elasticity and scale are improved, but latency increases and file system interface compatibility deteriorates
Solution Approach 1:
The patent introduces a cloud interface appliance as an intermediary layer between the ZFS file system and cloud object storage. This appliance translates file system operations into object storage operations, enabling legacy applications to access cloud storage with file-based semantics while maintaining ZFS functionalities like compression, encryption, and snapshots, thereby reducing latency and improving interface compatibility without sacrificing storage elasticity
2Quantity of substance
If cloud object storage is used directly, then storage scale is improved, but search capability deteriorates
Solution Approach 1:
The patent segments the storage system into a local ZFS file system layer that maintains full search capabilities and a cloud object storage layer for scalable capacity. The cloud interface appliance manages data distribution between these layers, allowing applications to leverage both the searchability of local storage and the scalability of cloud storage simultaneously
3Reliability
If cloud object storage encryption is used, then storage security is improved, but data vulnerability increases due to key management
Solution Approach 1:
The cloud interface appliance acts as a secure intermediary that manages encryption keys locally while providing cloud storage encryption. This approach maintains the security benefits of encrypted cloud storage while eliminating the key management vulnerabilities associated with cloud-managed encryption, as keys remain under local control through the appliance
4Adaptability or versatility
If POSIX interfaces are layered on cloud storage, then legacy application compatibility is improved, but system complexity increases
Solution Approach 1:
The cloud interface appliance provides a unified POSIX-compliant interface that transparently translates file system operations into cloud object storage operations. This single intermediary point of complexity allows legacy applications to access cloud storage without modification while the underlying complexity of interface translation, data synchronization, and consistency management is encapsulated within the appliance itself
Data Source
Figure 1
Figure 2
Figure 3A~3D
AI summary
Techniques described herein relate to systems and methods of data storage, and more particularly to providing layering of file system functionality on an object interface. In certain embodiments, file system functionality may be layered on cloud object interfaces to provide cloud-based storage while allowing for functionality expected from a legacy applications. For instance, POSIX interfaces and semantics may be layered on cloud-based storage, while providing access to data in a manner consistent with file-based access with data organization in name hierarchies. Various embodiments also may provide for memory mapping of data so that memory map changes are reflected in persistent storage while ensuring consistency between memory map changes and writes. For example, by transforming a ZFS file system disk-based storage into ZFS cloud-based storage, the ZFS file system gains the elastic nature of cloud storage.