Hybrid Cloud File System Layering for POSIX-Compatible Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional cloud-based storage systems, particularly object-based storage, face limitations such as restricted data access semantics, high latency, and incompatibility with local file system applications, making it difficult to integrate legacy applications without significant expense or complexity.
Innovation Solution
Layering file system functionality on cloud object interfaces, allowing for POSIX interfaces and semantics, enabling data organization in name hierarchies and memory mapping to ensure consistency between memory map changes and writes, thereby transforming disk-based storage systems like ZFS into cloud-based storage solutions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If cloud object storage is used, then storage capacity and scalability are improved, but data access latency increases and file system compatibility deteriorates
Solution Approach 1:
The patent segments cloud storage access into two layers: a fast local cache layer for frequently accessed data and a remote cloud object storage layer for bulk capacity. The file system is divided into cached portions and cloud-stored portions, allowing high-speed local access while maintaining scalable cloud capacity.
Solution Approach 2:
The patent introduces a file system layer as an intermediary between applications and cloud object storage. This layer provides POSIX-compatible file operations and translates them into object storage operations, masking the latency and incompatibility of direct cloud object access.
2Quantity of substance
If cloud object storage interfaces are used, then storage scalability is improved, but compatibility with legacy file system applications deteriorates
Solution Approach 1:
The patent implements a universal file system layer that provides POSIX compatibility while supporting cloud object storage backends. This layer serves multiple functions: providing file semantics to legacy applications, translating operations to object storage, and enabling both local and cloud storage through a unified interface.
Solution Approach 2:
The file system layer acts as an intermediary that translates POSIX file operations into cloud object storage operations. It provides name hierarchies, file locking, and other file system semantics that legacy applications expect, while the underlying storage uses scalable cloud object interfaces.
3Quantity of substance
If data is stored in cloud object storage, then storage capacity is improved, but data security and encryption control deteriorate
Solution Approach 1:
The patent extracts encryption key management from the cloud provider's control and places it in the client's local file system layer. Encryption keys are generated, stored, and managed locally, allowing the client to maintain full control over data security while using cloud storage capacity.
Solution Approach 2:
The patent applies encryption to data before it is uploaded to cloud object storage. The file system layer encrypts data locally using client-controlled keys, then stores the encrypted objects in the cloud. This preliminary encryption action ensures data security is maintained regardless of cloud storage location.
Data Source
AI summary
Techniques described herein relate to systems and methods of data storage, and more particularly to providing layering of file system functionality on an object interface. In certain embodiments, file system functionality may be layered on cloud object interfaces to provide cloud-based storage while allowing for functionality expected from a legacy applications. For instance, POSIX interfaces and semantics may be layered on cloud-based storage, while providing access to data in a manner consistent with file-based access with data organization in name hierarchies. Various embodiments also may provide for memory mapping of data so that memory map changes are reflected in persistent storage while ensuring consistency between memory map changes and writes. For example, by transforming a ZFS file system disk-based storage into ZFS cloud-based storage, the ZFS file system gains the elastic nature of cloud storage.