Hybrid Cloud File Mirroring with POSIX Layering for Low-Latency Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional cloud-based storage systems face limitations such as high latency, limited search capabilities, and incompatibility with local file system applications, making it costly and impractical to convert legacy applications to use object interfaces, while also compromising security due to encryption key management.
Innovation Solution
Layering file system functionality on cloud object interfaces, specifically using POSIX interfaces and semantics to enable file-based access and data organization in name hierarchies, while providing memory mapping and data services like compression, encryption, deduplication, snapshots, and clones, thereby transforming ZFS file system disk-based storage into cloud-based storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of stationary object
If cloud object storage is used, then storage capacity and scalability are improved, but latency and performance deteriorate
Solution Approach 1:
The patent segments storage into two distinct layers: a local storage layer (disk or flash) for high-speed access to frequently used data, and a cloud object storage layer for capacity expansion. The file system intelligently manages which files reside on which layer, allowing applications to benefit from both local speed and cloud capacity without the performance penalty of pure cloud storage.
Solution Approach 2:
The patent introduces a file system as an intermediary layer between applications and cloud object storage. This file system provides a POSIX-compatible interface that translates traditional file operations into cloud object operations, masking the latency of cloud storage from applications while enabling access to cloud capacity.
2Adaptability or versatility
If cloud object storage is used, then storage scalability is improved, but compatibility with legacy file system applications deteriorates
Solution Approach 1:
The file system acts as an intermediary that translates POSIX file operations into cloud object storage operations. This allows legacy applications to continue using familiar file system interfaces while the underlying cloud object storage provides scalable capacity, without requiring application modifications.
Solution Approach 2:
The patent creates a universal storage interface that works with both traditional file system applications and cloud object storage. The file system layer provides POSIX compatibility for legacy applications while simultaneously supporting cloud object storage protocols, making the system adaptable to multiple storage backends.
3Volume of stationary object
If cloud object storage encryption is used, then data is stored remotely, but security is compromised due to encryption key management
Solution Approach 1:
The patent extracts encryption key management from the cloud provider's control and keeps it locally on the customer's premises. Encryption is performed locally before data is uploaded to cloud object storage, and decryption keys remain local. This allows data to be stored remotely in the cloud while maintaining security control locally, preventing cloud providers from accessing encrypted data.
Data Source
Figure 1
Figure 2
Figure 3A~3D
AI summary
Techniques described herein relate to systems and methods of data storage, and more particularly to providing layering of file system functionality on an object interface. In certain embodiments, file system functionality may be layered on cloud object interfaces to provide cloud-based storage while allowing for functionality expected from a legacy applications. For instance, POSIX interfaces and semantics may be layered on cloud-based storage, while providing access to data in a manner consistent with file-based access with data organization in name hierarchies. Various embodiments also may provide for memory mapping of data so that memory map changes are reflected in persistent storage while ensuring consistency between memory map changes and writes. For example, by transforming a ZFS file system disk-based storage into ZFS cloud-based storage, the ZFS file system gains the elastic nature of cloud storage.