Cloud Object Storage File Hierarchies for POSIX Compatibility
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud object storage systems have limitations such as high latency, limited search capabilities, and non-compliance with local file system interfaces, making it difficult to integrate legacy applications with cloud-based storage without significant expense or complexity, and they also pose security risks due to encryption key management.
Innovation Solution
Layering file system functionality on cloud object interfaces, specifically using POSIX interfaces and semantics, to provide cloud-based storage that aligns with legacy applications, while ensuring data security through local key management and providing features like compression, encryption, deduplication, snapshots, and clones, thereby enabling seamless integration and enhanced performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If cloud object storage is used directly, then storage elasticity and scale are improved, but interface compatibility with legacy applications deteriorates
Solution Approach 1:
The patent introduces a cloud interface appliance as an intermediary layer between legacy applications and cloud object storage. This appliance translates file system interface calls into object storage operations, enabling legacy applications to access cloud storage without modification while maintaining the elasticity and scale benefits of object storage
2Adaptability or versatility
If cloud object storage interfaces are used, then storage scale is improved, but access latency increases
Solution Approach 1:
The cloud interface appliance pre-loads data objects into local cache memory before they are needed by applications. This preliminary action reduces access latency by having data ready in fast local storage rather than retrieving it from remote cloud storage when applications need it
Solution Approach 2:
The patent introduces a new dimension of local caching between cloud storage and applications. This creates a hierarchical storage architecture where frequently accessed data resides in fast local cache while less frequently accessed data remains in scalable cloud storage, optimizing both speed and scale
3Adaptability or versatility
If cloud object storage is used, then storage elasticity is improved, but data security deteriorates due to encryption key management
Solution Approach 1:
The patent extracts encryption key management from the cloud storage provider and implements it locally within the cloud interface appliance. This extraction of the key management function ensures that sensitive encryption keys never leave the customer's controlled environment, maintaining data security while still utilizing the elastic storage capacity of cloud object storage
4Adaptability or versatility
If legacy applications are converted to use object interface, then cloud storage integration is improved, but cost and complexity increase
Solution Approach 1:
The cloud interface appliance serves as a translator that converts object storage operations into file system interface calls. This allows legacy applications to continue using familiar file-based interfaces while seamlessly accessing cloud object storage, eliminating the need for expensive and complex application conversions
Data Source
Figure 1
Figure 2
Figure 3A~3D
AI summary
Techniques described herein relate to systems and methods of data storage, and more particularly to providing layering of file system functionality on an object interface. In certain embodiments, file system functionality may be layered on cloud object interfaces to provide cloud-based storage while allowing for functionality expected from a legacy applications. For instance, POSIX interfaces and semantics may be layered on cloud-based storage, while providing access to data in a manner consistent with file-based access with data organization in name hierarchies. Various embodiments also may provide for memory mapping of data so that memory map changes are reflected in persistent storage while ensuring consistency between memory map changes and writes. For example, by transforming a ZFS file system disk-based storage into ZFS cloud-based storage, the ZFS file system gains the elastic nature of cloud storage.