ZFS Cloud Storage Layering for POSIX Access and Local Key Security
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional cloud-based storage systems face limitations such as limited search capabilities, high latency, and non-compatibility with local file system applications, making it costly and impractical to convert legacy applications to use object interfaces, while also compromising security due to encryption key management.
Innovation Solution
Layering file system functionality on cloud object interfaces, specifically using POSIX interfaces and semantics, to provide cloud-based storage that maintains file-based access and organization, while ensuring data integrity and security through local key management and services like compression, encryption, deduplication, snapshots, and clones.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If cloud object storage is used, then storage elasticity and scale are improved, but latency increases and search capability is limited
Solution Approach 1:
The patent segments data into blocks that can be independently managed and cached. By dividing the storage system into cached portions (locally stored frequently accessed data blocks) and cloud-stored portions, the system achieves both cloud elasticity and local access speed, resolving the latency issue while maintaining storage scalability
Solution Approach 2:
The system performs preliminary actions by pre-caching data blocks that are likely to be accessed soon based on access patterns. This anticipatory caching reduces latency for future accesses while maintaining the ability to scale storage capacity through cloud integration
2Quantity of substance
If cloud object storage is used, then storage capacity is improved, but compatibility with local file system applications deteriorates
Solution Approach 1:
The patent introduces a file system layer as an intermediary between cloud object storage and local applications. This layer translates file system operations into object storage operations, enabling legacy applications to access cloud storage through familiar file interfaces without requiring conversion to object interfaces, thus maintaining compatibility while leveraging cloud capacity
Solution Approach 2:
The system provides universal access by implementing both object storage interfaces and traditional file system interfaces on the same cloud storage backend. This multi-functionality allows different applications to use their preferred interface type while accessing the same scalable cloud storage capacity
3Ease of operation
If cloud-managed encryption keys are used, then ease of operation is improved, but security deteriorates
Solution Approach 1:
The patent extracts encryption key management from the cloud provider's control and places it locally on the customer's premises. By taking out the key management function from the cloud environment, the system maintains ease of automated encryption operations while significantly improving security, as customers retain full control over their encryption keys and can enforce stricter security policies
Data Source
AI summary
Techniques described herein relate to systems and methods of data storage, and more particularly to providing layering of file system functionality on an object interface. In certain embodiments, file system functionality may be layered on cloud object interfaces to provide cloud-based storage while allowing for functionality expected from a legacy applications. For instance, POSIX interfaces and semantics may be layered on cloud-based storage, while providing access to data in a manner consistent with file-based access with data organization in name hierarchies. Various embodiments also may provide for memory mapping of data so that memory map changes are reflected in persistent storage while ensuring consistency between memory map changes and writes. For example, by transforming a ZFS file system disk-based storage into ZFS cloud-based storage, the ZFS file system gains the elastic nature of cloud storage.