POSIX Layer for Cloud Object Stores With File Integrity

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional cloud-based storage systems face limitations such as limited search capabilities, high latency, and non-compatibility with local file system applications, making it costly and impractical to convert legacy applications to use object interfaces, while also compromising security due to cloud object storage encryption practices.

Innovation Solution

Layering file system functionality on cloud object interfaces, specifically by implementing POSIX interfaces and semantics, allowing for memory mapping and data organization in name hierarchies, and integrating ZFS file system services like compression, encryption, deduplication, snapshots, and clones, to provide a hybrid cloud storage system that bridges traditional file systems with cloud object stores.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If cloud object storage is used, then storage elasticity and scale are improved, but interface compatibility with legacy file system applications deteriorates

Engineering Contradiction:

Improvestorage elasticityVSAvoidinterface compatibility

Core Design Contradiction:

Adaptability or versatilityVSEase of operation

Solution Approach 1:

The patent introduces a cloud interface appliance as an intermediary layer between the cloud object store and the file system. This appliance translates file system operations (read, write, seek, truncate) into object storage operations (put, delete, get), enabling legacy applications to access cloud storage through familiar file system interfaces while maintaining the elasticity benefits of object storage

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system segments the storage architecture into distinct layers: the cloud object store layer providing elasticity, the cloud interface appliance layer providing translation and compatibility, and the file system layer providing application interface. This segmentation allows each layer to optimize for its specific function while working together as a unified system

Inventive Principle:

Principle #1Segmentation

2Reliability

If cloud object storage encryption is implemented, then data security is improved, but key management vulnerability increases

Engineering Contradiction:

Improvedata securityVSAvoidkey management vulnerability

Core Design Contradiction:

ReliabilityVSObject-affected harmful factors

Solution Approach 1:

The patent extracts key management functionality from the cloud service provider's control and places it locally within the file system and cloud interface appliance. Encryption keys are generated and managed locally, allowing the system to maintain data security through encryption while eliminating the vulnerability of relying on cloud provider's key management

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The system implements self-service key management where the local file system and cloud interface appliance autonomously handle encryption and decryption operations using locally-stored keys. This self-managed approach to encryption eliminates dependency on the cloud provider's security infrastructure

Inventive Principle:

Principle #25Self-service

3Ease of operation

If POSIX interfaces are layered on cloud storage, then application compatibility is improved, but system complexity increases

Engineering Contradiction:

Improveapplication compatibilityVSAvoidsystem complexity

Core Design Contradiction:

Ease of operationVSDevice complexity

Solution Approach 1:

The cloud interface appliance serves as a mediator that absorbs the complexity of translating between object storage and POSIX interfaces. Applications interact with simple, familiar file system calls while the intermediary handles the complex translation to object storage operations, keeping the application side simple

Inventive Principle:

Principle #24Intermediary (Mediator)

4Speed

If memory mapping is implemented, then data access performance is improved, but data consistency challenges increase

Engineering Contradiction:

Improvedata access performanceVSAvoiddata consistency

Core Design Contradiction:

SpeedVSReliability

Solution Approach 1:

The system implements feedback mechanisms where the cloud interface appliance monitors and tracks memory mapping operations, ensuring that changes in memory are properly reflected in persistent storage. This feedback loop maintains consistency between volatile memory and persistent object storage

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS10642879B2Guaranteed file system hierarchy data integrity in cloud object stores

Publication Date: 2020.05.05 ORACLE INT CORP

AI summary

Techniques described herein relate to systems and methods of data storage, and more particularly to providing layering of file system functionality on an object interface. In certain embodiments, file system functionality may be layered on cloud object interfaces to provide cloud-based storage while allowing for functionality expected from a legacy applications. For instance, POSIX interfaces and semantics may be layered on cloud-based storage, while providing access to data in a manner consistent with file-based access with data organization in name hierarchies. Various embodiments also may provide for memory mapping of data so that memory map changes are reflected in persistent storage while ensuring consistency between memory map changes and writes. For example, by transforming a ZFS file system disk-based storage into ZFS cloud-based storage, the ZFS file system gains the elastic nature of cloud storage.