POSIX File Semantics Layer for Low-Latency Cloud Object Access

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional cloud-based storage systems, particularly object-based storage, face limitations such as high latency, limited search capabilities, and incompatibility with local file system applications, making it costly and impractical to convert legacy applications to use object interfaces, while also compromising security due to encryption key management.

Innovation Solution

Layering file system functionality on cloud object interfaces, specifically using POSIX interfaces and semantics, allows for consistent file-based access and data organization in name hierarchies, enabling memory mapping to ensure data consistency and security through ZFS file system transformation into cloud-based storage, preserving traditional file system services like compression, encryption, deduplication, snapshots, and clones.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If cloud object storage is used, then scalability and elasticity are improved, but latency increases and file system compatibility deteriorates

Engineering Contradiction:

ImprovescalabilityVSAvoidlatency

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The patent segments cloud storage access into two paths: a fast local cache for frequently accessed objects and remote cloud storage for bulk capacity. This segmentation allows the system to maintain scalability while reducing latency for common operations by serving them from local storage rather than remote cloud endpoints.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system performs preliminary actions by pre-caching frequently accessed cloud objects into local storage before they are actually needed. This anticipatory caching reduces latency when applications access these objects, as they are already available locally rather than requiring remote retrieval at the moment of access.

Inventive Principle:

Principle #10Preliminary action

2Quantity of substance

If cloud object storage is used, then storage capacity is improved, but interface compatibility with legacy applications deteriorates

Engineering Contradiction:

Improvestorage capacityVSAvoidinterface compatibility

Core Design Contradiction:

Quantity of substanceVSEase of operation

Solution Approach 1:

The patent introduces a cloud gateway as an intermediary layer between legacy applications and cloud object storage. This gateway translates traditional file system operations (read, write, open, close) into cloud-compatible object operations, allowing legacy applications to access cloud storage without modification while maintaining full storage capacity benefits.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The cloud gateway provides universal access by supporting multiple interface types simultaneously - it can serve both legacy file system protocols and modern cloud object protocols through a single unified layer, making the system adaptable to different application requirements while maintaining ease of operation for legacy systems.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Adaptability or versatility

If cloud object storage is used, then elasticity is improved, but data security deteriorates due to encryption key management

Engineering Contradiction:

ImproveelasticityVSAvoiddata security

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The patent extracts encryption key management from the cloud provider's control and places it locally within the customer's environment. By taking out the key management function from the centralized cloud system and implementing local key storage and management, the system maintains elasticity of cloud storage while improving data security through customer-controlled encryption.

Inventive Principle:

Principle #2Taking out (Extraction)

4Ease of operation

If POSIX interfaces are layered on cloud storage, then file system functionality is improved, but system complexity increases

Engineering Contradiction:

Improvefile system functionalityVSAvoidsystem complexity

Core Design Contradiction:

Ease of operationVSDevice complexity

Solution Approach 1:

The cloud gateway serves as an intermediary that absorbs the complexity of translating between POSIX file system interfaces and cloud object storage operations. This intermediary layer handles the complexity of interface translation, caching strategies, and protocol conversion internally, presenting a simple POSIX-compliant interface to applications while managing system complexity within the gateway itself.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS11714784B2Low-latency direct cloud access with file system hierarchies and semantics

Publication Date: 2023.08.01 ORACLE INT CORP

AI summary

Techniques described herein relate to systems and methods of data storage, and more particularly to providing layering of file system functionality on an object interface. In certain embodiments, file system functionality may be layered on cloud object interfaces to provide cloud-based storage while allowing for functionality expected from a legacy applications. For instance, POSIX interfaces and semantics may be layered on cloud-based storage, while providing access to data in a manner consistent with file-based access with data organization in name hierarchies. Various embodiments also may provide for memory mapping of data so that memory map changes are reflected in persistent storage while ensuring consistency between memory map changes and writes. For example, by transforming a ZFS file system disk-based storage into ZFS cloud-based storage, the ZFS file system gains the elastic nature of cloud storage.