Cloud File System Hierarchies for Low-Latency POSIX Access

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional cloud-based storage systems, particularly object-based storage, face limitations such as restricted search capabilities and high latency, which do not align with the needs of local file system applications, and require expensive and complex conversions to support legacy applications.

Innovation Solution

Layering file system functionality on cloud object interfaces, enabling POSIX interfaces and semantics, and providing memory mapping to ensure consistency between memory map changes and writes, thereby allowing legacy applications to access data as files and organize it in name hierarchies while leveraging cloud storage's elasticity and scale.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If cloud object storage interfaces are used, then elasticity and scale are improved, but latency increases and search capability is limited

Engineering Contradiction:

Improveelasticity and scaleVSAvoidlatency

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The patent introduces a file system layer as an intermediary between cloud object storage and applications. This file system layer provides POSIX-compliant interfaces, hierarchical namespace organization, and caching mechanisms that reduce latency while maintaining access to the elastic cloud storage backend. The intermediary translates file system operations into object storage operations, bridging the gap between application needs and cloud capabilities.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent segments the storage system into multiple layers: cloud object storage for bulk capacity, file system layer for organization and access optimization, and cache layer for frequent operations. This segmentation allows each layer to optimize for its specific function while collectively providing both elasticity and low latency performance.

Inventive Principle:

Principle #1Segmentation

2Quantity of substance

If cloud object storage is used, then storage capacity is improved, but POSIX interface compatibility deteriorates

Engineering Contradiction:

Improvestorage capacityVSAvoidPOSIX interface compatibility

Core Design Contradiction:

Quantity of substanceVSEase of operation

Solution Approach 1:

The file system layer acts as an intermediary that translates POSIX file system operations into cloud object storage operations. It maintains POSIX interface compatibility by implementing standard file system calls (open, read, write, close) while mapping these operations to object storage primitives (put, get, delete). This allows legacy applications to access cloud storage without modification.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The file system layer provides universal access to cloud storage by implementing multiple interface standards simultaneously - POSIX interfaces for legacy applications, hierarchical namespace for organization, and object storage interfaces for the backend. This multi-functionality enables a single system to serve diverse access patterns and application requirements.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Productivity

If legacy applications are converted to use object interface, then cloud storage efficiency is improved, but conversion cost and complexity increase

Engineering Contradiction:

Improvecloud storage efficiencyVSAvoidconversion cost and complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent creates a virtual copy of the traditional file system interface layer that sits between applications and cloud storage. Instead of modifying applications to use object interfaces directly, it copies the familiar file system API behavior and implements it on top of object storage. This allows legacy applications to continue using file system operations while benefiting from cloud storage efficiency.

Inventive Principle:

Principle #26Copying

4Reliability

If encryption is implemented in cloud object storage, then security is improved, but key management vulnerability increases

Engineering Contradiction:

ImprovesecurityVSAvoidkey management vulnerability

Core Design Contradiction:

ReliabilityVSObject-affected harmful factors

Solution Approach 1:

The file system layer serves as an intermediary for encryption key management, maintaining keys locally in secure memory and managing their lifecycle. It handles key generation, storage, rotation, and deletion without exposing them to the cloud provider, thereby maintaining security while reducing key management vulnerability compared to cloud-managed encryption.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS10884984B2Low-latency direct cloud access with file system hierarchies and semantics

Publication Date: 2021.01.05 ORACLE INT CORP

AI summary

Techniques described herein relate to systems and methods of data storage, and more particularly to providing layering of file system functionality on an object interface. In certain embodiments, file system functionality may be layered on cloud object interfaces to provide cloud-based storage while allowing for functionality expected from a legacy applications. For instance, POSIX interfaces and semantics may be layered on cloud-based storage, while providing access to data in a manner consistent with file-based access with data organization in name hierarchies. Various embodiments also may provide for memory mapping of data so that memory map changes are reflected in persistent storage while ensuring consistency between memory map changes and writes. For example, by transforming a ZFS file system disk-based storage into ZFS cloud-based storage, the ZFS file system gains the elastic nature of cloud storage.