Cloud File System Layering for POSIX Access and Lower Latency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Cloud object storage systems have limited search capabilities and high latency, making it difficult to align with the needs of local file system applications, and converting legacy applications to use an object interface is expensive and impractical.

Innovation Solution

Layering file system functionality on cloud object interfaces to provide cloud-based storage while preserving native application interfaces, allowing for POSIX compliance and data organization in name hierarchies, and enabling memory mapping to ensure consistency between memory map changes and writes.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If cloud object storage is used directly, then storage elasticity and scale are improved, but search capability and latency are worsened

Engineering Contradiction:

Improvestorage elasticityVSAvoidsearch capability

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The patent introduces a file system layer as an intermediary between cloud object storage and applications. This layer provides search capabilities and latency optimization that pure object storage lacks, while maintaining the elasticity and scale benefits of cloud storage through hierarchical data placement and caching strategies.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system segments data into different storage tiers (hot, warm, cold) based on access patterns. Frequently accessed data is cached in faster storage layers with better search capabilities, while less frequently accessed data resides in cost-effective cloud object storage, thus improving overall search performance without sacrificing storage elasticity.

Inventive Principle:

Principle #1Segmentation

2Adaptability or versatility

If cloud object storage is used directly, then storage elasticity and scale are improved, but latency is worsened

Engineering Contradiction:

Improvestorage scaleVSAvoidaccess latency

Core Design Contradiction:

Adaptability or versatilityVSSpeed

Solution Approach 1:

The system performs preliminary actions by pre-caching frequently accessed data objects in faster storage layers before they are actually needed. This anticipatory caching reduces latency for common operations while maintaining the ability to scale storage capacity in the cloud.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system dynamically adjusts data placement between cloud object storage and local/cache storage based on real-time access patterns. Hot data is automatically moved to faster storage, while cold data remains in cost-effective cloud storage, optimizing both latency and scale adaptively.

Inventive Principle:

Principle #15Dynamics

3Adaptability or versatility

If legacy applications are converted to use object interface, then cloud storage compatibility is improved, but application complexity and cost are worsened

Engineering Contradiction:

Improvecloud storage compatibilityVSAvoidapplication complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

Instead of converting applications to use object interfaces, the patent inverts the approach by providing a file system interface that translates to object storage operations. This allows legacy applications to continue using familiar file system interfaces while the underlying system handles object storage compatibility, reducing application complexity.

Inventive Principle:

Principle #13The other way round (Inversion)

4Adaptability or versatility

If file system functionality is layered on cloud object interfaces, then POSIX compliance is improved, but system complexity is worsened

Engineering Contradiction:

ImprovePOSIX complianceVSAvoidsystem complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent introduces a file system layer as an intermediary that implements POSIX compliance while translating operations to cloud object storage interfaces. This layer handles the complexity of file system semantics (directories, permissions, locking) internally, presenting a standard POSIX interface to applications without requiring them to understand the underlying object storage complexity.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS10552469B2File system hierarchy mirroring across cloud data stores

Publication Date: 2020.02.04 ORACLE INT CORP

AI summary

Techniques described herein relate to systems and methods of data storage, and more particularly to providing layering of file system functionality on an object interface. In certain embodiments, file system functionality may be layered on cloud object interfaces to provide cloud-based storage while allowing for functionality expected from a legacy applications. For instance, POSIX interfaces and semantics may be layered on cloud-based storage, while providing access to data in a manner consistent with file-based access with data organization in name hierarchies. Various embodiments also may provide for memory mapping of data so that memory map changes are reflected in persistent storage while ensuring consistency between memory map changes and writes. For example, by transforming a ZFS file system disk-based storage into ZFS cloud-based storage, the ZFS file system gains the elastic nature of cloud storage.