Cloud File System Layering for POSIX Access on Object Storage

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Cloud object storage systems have limited search capabilities and high latency, making it difficult to align with the needs of local file system applications, and converting legacy applications to use an object interface is costly and impractical.

Innovation Solution

Layering file system functionality on cloud object interfaces, allowing for POSIX interfaces and semantics, while providing memory mapping to ensure consistency between memory map changes and writes, effectively transforming ZFS file system disk-based storage into cloud-based storage, enabling elastic scalability.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If cloud object storage is used directly, then elasticity and scalability are improved, but search capability is limited and latency increases

Engineering Contradiction:

Improveelasticity and scalabilityVSAvoidlatency

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The patent introduces a file system layer as an intermediary between cloud object storage and applications. This file system layer provides familiar file operations, caching, and metadata management, thereby reducing latency for applications while maintaining the elasticity and scalability of underlying cloud object storage.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If cloud object storage is used directly, then elasticity and scalability are improved, but alignment with local file system application needs deteriorates

Engineering Contradiction:

Improveelasticity and scalabilityVSAvoidalignment with file system application needs

Core Design Contradiction:

Adaptability or versatilityVSEase of operation

Solution Approach 1:

The file system layer acts as a mediator that translates cloud object storage operations into familiar file system operations. It provides POSIX-compliant interfaces, directory structures, and file operations that local applications expect, while backend operations leverage cloud object storage's elasticity and scalability.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system segments functionality into distinct layers: the file system layer handles file operations, caching, and metadata management, while the cloud object storage layer handles bulk data storage. This segmentation allows each layer to optimize for its specific function while maintaining overall system elasticity and scalability.

Inventive Principle:

Principle #1Segmentation

3Adaptability or versatility

If legacy applications are converted to use object interface, then cloud storage compatibility is improved, but cost increases and practicality decreases

Engineering Contradiction:

Improvecloud storage compatibilityVSAvoidconversion cost and practicality

Core Design Contradiction:

Adaptability or versatilityVSEase of manufacture

Solution Approach 1:

Instead of converting legacy applications to use cloud object storage interfaces, the patent inverts the approach by bringing file system functionality to the cloud object storage interface. This allows legacy applications to continue using familiar file system interfaces while benefiting from cloud storage's elasticity and scalability.

Inventive Principle:

Principle #13The other way round (Inversion)

Data Source

PatentEP3566132B1File system hierarchy mirroring across cloud data stores

Publication Date: 2023.09.06 ORACLE INT CORP

Figure 1
Figure 2
Figure 3A~3D

AI summary

Techniques described herein relate to systems and methods of data storage, and more particularly to providing layering of file system functionality on an object interface. In certain embodiments, file system functionality may be layered on cloud object interfaces to provide cloud-based storage while allowing for functionality expected from a legacy applications. For instance, POSIX interfaces and semantics may be layered on cloud-based storage, while providing access to data in a manner consistent with file-based access with data organization in name hierarchies. Various embodiments also may provide for memory mapping of data so that memory map changes are reflected in persistent storage while ensuring consistency between memory map changes and writes. For example, by transforming a ZFS file system disk-based storage into ZFS cloud-based storage, the ZFS file system gains the elastic nature of cloud storage.