FUSE File System Transition Mechanism for Hierarchical to Object Store Migration

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Legacy applications dependent on hierarchical file systems face challenges in migrating data to object stores without requiring intrusive modifications, limiting their ability to leverage object store benefits and achieve application mobility.

Innovation Solution

A transition mechanism involving a base file system, a temporary file system, and an intermediate fuse file system allows seamless migration of data from a hierarchical file system to an object store, maintaining hierarchical access and enabling application mobility by using a fuse file system as an interface and storing metadata in a temporary directory and object store.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If data is migrated from hierarchical file system to flat object store, then object store benefits are achieved, but legacy applications cannot access data without intrusive modifications

Engineering Contradiction:

Improveobject store compatibilityVSAvoidapplication modification requirement

Core Design Contradiction:

Adaptability or versatilityVSEase of manufacture

Solution Approach 1:

A translation layer is introduced between legacy applications and the object store to bridge the compatibility gap. This intermediary component translates hierarchical file system operations into object store operations, allowing applications to continue using familiar file system interfaces while the underlying data is stored in object store format. The translation layer handles path resolution, object mapping, and operation translation transparently.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system is divided into distinct functional layers: the legacy application layer, the translation layer, and the object store layer. This segmentation allows each layer to operate independently with its own optimization, enabling the object store to provide its benefits while the application layer maintains its original functionality without modifications.

Inventive Principle:

Principle #1Segmentation

2Productivity

If hierarchical file system data is moved to object store, then data mobility is improved, but applications lose node-local data access

Engineering Contradiction:

Improvedata mobilityVSAvoidnode-local data access

Core Design Contradiction:

ProductivityVSEase of operation

Solution Approach 1:

The translation layer creates and maintains local copies of frequently accessed objects in the node's file system cache. This allows applications to access data with the same performance characteristics as node-local access while the primary data repository remains in the object store, enabling data mobility without sacrificing access speed.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The system pre-loads and caches objects that are likely to be accessed soon into local storage before they are actually needed. This preliminary action ensures that when applications need access to migrated data, it is already available locally, maintaining performance while enabling data mobility.

Inventive Principle:

Principle #10Preliminary action

3Device complexity

If file system metadata is stored in temporary directory, then migration process is simplified, but metadata persistence and reliability are reduced

Engineering Contradiction:

Improvemigration process complexityVSAvoidmetadata persistence

Core Design Contradiction:

Device complexityVSReliability

Solution Approach 1:

The translation layer continuously synchronizes metadata between the temporary directory and the object store. This continuous action ensures that metadata is maintained in both locations with minimal latency, providing the simplicity of temporary storage while ensuring the reliability and persistence of object store metadata.

Inventive Principle:

Principle #20Continuity of useful action

Solution Approach 2:

The system implements feedback mechanisms where metadata changes in the temporary directory trigger automatic updates to the object store, and vice versa. This feedback loop ensures consistency and reliability of metadata across both storage locations throughout the migration process.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS11347696B2System for transition from a hierarchical file system to an object store

Publication Date: 2022.05.31 ORACLE INT CORP

AI summary

Techniques are provided for transitioning from a hierarchical file system to an object store. A request is received to change file metadata of a hierarchical file system. In response to the request: in-cache file metadata of the hierarchical file system is modified. The in-cache file metadata includes a directory structure of the hierarchical file system and file metadata of the hierarchical file system. Additionally, an in-store metadata journal entry is generated. The in-store metadata journal entry indicates an update to in-store file metadata. The in-store file metadata includes the directory structure of the hierarchical file system and file metadata of the hierarchical file system and is stored in an object store. The in-store metadata journal entry is stored in the object store. Furthermore, in-store file metadata and in-store metadata journal entry may be used to reconstruct the in-cache file metadata on another node.