Replication Engine Mapping File System Entities to Object Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Preserving storage efficiency mechanisms such as deduplication when replicating file system data to object-based storage systems is challenging due to fundamental differences in data layout structure and naming schemas between hierarchical file systems and object-based storage systems.
Innovation Solution
A replication engine maps file system entities, including inodes and data blocks, to object-based storage objects, generating inode file objects, indirection objects, and data objects that logically associate file block numbers with data block numbers, thereby preserving deduplication and enabling efficient replication while maintaining storage efficiency mechanisms like deduplication and compression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If file system data is replicated to object-based storage using traditional methods, then data replication is achieved, but storage efficiency mechanisms such as deduplication are lost
Solution Approach 1:
The patent introduces an intermediary mapping layer that translates file system concepts (inodes, file block numbers) into object-based storage concepts (object keys, data blocks). This mapping mechanism preserves the logical structure and deduplication relationships from the file system while adapting them to the object-based storage environment, thereby maintaining storage efficiency during replication.
Solution Approach 2:
The patent changes the parameter representation by transforming file system parameters (inode numbers, file block numbers) into object-based storage parameters (object keys, data block numbers) through a mapping function. This parameter transformation allows the preservation of deduplication relationships by maintaining the logical associations between data blocks while adapting to the different storage paradigm.
2Productivity
If hierarchical file system structure is converted to flat object-based storage, then storage scalability is improved, but data layout structure and naming schema compatibility deteriorates
Solution Approach 1:
The patent segments the conversion process into distinct mapping components: inode mapping, file block number mapping, and data block mapping. By dividing the complex conversion task into manageable segments, the system can transform hierarchical file system structures into flat object-based storage while preserving the necessary logical relationships through structured mapping functions.
Solution Approach 2:
The mapping mechanism serves as an intermediary that bridges the hierarchical file system namespace and the flat object-based storage namespace. It translates hierarchical paths and names into flat object keys while maintaining the logical structure and relationships, enabling compatibility between the two different data layout paradigms.
3Loss of substance
If file system entities are mapped to object-based storage objects, then storage efficiency is preserved, but system complexity increases
Solution Approach 1:
The patent creates simplified copy representations of file system entities (inodes, file block numbers, data blocks) as object-based storage objects. Rather than implementing complex real-time transformations, the system copies the essential structural information into a format suitable for object-based storage, preserving storage efficiency while simplifying the implementation complexity.
Data Source
AI summary
A method and system for replicating file system entities in a manner that preserves block-level access and file system efficiency mechanisms such as deduplication are disclosed. In an embodiment, a replication engine receives a stream of file system entities that include a file system inodes and file system data blocks. The replication engine generates object-based storage (OBS) objects based on data and reference information specified by the file system entities. As part of generating the OBS objects, the replication engine generates at least one inode file object that associates file block numbers of a file system inode file and the inode numbers. The replication engine uses inode information to generate reference objects that logically associate file block numbers with data block numbers in per inode manner. The replication engine further generates data objects that contains the file system data blocks and that associates the data blocks with corresponding data block numbers.