File System Catalog for Replication Metadata Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data recovery methods from secondary storage systems are costly and resource-intensive, especially for heterogeneous replication scenarios, as they require substantial processing to locate and retrieve specific data items across different storage paradigms.
Innovation Solution
Implementing a file system catalog that collects and generates metadata during replication, allowing for efficient management and recovery of replicated data by processing replication operations and populating a metadata database, which facilitates targeted data access without traversing entire file system or object-based data layouts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If targeted recovery is limited to specific data items, then recovery cost is reduced, but substantial processing resources are required to locate and retrieve the desired data items
Solution Approach 1:
The patent applies preliminary action by collecting and cataloging metadata during the replication process itself, before recovery operations are needed. The metadata database is populated in advance with information about data item locations and characteristics, so that when targeted recovery is needed, the system can immediately query the pre-built catalog rather than traversing the entire storage structure at recovery time.
Solution Approach 2:
The patent introduces an intermediary metadata database that sits between the storage system and recovery operations. This metadata catalog acts as a mediator that stores pre-collected information about data locations, file system structures, and object mappings, allowing recovery operations to efficiently locate target data without directly traversing the complex underlying storage structures.
2Reliability
If entire file system volume is recovered, then data security is ensured, but recovery cost becomes unduly high
Solution Approach 1:
The patent applies the extraction principle by enabling recovery operations to extract and retrieve only the specific data items that are needed, rather than recovering the entire file system volume. The metadata catalog allows the system to identify and extract only the relevant portions of data, leaving the rest of the volume untouched.
Solution Approach 2:
The patent implements partial action by performing recovery operations on only the necessary subset of data rather than the complete volume. The metadata-driven approach allows the system to apply recovery resources partially and selectively to specific files, directories, or data blocks that require recovery, avoiding the excessive action of processing the entire volume.
3Adaptability or versatility
If heterogeneous replication is implemented, then storage flexibility is improved, but processing resources increase substantially for data location and retrieval
Solution Approach 1:
The patent uses an intermediary metadata catalog that maintains mapping information between different storage paradigms. When data is heterogeneously replicated from file system to object-based storage, the metadata database stores the mapping relationships, allowing efficient location and retrieval of data across different storage types without requiring complex real-time traversal or conversion operations.
Solution Approach 2:
The system performs preliminary actions by collecting and storing metadata about heterogeneous storage structures during the replication process. This pre-collected information about file system to object mapping, data locations, and storage characteristics is stored in the metadata database, eliminating the need for complex processing during data location and retrieval operations across heterogeneous storage systems.
Data Source
AI summary
A method and system for managing backup storage of file system entities. In an aspect, a file system catalog includes a database populator tool that generates records within a metadata table that may be maintained within a database. In response to detecting a replication cycle, the populator tool reads a stream of replication operations. For each of the replication operations, the populator tool determines the type of operation and in response to determining that a directory inode is an operand of the replication operation, the tool generates one or more catalog records. Each of the generated records includes and logically associates data entries corresponding to an inode number, a parent inode number, an entity type, a point-in-time-image (PTI) ID, an absolute path, and an operation.