Metadata Template Workflow for Deduplication Repository Scaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data deduplication systems face challenges in efficiently scaling to support large repositories, such as Petabytes of data, due to the time-consuming process of pre-allocating file systems, especially in systems like IBM ProtecTIER, which lacks support for offloading zeroing operations to storage controllers.
Innovation Solution
The solution involves capturing, exporting, and compressing metadata of a pre-allocated file system, then decompressing and importing it into a new or existing deduplication repository, allowing for efficient configuration and scaling without relying on back-end disk hardware, thereby accelerating the file system padding process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional pre-allocation methods are used to prepare file systems for large deduplication repositories, then the file system can support large data volumes, but the process becomes extremely time-consuming and resource-intensive
Solution Approach 1:
The patent applies preliminary action by pre-generating and storing metadata templates that represent pre-allocated file system structures. When scaling a deduplication repository, instead of performing time-consuming pre-allocation operations on the actual large repository, the system loads pre-prepared metadata templates that already contain the inode structures and allocation information for the desired repository size. This allows the system to bypass the lengthy pre-allocation process and directly establish the file system structure.
Solution Approach 2:
The patent uses copying by creating metadata templates from smaller pre-allocated file systems and then replicating these templates to configure larger repositories. The metadata (including inode structures, allocation maps, and directory information) from a pre-allocated source file system is exported, compressed, and imported into the target repository. This copying approach allows rapid deployment of large repositories without repeating the entire pre-allocation process.
2Productivity
If metadata is exported and compressed from an existing deduplication appliance, then scaling becomes more efficient, but additional processing steps are required
Solution Approach 1:
The patent merges multiple operations into a streamlined process: exporting metadata from the source appliance, compressing it for efficient transfer, transferring it to the target appliance, decompressing it, and importing it into the new repository. By combining these steps into an integrated metadata template workflow, the system achieves efficient scaling despite the additional processing steps, as the compression and template reuse significantly reduce overall time and resource requirements compared to traditional pre-allocation.
Data Source
AI summary
Various embodiments for repository management in a data deduplication system, by a processor device, are provided. Metadata of an inode structure of an entire pre-allocated file system is captured, exported, and compressed from an existing deduplication appliance, the pre-allocated file system comprising a fully padded file system. The exported and compressed metadata of the pre-allocated file system is decompressed and imported into a data deduplication repository of a new deduplication appliance having an identical file system size as within the existing deduplication appliance, to initially configure or subsequently scale the inode structure of a file system of the data deduplication repository of the new deduplication appliance efficiently.