File Metadata Migration to Secondary Storage Sites
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing disaster recovery systems face challenges in efficiently replicating and migrating files to secondary storage sites, leading to double network traffic and bandwidth consumption.
Innovation Solution
The system migrates only the file metadata to secondary storage sites, avoiding the replication of full file data, and uses a metadata container to distribute file metadata across primary and secondary storage sites.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If full file data is replicated to secondary storage sites for disaster recovery, then data redundancy and reliability are improved, but network bandwidth consumption increases significantly
Solution Approach 1:
The patent extracts only the essential metadata from files and replicates that to secondary storage sites, leaving the full file data at the primary site. This selective extraction of critical information (metadata containing file access paths, permissions, and identifiers) maintains disaster recovery capability while eliminating the need to transfer large volumes of actual file data over the network.
Solution Approach 2:
The patent creates lightweight metadata copies instead of full file data copies. Each metadata record contains sufficient information to reconstruct or access the original file from the primary site, enabling rapid replication and recovery without the bandwidth overhead of copying entire files, especially for large files.
2Productivity
If file metadata is replicated to secondary storage sites, then access efficiency and recovery speed are improved, but network traffic increases
Solution Approach 1:
The patent replicates only metadata records to secondary storage sites, which are exponentially smaller than full file data. This metadata copying enables fast recovery operations by allowing the system to quickly identify and reconstruct files without transferring large amounts of data, thus improving productivity while minimizing network traffic.
Solution Approach 2:
The patent performs preliminary replication of metadata to secondary sites before actual file recovery is needed. This advance preparation ensures that when disaster recovery is triggered, the metadata is already available at secondary sites, enabling immediate file reconstruction and access without delay, thus improving recovery speed.
3Reliability
If both file data and metadata are migrated to distributed file containers, then data availability is improved, but network bandwidth usage doubles
Solution Approach 1:
The patent separates file data from metadata and extracts only the metadata for migration to distributed file containers. The file data remains at the primary storage location, while the extracted metadata is distributed to multiple sites. This separation eliminates the duplication of data migration while maintaining data availability through metadata distribution.
Solution Approach 2:
The patent segments the file system into two distinct components: file data and metadata. By segmenting these components, the system can migrate only the metadata portion to distributed containers, reducing network bandwidth usage by approximately 90% or more for large files while still providing data availability through the distributed metadata structure.
Data Source
AI summary
Provided are a computer program product, system, and method for replicating and migrating files to secondary storage sites. Files in a fileset at the primary storage site are migrated to a file container, wherein data for the files migrated to the file container is distributed among the secondary storage sites. File metadata for each of the migrated files is updated to include access information on the migrated file in the file container. Data for the migrated files is deleted in a local storage. The file metadata for the migrated files is replicated to the secondary storage sites over the network to store in filesets at the secondary storage sites without replicating file data for the migrated files to store in the filesets the secondary storage sites.