Base File Relationship Determination for Data Replication

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing data backup systems face inefficiencies in replicating data backups across storage systems, particularly when network outages interrupt the replication process, leading to impractical and unnecessary data transfer of intervening generations.

Innovation Solution

The system determines and stores base file relationships between files, allowing for efficient replication by leveraging common data segments between generations, and constructs a base file relationship between the current generation and the last successfully replicated generation to minimize data transfer during resumed replication.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If the system replicates data backups across storage systems after network outages, then data redundancy and disaster recovery capabilities are improved, but the amount of data transfer increases significantly due to unnecessary transfer of common regions from intervening generations

Engineering Contradiction:

Improvedata redundancyVSAvoiddata transfer volume

Core Design Contradiction:

ReliabilityVSLoss of substance

Solution Approach 1:

The system performs preliminary determination of base file relationships between generations before replication occurs. By pre-establishing which files in the current generation are clones of which in previous generations, the system can prepare replication strategies that leverage these relationships to avoid transferring common data regions, thus reducing overall data transfer volume while maintaining redundancy.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system segments the replication process by identifying and separating common data regions from unique data regions across generations. By determining base file relationships, the system can segment the data transfer operation to only transmit unique portions while reusing common portions from previous generations, significantly reducing the data transfer volume required.

Inventive Principle:

Principle #1Segmentation

2Reliability

If the system transfers all data during replication resumption, then complete data redundancy is achieved, but replication time and network bandwidth consumption increase significantly

Engineering Contradiction:

Improvedata redundancyVSAvoidreplication time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system performs preliminary analysis of file relationships and determines which data regions are common across generations before initiating replication. This pre-computation of base file relationships allows the system to design an optimized replication plan that only transfers necessary unique data, dramatically reducing replication time while ensuring complete redundancy is achieved.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

Instead of transferring all data (excessive action), the system transfers only the necessary unique portions of data by leveraging base file relationships to identify and skip common regions. This partial action approach maintains complete redundancy since common data is reused from previous generations, while significantly reducing replication time and network bandwidth consumption.

Inventive Principle:

Principle #16Partial or excessive action

3Productivity

If the system stores base file relationships between all generations, then replication efficiency is improved, but storage space requirements increase

Engineering Contradiction:

Improvereplication efficiencyVSAvoidstorage space

Core Design Contradiction:

ProductivityVSQuantity of substance

Solution Approach 1:

The system extracts and stores only the essential base file relationship information (pointers or references indicating which files are clones of which) rather than storing complete data copies. This extraction of minimal relationship data enables efficient replication by allowing the system to determine common regions through relationship traversal, significantly reducing storage space requirements while maintaining high replication efficiency.

Inventive Principle:

Principle #2Taking out (Extraction)

Data Source

PatentUS10372683B1Method to determine a base file relationship between a current generation of files and a last replicated generation of files

Publication Date: 2019.08.06 EMC IP HLDG CO LLC

AI summary

A method for data replication between two file systems is disclosed. A base file relationship between the current generation of files and the last replicated generation may be constructed based on the generation-to-generation base file relationships of the intervening generations. The base file relationship between the current generation of files and the last replicated generation may be utilized in the replication of the current generation of files to the replica file system to reduce the amount of data that needs to be transferred while the files on the replica file system are more than one generation behind the current generation due to network outage or other issues.