State Consistent Replication via Journal-Based Snapshotting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current backup systems are time-consuming, labor-intensive, and inadequate for 24/7 networks, especially when dealing with frequently changing data, and they often lose file structure during storage, making data retrieval difficult and costly.
Innovation Solution
The system employs state consistent replication by generating snapshots, using a distributed metadata store, and performing asynchronous replication correctness validation to efficiently identify and transfer changed data, minimizing network bandwidth and enabling rapid recovery with snapshots and version history.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If incremental backups are used to reduce data transfer, then network bandwidth usage is reduced, but data consistency across multiple backup systems becomes difficult to maintain
Solution Approach 1:
The patent introduces a journal file as an intermediary mechanism that records all data changes in a centralized location. Backup systems read from this journal to determine what data to transfer, ensuring all systems have a consistent view of changes without needing to exchange complex metadata directly with each other.
Solution Approach 2:
The system implements feedback mechanisms where backup systems continuously monitor the journal file for new entries and automatically initiate transfers when changes are detected. This feedback loop ensures data consistency is maintained across distributed backup systems without manual intervention.
2Productivity
If multiple parallel I/O operations are performed to increase throughput, then backup speed is improved, but coordination complexity and potential for inconsistent states increases
Solution Approach 1:
The patent performs preliminary actions by taking snapshots of data before backup operations begin. This establishes a known good state that all parallel I/O operations can reference, allowing multiple threads to read and write simultaneously without complex coordination, as they all operate from the same baseline snapshot.
Solution Approach 2:
The backup process is segmented into distinct phases: snapshot creation, journal reading, parallel data transfer, and validation. Each phase can be executed by multiple threads independently, reducing coordination complexity while maintaining high throughput during the parallel transfer phase.
3Quantity of substance
If data is stored sequentially on tape media, then storage capacity is maximized, but file structure is lost and retrieval time increases
Solution Approach 1:
The patent creates a copy of the data structure in the form of a manifest file that is stored alongside the sequential tape data. This manifest contains metadata about file structures, locations, and relationships, allowing rapid retrieval of specific files without having to sequentially search through entire tape contents.
Solution Approach 2:
The manifest file acts as an intermediary index between the sequential tape storage and the user's retrieval requests. Instead of directly searching sequential tape data, the system queries the manifest first to locate desired files, then retrieves only those specific files from the tape, dramatically reducing retrieval time.
4Reliability
If complete data transfers are performed to ensure data integrity, then backup reliability is improved, but time consumption and network bandwidth usage increase significantly
Solution Approach 1:
The patent extracts only the changed portions of data by reading the journal file to identify what has modified since the last backup. Instead of transferring complete datasets, the system extracts and transfers only the necessary increments, maintaining data integrity through journal-based change tracking while dramatically reducing transfer time and bandwidth usage.
Data Source
AI summary
Systems and methods are disclosed for state consistent replication of client data set on a client computer by generating a snapshot of the client data set on a local volume; synchronizing with a remote server volume corresponding to the local volume to create a copy of the client data set on the remote server; performing a master to slave replication of the data set; and taking a snapshot of the server data set to create a mirror of the snapshot of the client data set on the server.