Transparent In-Memory Checkpointing via Dirty Page Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing checkpoint data backup process in server clusters is time-consuming and requires large storage due to the need to back up extensive amounts of data, necessitating a method to reduce the amount of data to be backed up while maintaining redundancy and transparency to application programs.
Innovation Solution
A method and apparatus using a File System or device driver to track modified data, ensuring only modified checkpoint data is backed up by monitoring page parameters or setting pages as read-only to identify changes, thereby reducing the data volume and enhancing backup efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all checkpoint data is replicated to backup servers periodically, then redundancy and high availability are maintained, but backup time and storage requirements increase significantly
Solution Approach 1:
The patent extracts only the modified portions of checkpoint data (dirty pages) for backup replication, rather than copying all checkpoint data. The file system identifies which pages have been modified since the last backup and replicates only those specific pages to backup servers, significantly reducing backup time and network bandwidth consumption while maintaining adequate redundancy for failover scenarios.
Solution Approach 2:
The patent segments the checkpoint data into individual pages and tracks modification status for each page separately. By dividing the large checkpoint data structure into manageable page-sized units and identifying only the modified segments, the system avoids the overhead of backing up entire datasets, thereby reducing backup time and storage requirements while preserving the ability to restore complete checkpoint state when needed.
2Reliability
If all checkpoint data is replicated to backup servers periodically, then redundancy is maintained, but storage requirements increase significantly
Solution Approach 1:
The patent extracts and replicates only the modified portions of checkpoint data (dirty pages) to backup servers, rather than copying all checkpoint data. This selective replication approach significantly reduces the quantity of data stored on backup servers while maintaining sufficient redundancy for failover scenarios, as only the changed portions need to be preserved for recovery.
Solution Approach 2:
The patent implicitly discards unchanged checkpoint data from backup operations by identifying and replicating only the modified pages. The file system tracks which pages have changed and sends only those to backup servers, effectively discarding the redundancy of repeatedly backing up unchanged data while preserving the ability to recover complete checkpoint state when failover is required.
3Productivity
If the file system monitors page parameters to identify modified pages, then only modified data is backed up reducing backup time, but the complexity of the file system increases
Solution Approach 1:
The patent implements self-service by leveraging the file system's existing page fault handling mechanisms to automatically identify modified pages. When a page is accessed or modified, the file system's built-in page fault handlers set appropriate bits in page table entries, which the backup software then monitors. This approach enables selective backup without adding complex external monitoring infrastructure, as the file system's own operational mechanisms provide the necessary modification detection.
Data Source
AI summary
A method and apparatus for performing transparent checkpoint data backup by identifying portions of memory that store the checkpoint data that have been modified since a prior in-memory checkpoint process and backing up only the identified portions of memory containing the modified checkpoint data.