Checkpoint System Using Sparse Address Space Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional checkpointing methods face issues such as fragmentation of checkpoint files and unbounded disk space requirements, especially in high-performance computing environments with 64-bit addressing schemes, where every modified memory page is recorded across intervals without removal.
Innovation Solution
The method involves organizing data from a sparsely populated address space into a dense structure using a constant access container and fragmentation tracking, allowing for efficient checkpointing by marking available openings, extending the structure as needed, and managing page additions and deletions to minimize fragmentation and disk usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If incremental checkpoints are stored as a series of patches recording every modified page, then runtime complexity remains low and fragmentation is avoided, but disk space requirements become unbounded
Solution Approach 1:
The patent extracts only the essential information needed for checkpointing - specifically tracking which pages are currently in use and their mappings - rather than storing redundant patch data for every modification. This selective extraction of critical data reduces storage requirements while maintaining checkpointing functionality.
Solution Approach 2:
The patent discards redundant checkpoint data that has already been superseded by newer checkpoints, and recovers storage space by overwriting or removing old patch files. This allows the system to maintain bounded disk space usage while preserving the ability to restore to any checkpoint state.
2Reliability
If all modified pages are recorded in every checkpoint interval, then complete restoration is ensured, but checkpoint file size grows without bound
Solution Approach 1:
The patent merges multiple checkpoint intervals into a single bounded data structure that tracks page usage across all intervals. By combining the information from multiple checkpoints into one coherent mapping structure, the system ensures complete restoration capability while maintaining constant file size regardless of the number of intervals.
Solution Approach 2:
The patent implements a dynamic data structure that adapts to page allocation and deallocation patterns. The checkpoint system dynamically tracks which pages are in use and updates mappings accordingly, allowing the checkpoint file size to remain proportional to actual memory usage rather than growing with the number of checkpoints.
3Ease of operation
If a sparse address space is maintained to track all possible pages, then page tracking is simplified, but memory and processing overhead increase significantly
Solution Approach 1:
The patent segments the large address space into manageable chunks or regions, tracking only the pages that are actually in use within each segment. This segmentation approach maintains the simplicity of tracking while reducing the overall data structure overhead by focusing computational resources on active pages rather than the entire address space.
Data Source
AI summary
Approaches to efficiently creating a checkpoint of the process are described. In one approach, a method of performing a checkpoint operation on a process involves detecting a change in the contents of a memory page associated with the process. This change occurred after a preceding checkpoint operation. The method also involves modifying a data structure, at a location corresponding to the contents of that memory page.