Adaptive Checkpoint Interval for High-Performance Computing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In high-performance computing systems with multiple processing nodes, checkpoint overhead increases due to latency in slower bandwidth paths, reducing overall processing throughput and complicating error recovery.
Innovation Solution
A method is introduced to dynamically determine a checkpoint interval based on monitoring block size, checkpoint bandwidth, and failure rate, allowing for efficient creation and restoration of checkpoints to manage task execution and error recovery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If checkpoints are established periodically to support error recovery, then reliability is improved, but checkpoint overhead increases and processing throughput decreases
Solution Approach 1:
The patent applies dynamics by making the checkpoint interval adaptive rather than fixed. The system dynamically adjusts the checkpoint interval based on monitored error rates and task characteristics, allowing checkpoints to be created more frequently when error rates are high and less frequently when error rates are low, thus optimizing the balance between reliability and processing throughput
Solution Approach 2:
The patent changes the parameter of checkpoint interval from a static value to a dynamic value that varies based on system conditions. By monitoring error rates and adjusting the checkpoint interval parameter accordingly, the system optimizes both error recovery capability and processing throughput under different operating conditions
2Reliability
If checkpoint frequency is increased to improve error recovery, then reliability is improved, but checkpoint overhead and latency increase
Solution Approach 1:
The system dynamically adjusts checkpoint frequency based on monitored error rates rather than using a fixed high frequency. This allows the system to achieve adequate error recovery capability while minimizing checkpoint latency by creating checkpoints only when necessary based on actual system conditions
3Reliability
If checkpoint size is increased to capture more state data, then reliability is improved, but resource utilization and checkpoint overhead increase
Solution Approach 1:
The patent extracts only the necessary state data required for error recovery rather than capturing all possible state information. By identifying and storing only the critical state variables needed to resume task execution, the system reduces checkpoint size and memory resource utilization while maintaining adequate error recovery capability
Solution Approach 2:
The system applies partial action by capturing a subset of state data that is sufficient for error recovery rather than capturing complete system state. This partial checkpointing approach reduces resource overhead while providing adequate reliability for task resumption
Data Source
AI summary
According to an aspect, a method for triggering creation of a checkpoint in a computer system includes executing a task in a processing node of the computer system. A monitoring block size is determined for the checkpoint. A checkpoint interval is determined based on the monitoring block size, a checkpoint bandwidth, and a failure rate of the computer system. Based on determining that the checkpoint interval has elapsed, the checkpoint including state data of the task is created to enable restarting execution of the task upon a restart operation. The state data of the checkpoint is restored from a memory responsive to detecting an error condition at the processing node. Execution of the task is restarted in the processing node based on the state data restored from the memory.