Application Checkpoint Scheduling from Resource Anomaly Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The conventional frequency-based checkpoint/restart mechanism in high-performance computing (HPC) systems is resource-intensive and time-consuming, leading to significant data transfer and storage costs and performance impacts, which outweigh the benefits of data recovery in the event of failures.
Innovation Solution
Implementing a checkpoint mechanism based on failure prediction, triggering checkpoints only when a failure is anticipated, rather than at regular intervals, using predictive algorithms to minimize the frequency of checkpoints and reduce performance impact.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If frequency-based checkpoint mechanism is implemented, then data recovery capability is improved, but execution performance deteriorates due to significant data transfer and storage overhead
Solution Approach 1:
The checkpoint mechanism transitions from a static frequency-based approach to a dynamic event-driven approach. Checkpoints are triggered dynamically based on detected failure events or anomaly predictions rather than executing at fixed intervals. This allows the system to adapt checkpoint frequency to actual system conditions, performing checkpoints only when necessary to maintain reliability while minimizing performance overhead during normal operation.
Solution Approach 2:
The system changes the triggering parameter for checkpoints from time-based (frequency) to event-based (failure detection or anomaly prediction). By monitoring system state parameters and triggering checkpoints only when specific failure conditions or predicted anomalies are detected, the system maintains data recovery capability while dramatically reducing the number of unnecessary checkpoint operations that degrade execution performance.
2Reliability
If checkpoint is executed at regular intervals, then failure recovery is enabled, but energy consumption increases due to unnecessary data transfers
Solution Approach 1:
The checkpoint mechanism transitions from a static frequency-based approach to a dynamic event-driven approach. Checkpoints are triggered dynamically based on detected failure events or anomaly predictions rather than executing at fixed intervals. This allows the system to adapt checkpoint frequency to actual system conditions, performing checkpoints only when necessary to maintain reliability while minimizing performance overhead during normal operation.
Solution Approach 2:
The system uses failure detection mechanisms and anomaly prediction as triggers to convert potentially harmful failures into beneficial early checkpoint actions. By detecting signs of impending failure or actual failures, the system triggers checkpoints that prevent data loss without requiring continuous checkpointing. This transforms the monitoring infrastructure into a benefit that reduces unnecessary energy-consuming checkpoint operations while maintaining reliability.
3Reliability
If checkpoint frequency is increased, then data loss prevention is improved, but bandwidth consumption increases due to repeated data transfers
Solution Approach 1:
The checkpoint mechanism transitions from a static frequency-based approach to a dynamic event-driven approach. Checkpoints are triggered dynamically based on detected failure events or anomaly predictions rather than executing at fixed intervals. This allows the system to adapt checkpoint frequency to actual system conditions, performing checkpoints only when necessary to maintain reliability while minimizing performance overhead during normal operation.
Solution Approach 2:
The system performs preliminary failure detection and anomaly prediction to identify when checkpoints will be necessary before actual failures occur. By detecting early signs of resource anomalies or system failures, the system can trigger checkpoints in advance to save critical data states, preventing data loss without requiring continuous or high-frequency checkpointing that would consume excessive bandwidth.
4Reliability
If checkpoint mechanism is implemented, then application restart capability is improved, but time consumption increases due to data read, transfer and write operations
Solution Approach 1:
The checkpoint mechanism transitions from a static frequency-based approach to a dynamic event-driven approach. Checkpoints are triggered dynamically based on detected failure events or anomaly predictions rather than executing at fixed intervals. This allows the system to adapt checkpoint frequency to actual system conditions, performing checkpoints only when necessary to maintain reliability while minimizing performance overhead during normal operation.
Solution Approach 2:
The system implements feedback through failure detection mechanisms and anomaly prediction that monitor system state and provide information about impending or actual failures. This feedback loop allows the system to intelligently trigger checkpoints only when failure risk is detected, ensuring that checkpoint operations occur at optimal moments to enable application restart while minimizing unnecessary time consumption from routine checkpointing that would not lead to actual recovery events.
Data Source
AI summary
The invention relates to a computer-implemented method for programming a checkpoint of at least one application executed on a computing infrastructure. The computing infrastructure includes a plurality of resources allocated to the execution of the application. The method includes determining a date of failure of the application as a function of a prediction of an anomaly of at least one resource of the plurality of resources, and programming the checkpoint as a function of the date of failure.