Active Failure Recovery Model for Distributed Computing

Abstract

The present disclosure generally describes methods and systems, including computer-implemented methods, computer program products, and computer systems, for providing an active failure recovery model for distributed computing. A computer-implemented method includes: constructing a virtual tree-like computing structure of a plurality of computing nodes; for each computing node of the virtual tree-like computing structure, executing a node failure prediction model by a hardware processor to calculate a correlation with the computing node connected mean time between failures (MTBF); determine whether to perform a checkpoint of the compute node based on a comparison between the computed MTBF and maximum and minimum thresholds; migrate the process from the compute node to a different one as a recovery node Compute node; and continue the process execution on a different compute node.

Description

[0001] priority claim [0002] This application claims priority to US Patent Application No. 14 / 445,369, filed July 29, 2014, the entire contents of which are incorporated herein by reference. Background technique [0003] Execute critical / real-time scientific applications (e.g. seismic data processing, 3D reservoir Uncertainty modeling and simulation) require high-end computing power, which can take days or weeks to process data to generate the desired solution. The success of longer job executions depends on the reliability of the system. Since most scientific applications deployed on supercomputers can fail if only one of the processes fails, fault tolerance in distributed systems is an important feature in complex computing environments. Tolerating computer processing failures of any type reactively generally involves the choice of whether to allow periodic checkpointing of the state of one or more processes - an effective technique that can be widely applied in high per...

AI Technical Summary

Problems solved by technology

However, this technique has overhead issues associated with choosing the optimal checkpoint interval and stable storage location for the checkpoint data

Furthermore, current failure recovery models are usually limited to a few types of computational failures and are invoked manually in case of computational failure, which limits their usefulness and efficiency

Images