Proactive failure recovery model for distributed computing

Abstract

This disclosure generally describes methods and systems, including computer-implemented methods, computer-program products, and computer systems, for providing a proactive failure recovery model for distributed computing. One computer-implemented method includes building a virtual tree-like computing structure of a plurality of computing nodes, for each computing node of the virtual tree-like computing structure, performing, by a hardware processor, a node failure prediction model to calculate a mean time between failure (MTBF) associated with the computing node, determining whether to perform a checkpoint of the computing node based on a comparison between the calculated MTBF and a maximum and minimum threshold, migrating a process from the computing node to a different computing node acting as a recovery node, and resuming execution of the process on the different computing node.

Description

[0001] priority claim [0002] This application claims priority to US Patent Application No. 14 / 445,369, filed July 29, 2014, the entire contents of which are incorporated herein by reference. Background technique [0003] Critical / real-time scientific applications (e.g. seismic data processing, 3D reservoir Uncertainty modeling and simulation) require high-end computing power, which can take days or weeks to process data to generate the desired solution. The success of longer job executions depends on the reliability of the system. Since most scientific applications deployed on supercomputers can fail if only one of the processes fails, fault tolerance in distributed systems is an important feature in complex computing environments. Tolerating computer processing failures of any type reactively generally involves the choice of whether to allow periodic checkpointing of the state of one or more processes - an effective technique that can be widely applied in high performance...

AI Technical Summary

Problems solved by technology

However, this technique has overhead issues associated with choosing the optimal checkpoint interval and stable storage location for the checkpoint data

Furthermore, current failure recovery models are usually limited to a few types of computational failures and are invoked manually in case of computational failure, which limits their usefulness and efficiency

Images