Hybrid neural network fault prediction method and system for high-performance computer

Abstract

The invention discloses a hybrid neural network fault prediction method and system for a high-performance computer, and the method comprises the steps: collecting the log data of the high-performance computer, wherein the log data comprises a log event id, a timestamp when a corresponding log event occurs, and a log event level; performing data cleaning and feature selection on the collected log data to obtain initial feature data; constructing a fault prediction model by using a random forest algorithm, inputting the obtained initial feature data into the fault prediction model, calculating feature importance by using the random forest algorithm, and performing feature selection to obtain feature sample data; and inputting the obtained feature sample data into an LSTM network model, and predicting whether a fault event exists in the feature sample by using the LSTM network model. According to the method, the log data features are scored and selected through the random forest, and the dimensionality is reduced, so that the training complexity can be reduced, and the training degree is accelerated.

Description

technical field [0001] The invention belongs to the technical field of reliability and availability of storage systems, and in particular relates to a hybrid neural network fault prediction method and system for high-performance computers. Background technique [0002] High Performance Computing (High Performance Computing, HPC) is a computing method that uses parallel processing to run applications. This computing method is efficient, fast and reliable. High-performance computing systems allow the execution of computationally intensive applications on large numbers of high-end processors interconnected by rapid networking. At the same time, high-performance computing systems are now widely used in large fields, such as climate simulation, molecular dynamics, fluid dynamics, medical imaging, and so on. For this reason, various countries are vigorously developing and researching high-performance computing. [0003] With the vigorous development of high-performance computers...

AI Technical Summary

Problems solved by technology

[0005] The fault prediction method generally predicts the state of a high-performance computer for a period of time by analyzing the current state of the high-performance computer and the state of the past period of time, especially whether a fault will occur in the next period of time, so that certain processing methods are allowed to reduce the loss caused by the fault. The fault prediction method is feasible, but it also brings two major challenges to our fault prediction. One is how to design a fault prediction model with a relatively high accuracy rate. minimize the loss

Images