System and method for intelligently monitoring large-scale data center cluster computing nodes

Abstract

The invention provides a system and a method for intelligently monitoring large-scale data center cluster computing nodes. Hardware microarchitecture data indexes of the computing nodes and data indexes related to processes of running applications are acquired through monitoring nodes in the system, the data indexes are transmitted to monitoring equipment in the system, big data analysis is executed via the monitoring equipment, and results are sent to customer premise equipment to be displayed to a user. By the system and the method, the microarchitecture data indexes of the computing nodes and the data indexes of the processes of the running applications are acquired, so that intelligent big data analysis is realized, the faulted computing nodes are positioned automatically, and fault causes are provided.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a system and method for intelligently monitoring large-scale data center cluster computing nodes. Background technique [0002] With the continuous progress of human society and the development of science and technology, people not only have a wider understanding of the natural world, but also have an increasingly urgent need to explore the unknown world. This has led to a sharp increase in the amount of information and data held by human beings. At the same time, these massive information and data need to be analyzed and processed in a timely manner. For example, a large astronomical radio telescope array can generate more than 100GB of cosmic microwave data in one second, and these data need to be analyzed in time; Data is also measured in terabytes; in addition, fields such as human genome engineering, oil exploration, weather forecasting, etc. have also put forward higher ...

AI Technical Summary

Problems solved by technology

The personnel in charge of operation and maintenance need to check the running status of the cluster in real time. Once a problem occurs, although the location of the node can sometimes be located, it is often impossible to accurately locate the faulty device. It also takes time and effort to judge and troubleshoot through the experience of the staff; the cluster Although many users can understand their own job status through many job scheduling software, they can rarely count the historical analysis of jobs; moreover, cluster decision makers often cannot directly obtain relevant expenses, usage efficiency, and personnel work efficiency from the cluster. , cost-benefit ratio and other relevant decision-making information can only be made through manual analysis of a large amount of data, which is time-consuming and labor-intensive

In addition, application developers are often unable to obtain information such as hardware microarchitecture, system process, stack, and module error and crash statistics that are urgently needed to optimize application software from the cluster. They need to obtain it through a large number of experiments based on experience, which is time-consuming and laborious.

Images