A distributed hadoop cluster fault automatic diagnosis and repair system

Abstract

The invention discloses a distributed hadoop cluster fault automatic diagnosis and restoration system which comprises a cluster file system monitoring module for collecting and obtaining cluster node information and a database file; a work and task monitoring module for collecting information of work and tasks; a physical node monitoring module for monitoring resource consumption information of each physical node; a data storage and analysis and processing module for storing monitoring data to a database, setting monitoring alarm rules and configuring alarm ID, level and reasons in advance; and an automatic restoration module for defining and configuring various common alarm faults in advance and making a preprocessing script for each alarm fault, matching the fault happened at present with the alarm faults defined and configured in advance when monitoring a fault, and calling the corresponding preprocessing script to finish automatic restoration of the fault. The method can diagnose and restore system fault automatically to allow maintenance to become easier, and performance data and node state to be clearer and more obvious.

Description

technical field [0001] The invention relates to a cluster fault automatic diagnosis and repair system, in particular to a distributed hadoop cluster fault automatic diagnosis and repair system. Background technique [0002] There is no solution in the industry to automatically analyze and solve problems found in hadoop cluster monitoring. Currently, the solution to hadoop cluster failures is to pre-configure key operation and maintenance monitoring indicators, check the health of hadoop clusters and related projects, and perform job and task execution at the same time. Analysis, the monitoring information is exposed, and the maintenance personnel log in to the web platform to check the nodes with problems and their performance, log in to the nodes to analyze the logs, and repair the cluster. [0003] The main disadvantages of the existing technology are as follows: 1. Real-time alarm of faults cannot be realized, and each configured monitoring indicator requires maintenance ...

AI Technical Summary

Problems solved by technology

[0003] The main disadvantages of the existing technology are as follows: 1. Real-time fault alarm cannot be realized, and each configured monitoring indicator requires maintenance personnel to log in to the web platform to check the problematic nodes; 2. Cluster monitoring can only view the current monitoring data without storing and historical query function, it is impossible to analyze the monitoring data and establish a predictive model; 3. It is impossible to automatically analyze the monitoring data and perform automatic repairs based on the analysis results. It is necessary to manually log in to the problem node to analyze the error log and solve the fault based on experience

Images