A method and system for automatic labeling of fault indicators based on logs

Abstract

The invention relates to a log-based fault index automatic tagging system and method. The method comprises the steps of screening out failure / fault logs according to event levels of system logs, allocating a failure / fault type to each failure / fault log according to information contents of each failure / fault log, and determining an effective time window of performance index data according to each failure / fault type; performing modeling on all the performance index data corresponding to the effective time window of each failure / fault type, and building a fault index model; and performing automatic tagging on the performance index data according to the fault index model so as to judge whether the performance index data is a fault index or not. According to the system and the method, the time and energy of manually tagging the fault index can be shortened and reduced; the time and manpower resources are saved; the workload is reduced; a manager can quickly check system faults and perform fault diagnosis; and whether the system is in a fault / failure state or not in a time period can be estimated according to a feature of the index, so that corresponding measures can be adopted in time.

Description

technical field [0001] The invention relates to the field of distributed environment reliability, in particular to a method and system for automatic labeling of fault indicators based on logs. Background technique [0002] Troubleshooting is a technique used to discover abnormalities in a system or system component. As the scale of software systems continues to grow, the logic becomes more complex, and fault diagnosis becomes more and more difficult. On the one hand, detailed monitoring capabilities may not be available in large-scale systems; on the other hand, due to the existence of some fault-tolerant mechanisms, faults sometimes do not appear intuitively. Fault diagnosis technology can mainly be used to discover the deficiencies of the system. [0003] At present, fault diagnosis technology is constantly incorporating new computing technology and mathematical methods, including artificial intelligence, machine learning, stochastic process, Bayesian inference, graph th...

AI Technical Summary

Problems solved by technology

[0008] In the existing technology, in the process of using system index data for fault detection and automatic diagnosis, when detecting various index data generated by different applications during operation, it is difficult to define which indexes correspond to system failure or fault, and usually need to pass Manually view system behavior and index timing diagrams for targeted analysis, and even though users can pre-set thresholds for abnormal or fault indicators based on their own experience, different indicators have different units, such as cpu utilization and memory The utilization rate is a percentage value, the disk access bandwidth and network transmission bandwidth are MB / s, etc., so when faced with a large number of system indicators, how to choose the threshold of failure indicators is also a difficult problem for users.

Images