Log Monitoring Method Based on Behavioral Anomaly Detection

Abstract

The invention provides a log monitoring method based on abnormal behavior detection. The log monitoring method includes the steps of log preprocessing and log anomaly detection. According to the log preprocessing step, log structures are unified, and logs are processed in a clustered mode; according to the anomaly detection step, log flow is converted into a behavior sequence according to the log clustering result, a behavior mode is generated, and anomaly indexes of the real-time log flow are obtained; anomaly indexes and an anomaly threshold value are compared, and whether to give out anomaly early warning or not is determined according to the comparison result. According to the method, starting from the log type characteristics, a generating rule of the logs of different types is analyzed, the problem that the data size is too large is solved in a log information clustering mode, log content is effectively analyzed in real time, the data can be automatically processed conveniently, and the method is in high universality and anomaly detection success rate.

Description

technical field [0001] The invention relates to the technical field of software monitoring, in particular to a log monitoring method based on abnormal behavior detection. Background technique [0002] Today's software systems are getting bigger and bigger, and the structure is getting more and more complex, and the occurrence of exceptions and errors becomes unavoidable. At present, there are generally two ways to avoid software exceptions: one is to prevent software exceptions during the development process, including good programming habits, structured and scientific software development processes, and a large number of essential unit tests, end-to-end Testing, etc.; the second is to use the monitoring system to monitor the software status in real time during the system operation, to detect abnormal behaviors that have occurred or are about to occur in time, and leave valuable time for taking corresponding measures. [0003] Log monitoring is a series of processes of coll...

AI Technical Summary

Problems solved by technology

Considering the rate of log generation, the time required to process each log, and hardware overhead, it is difficult for existing analysis technologies to achieve real-time and effective analysis of log content

[0008] 2) The log structure is not fixed

[0009] Logs do not follow a specific format, the payload section is still manually written by the programmer, and can create inconsistencies that hinder automated processing

[0010] 3) Logging systems vary widely

[0011] Facing different businesses in different industries, the accuracy of abnormal detection in the general monitoring system is not high

Images