Log clustering method based on graph structure

Abstract

The invention provides a log clustering method based on a graph structure. The method comprises the following steps of clustering logs based on text segmentation, vector similarity and a maximum connected sub-graph in order to obtain a feature library; and carrying out class labelling on the massive logs according to the class features in the feature library. The method can automatically recognize the most appropriate class number in the massive logs without manually assigning the clustering number; in addition, the method can classify the logs precisely to lay a foundation for mining of massive log data.

Description

technical field [0001] The invention relates to the field of text clustering, in particular to a log clustering method based on a graph structure. Background technique [0002] With the rapid development of information technology and the continuous expansion of cluster scale, massive log data is generated, but there is no effective analysis and mining of log data. Log data records the operating information of the system, and mining log data is of great significance. For example, by analyzing log data, we can build an intelligent operation and maintenance system to complete functions such as fault location and fault warning. Accurate category labeling of logs is an important direction of log data mining. Based on this, we automatically identify the appropriate number of categories for logs by clustering massive logs. By extracting the features of each category, a log category feature library is generated, and new logs are marked according to the category of the feature libr...

AI Technical Summary

Problems solved by technology

Traditional clustering algorithms cannot meet the needs of massive log clustering

For example, the traditional K-Means and K-Medoid clustering algorithms require specifying the number of clusters and cannot automatically identify the appropriate number of categories for logs

In order to obtain a better clustering effect, the traditional Denclue clustering algorithm needs continuous experiments to obtain the appropriate number of clusters. The parameters are difficult to control, the amount of calculation is too large, and the clustering cannot guarantee the real number of categories.

Images