Graph theory correlation theory-based anomaly detection method

Abstract

The invention discloses a graph theory correlation theory-based anomaly detection method, which specifically comprises the following steps: 1, performing clustering operation on an original data set,and dividing the data set into different clusters; 2, calculating the mean value density of the original data set, and comparing the mean value density of the original data set as a threshold value with the cluster density to simplify the data set; 3, carrying out feature extraction and spatial distance calculation on the data set, and carrying out datamation operation on a result; 4, distributingall data points of the effectively detected data cluster according to the calculated weight values to construct an undirected connected graph; and 5, searching the shortest path of the correspondingcluster by adopting a Floyd algorithm. In terms of preprocessing of the data set, a method of simplifying the data set for the second time is adopted, dimension reduction operation is carried out on the data set according to different reference information, a large number of useless data sets can be effectively reduced, and time complexity and space complexity in the anomaly detection process arereduced to a great extent.

Description

Technical field [0001] The invention relates to an abnormality detection method, in particular to an abnormality detection method based on graph theory related theories, and belongs to the technical field of abnormal detection method application. Background technique [0002] At present, the closest existing technology: Among the commonly used outlier detection methods, there are many classic methods, which are cut from different angles for anomaly detection. A method for anomaly detection using random forest is to randomly select from training data. Ψ point sample points are put into the root node of the tree as a subsample, and then a dimension is randomly specified, a cutting point p is randomly generated in the current node data, and the cutting point is generated between the maximum and minimum values ​​of the specified dimension in the current node data Between, this cutting point generates a hyperplane, and then divides the current node data space into 2 subspaces: put the...

AI Technical Summary

Problems solved by technology

[0016] (1) The existing technology does not perform effective data preprocessing on the original data set

[0017] (2) The existing anomaly detection method is still relatively simple in the basis of anomaly judgment, which is relatively easy to cause misjudgment

[0018] (3) The existing anomaly detection methods do not label the feature information when performing anomaly detection

[0020] When faced with a data set with a high dimensionality and a large amount of total data, if we adopt a method of complete traversal for anomaly detection and do not perform effective data preprocessing on the original data set, it will greatly cause abnormalities in the normal data points The detection process is redundant, which will not only increase the time complexity of the anomaly detection method but also reduce the accuracy of the anomaly detection method

The traditional dimension reduction method is only based on a single information feature, such as PCA, and uses the size of the feature value to perform the dimension reduction process. During this process, there is no operation of abnormal judgment to compare data-related information, which can easily cause the loss of abnormal points.

Images