Abnormity detection method and system based on self-adversarial variational auto-encoder

Abstract

The invention belongs to the field of anomaly detection, and particularly relates to an anomaly detection method and system based on a self-adversarial variational auto-encoder, and the method comprises the steps: constructing an anomaly detection model based on the self-adversarial variational auto-encoder, training the anomaly detection model, and obtaining a standard anomaly score set of training data, automatically calculating a threshold value according to a kernel density estimation method and the standard anomaly score set, obtaining to-be-detected data from the online database, preprocessing the to-be-detected data, inputting the preprocessed to-be-detected data into the trained anomaly detection model to obtain a detection score, comparing the detection score with the threshold value, and outputting a detection result; according to the method, the advantages of an unsupervised anomaly detection method based on a variational auto-encoder and adversarial training are combined, the limitation of each technology is made up, the anomaly detection accuracy is improved, and the problem that the false alarm rate or the missing report rate is high due to a fixed threshold value method is effectively relieved by designing an automatic threshold value selection module.

Description

technical field [0001] The invention belongs to the field of abnormality detection, and particularly relates to an abnormality detection method and system based on a self-confrontation variational autoencoder. Background technique [0002] Anomaly detection, by definition, is a technology that identifies abnormal situations and mines non-logical data. In the era of big data, the speed of manual data processing is far behind that of machines, so detecting anomalies in data faster has become a a very important task at the moment. At present, there are some problems in the known anomaly detection algorithms. First, some anomaly detection algorithms have high requirements on the labels of data sets. The core area and the edge area of ​​the original center point clustering are processed separately to realize the dynamic update of the contour and improve the timeliness of the anomaly detection method based on CFSFDP. However, the clustering method has relatively high requirement...

AI Technical Summary

Problems solved by technology

[0003] To sum up, the difficulty of anomaly detection lies in three aspects. First, real-world data sets lack labels. Algorithms with high label requirements have the problems of high labeling costs and complex calculations for data sets. Often unsupervised learning The method is the most worthy of consideration, and in order to solve the problem of high false positive rate, it is necessary to automatically set the threshold; secondly, the existing deep generative model does not consider the time dependence of data, the correlation between multivariate attributes, and the logarithm Gaussian distribution of input data; the third aspect, ensuring the high accuracy of anomaly detection while considering time loss

Images