Data exception detection method, device, equipment and storage medium

Abstract

The invention discloses a data exception detection method, a device, equipment and a storage medium. The method comprises the steps of obtaining unmarked data; extracting primary abnormal data from the unmarked data according to a preset query strategy; identifying and marking the primary abnormal data, storing the primary abnormal data into a marked first data set to form a second data set, and training a pre-trained hypersphere classification model through the second data set; identifying whether the hyper-sphere classification model meets a training termination condition or not; and when the training termination condition is reached, inputting the unmarked data into the hypersphere classification model under the training termination condition for classification screening to obtain target abnormal data. According to the method, a small amount of marked data is used for training the classification model, the classification model is used for classifying the unmarked data after the training termination condition is met, original distribution of the data is not limited, the data volume needing to be marked by an operator is reduced, and the classification result accuracy is high.

Description

technical field [0001] The present application relates to the technical field of data processing in artificial intelligence, and in particular to a data anomaly detection method, device, equipment and storage medium. Background technique [0002] The monitoring of computer systems is an important part of AIOps. In the process of monitoring computer systems, the CPU and disk of the computer system will generate a large amount of index data, which will also contain some abnormal values. Through the branching of abnormal points, the cause of system abnormality can be found out, and suggestions for subsequent operations can be provided. Therefore, anomaly detection technology plays an important role in the field of intelligent operation. [0003] Traditional anomaly detection includes statistical and density-based methods. [0004] Statistics-based methods often find out suspected outliers by training a large amount of labeled data, which belongs to supervised learning. Accor...

AI Technical Summary

Problems solved by technology

[0005] 1. Most of the massive data generated during the running of the program is unlabeled, and data labeling often requires professionals, so it takes a lot of manpower, material and financial resources to obtain enough data labels

[0006] 2. The proportion of abnormal data is small, and it is more cumbersome to find potential abnormal points and their corresponding classification from a large amount of data

[0007] The density-based method belongs to unsupervised learning, which can be completed without data labeling, but the detection accuracy is usually not high, and there is no theoretical support for the classification results

Images