Intrusion detection method based on observational learning

Abstract

The invention discloses an intrusion detection method based on observational learning. According to the intrusion detection method, the observational learning and semi-supervised ensemble learning are combined, unlabeled data is labeled to form a labeled data set with an ensemble result by individual classifier by observing output results to an unlabeled data set by other classifiers in network data detection, and the labeled data set is added to the original labeled data set to train the classifier again. According to the intrusion detection method, the unlabeled data is fully used and is translated to the labeled data to further improve the performance of the classifiers, and the intrusion detection method also has good detection effect in the case of a small amount of the labeled data. According to the intrusion detection method, each classifier generates virtual data per se without help of any external forces in the learning process, the virtual data and initial data set are trained, re-observed and re-trained together, and the identification performance of a detection system is effectively improved. The intrusion detection method can be expanded for processing a large amount of unlabeled training data, such as the application fields of medical images, web page classification, remote sensing image processing, face identification, and the like.

Description

technical field [0001] The invention belongs to the technical field of pattern recognition and machine learning, and relates to an intrusion detection method based on observation learning, in particular to an intrusion detection method based on observation learning, which can be used to further improve the recognition rate of a detection system when there is only a small amount of marked data . The proposed intrusion detection method can be extended to applications with less labeled data and more unlabeled data, such as medical image processing, webpage classification, remote sensing image processing, face recognition, etc. Background technique [0002] In the real world, there are many problems that require a large amount of labeled data to train high-precision classifiers, but it is very difficult to obtain labeled data, and even consumes a lot of manpower and material resources. However, with the rapid development of data collection and storage technology, the acquisitio...

AI Technical Summary

Problems solved by technology

However, just like general machine learning problems, semi-supervised learning also suffers from the problem of "selective superiority", that is, each method shows better performance in some domains, but not all domains.

Thus, a dilemma arises: Which method should be used for a certain application task? If you want to select the algorithm with the best performance, the process is tedious and tedious; if you want the selection process to be simple, the performance of the selected algorithm may not be the best

Based on this idea, people have proposed many intrusion detection methods, but most of the existing methods have defects of varying degrees, such as high false positive rate, high false negative rate, etc., and most of them have not effectively utilized a large number of unlabeled data. The same problem exists in pure observation learning.

Images