A pseudo-label-guided multi-view consensus graph semi-supervised network intrusion detection method

Abstract

The present invention discloses a multi-perspective consensus graph semi-supervised network intrusion detection system guided by pseudo-labels, including determining high-confidence pseudo-labels for unlabeled intrusion data, and using pseudo-labels to guide the learning process of the consensus graph. First, the K-nearest neighbor method is used to assign multiple pseudo-labels to the unlabeled intrusion data at each perspective of the intrusion data, and then the pseudo-label information corresponding to the unlabeled intrusion data with consistent pseudo-labels in all perspectives is selected to introduce into the learning optimization process of the multi-perspective consensus graph. The optimal consensus graph finally learned is used to propagate label information to obtain the final network intrusion detection result. The present invention makes up for the defect that the existing semi-supervised multi-perspective network intrusion detection system cannot fully utilize the supervisory information of limited label intrusion data, and can generate additional supervisory information to ultimately obtain better network intrusion detection performance.

Description

Technical Field

[0001] The invention relates to a pseudo-label-guided multi-view consensus graph semi-supervised network intrusion detection method, belonging to the field of data classification. Background Art

[0002] With the advent of the information age and the popularization of the Internet, cyber attacks pose an increasingly serious threat to personal, corporate and national information security. How to accurately identify whether the current access is an abnormal access and determine the attack category to which the abnormal access belongs in order to quickly deploy corresponding defense measures is the main research content of network intrusion detection. However, due to the existence of massive network access data, labeling massive network data requires a lot of manpower and material resources, so it is not feasible in actual operation. Semi-supervised learning has become one of the main means of network intrusion detection in recent years because it can classify a large number of unsupervised samples using only the supervised information of a small number of labeled samples.

[0003] Different types of feature extractors can collect multiple types of data features for the same network intrusion sample to fully describe the sample target, where each type of feature can be regarded as a perspective of the sample. Proper use of such heterogeneous multi-perspective data can greatly improve the accuracy of intrusion detection. How to reasonably use the features of multiple perspectives in a limited label sample scenario is the key to designing a semi-supervised multi-perspective network intrusion detection system.

[0004] The design of semi-supervised multi-view network intrusion system usually follows at least one of the following three principles: the first principle is that two network intrusion samples that are close in feature space usually belong to the same attack type; the second principle is that samples in the same cluster structure have the same label; the third principle is that network intrusion samples are distributed in a manifold space with a dimension much lower than the original feature space, and this manifold space represents the relationship between samples through a graph. Due to the good performance shown by the graph-based semi-supervised classification method, the present invention is committed to learning a high-quality multi-view consensus graph to perform network intrusion detection tasks. Summary of the invention

[0005] Technical problem: In view of the problem that the existing network intrusion detection system cannot fully utilize the supervisory information of existing labeled samples, the present invention provides a pseudo-label-guided multi-view consensus graph semi-supervised network intrusion detection method. The invention uses the supervisory information of limited-label network intrusion samples to generate additional supervisory information and introduces it into the learning process of the multi-view consensus graph, thereby obtaining a multi-view consensus graph that can better estimate the sample manifold structure to perform network intrusion detection tasks.

[0006] Technical solution: First, a small part of the collected network intrusion samples is randomly selected and manually labeled as the labeled sample set, and the remaining network intrusion samples are the unlabeled sample set; secondly, in each perspective of the sample, based on the labeled samples and using the nearest neighbor method, the pseudo labels of the corresponding perspective are assigned to the unlabeled samples; then, the unlabeled samples with consistent pseudo labels in all perspectives are selected as high-confidence samples, and the corresponding consistent pseudo label information is used as additional supervision information; then, the self-representative subspace learning method is used to learn the perspective-specific graph in each perspective, and the maximum alignment method is used to dynamically learn the multi-perspective consensus graph with the minimum difference from all learned perspective-specific graphs during the optimization process by combining the supervision information of the original labeled samples with the additional supervision information; at the same time, the learned multi-perspective consensus graph can propagate the supervision information of the labeled samples to the unlabeled samples, and obtain the indicator label matrix; finally, the above steps are jointly integrated in a unified learning framework, and the coordinate descent method is used to obtain the optimal solution of each module, and finally the classification result of the unlabeled samples is obtained.

[0007] The technical solution adopted by the present invention to solve its technical problem can be further refined. The pseudo labels of each view of the unlabeled sample are determined by the label of the labeled sample with the closest Euclidean distance in the original feature space. In practice, other classic single-view classification models can be trained based on the labeled samples to obtain higher-quality pseudo labels, thereby further improving the quality of the final high-confidence pseudo labels.

[0008] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0009] Without introducing additional high computational costs, the supervisory information of limited existing labeled network intrusion samples is used, and high-confidence pseudo labels are screened according to the consistency principle of multi-view learning as additional supervisory information and introduced into the learning process of the multi-view consensus graph. Different from the existing methods that only use the supervisory information of limited labeled samples, the multi-view consensus graph learned by combining the existing label information and the additional label information can better estimate the manifold structure of the sample, thereby achieving more accurate network intrusion detection.

[0010] Different from the traditional method which usually adopts Gaussian kernel function to construct a graph representing sample relationship, the present invention adopts self-representative subspace learning method to learn the view-specific similarity graph of each view, thus avoiding the manual cost caused by adjusting the Gaussian kernel width parameter.

[0011] The present invention adopts the maximum alignment method, combines the supervision information of the original labeled samples with the additional supervision information, and dynamically learns the multi-view consensus graph with the minimum difference from all learned perspective-specific graphs during the optimization process, making full use of the supervision information of the samples. Each module finds the optimal solution in a mutually coupled and mutually enhanced method, greatly improving the accuracy of the network intrusion detection system. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a flow chart of the semi-supervised network intrusion detection system based on multi-view consensus graph guided by pseudo labels of the present invention.

[0013] Figure 2 It is a flow chart of high-confidence pseudo-label generation of the present invention. DETAILED DESCRIPTION

[0014] In order to more clearly describe the content of the present invention, the following is further explained in conjunction with examples and drawings of the specification. The examples mentioned below are not intended to limit the scope of the present invention. The pseudo-label-guided multi-view consensus graph semi-supervised network intrusion detection system of the present invention includes the following steps:

[0015] Step 1: Input network intrusion detection dataset Where n is the total number of samples, V is the total number of viewing angles, and d v represents the feature dimension of the v-th perspective, Represent the labeled sample set and the unlabeled sample set respectively, and input the one-hot matrix Y of the labeled sample labels.

[0016] Step 2: First, obtain the pseudo labels of the unlabeled network intrusion samples. The specific steps are as follows:

[0017] Step 2.1: Use the label network to intrude the sample and assign an initial pseudo label to all unlabeled samples using the nearest neighbor method in all view angles:

[0018] Step 2.1.1: Use the following formula to calculate the Euclidean distance between labeled samples and unlabeled samples:

[0019]

[0020] is the sample x i and x j The distance, d represents the sample dimension, and the superscript v represents the vth viewing angle;

[0021] Step 2.1.2: Based on the distance calculated in the previous step, select the nearest labeled sample to assign a pseudo label to the unlabeled sample.

[0022] Step 2.2: For all unlabeled samples, compare whether the pseudo labels assigned in each perspective are consistent, and select the unlabeled samples with consistent pseudo labels as high confidence samples.

[0023] Step 2.3: Output the pseudo labels of high-confidence samples as additional supervision information.

[0024] Step 3: For each view of the sample, a self-representation learning method is used to dynamically learn the view-specific graph. The specific operations are:

[0025]

[0026] Where: X v and S v denote the sample corresponding to the vth view and the learned view-specific map, respectively. V denotes the total number of views, and ||·|| F represents the F-norm of the corresponding matrix, λ represents a regularization parameter greater than 0, diag(·) represents the diagonal elements of the corresponding matrix, and 1 represents a vector of all 1s.

[0027] Step 4: Use the maximum alignment method and combine the label information of the existing labeled samples and additional supervision information to dynamically learn a multi-view consensus map with the minimum difference from all view-specific maps. The specific operations are:

[0028]

[0029] sA T 1=1,A≥0

[0030] Among them, A represents the learned multi-view consensus graph, F i is the i-th row of the indicator label matrix, A ij is the i-th row and j-th column element of the consensus graph, and Y′ is the additional supervisory information matrix determined based on the existing labeled samples. (3) The first term in the formula is used to constrain the consensus graph A and the view-specific graph S of each view. v There is a minimum difference. The second constraint makes the distance between the labels corresponding to two samples with greater similarity closer. The third and fourth constraints are used to constrain the final propagated labels to be closer to the real labels and pseudo labels.

[0031] Step 5: Unify the above mentioned modules into a joint optimization framework:

[0032]

[0033] Among them, λ, α, β, μ, and γ are regularization parameters of the corresponding modules. The framework is optimized using the coordinate descent method. The specific steps are:

[0034] Step 5.1: Optimize parameter Sv , and treat the other parameters as constants:

[0035] Step 5.1.1: Ignore S v The constraint condition for formula (4) is v Taking the partial derivative and setting it to 0, we can get S v The solution without considering the constraints is:

[0036]

[0037] Step 5.1.2: Consider S v Constraints, solve the following equation:

[0038]

[0039] By constructing the Lagrangian function and combining the KKT condition, we can get S v The optimal solution is:

[0040]

[0041] in And S i ' i =0, It should be noted that, in order to ensure the quality of the graph, we only calculate the similarity values ​​of the 10 nearest neighbor samples of the i-th sample.

[0042] Step 5.2: Optimize parameter A and treat other parameters as constants: Solve the following equation:

[0043]

[0044] sA T 1=1,A≥0

[0045] in By constructing the Lagrangian function and combining the KKT condition, the optimal solution of A can be obtained:

[0046]

[0047] in And A i ' i =0,

[0048] Step 5.3: Take the partial derivative of formula (4) with respect to F and set it to 0, and you can get the closed solution of F at the current iteration step:

[0049]

[0050] Step 5.4: Repeat steps 5.1-5.3 until the value of formula (4) converges, and the optimization is completed.

[0051] Step 6: Output the classification results of unlabeled samples according to the following formula:

[0052]

[0053] The above describes the specific implementation of the present invention according to the accompanying drawings. However, those skilled in the art will understand that, without departing from the spirit and principle of the present invention, several improvements and equivalent substitutions may be made. The technologies and solutions after improvements and equivalent substitutions in the claims of the present invention all fall within the protection scope of the present invention.

[0054] Experimental design

[0055] Experimental data set selection: The present invention uses the conventional network security data set NSL-KDD to evaluate the effectiveness of the system in completing the network intrusion detection task. The detailed information of the data set is shown in the following table:

[0056] Dataset Number of viewing angles Number of categories Number of samples Dimensions NSL-KDD 3 5 2526 464 / 1144 / 104

[0057] The present invention adopts an adaptive method to determine the regularization parameters λ and α, that is, for the i-th sample, the corresponding regularization parameter value is the sum of the similarity values ​​of the 11th nearest neighbor minus the similarity values ​​of the first 10 nearest neighbors, and the final regularization parameter value is the average of the regularization parameter values ​​of all samples; for the regularization parameters β and γ, a grid search method is used to select the regularization parameters in [10 -3 ; 10 -2 ; 10 -1 ] and [10 0 ; 10 1 ; 10 2 ; 10 3 ] to determine the optimal regularization parameter; μ is set to 1.

[0058] Comparison Models: The proposed system is named PMvC. We compare the performance among LP, AMMSS, SMGI, AMGL, MLAN and MVCSD.

[0059] Performance measurement method: This experiment uses accuracy ACC as an indicator to measure model performance. For a given true label Y i and the predicted label F i , ACC is defined as follows:

[0060]

[0061] Among them, δ(F i ,Y i ) Only in F i=Y i 1 if the value is 0, otherwise it is 0.

[0062] Experimental strategy: For the present invention and other comparison models, we respectively use their optimal parameters, run the system on data sets containing 10%, 15%, 20%, 25% and 30% labeled samples, and report the average and standard deviation of the results of 10 runs. In order to avoid randomness, the labeled samples are randomly selected in all 10 experiments.

[0063] The experimental results are shown in the following table. The data in the table correspond to the mean and standard deviation of the ACC indicator obtained from 10 independent experimental results. Each row corresponds to an algorithm, and each column corresponds to a data set. The best results on each data set are marked in bold.

[0064]

[0065] It can be seen from the results in the table that the PMvC proposed in the present invention can achieve stable and optimal performance on network security datasets containing labeled samples in different proportions, and excellently completes the network intrusion detection task.

Claims

1. A pseudo-label-guided multi-view consensus graph semi-supervised network intrusion detection method, the specific steps of which include: 1) Data processing: First, randomly select a small portion of the collected network intrusion data and manually label it. The labeled samples and the remaining unlabeled samples will participate in the subsequent model training together; 2) Data processing step 2: In each view of the sample, calculate the distance between each unlabeled sample and all labeled samples, select the nearest labeled sample and assign a pseudo label of the corresponding view to the unlabeled sample based on the label of the labeled sample; 3) The first step of training: select unlabeled samples with consistent pseudo labels in all views as high-confidence samples, and use the corresponding consistent pseudo label information as additional supervision information; 4) Training step 2: The self-representation subspace learning method is used to learn the view-specific graph in each view: in: X v and S v denote the sample corresponding to the vth view and the learned view-specific map, respectively. V denotes the total number of views, and ||·|| F represents the F-norm of the corresponding matrix, λ represents a regularization parameter greater than 0, diag(·) represents the diagonal elements of the corresponding matrix, and 1 represents a vector of all 1s; 5) The third step of training: adopt the maximum alignment method, and combine the supervision information of the original label samples with the additional supervision information obtained in the first step of training to dynamically learn the multi-view consensus map with the minimum difference from all learned view-specific maps during the optimization process: in: A represents the learned multi-view consensus graph, F i is the i-th row of the indicator label matrix, A ij is the i-th row and j-th column element of the consensus graph, Y is the one-hot supervision information matrix corresponding to the existing labeled samples, and Y′ is the additional supervision information matrix determined based on the existing labeled samples; 6) Training step 4: The learned multi-view consensus graph propagates the supervision information of the labeled samples to the unlabeled samples and obtains the indicator label matrix; 7) Network Intrusion Detection Process: The above steps are integrated into a unified network intrusion detection system, which is specifically represented as follows: in: λ, α, β, μ, and γ are regularization parameters of the corresponding modules respectively. The coordinate descent method is used to iteratively obtain the optimal solution of each module. Finally, the category with the largest probability value of the unlabeled sample corresponding to the category in the indicator label matrix is ​​the category of the final unlabeled network intrusion data, specifically:

Images