A hybrid intrusion detection method based on dynamic graph boundary manifold embedding

Abstract

The application relates to the technical field of network intrusion detection, and discloses a hybrid intrusion detection method based on dynamic graph boundary manifold embedding, which comprises the following steps: obtaining network flow data of a hybrid intrusion behavior to be detected; according to the network flow data of the hybrid intrusion behavior to be detected, adopting a dynamic graph boundary manifold embedding method to extract principal component score features of various types of intrusion behaviors to be detected; adopting a support vector data description method to respectively construct detection sub-models of various types of intrusion behaviors, and taking the principal component score features of various types of intrusion behaviors to be detected as inputs to perform intrusion behavior detection and intrusion behavior category identification. The application can overcome the application limitations of a conventional anomaly detection model based on SPE statistics, and realizes synchronous detection of intrusion behaviors and intrusion behavior categories.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network intrusion detection, and particularly relates to a hybrid intrusion detection method based on dynamic graph boundary manifold embedding. BACKGROUND

[0002] In practical applications, real-time monitoring of network transmission is completed by constructing an intrusion detection system (IDS), and if suspicious transmission is found, a security alarm is sent, and a corresponding security module is started according to the category of intrusion to eliminate the threat of intrusion. Intrusion detection is an important part of the network security protection system and is one of the key research contents in the field of network security.

[0003] According to the adopted strategy, intrusion detection can be generally divided into misuse detection and anomaly detection. Misuse detection is a feature-based static detection technology, and its characteristics are that an intrusion feature database needs to be established in advance according to past intrusion experience, and the existing intrusion types are detected according to this database. New intrusion types cannot be detected before the intrusion feature database is updated, so the false negative rate of misuse detection is relatively high. Anomaly detection is a behavior-based dynamic detection technology, and its characteristics are that the main analysis target is normal network behavior, and the abnormal behavior that does not conform to the behavior specification of the current system and network is found out by drawing the boundary of the normal network behavior characteristics, and is monitored and analyzed. According to the detection principle, the false negative rate of anomaly detection is generally low, but not all abnormal behaviors belong to intrusion behaviors, so the false positive rate of anomaly detection is higher than that of misuse detection.

[0004] In recent years, the complexity and diversity of intrusion behaviors have increased day by day. Through the adoption of fuzzy processing and data encryption technologies, various forms of intrusion such as polymorphic viruses, metamorphic viruses and code obfuscation have been produced. It is difficult to cope with the increasingly complex intrusion detection tasks in the future by relying only on feature-based static intrusion detection. The two intrusion detection strategies have their own advantages, and the future intrusion detection system will gradually develop towards the direction of hybrid intrusion strategy.

[0005] The intrusion detection task is to distinguish normal behavior from abnormal behavior (anomaly detection or binary classification task), and to judge the specific intrusion behavior according to the characteristics (multiclassification task), so intrusion detection belongs to a standard classification problem. Machine learning, as the core of artificial intelligence, has been proved to be able to efficiently complete various different classification tasks, which makes how to use machine learning algorithms to improve the effect of intrusion detection become a research hotspot in the field of intrusion detection today. SUMMARY

[0006] In view of the above problems in the prior art, the application provides a hybrid intrusion detection method based on dynamic graph boundary manifold embedding, which integrates anomaly detection function and classification identification function, and overcomes the shortcomings of misuse detection and anomaly detection two single intrusion detection strategies.

[0007] In order to achieve the above-mentioned application purposes, the technical scheme adopted by the application is:

[0008] A hybrid intrusion detection method based on dynamic graph boundary manifold embedding comprises the following steps:

[0009] Obtaining network flow data of a hybrid intrusion behavior to be detected;

[0010] According to the network flow data of the hybrid intrusion behavior to be detected, a dynamic graph boundary manifold embedding method is used to extract principal component score features of each type of intrusion behavior to be detected;

[0011] A support vector data description method is used to construct a detection sub-model of each type of intrusion behavior, and the principal component score features of each type of intrusion behavior to be detected are used as input to perform intrusion behavior detection and intrusion behavior classification identification.

[0012] Further, according to the network flow data of the hybrid intrusion behavior to be detected, a dynamic graph boundary manifold embedding method is used to extract principal component score features of each type of intrusion behavior to be detected, comprising:

[0013] Determining a graph boundary manifold according to the network flow data of the hybrid intrusion behavior to be detected;

[0014] Constructing an intrinsic graph and a penalty graph according to the graph boundary manifold;

[0015] Establishing an objective function of minimizing intra-class reconstruction error and maximizing inter-class reconstruction error according to the intrinsic graph and the penalty graph;

[0016] Solving the objective function to obtain a projection matrix of embedding the feature sample into a low-dimensional space;

[0017] Calculating principal component score features according to the projection matrix and a two-dimensional graph matrix of the graph boundary manifold.

[0018] Further, determining a graph boundary manifold according to the network flow data of the hybrid intrusion behavior to be detected, comprising:

[0019] Standardizing the network flow data of the hybrid intrusion behavior to be detected to construct a feature matrix of original samples;

[0020] Extracting the original samples by using a sliding window to construct two-dimensional graph matrix samples;

[0021] Calculating the Euclidean distance between each two-dimensional graph matrix sample, and determining the graph boundary manifold according to the Euclidean distance.

[0022] Further, the graph boundary manifold is determined according to the Euclidean distance, comprising:

[0023] The set number of different-class samples closest to the current sample point are selected to determine the different-class boundary neighborhood;

[0024] The mean value of each sample point in the different-class boundary neighborhood is calculated, and the set number of same-class samples closest to the mean value point is selected to determine the same-class boundary neighborhood.

[0025] Further, the intrinsic graph and the penalty graph are constructed according to the graph boundary manifold, comprising:

[0026] The intrinsic graph is constructed according to the same-class graph boundary manifold sample point in the graph boundary manifold, and the intra-class weight matrix is constructed according to the same-class boundary neighborhood;

[0027] The penalty graph is constructed according to the different-class graph boundary manifold sample point in the graph boundary manifold, and the inter-class weight matrix is constructed according to the different-class boundary neighborhood.

[0028] Further, the objective function of minimizing the intra-class reconstruction error and maximizing the inter-class reconstruction error is established according to the intrinsic graph and the penalty graph, comprising:

[0029] The intra-class reconstruction error and the inter-class reconstruction error of the two-dimensional graph matrix sample embedded in the low-dimensional space through the projection matrix are calculated respectively;

[0030] The objective function is established with the optimization target of minimizing the intra-class reconstruction error and maximizing the inter-class reconstruction error.

[0031] Further, the projection matrix of the feature sample embedded in the low-dimensional space is obtained by solving the objective function, comprising:

[0032] The first projection matrix and the second projection matrix are initialized by using the unit matrix;

[0033] The generalized eigenvalue of the first projection matrix is calculated, and the first projection matrix is updated by using the eigenvector corresponding to each generalized eigenvalue of the first projection matrix;

[0034] The generalized eigenvalue of the second projection matrix is calculated, and the second projection matrix is updated by using the eigenvector corresponding to each generalized eigenvalue of the second projection matrix;

[0035] The first projection matrix and the second projection matrix are iteratively updated until the optimal first projection matrix and the optimal second projection matrix are obtained when the first projection matrix and the second projection matrix stop updating.

[0036] Further, the principal component score feature is calculated according to the projection matrix and the two-dimensional graph matrix of the graph boundary manifold, comprising:

[0037] The principal component score feature is calculated according to the product of the transpose matrix of the first projection matrix, the two-dimensional graph matrix of the graph boundary manifold and the second projection matrix.

[0038] Further, the support vector data description method is used to construct detection sub-models of various intrusion behaviors, including:

[0039] According to the network flow data of various known intrusion behaviors, samples of each known intrusion behavior state are divided, and an independent intrusion behavior state detection sub-model is established for each known intrusion behavior state;

[0040] The principal component score feature of the sample corresponding to each known intrusion behavior state is taken as an input feature, and the support vector data description method is used to determine the class center and control limit of each intrusion behavior state.

[0041] Further, the principal component score feature of each to-be-detected intrusion behavior is taken as an input for intrusion behavior detection and intrusion behavior category recognition, including:

[0042] Each intrusion behavior state detection sub-model is selected in turn, and the Euclidean distance between the principal component score feature of each to-be-detected intrusion behavior and the class center corresponding to the current intrusion behavior state detection sub-model is calculated;

[0043] It is judged whether the calculated Euclidean distance is greater than the control limit corresponding to the current intrusion behavior state detection sub-model; if yes, the score of the current intrusion behavior state detection sub-model is set to 0, and the next intrusion behavior state detection sub-model is selected; otherwise, the score of the current intrusion behavior state detection sub-model is set to 1, and the intrusion behavior detection result and the intrusion behavior category recognition result are obtained.

[0044] The present application has the following beneficial effects:

[0045] The dynamic graph boundary manifold embedding method is adopted to maximize the class distinction ability, so that the key information representing the characteristics of each intrusion state can be more easily obtained in the intrusion detection and recognition process, and the intrusion behavior detection and classification recognition precision is improved; and the abnormal state detection model established in combination with SVDD can overcome the application limitation of the conventional abnormal detection model based on SPE statistics, and realize the synchronous detection of intrusion behaviors and intrusion behavior categories. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 It is a flowchart of a hybrid intrusion detection method based on dynamic graph boundary manifold embedding;

[0047] Figure 2 It is a schematic diagram for finding a graph boundary manifold;

[0048] Figure 3Fig. 1 is a schematic diagram of an intrusion behavior detection and identification process of DGBME;

[0049] Figure 4 Fig. 4 is a schematic diagram of a detection result of Moddle1;

[0050] Figure 5 Fig. 5 is a schematic diagram of a model score result. DETAILED DESCRIPTION

[0051] The specific embodiments of the present application are described below to facilitate the understanding of the present application for those skilled in the art, but it should be clear that the present application is not limited to the scope of the specific embodiments, and for those skilled in the art, it is obvious that various changes are within the spirit and scope of the present application defined and determined by the appended claims, and all the inventions utilizing the concept of the present application are within the scope of protection.

[0052] The present application proposes a new dynamic graph boundary manifold embedding (DGBME) feature extraction method to obtain low-dimensional features with strong class discrimination ability; secondly, in the low-dimensional feature space, the class centers and control limits of each class of intrusion state are determined by using support vector data description (SVDD), and the detection sub-models of each class of intrusion state are established; finally, the detection results of each detection sub-model are integrated to realize intrusion behavior detection and intrusion behavior class identification.

[0053] As shown in Figure 1 Fig. 1, the mixed intrusion detection method based on dynamic graph boundary manifold embedding provided by the embodiment of the present application comprises the following steps S1 to S3:

[0054] S1, obtaining network flow data of a mixed intrusion behavior to be detected;

[0055] In an optional embodiment of the present application, in a real application scenario, the PCAP format file of the original network flow is obtained by using the PCAP function library capable of capturing network traffic, and the original network flow PCAP file is processed into network flow units of five tuples by using analysis tools such as TCPdump, snort and wireshark, and then the network traffic features are extracted by a tool similar to CICFlowMeter-V3 to generate a CSV network flow feature file, which is a feature set obtained by analyzing and converting the network flow.

[0056] Before processing by machine learning, these features need to be preprocessed into numerical features. The preprocessing process is as follows:

[0057] 1: Symbolic feature digitization.

[0058] The character features are converted in one-hot encoding form: 1) protocol_type is the protocol type, and there are 3 types: TCP, UDP and ICMP, so the number type dimension corresponding to protocol_type after one-hot encoding is 3; 2) service is the network service type of the target host, and there are 70 types, so the number type dimension corresponding to service after one-hot encoding is 70; 3) flag is the state of normal or error connection, and there are 11 types, so the number type dimension corresponding to flag after one-hot encoding is 11.

[0059] 2: Numerical feature normalization.

[0060] The value ranges of various numerical features in the original data set are different, and before feature extraction, the original feature quantity is processed by arctangent transformation to eliminate the difference between orders of magnitude, and then normalized to make the feature value range in the [0, 1] interval.

[0061] 3: The dimension of a single sample data after the conversion of the previous two steps is 123, of which the feature dimension is 122 and the category dimension is 1. In order to eliminate the influence of redundant features on the algorithm performance and operation speed, the principal component analysis method is used to remove a large number of redundant features with an information retention degree greater than 99%, and the final sample feature space is obtained , wherein is the feature space of the training set, is the feature space of the test set.

[0062] S2, according to the network flow data of the mixed intrusion behavior to be detected, a principal component score feature of each type of intrusion behavior to be detected is extracted by using a dynamic graph boundary manifold embedding method;

[0063] In an optional embodiment of the present application, step S2 learns the idea of boundary manifold embedding MME algorithm, that is, according to the label information and distance information of the sample points, the heterogeneous boundary manifold and the homogeneous boundary manifold of each sample are found, and a new dynamic graph boundary manifold embedding (Dynamic graph boundary manifold embedding, DGBME) method is proposed. The method takes a two-dimensional graph matrix as input, can extract features from two-dimensional matrix data, is also suitable for analysis scenarios of multivariate and multi-step input time series data, such as power load prediction, power generation power prediction in the field of power system, and is also suitable for analyzing the aging evolution trend of battery equipment or online anomaly detection with time development. The DGBME method is dedicated to mining features with discriminative significance to improve intra-class cohesion and inter-class discriminability.

[0064] This embodiment extracts principal component score features for various types of intrusion behaviors based on network flow data of the mixed intrusion behaviors to be detected using a dynamic graph boundary manifold embedding method, including the following steps S201 to S205:

[0065] S201. Determine the graph boundary manifold based on the network flow data of the mixed intrusion behavior to be detected;

[0066] In this embodiment, step S201 determines the graph boundary manifold based on the network flow data of the mixed intrusion behavior to be detected, including:

[0067] The network stream data of the mixed intrusion behavior to be detected is standardized to construct the feature matrix of the original samples; specifically, the original feature matrix obtained by preprocessing the network stream data of the mixed intrusion behavior to be detected in step S1 is processed. After standardization, it is represented as:

[0068] ;

[0069] in, It is the mean vector. Let x be the standard deviation. m Let m be the m-th feature, and T be the transpose symbol.

[0070] A sliding window is used to extract data from the original samples, constructing a two-dimensional graph matrix sample. Specifically, the sliding window technique is used, with a window size of l and a step size of 1. Let the i-th sample in the feature matrix X of the original samples be... Influenced by the first l-1 samples, construct samples Two-dimensional graph matrix sample , is represented as:

[0071] .

[0072] Calculate the Euclidean distance between each two-dimensional graph matrix sample, and determine the graph boundary manifold based on the Euclidean distance. Specifically, calculate the two-dimensional graph matrix sample X. i With two-dimensional graph matrix sample X j The Euclidean distance d between them ij , is represented as:

[0073] .

[0074] in, This is the norm symbol.

[0075] like Figure 2 The diagram shown is a schematic of the boundary manifold. First, the distance to the sample point is calculated. The k nearest outlier samples are identified as the outlier boundary neighborhood; then, the mean value mb of the outlier boundary points is calculated, and the k nearest sample points to the mean value mb are found. Samples of the same type are identified as the boundary neighborhood of the same type.

[0076] S202. Construct eigenmaps and penalty maps based on the boundary manifolds of the graph;

[0077] In this embodiment, step S202, which constructs an eigengraph and a penalty graph based on the graph boundary manifold, includes:

[0078] An intrinsic graph is constructed based on sample points of similar graph boundary manifolds within the same boundary manifold, and an intra-class weight matrix is ​​constructed based on the similar boundary neighborhoods. Specifically, constructing the intrinsic graph using sample points of similar graph boundary manifolds aims to better preserve the local neighborhood structure of the boundary manifolds with similar sample points. An intra-class weight matrix is ​​then constructed based on the similar boundary neighborhoods. :

[0079] ;

[0080] in, Represents a two-dimensional graph matrix sample X j Sample X belongs to a two-dimensional graph matrix i The first k nearest neighbors, Represents a two-dimensional graph matrix sample X j With two-dimensional graph matrix sample X i The intra-class weights, min denotes the minimum function, st denotes the constraint condition, and the intra-class weight matrix. The weights of the first k nearest neighbor samples are: The default value is 0.01.

[0081] A penalty graph is constructed based on sample points of dissimilar graph boundary manifolds, and an inter-class weight matrix is ​​constructed based on the neighborhood of the dissimilar boundaries. Specifically, the penalty graph is constructed using sample points of dissimilar graph boundary manifolds to increase the distance between sample points of the same class and those of different classes. An inter-class weight matrix is ​​then constructed based on the neighborhood of the dissimilar boundaries. :

[0082] .

[0083] in, Represents a two-dimensional graph matrix sample X j With two-dimensional graph matrix sample X i Inter-class weights, inter-class weight matrix The weights of the first k nearest neighbor samples are: The default value is 0.01.

[0084] S203, establishing an objective function of minimizing the intra-class reconstruction error and maximizing the inter-class reconstruction error according to the eigenmap and the penalty map;

[0085] In the embodiment, step S203 establishes an objective function of minimizing the intra-class reconstruction error and maximizing the inter-class reconstruction error according to the eigenmap and the penalty map, comprising:

[0086] The intra-class reconstruction error and the inter-class reconstruction error of the two-dimensional graph matrix sample embedded in the low-dimensional space through the projection matrix are calculated respectively; specifically, the intra-class reconstruction error and the inter-class reconstruction error of the two-dimensional graph matrix sample embedded in the low-dimensional space through the projection matrix are calculated respectively. through the projection matrix and embedded in the low-dimensional space.

[0087] The intra-class reconstruction error is:

[0088] ;

[0089] The inter-class reconstruction error is:

[0090] ;

[0091] The objective function is established by taking the minimization of the intra-class reconstruction error and the maximization of the inter-class reconstruction error as the optimization target.

[0092] ;

[0093] Wherein, J is the objective function, the intermediate variable , is a harmonic parameter, used to adjust the decisive role of the intra-class reconstruction error and the inter-class reconstruction error on the final result.

[0094] Solving the objective function is equivalent to simultaneously minimizing the following two formulas:

[0095] ;

[0096] ;

[0097] Wherein: tr(.) represents the trace operation of the matrix; the intermediate variable , , , , .

[0098] S204, solving the objective function to obtain the projection matrix of the characteristic sample embedded in the low-dimensional space;

[0099] In this embodiment, step S204, solving the objective function to obtain the projection matrix of the feature sample embedding in the low-dimensional space, includes:

[0100] The first projection matrix U and the second projection matrix V are initialized using the identity matrix;

[0101] Calculate the generalized eigenvalues ​​of the first projection matrix U, and update the first projection matrix U using the eigenvectors corresponding to each of the generalized eigenvalues ​​of the first projection matrix U; specifically, fix the second projection matrix. V Solve the equation The generalized eigenvalues ​​are obtained, resulting in n1 eigenvalues. The corresponding eigenvector u, let the first projection matrix ,in For the n1th eigenvalue The corresponding feature vectors are used to update intermediate variables. ;

[0102] Calculate the generalized eigenvalues ​​of the second projection matrix V, and update the second projection matrix V using the eigenvectors corresponding to each generalized eigenvalue of the second projection matrix V; specifically, fix the first projection matrix U, and solve the equation Generalized eigenvalues: We obtain l1 eigenvalues The corresponding eigenvector v, let the second projection matrix ,in For the l1th eigenvalue The corresponding feature vectors are used to update intermediate variables. ;

[0103] Iteratively update the first projection matrix U and the second projection matrix V until the first projection matrix U and the second projection matrix V stop updating, and then obtain the optimal first projection matrix and second projection matrix.

[0104] S205. Calculate the principal component score features based on the projection matrix and the two-dimensional graph matrix of the graph boundary manifold.

[0105] In this embodiment, step S205 calculates the principal component score features based on the projection matrix and the two-dimensional graph matrix of the graph boundary manifold, including:

[0106] The principal component score features are calculated based on the product of the transpose of the first projection matrix, the two-dimensional graph matrix of the graph boundary manifold, and the second projection matrix. Specifically, for any new sample... After normalization and graph matrix transformation, it becomes a graph matrix sample. Then, the projection matrices U and V are obtained, and the final principal component scores are calculated. The calculation method is as follows:

[0107] .

[0108] S3, respectively constructing detection sub-models of each type of intrusion behavior by using support vector data description method, and taking principal component score features of each type of to-be-detected intrusion behavior as input to perform intrusion behavior detection and intrusion behavior category identification.

[0109] In an optional embodiment of the present application, step S3 respectively constructs detection sub-models of each type of intrusion behavior by using support vector data description method, including:

[0110] According to network flow data of each type of known intrusion behavior, samples of each type of known intrusion behavior state are divided, and an independent intrusion behavior state detection sub-model is established for each type of known intrusion behavior state;

[0111] Taking principal component score features of samples corresponding to each type of known intrusion behavior state as input features, class centers and control limits of each type of intrusion behavior state are respectively determined by using support vector data description method.

[0112] Step S3 takes principal component score features of each type of to-be-detected intrusion behavior as input to perform intrusion behavior detection and intrusion behavior category identification, including:

[0113] Each intrusion behavior state detection sub-model is selected in turn, and the Euclidean distance between principal component score features of each type of to-be-detected intrusion behavior and the class center corresponding to the current intrusion behavior state detection sub-model is calculated;

[0114] It is judged whether the calculated Euclidean distance is greater than the control limit corresponding to the current intrusion behavior state detection sub-model; if yes, the score of the current intrusion behavior state detection sub-model is set to 0, and the next intrusion behavior state detection sub-model is selected; otherwise, the score of the current intrusion behavior state detection sub-model is set to 1, and intrusion behavior detection results and intrusion behavior category identification results are obtained.

[0115] In this embodiment, dynamic graph boundary manifold embedding (DGBME) is essentially a feature dimension reduction method, which is similar to PCA and KPCA applied to the abnormal detection scene, and needs to set a statistic and calculate a control limit. Common statistics are Hotelling-T2 and squared prediction error (SPE). Since Hotelling-T2 and SPE statistics need to be established on the premise that samples are independent and variables follow Gaussian distribution, they can effectively monitor the process development trend, therefore, the SVDD method is used to determine the statistic and the control limit. The general implementation process is as follows: DGBME is used to determine a low-dimensional feature space, and then SVDD is used to establish detection sub-models of each type of behavior state in the low-dimensional space, that is, each type of behavior state corresponds to a detection model.

[0116] The conventional anomaly detection model only uses normal samples and the limit of normal sample statistics, i.e., the normal original boundary, in the offline modeling stage. In the online detection stage, whether an abnormal state occurs is determined by comparing whether the test sample statistics exceeds the normal value boundary. It can be seen that the conventional anomaly detection model used for intrusion detection is an anomaly-based intrusion detection, and cannot determine the specific intrusion behavior type.

[0117] The mixed intrusion detection method based on DGBME proposed in the embodiment can not only determine whether an intrusion behavior occurs, but also distinguish the specific intrusion behavior type. The method is divided into two stages of offline training and online identification, and the process is as shown in Figure 3 .

[0118] In the data generation stage, a sample data set covering each type of intrusion behavior is constructed , wherein is the principal component score of the training set, is the principal component score of the test set; and the original data is preprocessed to obtain a feature matrix , which provides a data basis for subsequent analysis.

[0119] In the offline training stage, first, the samples of each type of intrusion behavior are divided, and a training set feature space containing c intrusion behavior categories is constructed , wherein represents the feature expression of the ith intrusion sample. The DGBME method is used for feature extraction to obtain the projection matrix U, V and the principal component score , which constitutes a low-dimensional feature space, wherein represents the feature expression of the ith intrusion sample after DGBME feature extraction. The specific steps are as follows: the training samples containing various intrusion behaviors are selected, an intrusion behavior feature matrix is constructed, and c is the number of categories, wherein represents the feature set expression of the ith sample, and x mi represents the mi-th sample feature, and mi is the number of the ith sample; the intrusion behavior feature matrix is normalized, a sliding window size T=20 is set and a two-dimensional graph matrix sample is constructed, then the objective function is solved to obtain the projection matrix U and V, and further to obtain the low-dimensional feature expression of all categories of samples, i.e., the principal component score matrix: , represents the principal component score matrix of the ith sample, and y mi represents the principal component score of the mi-th sample.

[0120] Then, the samples of each type of intrusion behavior state are divided, and a separate intrusion behavior state detection sub-model is established for each intrusion behavior state, wherein each detection sub-model is denoted as {Moddle1, Moddle2,..., Moddle c}, c is the number of categories. The specific steps are as follows: taking the intrusion behavior state F i as an example, the intrusion behavior state F i is divided into c categories, and the i-th category is represented by F i (i=1, 2,..., c). i The principal component scores of the corresponding samples are determined by using the SVDD method as input. i The ball center O pi and the 99% control limit R new of the intrusion behavior state F c are determined.

[0121] In the online identification stage, the principal component score matrix of all new samples or test samples is first calculated according to the projection matrix U and V obtained in the offline modeling stage. , wherein represents the principal component score matrix of the i-th category test sample, and y pi represents the principal component score of the pi-th test sample, and pi is the number of the i-th category test sample, and y is the principal component score of any sample to be detected.

[0122] Then, y new is input into each detection sub-model {Moddle1, Moddle2,..., Moddle c} to realize intrusion behavior detection and identification, and the specific steps are as follows: taking the Euclidean distance D inew between y inew and O i as a statistical value; determining whether D inew > R inew is satisfied, if yes, f inew =0, otherwise f i =1. Wherein f inew is the score of the model Moddle test , f i =1 means that x inew belongs to the intrusion behavior F test , and f i =0 means that x train does not belong to the intrusion behavior F c .

[0123] The method of the present application will be analyzed and described below in combination with a specific intrusion detection example.

[0124] Data generation stage:

[0125] In real application scenarios, the PCAP format file of the original network stream is obtained by using the PCAP function library capable of capturing network traffic, and the original network stream PCAP file is processed into a network stream in units of five tuples by using analysis tools such as TCPdump, snort and wireshark, and then the network traffic features are extracted by a tool similar to CICFlowMeter-V3 to generate a CSV network stream feature file. The network stream feature file is a feature set obtained by analyzing and converting the network stream. In the experiment of the present application, the intrusion detection benchmark data set NSL-KDD is selected. Each record in the network stream feature file corresponding to the data set contains 41 features and a label indicating normal or specific attack behavior. The 41 features are derived from the basic features of TCP / IP protocol connection and the cumulative traffic features in a window interval or a plurality of connections:

[0126] ① Network connection basic features; (a total of 9, serial numbers 1-9);

[0127] ② Network connection content features; (a total of 13, serial numbers 10-22);

[0128] ③ Time-based network traffic features; (a total of 9, serial numbers 23-31);

[0129] ④ Host-based network traffic features; (a total of 10, serial numbers 32-41).

[0130] Among the 41 features, there are 3 character features (protocol_type, service and flag, respectively), 38 numerical features, 4 of which are discrete binary features and 34 of which are continuous features. Before processing by machine learning, these features need to be preprocessed into numerical features.

[0131] The preprocessing process is as follows:

[0132] 1: Symbolic feature digitization.

[0133] Character features are converted using one-hot encoding: ①protocol_type is the protocol type, which has 3 types: TCP, UDP, ICMP, so after one-hot encoding, the numerical type dimension corresponding to protocol_type is 3; ②service is the network service type of the target host, which has 70 types, so after one-hot encoding, the numerical type dimension corresponding to service is 70; ③flag is the state of normal or error connection, which has 11 types, so after one-hot encoding, the numerical type dimension corresponding to flag is 11.

[0134] 2: Numerical feature normalization.

[0135] The 38 numerical features in the original data set have different value ranges. Before feature extraction, the inverse tangent transformation is used to eliminate the difference between the orders of magnitude, and then the normalization processing is performed, so that the feature value range is in the interval [0, 1].

[0136] 3: The single sample data dimension after the conversion of the previous two steps is 123, wherein the feature dimension is 122, and the category dimension is 1. In order to eliminate the influence of redundant features on the algorithm performance and operation speed, the principal component analysis method is used to eliminate a large number of redundant features with an information retention degree greater than 99%, and the final sample feature space is obtained. Thus, the data generation stage is completed.

[0137] The complete NSL-KDD data set is large. Based on the NSL-KDD data set, the present application screens samples to construct an NSL-KDD data subset for experiments, as shown in Table 1. The training set contains 1 normal class and 22 attack behaviors, a total of 23 classes, the test set contains 1 normal class and 20 attack behaviors in the training set and 16 new attack behaviors, a total of 37 classes, and the purpose is to test the generalization ability of the machine learning algorithm to adapt to unknown type sample data. The above attack behaviors are divided into four types of intrusion behaviors, including scanning Probe (6 attack behaviors), denial of service DoS (11 attack behaviors), remote user attack R2L (15 attack behaviors) and privilege escalation attack U2R (7 attack behaviors). The present application takes the four types of intrusion behaviors plus one normal type as the class benchmark (i.e. c = 5), tests the performance of the mixed intrusion detection method based on DGBME proposed by the present application, and the related data division is shown in Table 1. Since the U2R and R2L two types of intrusion classes have fewer samples in the data set, it is difficult to improve the detection accuracy of the U2R and R2L two types of intrusion classes.

[0138] Table 1 NSL-KDD data subset for experiments

[0139]

[0140] Intrusion behavior detection and recognition ability test:

[0141] According to the process shown in Figure 3 , the obtained in the data generation stage is input, the feature is extracted by using the DGBME method proposed by the present application to obtain , and then Y train is used to complete the training of each test sub-model {Moddle1, Moddle2,..., Moddle c}, and Y test is used for performance test.

[0142] 1. Intrusion behavior detection capability test:

[0143] Take the detection sub-model Moddle1 in the normal state F1 as an example, combine the F1 test sample with the intrusion behavior F4 test sample to construct a test set 1 (i.e. the first 100 samples are F1 and the last 30 samples are F4).

[0144] After the data of the test set 1 is subjected to DGBME feature extraction, the test result obtained by inputting Moddle1 is as shown in Figure 4 , wherein the abscissa is the sample number of the test set 1 and the ordinate is the model score of Moddle1. From Figure 4 , it can be seen that Moddle1 determines that all the samples after the serial number 101 are out of limits, that is, the intrusion behavior is detected, which is consistent with the actual situation, but it is impossible to further determine the type of intrusion behavior based on Moddle1 only.

[0145] 2. Intrusion behavior identification capability test:

[0146] According to the flowchart shown in Figure 3 , the test samples of each type of intrusion behavior in Table 1 are sequentially input into each detection model {Moddle1, Moddle2,..., Moddle c}, and the model scores {f1, f2,..., f c} are recorded, the intrusion behavior is identified according to the scores, and the model score results of the test set are as shown in Figure 5 , and the final classification identification effect is as shown in Table 2.

[0147] Table 2. Intrusion state identification result table

[0148]

[0149] From the results of Table 2, it can be seen that the difference between the training set and the test set is large, which is due to the fact that the data set is not balanced (the U2R and R2L two types of samples are less), and the test set adds 16 new attack behavior samples that are not in the training phase; the recall rate of DoS and R2L two types on the training set and the test set is the most obvious, because most of the 16 new attack behaviors added in the test set belong to DoS and R2L two types (see Table 1), and the detection sub-model trained in the offline stage does not learn the characteristics of new samples, so it is difficult to identify new samples, so most of the new attack behaviors in the test set are judged as new type or new concept samples; the recall rate and the precision rate of the normal class are very high, which shows that the normal behavior boundary learned by the detection sub-model established by the application is relatively accurate, and can effectively distinguish between normal and abnormal behaviors, that is, the false alarm rate and the false alarm rate of the abnormal behavior detection are relatively low; the precision rate of the test set R2L is relatively high and the recall rate is low, which is due to two factors, one is that the sample number is small during offline training, which leads to the model established in the offline modeling stage being more likely to be divided into other categories, and the other is that a large number of unknown new types are added in the test set R2L, and the false alarm rate of these new types is high, that is, the recall rate of the test set is reduced; the overall accuracy and precision of the recognition method of the application are relatively high, which shows that each detection sub-model established by the application can effectively represent the feature boundary of the normal state and each intrusion behavior, so that the false alarm rate of the abnormal behavior is low, and the false alarm rate of the intrusion behavior recognition is also low.

[0150] 3. Comparison verification:

[0151] The mixed intrusion detection method based on DGBME of the application includes intrusion behavior detection and recognition, and the accuracy of the intrusion behavior recognition result depends on the effectiveness of each detection sub-model. Therefore, only the detection model performance is compared.

[0152] The intrusion behavior detection method proposed in the application is denoted as DGBME+SVDD, which is compared with the conventional intrusion behavior detection methods PCA, KPCA and SVDD, wherein PCA and KPCA use SPE, T 2 as statistical quantities, and SVDD uses the hyper-sphere radius as the statistical quantity. The missed detection rate (MDR), the false alarm rate (FAR) and the detection time (DT) are used as evaluation indexes. Missed detection means that the intrusion behavior sample is judged as normal, and the missed detection rate is the ratio of the number of missed detections to the number of intrusion behavior samples; false detection means that the normal is judged as intrusion behavior, and the false detection rate is the ratio of the number of false detections to the number of normal samples.

[0153] The training data and test data of each detection model are the same. The test data in this stage is test set 2, which is constructed by combining different numbers of normal state F1 test samples and test samples corresponding to each intrusion behavior F2, F3, F4 and F5. The first 100 time points are actually normal states, and after 101-430 time points, each type of intrusion behavior state appears. Therefore, the closer the intrusion behavior detection time DT of the model to 100, and the smaller the overall missed detection rate (MDR) and the overall false alarm rate (FAR), the stronger the detection capability.

[0154] The intrusion behavior detection results of the four models are shown in Table 3.

[0155] Table 3 Intrusion behavior detection result table

[0156]

[0157] As can be seen from Table 3, the intrusion behavior detection time corresponding to the DGBME+SVDD detection model proposed by the application is consistent with the actual situation, and the missed detection rate and the false alarm rate are the lowest, indicating that the method proposed by the application has better intrusion behavior detection performance.

[0158] The application proposes a hybrid intrusion detection method based on DGBME. Through example verification and comparative analysis, the following conclusions are obtained:

[0159] ① The DGBME feature extraction method takes maximizing class distinction ability as the goal, and is more likely to obtain key information representing the characteristics of each intrusion state in the intrusion detection and recognition process, which is conducive to improving the intrusion behavior detection and classification recognition accuracy.

[0160] ② The detection model established by combining DGBME and SVDD method has certain advantages compared with the commonly used PCA, KPCA and SVDD detection models.

[0161] ③ The hybrid intrusion behavior recognition strategy proposed by the application can achieve good recognition accuracy, and can detect new concept samples, providing basis and guidance for updating the model in the later period, overcoming the disadvantage that the traditional classification model will forcibly divide it into existing types.

[0162] The application is described with reference to the flowcharts and / or block diagrams of the method, device (system) and computer program product according to the embodiments of the application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device realize the functions in the flowcharts and / or block diagrams. Figure 1one or more processes and / or blocks Figure 1 an apparatus for performing the functions specified in the flowchart

[0163] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flowchart Figure 1 one or more processes and / or blocks Figure 1 an apparatus for performing the functions specified in the flowchart

[0164] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart Figure 1 one or more processes and / or blocks Figure 1 an apparatus for performing the functions specified in the flowchart

[0165] The principles and implementations of the present application are described in the embodiments, the above description of the embodiments is only for helping to understand the method of the present application and its core idea; meanwhile, for the ordinary skilled in the art, according to the idea of the present application, the specific implementation and application range will have changes, and the above description of the present application should not be understood as the limitation of the present application.

[0166] The person skilled in the art will understand that the embodiments described herein are for helping the reader to understand the principles of the present application, and should be understood as the protection scope of the present application not being limited to such specific statements and embodiments. The person skilled in the art can make various other specific modifications and combinations according to the technical inspirations disclosed in the present application without departing from the essence of the present application, and these modifications and combinations still fall within the protection scope of the present application.

Claims

1. A hybrid intrusion detection method based on dynamic graph boundary manifold embedding, characterized in that, The method comprises the following steps: obtaining network flow data of mixed intrusion behaviors to be detected; extracting principal component score features of each type of intrusion behavior to be detected by using a dynamic graph boundary manifold embedding method according to the network flow data of the mixed intrusion behaviors to be detected, comprising: determining a graph boundary manifold according to the network flow data of the mixed intrusion behaviors to be detected; comprising: performing standardization processing on the network flow data of the mixed intrusion behaviors to be detected, and constructing a feature matrix of original samples; extracting the original samples by using a sliding window, and constructing two-dimensional graph matrix samples; calculating the Euclidean distances between each two-dimensional graph matrix sample, and determining the graph boundary manifold according to the Euclidean distances; comprising: selecting a set number of different-class samples closest to the current sample point as different-class boundary neighbors; calculating the mean value of each sample point in the different-class boundary neighbors, and selecting a set number of same-class samples closest to the mean value point as same-class boundary neighbors; constructing an intrinsic graph and a penalty graph according to the graph boundary manifold; comprising: constructing the intrinsic graph according to the same-class graph boundary manifold sample points in the graph boundary manifold, and constructing an intra-class weight matrix according to the same-class boundary neighbors; constructing the penalty graph according to the different-class graph boundary manifold sample points in the graph boundary manifold, and constructing an inter-class weight matrix according to the different-class boundary neighbors; establishing an objective function of minimizing intra-class reconstruction error and maximizing inter-class reconstruction error according to the intrinsic graph and the penalty graph; solving the objective function to obtain a projection matrix of embedding the feature samples into a low-dimensional space; calculating principal component score features according to the projection matrix and the two-dimensional graph matrix of the graph boundary manifold; constructing detection sub-models of each type of intrusion behavior by using a support vector data description method, comprising: dividing samples of each known intrusion behavior state according to network flow data of each known intrusion behavior, and respectively establishing an independent intrusion behavior state detection sub-model for each known intrusion behavior state; taking the principal component score features of the samples corresponding to each known intrusion behavior state as input features, and respectively determining a class center and a control limit of each intrusion behavior state by using a support vector data description method; and taking the principal component score features of each type of intrusion behavior to be detected as input to perform intrusion behavior detection and intrusion behavior category recognition, comprising: selecting each intrusion behavior state detection sub-model in turn, and calculating the Euclidean distances between the principal component score features of each type of intrusion behavior to be detected and the class center corresponding to the current intrusion behavior state detection sub-model; judging whether the calculated Euclidean distance is greater than the control limit corresponding to the current intrusion behavior state detection sub-model; if yes, setting the score of the current intrusion behavior state detection sub-model to 0, and continuing to select the next intrusion behavior state detection sub-model; otherwise, setting the score of the current intrusion behavior state detection sub-model to 1, and obtaining an intrusion behavior detection result and an intrusion behavior category recognition result.

2. The hybrid intrusion detection method based on dynamic graph boundary manifold embedding according to claim 1, wherein, establishing an objective function of minimizing intra-class reconstruction error and maximizing inter-class reconstruction error according to the intrinsic graph and the penalty graph, comprising: respectively calculating the intra-class reconstruction error and the inter-class reconstruction error of embedding the two-dimensional graph matrix samples into a low-dimensional space through the projection matrix; establishing an objective function with the optimization target of minimizing the intra-class reconstruction error and maximizing the inter-class reconstruction error.

3. The hybrid intrusion detection method based on dynamic graph boundary manifold embedding according to claim 2, wherein, Solving the objective function, obtaining the projection matrix of the characteristic sample embedding low-dimensional space, comprising: Adopting unit matrix to initialize the first projection matrix and the second projection matrix; Calculating the generalized eigenvalue of the first projection matrix, updating the first projection matrix with the eigenvector corresponding to each generalized eigenvalue of the first projection matrix; Calculating the generalized eigenvalue of the second projection matrix, updating the second projection matrix with the eigenvector corresponding to each generalized eigenvalue of the second projection matrix; Iteratively updating the first projection matrix and the second projection matrix until the optimal first projection matrix and the second projection matrix are obtained when the first projection matrix and the second projection matrix stop updating.

4. The hybrid intrusion detection method based on dynamic graph boundary manifold embedding according to claim 3, wherein, According to the projection matrix and the two-dimensional graph matrix of the graph boundary manifold, calculating the principal component score feature, comprising: According to the product of the transpose matrix of the first projection matrix, the two-dimensional graph matrix of the graph boundary manifold and the second projection matrix, calculating the principal component score feature.

Images