Method and device for detecting and classifying intrusion traffic, electronic device, and storage medium

Abstract

The present disclosure discloses a method and apparatus for detecting and classifying intrusion traffic, an electronic device, and a storage medium, relating to the technical field of data processing. The main technical solutions include: First, preprocess the intrusion traffic set, and divide the traffic in the intrusion traffic set into a training set, a test set, and an open set. Secondly, train an intrusion traffic classification model based on the training set, the test set, incompressible noise, and latent vectors, replace the activation function of the intrusion traffic classification model, and adjust the parameter values of the replaced activation function. Finally, perform a performance test on the intrusion traffic classification model based on the open set; after training the intrusion traffic classification model through the intrusion traffic set to enable the intrusion traffic classification model to have the function of identifying known types of intrusion traffic, replace the activation function in the intrusion traffic classification model to enable the intrusion traffic classification model to have the ability to detect unknown types of intrusion traffic.

Description

Technical Field

[0001] The present disclosure relates to the technical field of data processing, and in particular, to a method and device for detecting and classifying intrusion traffic, an electronic device, and a storage medium. Background Art

[0002] Intrusion detection refers to a security monitoring and guarantee carried out while a computer and a data network are operating normally and openly. The ways of network intrusion can be divided into the following several types: one is that non-administrator users inside the system access unauthorized files or data; the second is that external attackers of the system illegally access or damage system data; the third is the chaotic management of user permissions inside the system. The goal of intrusion detection is to perform real-time detection and identification for the above intrusion methods. At present, compared with traditional network defense technologies such as firewalls, a Network Intrusion Detection System (NIDS) can better detect and identify abnormal network traffic, thereby preventing the network from being possibly invaded to ensure its confidentiality, integrity, and availability.

[0003] The NIDS system adopting misuse detection technology performs intrusion detection through the characteristics of known types of intrusion traffic, and realizes the detection and classification of attack traffic through feature comparison, and the false alarm rate of this method is low.

[0004] Although the above NIDS system adopting misuse detection technology can achieve intrusion detection, this detection scheme cannot discover unknown types of attack traffic (i.e., zero-day attack traffic), so the database needs to be updated manually frequently. Summary of the Invention

[0005] The present disclosure provides a method and device for detecting and classifying intrusion traffic, an electronic device, and a storage medium. Its main purpose is to classify known types of intrusion traffic and detect unknown types of intrusion traffic under open set conditions.

[0006] According to the first aspect of the present disclosure, a method for detecting and classifying intrusion traffic is provided, which includes:

[0007] Preprocess the intrusion traffic set, and divide the traffic in the intrusion traffic set into a training set, a test set, and an open set. The intrusion traffic types in the training set and the test set are the same, and the open set contains intrusion traffic types different from those in the training set and the test set;

[0008] Train an intrusion traffic classification model based on the training set, the test set, incompressible noise, and latent vectors. The intrusion traffic classification model includes a generator, a discriminator, and a classifier;

[0009] Replace the activation function of the intrusion traffic classification model and adjust the parameter values of the replaced activation function;

[0010] Perform performance testing on the intrusion traffic classification model based on the open set.

[0011] Optionally, after performing performance testing on the intrusion traffic classification model based on the open set, the method further includes:

[0012] Record the unknown intrusion traffic categories and data in the open set based on the intrusion traffic classification model.

[0013] Optionally, the preprocessing of the intrusion traffic set includes:

[0014] Replace or delete the values in the intrusion traffic that cannot be read by the intrusion detection algorithm;

[0015] Delete the all-zero features in the intrusion traffic;

[0016] Delete the irrelevant network flow features in the intrusion traffic;

[0017] Normalize and supplement the dimensions of the intrusion traffic according to the input size of the intrusion traffic classification model.

[0018] Optionally, the training of the intrusion traffic classification model based on the training set, test set, incompressible noise, and latent vector further includes:

[0019] Input the training set data into the discriminator, and input the incompressible noise and latent vector into the generator;

[0020] The generator generates pseudo-data based on the incompressible noise and latent vector, and inputs the pseudo-data into the discriminator and classifier.

[0021] Optionally, the training of the intrusion traffic classification model based on the training set, test set, incompressible noise, and latent vector further includes:

[0022] Fix the network parameters of the generator and train the discriminator and classifier;

[0023] According to the training results, calculate the loss functions of the discriminator and the classifier respectively;

[0024] Adjust the network parameters of the discriminator and the classifier based on the loss functions.

[0025] Optionally, the training of the intrusion traffic classification model based on the training set, test set, incompressible noise, and latent vector further includes:

[0026] After the number of training times is greater than or equal to a preset round threshold, fix the network parameters of the discriminator and the classifier, and train the generator;

[0027] Calculate the loss function of the generator according to the training results, and adjust the network parameters of the generator based on the loss function.

[0028] Optionally, the training of the intrusion traffic classification model based on the training set, the test set, the incompressible noise, and the latent vector further includes:

[0029] Judge whether the generator, the discriminator, and the classifier meet the convergence conditions respectively according to the loss functions of the generator, the discriminator, and the classifier;

[0030] When the generator, the discriminator, and the classifier simultaneously meet the convergence conditions, complete the training of the intrusion traffic classification model.

[0031] Optionally, the replacement of the activation function of the intrusion traffic classification model and the adjustment of the parameter values of the replaced activation function include:

[0032] Replace the activation function of the intrusion traffic classification model from the SoftMax layer with the OpenMax layer;

[0033] Extract the processing results of the Dense layer before the SoftMax layer as the activation vectors of each traffic sample;

[0034] Determine the tail size and the number of traffic types for adjusting the activation vectors, and calculate the parameters of the Weibull distribution;

[0035] Adjust the activation vector values of the known intrusion traffic categories of the intrusion traffic classification model, and calculate the activation vector values and estimated probabilities of the unknown intrusion traffic categories.

[0036] Optionally, the method is used for misuse detection technology, including:

[0037] Use the intrusion traffic classification model based on the SoftMax layer and the intrusion traffic classification model based on the OpenMax layer to detect and classify intrusion traffic.

[0038] Optionally, the method is used for anomaly detection technology, and the preprocessing of the intrusion traffic set further includes:

[0039] Classify the normal traffic based on the behavior patterns of the normal traffic and label it, and the label is used to distinguish different behavior patterns of the normal traffic.

[0040] According to the second aspect of the present disclosure, there is provided a detection and classification device for intrusion traffic, including:

[0041] A processing unit, configured to preprocess the intrusion traffic set and divide the traffic in the intrusion traffic set into a training set, a test set, and an open set, where the intrusion traffic types in the training set and the test set are the same, and the open set contains intrusion traffic types different from those in the training set and the test set;

[0042] A training unit, configured to train an intrusion traffic classification model based on the training set, the test set, the incompressible noise, and the latent vector, where the intrusion traffic classification model includes a generator, a discriminator, and a classifier;

[0043] A replacement unit, configured to replace the activation function of the intrusion traffic classification model and adjust the parameter values of the replaced activation function;

[0044] A testing unit, configured to perform a performance test on the intrusion traffic classification model based on the open set.

[0045] Optionally, the apparatus further includes:

[0046] A recording unit, configured to record the unknown intrusion traffic categories and data in the open set based on the intrusion traffic classification model after the testing unit performs a performance test on the intrusion traffic classification model based on the open set.

[0047] Optionally, the processing unit includes:

[0048] A first deletion module, configured to replace or delete the values in the intrusion traffic that cannot be read by the intrusion detection algorithm;

[0049] A second deletion module, configured to delete the all-zero features in the intrusion traffic;

[0050] A third deletion module, configured to delete the irrelevant network flow features in the intrusion traffic;

[0051] A processing module, configured to perform normalization processing and dimension expansion on the intrusion traffic according to the input size of the intrusion traffic classification model.

[0052] Optionally, the training unit further includes:

[0053] An input module, configured to input the training set data into the discriminator and input the incompressible noise and the latent vector into the generator;

[0054] A generation module, configured to generate pseudo data by the generator based on the incompressible noise and the latent vector, and input the pseudo data into the discriminator and the classifier.

[0055] Optionally, the training unit further includes:

[0056] A first training module for fixing the network parameters of the generator and training the discriminator and the classifier;

[0057] A calculation module for calculating the loss functions of the discriminator and the classifier respectively according to the training results;

[0058] A second adjustment module for adjusting the network parameters of the discriminator and the classifier based on the loss functions.

[0059] Optionally, the training unit further includes:

[0060] A second training module for fixing the network parameters of the discriminator and the classifier and training the generator after the number of training times is greater than or equal to a preset round threshold;

[0061] A second adjustment module for calculating the loss function of the generator according to the training results and adjusting the network parameters of the generator based on the loss function.

[0062] Optionally, the training unit further includes:

[0063] A judgment module for respectively judging whether the generator, the discriminator and the classifier meet the convergence conditions according to the loss functions of the generator, the discriminator and the classifier;

[0064] A completion module for completing the training of the intrusion traffic classification model after the generator, the discriminator and the classifier all converge.

[0065] Optionally, the replacement unit includes:

[0066] A replacement module for replacing the activation function of the intrusion traffic classification model from the SoftMax layer to the OpenMax layer;

[0067] An extraction module for extracting the processing result of the Dense layer before the SoftMax layer as the activation vector of each traffic sample;

[0068] A determination module for determining the tail size and the number of traffic types for adjusting the activation vector and calculating the parameters of the Weibull distribution;

[0069] An adjustment module for adjusting the activation vector values of the known intrusion traffic categories of the intrusion traffic classification model, calculating the activation vector values and estimated probabilities of the unknown intrusion traffic categories.

[0070] Optionally, the device is used for misuse detection technology, including:

[0071] A detection unit for detecting and classifying intrusion traffic using an intrusion traffic classification model based on a SoftMax layer and an intrusion traffic classification model based on an OpenMax layer.

[0072] Optionally, the device is used for anomaly detection technology, and the preprocessing of the intrusion traffic set further includes:

[0073] A classification unit for classifying the normal traffic based on the behavior patterns of the normal traffic and labeling the tags, where the tags are used to distinguish the behavior patterns of different normal traffic.

[0074] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0075] At least one processor; and

[0076] A memory communicatively connected to the at least one processor; wherein,

[0077] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in the foregoing first aspect.

[0078] According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method described in the foregoing first aspect.

[0079] According to a fifth aspect of the present disclosure, there is provided a computer program product including a computer program, where the computer program implements the method described in the foregoing first aspect when executed by a processor.

[0080] The detection and classification method, device, electronic device, and storage medium for intrusion traffic provided by the present disclosure mainly include the following technical solutions: First, preprocess the intrusion traffic set, and divide the traffic in the intrusion traffic set into a training set, a test set, and an open set. The intrusion traffic types in the training set and the test set are the same, and the open set contains intrusion traffic types different from those in the training set and the test set. Secondly, train the intrusion traffic classification model based on the training set, the test set, the incompressible noise, and the latent vector. The intrusion traffic classification model includes a generator, a discriminator, and a classifier. Replace the activation function of the intrusion traffic classification model and adjust the parameter values of the replaced activation function. Finally, perform a performance test on the intrusion traffic classification model based on the open set. Compared with the related technologies, in the embodiments of the present application, the intrusion traffic classification model is trained through the intrusion traffic set. After the intrusion traffic classification model has the function of identifying known types of intrusion traffic, the activation function in the intrusion traffic classification model is replaced, so that the intrusion traffic classification model has the ability to detect unknown types of intrusion traffic.

[0081] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0083] Figure 1 is a flowchart of a method for detecting and classifying intrusion traffic provided by an embodiment of the present disclosure;

[0084] Figure 2 is a block diagram of the overall network structure of an intrusion traffic classification model provided by an embodiment of the present disclosure;

[0085] Figure 3 is a flowchart of a method for preprocessing intrusion traffic data provided by an embodiment of the present disclosure;

[0086] Figure 4 is a structure diagram of an intrusion traffic classification model provided by an embodiment of the present disclosure;

[0087] Figure 5 is a flowchart of a method for training an intrusion traffic classification model provided by an embodiment of the present disclosure;

[0088] Figure 6 is a network structure diagram of a classifier and a discriminator provided by an embodiment of the present disclosure;

[0089] Figure 7Schematic diagram of a generator network structure provided by an embodiment of the present disclosure;

[0090] Figure 8 Schematic flowchart of a method for replacing an activation function provided by an embodiment of the present disclosure;

[0091] Figure 9 Structural diagram of an O-S open-set traffic detection and classification model based on misuse detection technology provided by an embodiment of the present disclosure;

[0092] Figure 10 Schematic diagram of the structure of an OpenMax open-set traffic detection and classification model based on anomaly detection technology provided by an embodiment of the present disclosure;

[0093] Figure 11 Schematic diagram of the structure of a detection and classification device for intrusion traffic provided by an embodiment of the present disclosure;

[0094] Figure 12 Schematic diagram of the structure of another detection and classification device for intrusion traffic provided by an embodiment of the present disclosure;

[0095] Figure 13 Schematic block diagram of an exemplary electronic device 600 provided by an embodiment of the present disclosure. Detailed implementation manners

[0096] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0097] The following describes a method, device, electronic device, and storage medium for detecting and classifying intrusion traffic according to embodiments of the present disclosure with reference to the accompanying drawings.

[0098] Figure 1 Schematic flowchart of a method for detecting and classifying intrusion traffic provided by an embodiment of the present disclosure.

[0099] As Figure 1 shown, the method includes the following steps:

[0100] Step 101: Preprocess the intrusion traffic set, and divide the traffic in the intrusion traffic set into a training set, a test set, and an open set. The intrusion traffic types in the training set and the test set are the same, and the open set contains intrusion traffic types different from those in the training set and the test set.

[0101] Please refer to Figure 2 ,Figure 2 It is a block diagram of the overall network structure of an intrusion traffic classification model provided by an embodiment of the present application; the intrusion traffic collectively includes normal traffic and various intrusion traffic categories. In the embodiment of the present application, the CICIDS2017 intrusion traffic dataset is used as the intrusion traffic set for illustration. However, it should be noted that this illustration method is not a specific limitation on the intrusion traffic set; the CICIDS2017 intrusion traffic dataset is an 85-dimensional intrusion traffic feature set. The intrusion traffic in the intrusion traffic feature set contains some features that cannot be applied in intrusion detection and some data that is meaningless for the training of the intrusion traffic classification model, such as collection time, number, etc. Therefore, it is necessary to process the intrusion traffic dataset and remove the above relevant data.

[0102] The intrusion traffic categories included in the training set and the test set are the same. The intrusion traffic data of each category is independent and non-repetitive, and there is no identical data between the training set and the test set; the open set adds new intrusion traffic categories and data compared to the previous two. The training set data has no labels, and the test set and open set data are labeled.

[0103] Step 102 trains the intrusion traffic classification model based on the training set, the test set, the incompressible noise, and the latent vector. The intrusion traffic classification model includes a generator, a discriminator, and a classifier.

[0104] The intrusion traffic classification model uses an improved network of Generative Adversarial Networks (GAN) - Information Maximizing GAN (Info GAN) as the intrusion traffic classification model to reduce the dependence on labeled data and improve the classification performance of the model.

[0105] Please refer to Figure 2 , the trained intrusion traffic classification model has the function of detecting and classifying known categories of intrusion traffic in a closed-set environment.

[0106] Step 103 replaces the activation function of the intrusion traffic classification model and adjusts the parameter values of the replaced activation function.

[0107] Please continue to refer to Figure 2, replace the SoftMax layer of the auxiliary classification network of the intrusion traffic classification model, and use the improved OpenMax layer to replace the SoftMax layer. Use the traffic of the test set that can be correctly classified by the intrusion traffic classification model as the sample data for calculating the average activation vector of each type of traffic by the open set detection model, and adjust the calculation of the activation vector of the known class traffic by the model, so that the probability calculation finally output by the classifier includes the estimated probability of the unknown class. Adjust the parameter values of the OpenMax algorithm, and use the open set data labeled with labels to test the unknown class detection and classification performance of the intrusion traffic detection model.

[0108] The intrusion traffic classification model after replacing the activation function has the detection and classification functions for identifying unknown types of intrusion traffic in an open set environment.

[0109] Step 104, perform a performance test on the intrusion traffic classification model based on the open set.

[0110] Please continue to refer to Figure 2 , the open set includes intrusion traffic of known types and intrusion traffic of unknown types in the intrusion traffic classification model. Based on the open set, the detection and classification effects of the intrusion traffic classification model for intrusion traffic of known types and unknown types can be detected simultaneously.

[0111] The main technical solutions of the detection and classification method for intrusion traffic provided by the present disclosure include: First, preprocess the intrusion traffic set, and divide the traffic in the intrusion traffic set into a training set, a test set, and an open set. The intrusion traffic types in the training set and the test set are the same, and the open set includes intrusion traffic types different from those in the training set and the test set; Secondly, train the intrusion traffic classification model based on the training set, the test set, the incompressible noise, and the latent vector. The intrusion traffic classification model includes a generator, a discriminator, and a classifier; replace the activation function of the intrusion traffic classification model, and adjust the parameter values of the replaced activation function; Finally, perform a performance test on the intrusion traffic classification model based on the open set; Compared with the related art, in the embodiment of the present application, the intrusion traffic classification model is trained through the intrusion traffic set, so that the intrusion traffic classification model has the function of identifying intrusion traffic of known types, and then the activation function in the intrusion traffic classification model is replaced, so that the intrusion traffic classification model has the ability to detect intrusion traffic of unknown classes.

[0112] As an extension of the above application embodiments, after performing performance testing on the intrusion traffic classification model based on the open set, it further includes: recording the unknown intrusion traffic categories and data in the open set based on the intrusion traffic classification model; when the detection ends, the open set detection model records the newly obtained intrusion traffic categories and data into its own database, turning the unknown intrusion traffic into known traffic. This realizes the continuous update and expansion of the model database, and the purpose that the intrusion traffic classification model can identify an increasing number of traffic categories and continuously improve the intrusion detection ability.

[0113] In the adversarial training of the generator and discriminator in the original GAN, the purpose is to train the generator to have the ability to generate pseudo-data consistent with the real data distribution. There is a problem with this training method, that is, the input of the generator is a random noise signal z that follows a normal distribution, and there are no other constraints. After training, it is impossible to correspond the individual dimensions of z with the semantic features of the output data. This problem results in poor interpretability of GAN and it is difficult to verify whether its representation is meaningful. Info GAN solves this problem by improving the input noise vector: decomposing the input noise into two parts, one part is the incompressible noise z; the other part is the latent vector c, which is used for the structured semantic features of the data distribution.

[0114] c = {c 1 , c 2 ,..., c L} represents the set of input latent vectors. The probability distributions of the latent vectors are independent, and the value c ∈ [0, m - 1], c ∈ Z, where m represents the number of data classification categories, and the value of c follows an equiprobable distribution, P c = 1 / m. The distributions of the latent vectors are independent of each other and satisfy the following relationship:

[0115]

[0116] The value of z follows a standard normal distribution. Inputting the incompressible noise z and the latent vector c into the generator, the generated data is expressed as G(z, c). If calculated using the objective function of the original GAN, it will cause the generator to ignore the latent vector part, resulting in P G (x|c) = P G (x). Use the information regularization method to solve this problem, that is, the objective function needs to ensure that the mutual information entropy value between the latent vector and the generated data should be relatively high. The calculation formula for the mutual information is as follows:

[0117] I(X; Y) = H(X) - H(X|Y) = H(Y) - H(X|Y) (2)

[0118] Among them, H(·) represents the information entropy. I(X; Y) can be described as the uncertainty of X under the condition that Y is known. When X and Y are independent of each other, I(X; Y) = 0; when X and Y are associated by a deterministic invertible function, the value of I(X; Y) reaches the maximum. The intrusion traffic classification model uses a latent vector to improve the problems that the original GAN input noise cannot correspond to the semantics of the generated data and has poor interpretability. The latent vector c and the generated data G(z, c) should have a high degree of correlation. Therefore, given P G (x), P G (x|c) should have a small entropy. The optimization objective function adopted can be expressed as:

[0119]

[0120] Among them, λ represents a hyperparameter, and its value can be 1. In practice, the unknown of P(c|x) makes it difficult to directly maximize I(c; G(z, c)). Therefore, an auxiliary distribution Q(c|x) is introduced, and its calculation process is as follows:

[0121] I(c; G(z, c)) = H(c) - H(c|G(z, c)) = E x～G(z,c) E c'～P(c|x) logP(c'|x) + H(c) (4)

[0122] By constructing the KL divergence (Kullback–Leibler Divergence) of logP(c'|x), using Q(c|x) to substitute P(c'|x) gives:

[0123]

[0124] The conclusion can be obtained as in formula (6). Since the value on the right side of the inequality is related to the generator and the auxiliary distribution Q(c|x), it is abbreviated as L 1 (G, Q). This formula shows that the maximization of the auxiliary distribution can be used to approximate the lower bound of the mutual information I(c; G(z, c)), and L 1 (G, Q) can be estimated by Monte Carlo sampling.

[0125]

[0126] In summary, the objective function of Info GAN can be expressed as follows:

[0127]

[0128] The intrusion traffic needs to be preprocessed before being input into the intrusion traffic classification model for use. Please refer to Figure 3 , Figure 3The flowchart of a method for preprocessing intrusion traffic data provided by an embodiment of the present disclosure includes:

[0129] Step 201 replaces or deletes the values in the intrusion traffic that cannot be read by the intrusion detection algorithm.

[0130] Not all sample features in the dataset have corresponding values. Some data has NAN values in the feature "Bwd PacketLength Max" and Infinity values in the features "Bwd Packet Length Max" and "Bwd Packet LengthMin". These values cannot be read by the intrusion detection algorithm and need to be deleted or replaced. In the embodiments of the present application, the NAN values are respectively replaced with the average values of the data of their respective types in the feature dimension, and the Infinity values are respectively replaced with the maximum values of the data of their respective types in the feature dimension to achieve data cleaning of outliers. The "fwd_header_length" feature appears twice in the dataset, and the data values of each sample are the same, so redundant columns need to be removed.

[0131] Step 202 deletes the all-zero features in the intrusion traffic.

[0132] There are 10 feature dimensions in the dataset with values of 0 for all samples, which cannot play a role as features in intrusion detection. Such features are called all-zero features, and they are: Bwd PSH Flags, Fwd URG Flags, Bwd URG Flags, CWEFlags Count, Fwd Avg Bytes / Bulk, Fwd Avg Packets / Bulk, Fwd Avg Bulk Rate, Bwd AvgBytes / Bulk, Bwd Avg Packets / Bulk, Bwd Avg Bulk Rate. Removing the all-zero features helps to improve the model accuracy, so the above features are removed.

[0133] Step 203 deletes the irrelevant network flow features in the intrusion traffic.

[0134] Some features include the source address, destination address, source port number, destination port number, collection time, number, etc. of traffic collection, which are used to distinguish network traffic. Their formats are not suitable for being read by the intrusion detection model. These features are mainly Flow ID, Source IP, Destination IP, Timestamp, Source Port, Destination Port in the dataset.

[0135] Step 204: Normalize and expand the dimension of the intrusion traffic according to the input size of the intrusion traffic classification model.

[0136] After removing the above features in Step 201, Step 202, and Step 203, the dataset features are reduced from 85 dimensions to 69 dimensions. Normalize and expand the dimension of the samples of the remaining features according to Equation (8). To adapt to the input size of the intrusion detection model, zero-padding is performed on the processed samples to expand the data dimension, and label random rearrangement and two-dimensional size conversion are performed.

[0137]

[0138] where x mi n is the minimum value in each dimension of the data, and x max is the maximum value in each dimension of the data. x and x* respectively represent the current data value and the current data value after normalization. After processing, the data features are combined with the one-hot encoding of the non-data features. To unify the input dataset feature dimension without affecting feature extraction, the input feature dimension is set to 121 dimensions, and samples with insufficient features are supplemented with 0 for dimension. To better exert the operation effect of the convolutional layer, it can be converted into a two-dimensional feature vector with a size of 11×11.

[0139] As an extension of the above application embodiment, when training the intrusion traffic classification model, it further includes: inputting the training set data into the discriminator, and inputting the incompressible noise and the latent vector into the generator; the generator generates pseudo data based on the incompressible noise and the latent vector, and inputs the pseudo data into the discriminator and the classifier; please refer to Figure 4 , Figure 4 which is a structural diagram of an intrusion traffic classification model provided by an embodiment of the present disclosure; determine the value range of the latent vector according to the number of traffic types in the dataset, set the label label = 1 for the training set data, send the latent vector and the noise into the generator to synthesize pseudo data, and set the label label = 0 for the pseudo data. Send the input data, the pseudo data, and the labels into the discriminator for type discrimination, output the discrimination result, calculate the loss function, and respectively obtain the loss functions loss of the generator and the discriminator, and use the loss function to adjust the network parameter distributions of the discriminator and the generator. At the same time, send the synthesized pseudo data into the classifier, the classifier judges the type of the latent vector in the pseudo data and performs classification output, compares the output result with the input latent vector c, and calculates the loss function loss. And adjust the network parameter distribution of the classifier according to the loss function.

[0140] The intrusion traffic classification model in the embodiments of the present application is an improved network of a generative adversarial network. Therefore, when training the intrusion traffic classification model, a method of training one party's network parameters while fixing the network parameters of the other party is adopted; as Figure 5 shown, Figure 5 FIG. is a schematic flowchart of a method for training an intrusion traffic classification model provided by an embodiment of the present application, including:

[0141] Step 301: Fix the network parameters of the generator and train the discriminator and the classifier.

[0142] Before fixing the generator, first train the generator, discriminator, and classifier for a certain number of rounds based on incompressible noise, latent vectors, and training set data. In practical applications, the number of training times can be set according to actual situations, and the embodiments of the present application do not limit this.

[0143] Step 302: Calculate the loss functions of the discriminator and the classifier respectively according to the training results.

[0144] Step 303: Adjust the network parameters of the discriminator and the classifier based on the loss functions.

[0145] Use the loss functions to adjust the network parameter distributions of the discriminator and the generator, and judge the training results of the discriminator and the generator based on the loss functions.

[0146] As an extension of the above embodiments of the application, after training the discriminator and the classifier by fixing the network parameters of the generator for a number of times greater than or equal to a preset round threshold, at this time, the discrimination and classification capabilities of the discriminator and the classifier are much greater than the ability of the generator to generate pseudo data. At this time, the effect of continued training will be greatly reduced. Therefore, after training for a certain number of rounds, swap the positions of the generator, discriminator, and classifier, fix the network parameters of the discriminator and the classifier, and train the generator. Calculate the loss function of the generator according to the training results, and adjust the network parameters of the generator based on the loss function; and train the generator, discriminator, and classifier in this order, and judge whether the generator, discriminator, and classifier meet the convergence conditions according to the loss functions of the generator, discriminator, and classifier; when the generator, discriminator, and classifier all meet the convergence conditions, complete the training of the intrusion traffic classification model.

[0147] As Figure 6 , Figure 7 shown, Figure 6 FIG. is a schematic diagram of the network structure of a classifier and a discriminator provided by an embodiment of the present application; Figure 7Schematic diagram of a generator network structure provided by an embodiment of the present application; the classifier and the discriminator share a 3-layer convolutional layer structure. The activation function used in the convolutional layer is LeakyReLU, and the input of the network is preprocessed training set traffic data. After the Flatten layer, the discriminator uses a Dense layer with 1 channel to discriminate the authenticity of the samples, and the activation function uses the Sigmoid function; after passing through the Flatten layer, the classifier passes through a Dense layer with 128 channels, a batch normalization layer, and the activation function is LeakyReLU. Then, after passing through a Dense layer with c channels, the latent vector is divided into c categories, and the activation function is the SoftMax function. The input of the generator is random noise. After batch normalization processing, the size is converted to 4×4×512. Through the processing of 4 layers of transposed convolutional layers, the size becomes 11×11×1, the activation function is Tanh, and the output is fake data for the discriminator to identify.

[0148] The intrusion traffic classification model under the closed set condition does not have the ability to detect unknown category intrusion traffic because the number of channels in the Dense layer of the category output of the model is equal to the number of categories to be classified by the model, and the SoftMax function is usually used as the activation function for this layer. By replacing the activation function, the intrusion traffic classification model can detect the intrusion traffic of the location category; as Figure 8 shown, Figure 8 Schematic flowchart of a method for replacing an activation function provided by an embodiment of the present application, including:

[0149] Step 401, replace the activation function of the intrusion traffic classification model from the SoftMax layer to the OpenMax layer.

[0150] Let the number of categories of data in the closed set be y = 1, 2,..., N. The SoftMax function is a gradient logarithmic normalization tool in the category probability distribution, and its function is to transform the vector z into a vector σ(z) with element values ∈[0, 1] and the sum of element values equal to 1. The input of the SoftMax function is an N-dimensional node weight vector processed by the Dense layer, called the activation vector, which can be expressed as: v(x) = {v 1 (x), v 2 (x),..., v N (x)}. The calculation formula of the SoftMax function is as follows:

[0151]

[0152] In the formula, the denominator is the sum of e v(x)Perform summation to ensure that the sum of the probabilities P(y = j|x) output for all categories is 1. In the open-set recognition task, since there is data of unknown types, it is obviously inappropriate to make the sum of the estimated probabilities that the category belongs to the known set equal to 1. Therefore, it is necessary to replace the SoftMax function and introduce the estimated probability for samples of unknown classes while maintaining its original function. The above purpose is achieved through the OpenMax structure. The open-set recognition algorithm based on the OpenMax structure uses the activation vector v(x) of the layer before the SoftMax layer as the input, thus avoiding the logical normalization processing of the activation vector by the SoftMax function. According to the analysis of extreme value theory, the activation vector of the sample follows the Weibull distribution, and its cumulative probability density function is expressed as follows:

[0153]

[0154] where x represents the random variable, λ represents the scale parameter, and k represents the shape parameter.

[0155] Step 402: Extract the processing result of the Dense layer before the SoftMax layer as the activation vector of each traffic sample.

[0156] Select the model and test set traffic under the closed-set condition, and retain the test set samples that the model can correctly classify. Respectively take the activation vectors v(x) = {v 1 (x),v 2 (x),...,v N (x)} of these samples before the SoftMax structure of the closed-set model. Use S to represent the activation vectors of this part of the sample traffic, where N represents the number of classifications of the closed-set model. S i,j ={v 1 (x i,j ),v 2 (x i,j ),...,v N (x i,j )} represents the activation vector of the i-th sample in the j-th type of traffic, where j ∈ [1, N] represents the category to which the sample traffic belongs, and i represents the sample number of this type of traffic.

[0157] Step 403: Determine the tail size and the number of traffic types for adjusting the activation vector, and calculate the parameters of the Weibull distribution.

[0158] Classify the traffic in S according to its label category, and calculate the average activation vector (Mean Activation Vector, MAV) of all known category traffic respectively, that is where n represents the total number of samples of this type of traffic.

[0159] Determine the tail size η and the alpha rank α, and calculate the parameters of the Weibull distribution. This step estimates the parameters by calling the FitHigh function in the libMR package of Python. The alpha rank represents the number of traffic types for which the activation vectors are to be adjusted, and α ∈ (0, N]. The method for selecting the tail is as follows: Calculate the distance ||S i,j - μ j || between the activation vector of each sample in S and the average activation vector of that class. Sort ||S i,j - μ j || in descending order, and select the top η distance values to form the tail. η is called the tail size. When the number of samples of a certain type is less than η, all the sample traffic is taken to participate in the calculation of the FitHigh function. The calculation method of the parameters of the Weibull distribution is as follows: First, determine the traffic class j. Take the logarithm of all the distance values to get x i = ln|S i,j - μ j ||, and x = {x 1 , x 2 ,..., x n}. Normalize the distance values and calculate the average distance of the sample traffic of this type:

[0160]

[0161] The expressions for the scale parameter λ and the shape parameter κ can be represented as:

[0162] λ = exp[(max(x) - min(x))·muhat + max(x)]

[0163]

[0164] where the expressions for sigmahat and muhat are as follows:

[0165]

[0166]

[0167] After determining the value of alpha rank α, the adjustment weight rank_α can be solved, and its expression is as follows:

[0168]

[0169] where j represents the class number of the known traffic, and j ∈ [1, N].

[0170] Step 404: Adjust the activation vector values of the intrusion traffic classification model for known intrusion traffic categories, and calculate the activation vector values and estimated probabilities for unknown intrusion traffic categories.

[0171] With reference to the probability output of the SoftMax layer, sort the probability values in descending order. Denote the index value of each probability before sorting as s(j), and the index value of each probability after sorting as s'(j). Each probability value corresponds to the probability that the SoftMax layer determines the traffic sample belongs to that type. Calculate the weight ω(x) for each category of activation vector modified by OpenMax, and its expression is as follows:

[0172]

[0173] Adjust the activation vector v s(i) (x) of the known class to The calculation formula is as follows:

[0174]

[0175] Calculate the activation vector of the unknown class The calculation formula is as follows:

[0176]

[0177] Calculate the estimated probabilities of the adjusted known class and unknown class. The calculation formula is as follows:

[0178]

[0179] Among them, j = 0 represents the estimated probability of unknown traffic, and j ∈ [1, N] represents the estimated probability of known traffic. After the calculation is completed, select the maximum value of the estimated probabilities in j ∈ [0, N], and output the corresponding category as the classification result.

[0180] In practical applications, the intrusion traffic classification model can also be fine-tuned according to actual needs, or used together with other models to improve the accuracy of intrusion traffic recognition; as Figure 9 shown, Figure 9 is the structure diagram of an O-S open-set traffic detection and classification model based on misuse detection technology provided by an embodiment of this application. When the intrusion traffic detection model is used in an intrusion detection and classification system based on misuse detection technology, the following method can be adopted: Use the intrusion traffic classification model based on the SoftMax layer and the intrusion traffic classification model based on the OpenMax layer to detect and classify intrusion traffic.

[0181] When the closed-set intrusion traffic classification model has the ability to classify normal traffic and known-class intrusion traffic, the classification recall rate of the SoftMax-based intrusion traffic classification model for known-type traffic is generally higher than that of the OpenMax-based intrusion traffic classification model. This indicates that using only the OpenMax layer for open-set intrusion traffic detection and classification will cause false alarms in the model's detection of unknown traffic and misclassify known-class traffic as unknown-class traffic. To solve this problem, the O-S open-set detection model uses both the OpenMax-based open-set intrusion traffic classification model and the SoftMax-based closed-set intrusion traffic classification model to detect and classify open-set traffic. Figure 9 In [figure], a1 and a2 represent first using the OpenMax-based open-set intrusion traffic classification model to classify known traffic and detect unknown traffic in open-set traffic. There is a false alarm problem in the classification results of this part of the unknown traffic, that is, some known-class traffic will be misclassified as unknown traffic, resulting in poor overall recognition performance; the unknown traffic and labels output by part a2 are input into the closed-set intrusion traffic classification model obtained using the same training set, that is, b1. The SoftMax-based model mainly misclassifies unknown-class traffic as normal traffic. Therefore, part b2 represents discarding the results determined to be normal by the closed-set intrusion traffic classification model, that is, retaining the original labels and classifying them as unknown-type traffic; part b3 represents retaining the results determined to be other types of known traffic and replacing the original unknown labels, and classifying this part of the traffic into the known traffic results according to the new labels.

[0182] In the O-S open-set intrusion traffic classification model, unknown-class traffic is first detected by the OpenMax-based open-set detection model, and then the results are input into the closed-set intrusion traffic classification model to use SoftMax to remove misclassified traffic with false alarms; a part of the known-class traffic is identified and classified by the OpenMax-based open-set intrusion traffic classification model, and the remaining part comes from the correction and classification of false-alarm traffic by the closed-set intrusion traffic classification model.

[0183] As an extension of the embodiment of the present application, when the intrusion traffic classification model is used in an intrusion detection and classification system based on anomaly detection technology, the preprocessing of the intrusion traffic set further includes: classifying the normal traffic based on the behavior patterns of the normal traffic and labeling the labels, where the labels are used to distinguish the behavior patterns of different normal traffic. Please refer to Figure 10 , Figure 10Schematic diagram of the structure of an OpenMax open-set traffic detection and classification model based on anomaly detection technology provided by an embodiment of this application; the feature of anomaly detection technology is that the system only records normal traffic and behaviors, and detects through the deviation between normal traffic and intrusion traffic. In this case, the intrusion traffic classification model based on OpenMax and the O-S intrusion traffic classification model will misclassify unknown traffic as normal traffic, resulting in missed alarms for intrusion traffic and being unable to establish an effective anomaly detection model, thus bringing the risk of being invaded to the user system. The reason for the missed alarm problem is that normal traffic may have multiple behavior patterns. For an intrusion traffic classification model with a high classification accuracy for normal traffic, its classification decision range usually covers normal traffic with multiple behavior patterns. Although this can ensure a high recognition rate of normal traffic in the test set for the intrusion traffic classification model, it also leads to the distance confusion problem. The distance confusion problem in the intrusion traffic classification model is reflected in that the distance between normal traffic may be greater than the distance between normal traffic and unknown traffic. In this case, in order to maintain a high recognition accuracy, the SoftMax algorithm of the closed set has too wide a classification decision range for normal traffic, resulting in overconfidence in the classification of normal traffic. The unknown traffic caught in the distance confusion problem will be misclassified as normal traffic, ultimately resulting in a low detection recall rate of the intrusion traffic classification model for unknown traffic.

[0184] To address this problem, the abnormal traffic detection and classification model adopts a method based on fine-grained classification to solve the distance confusion problem, that is, by clustering normal traffic into different subclasses, so that normal traffic with similar behavior patterns is grouped together. Different labels are assigned to each subclass, and the intrusion traffic classification model under the closed set is trained using normal traffic with subclass labels. The characteristics of this subclass label are: during the training and testing process of the closed set model, the performance of the model for each subclass classification of normal traffic is evaluated according to the subclass label; while in the classification and detection of open-set traffic, when the model is confused about the subclass classification of normal traffic, it will not lead to a decrease in the model's recognition performance for normal traffic. The purpose of the above method is to limit the classification decision range of the intrusion traffic classification model for normal traffic, and enable the model to learn more fine-grained features of normal traffic by classifying sub-traffic. Figure 10Among them, first, two-dimensional PCA visualization processing is performed on the normal traffic to observe the distribution of the normal traffic and determine the number of subclasses that the normal traffic can be divided into. The unsupervised clustering algorithm K-means is used to cluster the normal traffic in the test set and the open set to obtain the subclass labels of each traffic, which are used to test the classification performance of the intrusion traffic classification for the subclass traffic. The intrusion traffic classification model under the closed-set condition is trained through the training set and the test set of the Normal type traffic. When the model converges, the test set of the Normal traffic is used to calculate the MAV of each subclass traffic. According to the algorithm steps of OpenMax, the estimated probability of the unknown traffic is calculated. After the intrusion traffic classification model converges, the open set is used to evaluate the performance of the model.

[0185] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0186] (1) An intrusion traffic classification algorithm and an open-set traffic detection and classification algorithm are constructed for the intrusion traffic classification model by using the unsupervised learning algorithm, reducing the dependence of the training set on the labeled data, avoiding the heavy work of data annotation, and expanding the data range available for training the model. The model maintains a high accuracy in classifying various types of intrusion traffic.

[0187] (2) An intrusion detection and classification model based on misuse detection technology is proposed. By combining the intrusion traffic classification model based on the SoftMax layer and the intrusion traffic classification model based on the OpenMax layer, the false alarm problem caused by the OpenMax algorithm misclassifying the known traffic as the unknown class is effectively solved. The intrusion detection and classification model based on misuse detection technology can effectively detect various types of unknown traffic, and its classification performance for the known traffic is better than that of the intrusion traffic classification model based on the OpenMax layer.

[0188] (3) An intrusion detection and classification model based on anomaly detection technology is proposed. The model performs fine-grained classification on the training set and uses the subclasses of the normal traffic to train the intrusion traffic classification model to learn more refined features of the normal traffic. The intrusion detection and classification model based on anomaly detection technology solves the problem of distance confusion of the normal traffic by the intrusion traffic classification model under the closed-set condition, prevents the intrusion traffic classification model from missing alarms in detecting unknown traffic, and improves the detection accuracy of the intrusion traffic classification model for unknown traffic.

[0189] Corresponding to the above-mentioned intrusion traffic detection and classification method, the present invention also proposes an intrusion traffic detection and classification device. Since the device embodiment of the present invention corresponds to the above-mentioned method embodiment, for the details not disclosed in the device embodiment, reference may be made to the above-mentioned method embodiment, and no further description will be given in the present invention.

[0190] Figure 11The structural schematic diagram of a detection and classification device for intrusion traffic provided by an embodiment of the present disclosure is as follows Figure 11 shown, including:

[0191] A processing unit 51, configured to preprocess an intrusion traffic set and divide the traffic in the intrusion traffic set into a training set, a test set, and an open set, where the intrusion traffic types in the training set and the test set are the same, and the open set contains intrusion traffic types different from those in the training set and the test set;

[0192] A training unit 52, configured to train an intrusion traffic classification model based on the training set, the test set, non-compressible noise, and latent vectors, where the intrusion traffic classification model includes a generator, a discriminator, and a classifier;

[0193] A replacement unit 53, configured to replace the activation function of the intrusion traffic classification model and adjust the parameter values of the replaced activation function;

[0194] A testing unit 54, configured to perform performance testing on the intrusion traffic classification model based on the open set.

[0195] The detection and classification device for intrusion traffic provided by the present disclosure, the main technical solutions include: First, preprocess the intrusion traffic set and divide the traffic in the intrusion traffic set into a training set, a test set, and an open set, where the intrusion traffic types in the training set and the test set are the same, and the open set contains intrusion traffic types different from those in the training set and the test set; Second, train an intrusion traffic classification model based on the training set, the test set, non-compressible noise, and latent vectors, where the intrusion traffic classification model includes a generator, a discriminator, and a classifier; replace the activation function of the intrusion traffic classification model and adjust the parameter values of the replaced activation function; Finally, perform performance testing on the intrusion traffic classification model based on the open set; Compared with the related art, in the embodiment of the present application, the intrusion traffic classification model is trained through the intrusion traffic set, so that the intrusion traffic classification model has the function of identifying known types of intrusion traffic, and then the activation function in the intrusion traffic classification model is replaced, so that the intrusion traffic classification model has the ability to detect unknown types of intrusion traffic.

[0196] Further, in a possible implementation manner of this embodiment, as Figure 12 shown, the device further includes:

[0197] A recording unit 55, configured to record the unknown intrusion traffic categories and data in the open set based on the intrusion traffic classification model after the testing unit performs performance testing on the intrusion traffic classification model based on the open set.

[0198] Further, in a possible implementation manner of this embodiment, as Figure 12 shown, the processing unit 51 includes:

[0199] A first deletion module 511, configured to replace or delete values in the intrusion traffic that cannot be read by the intrusion detection algorithm;

[0200] A second deletion module 512, configured to delete all-zero features in the intrusion traffic;

[0201] A third deletion module 513, configured to delete irrelevant network flow features in the intrusion traffic;

[0202] A processing module 514, configured to perform normalization processing and dimension supplementation on the intrusion traffic according to the input size of the intrusion traffic classification model.

[0203] Further, in a possible implementation manner of this embodiment, as Figure 12 shown, the training unit 52 further includes:

[0204] An input module 521, configured to input the training set data into the discriminator, and input the incompressible noise and the latent vector into the generator;

[0205] A generation module 522, configured to generate pseudo data based on the incompressible noise and the latent vector by the generator, and input the pseudo data into the discriminator and the classifier.

[0206] Further, in a possible implementation manner of this embodiment, as Figure 12 shown, the training unit 52 further includes:

[0207] A first training module 523, configured to fix the network parameters of the generator and train the discriminator and the classifier;

[0208] A calculation module 524, configured to calculate the loss functions of the discriminator and the classifier respectively according to the training results;

[0209] A second adjustment module 525, configured to adjust the network parameters of the discriminator and the classifier based on the loss functions.

[0210] Further, in a possible implementation manner of this embodiment, as Figure 12 shown, the training unit 52 further includes:

[0211] A second training module 526, configured to fix the network parameters of the discriminator and the classifier and train the generator after the number of training times is greater than or equal to a preset round threshold;

[0212] The second adjustment module 527 is configured to calculate the loss function of the generator according to the training result, and adjust the network parameters of the generator based on the loss function.

[0213] Further, in a possible implementation manner of this embodiment, as Figure 12 shown, the training unit 52 further includes:

[0214] A judgment module 528, configured to judge whether the generator, discriminator, and classifier meet the convergence conditions respectively according to the loss functions of the generator, discriminator, and classifier;

[0215] A completion module 529, configured to complete the training of the intrusion traffic classification model after the generator, discriminator, and classifier all converge.

[0216] Further, in a possible implementation manner of this embodiment, as Figure 12 shown, the replacement unit 53 includes:

[0217] A replacement module 531, configured to replace the activation function of the intrusion traffic classification model from the SoftMax layer with the OpenMax layer;

[0218] An extraction module 532, configured to extract the processing result of the Dense layer before the SoftMax layer as the activation vector of each traffic sample;

[0219] A determination module 533, configured to determine the tail size and the number of traffic types for adjusting the activation vector, and calculate the parameters of the Weibull distribution;

[0220] An adjustment module 534, configured to adjust the activation vector values of the known intrusion traffic categories of the intrusion traffic classification model, and calculate the activation vector values and estimated probabilities of the unknown intrusion traffic categories.

[0221] Further, in a possible implementation manner of this embodiment, as Figure 12 shown, the device for misuse detection technology includes:

[0222] A detection unit 56, configured to detect and classify intrusion traffic by using an intrusion traffic classification model based on the SoftMax layer and an intrusion traffic classification model based on the OpenMax layer.

[0223] Further, in a possible implementation manner of this embodiment, as Figure 12 shown, for the device for anomaly detection technology, the preprocessing of the intrusion traffic set further includes:

[0224] Classification unit 57 is used to classify the normal traffic based on the behavior patterns of the normal traffic and label it, and the label is used to distinguish the behavior patterns of different normal traffic.

[0225] It should be noted that the foregoing explanation of the method embodiments also applies to the devices in this embodiment, with the same principle, and will not be limited in this embodiment.

[0226] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0227] Figure 13 The schematic block diagram of an example electronic device 600 that can be used to implement the embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0228] As Figure 13 shown, the device 600 includes a computing unit 601, which can execute various appropriate actions and processes according to the computer program stored in the ROM (Read-Only Memory) 602 or the computer program loaded from the storage unit 608 into the RAM (Random Access Memory) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. The I / O (Input / Output) interface 605 is also connected to the bus 604.

[0229] A plurality of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disc, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0230] The computing unit 601 can be various general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, CPU (Central Processing Unit), GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSP (Digital Signal Processor), and any suitable processor, controller, microcontroller, etc. The computing unit 601 executes the various methods and processes described above, such as the method for detecting and classifying intrusion traffic. For example, in some embodiments, the method for detecting and classifying intrusion traffic can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the method described above can be performed. Alternatively, in other embodiments, the computing unit 601 can be configured to execute the aforementioned method for detecting and classifying intrusion traffic in any other suitable manner (e.g., by means of firmware).

[0231] Various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chip), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special or general programmable processor that receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0232] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing devices, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program codes can be executed entirely on the machine, partially on the machine, executed partially on the machine as an independent software package and partially on a remote machine, or executed entirely on a remote machine or server.

[0233] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a RAM, a ROM, an EPROM (Electrically Programmable Read-Only-Memory), or a flash memory, an optical fiber, a CD-ROM (Compact Disc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0234] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or an LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0235] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with embodiments of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0236] A computer system can include a client and a server. The client and the server are generally far from each other and usually interact through a communication network. The client-server relationship is created by computer programs running on respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services ("Virtual Private Server", or simply "VPS" for short). The server can also be a server of a distributed system, or a server combined with blockchain.

[0237] Among them, it should be noted that artificial intelligence is a discipline that studies how to make a computer simulate certain thinking processes and intelligent behaviors of humans (such as learning, reasoning, thinking, planning, etc.), and it has both hardware-level technologies and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include several major directions such as computer vision technology, speech recognition technology, natural language processing technology, and machine learning / deep learning, big data processing technology, and knowledge graph technology.

[0238] It should be understood that various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and no limitation is imposed herein.

[0239] The above specific embodiments do not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. A method for detecting and classifying intrusion traffic, characterized in that, it includes: Preprocess the intrusion traffic set, and divide the traffic in the intrusion traffic set into a training set, a test set, and an open set. The intrusion traffic types in the training set and the test set are the same, and the open set contains intrusion traffic types different from those in the training set and the test set; Train the intrusion traffic classification model based on the training set, the test set, the incompressible noise, and the latent vector. The intrusion traffic classification model includes a generator, a discriminator, and a classifier; Replace the activation function of the intrusion traffic classification model and adjust the parameter values of the replaced activation function; Perform performance testing on the intrusion traffic classification model based on the open set; The replacing the activation function of the intrusion traffic classification model and adjusting the parameter values of the replaced activation function includes: Replace the activation function of the intrusion traffic classification model from the SoftMax layer to the OpenMax layer; Extract the processing result of the Dense layer before the SoftMax layer as the activation vector of each traffic sample; Determine the tail size and the number of traffic types for adjusting the activation vector, and calculate the parameters of the Weibull distribution; Adjust the activation vector values of the known intrusion traffic categories of the intrusion traffic classification model, and calculate the activation vector values and estimated probabilities of the unknown intrusion traffic categories.

2. The method according to claim 1, characterized in that, After performing performance testing on the intrusion traffic classification model based on the open set, the method further includes: Record the unknown intrusion traffic categories and data in the open set based on the intrusion traffic classification model.

3. The method according to claim 1, characterized in that, The preprocessing of the intrusion traffic set includes: Replace or delete the numerical values in the intrusion traffic that cannot be read by the intrusion detection algorithm; Delete the all-zero features in the intrusion traffic; Delete the irrelevant network flow features in the intrusion traffic; Normalize and expand the dimensions of the intrusion traffic according to the input size of the intrusion traffic classification model.

4. The method according to claim 1, characterized in that, The training of the intrusion traffic classification model based on the training set, the test set, the incompressible noise, and the latent vector further includes: Input the training set data into the discriminator, and input the incompressible noise and the latent vector into the generator; The generator generates pseudo data based on the incompressible noise and the latent vector, and inputs the pseudo data into the discriminator and the classifier.

5. The method according to claim 4, characterized in that, The training of the intrusion traffic classification model based on the training set, the test set, the incompressible noise, and the latent vector further includes: Fix the network parameters of the generator, and train the discriminator and the classifier; According to the training results, calculate the loss functions of the discriminator and the classifier respectively; Adjust the network parameters of the discriminator and the classifier based on the loss functions.

6. The method according to claim 5, characterized in that, Training the intrusion traffic classification model based on the training set, test set, incompressible noise, and latent vector further includes: After the number of training times is greater than or equal to a preset round threshold, fix the network parameters of the discriminator and classifier, and train the generator; Calculate the loss function of the generator according to the training results, and adjust the network parameters of the generator based on the loss function.

7. The method according to claim 4 or 5, wherein, Training the intrusion traffic classification model based on the training set, test set, incompressible noise, and latent vector further includes: Judge whether the generator, discriminator, and classifier meet the convergence conditions according to the loss functions of the generator, discriminator, and classifier respectively; When the generator, discriminator, and classifier simultaneously meet the convergence conditions, complete the training of the intrusion traffic classification model.

8. The method according to any one of claims 1-6, wherein, The method is used for misuse detection technology, including: Using the intrusion traffic classification model based on the SoftMax layer and the intrusion traffic classification model based on the OpenMax layer to detect and classify intrusion traffic.

9. The method according to any one of claims 1-6, wherein, The method is used for anomaly detection technology, and the preprocessing of the intrusion traffic set further includes: Classify the normal traffic based on the behavior patterns of normal traffic and label it, and the label is used to distinguish the behavior patterns of different normal traffic.

10. A detection and classification device for intrusion traffic, wherein, includes: A processing unit for preprocessing the intrusion traffic set and dividing the traffic in the intrusion traffic set into a training set, a test set, and an open set. The intrusion traffic types in the training set and the test set are the same, and the open set contains intrusion traffic types different from those in the training set and the test set; A training unit for training an intrusion traffic classification model based on the training set, test set, incompressible noise, and latent vector. The intrusion traffic classification model includes a generator, a discriminator, and a classifier; A replacement unit for replacing the activation function of the intrusion traffic classification model and adjusting the parameter values of the replaced activation function; A testing unit for performing performance testing on the intrusion traffic classification model based on the open set; The replacement unit includes: A replacement module for replacing the activation function of the intrusion traffic classification model from the SoftMax layer to the OpenMax layer; An extraction module for extracting the processing results of the Dense layer before the SoftMax layer as the activation vectors of each traffic sample; A determination module for determining the tail size and the number of traffic types for adjusting the activation vectors, and calculating the parameters of the Weibull distribution; An adjustment module for adjusting the activation vector values of the known intrusion traffic categories of the intrusion traffic classification model, calculating the activation vector values and estimated probabilities of the unknown intrusion traffic categories.

11. The device according to claim 10, wherein, The device further includes: A recording unit, configured to record the unknown intrusion traffic categories and data in the open set based on the intrusion traffic classification model after the test unit performs a performance test on the intrusion traffic classification model based on the open set.

12. The apparatus according to claim 10, wherein, the processing unit includes: a first deletion module, configured to replace or delete the values in the intrusion traffic that cannot be read by the intrusion detection algorithm; a second deletion module, configured to delete the all-zero features in the intrusion traffic; a third deletion module, configured to delete the irrelevant network flow features in the intrusion traffic; a processing module, configured to perform normalization processing and dimension expansion on the intrusion traffic according to the input size of the intrusion traffic classification model.

13. The apparatus according to claim 10, wherein, the training unit further includes: an input module, configured to input the training set data into a discriminator, and input the incompressible noise and the latent vector into a generator; a generation module, configured to generate pseudo data based on the incompressible noise and the latent vector by the generator, and input the pseudo data into the discriminator and the classifier.

14. The apparatus according to claim 13, wherein, the training unit further includes: a first training module, configured to fix the network parameters of the generator, and train the discriminator and the classifier; a calculation module, configured to calculate the loss functions of the discriminator and the classifier respectively according to the training results; a second adjustment module, configured to adjust the network parameters of the discriminator and the classifier based on the loss functions.

15. The apparatus according to claim 14, wherein, the training unit further includes: a second training module, configured to fix the network parameters of the discriminator and the classifier after the number of training times is greater than or equal to a preset round threshold, and train the generator; a second adjustment module, configured to calculate the loss function of the generator according to the training results, and adjust the network parameters of the generator based on the loss function.

16. The apparatus according to claim 13 or 14, wherein, the training unit further includes: a judgment module, configured to judge whether the generator, the discriminator and the classifier meet the convergence conditions respectively according to the loss functions of the generator, the discriminator and the classifier; a completion module, configured to complete the training of the intrusion traffic classification model after the generator, the discriminator and the classifier all converge.

17. The apparatus according to any one of claims 10-15, wherein, the apparatus is used for misuse detection technology, including: a detection unit, configured to detect and classify intrusion traffic by using an intrusion traffic classification model based on a SoftMax layer and an intrusion traffic classification model based on an OpenMax layer.

18. The apparatus according to any one of claims 10-15, wherein, the apparatus is used for anomaly detection technology, and the preprocessing of the intrusion traffic set further includes: A classification unit is used to classify the normal traffic based on the behavior patterns of the normal traffic and label it, and the label is used to distinguish the behavior patterns of different normal traffics.

19. An electronic device, characterized in that, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of claims 1-9.

20. A computer-readable storage medium stores a computer program, characterized in that, the computer program is executable by a processor to implement the method steps according to any one of claims 1-9.

21. A computer program product, characterized in that, comprising a computer program, and the computer program implements the method according to any one of claims 1-9 when executed by a processor.

Images