Anti-noise-pollution high-robustness network traffic classification method
The clustering algorithm is used to clean and add Gaussian noise, combined with the autoencoder model to process network traffic data, which solves the problem of low classification accuracy caused by traffic data noise in deep learning technology, and significantly improves the accuracy of network traffic classification. performance and robustness.
Patent Information
- Application Number
- CN202411458761.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-18
- Publication Date
- 2025-01-21
AI Technical Summary
The main difficulty facing deep learning technology in network traffic classification is that the traffic data is too noisy in the training data, resulting in a low classification accuracy rate of the model in practical applications.
Using a highly robust network traffic classification method that is resistant to noise pollution, the traffic data is cleaned through a clustering algorithm, Gaussian noise is added to the cleaned data samples, and the autoencoder model is used to train the data samples to filter the noise and achieve Classification of traffic data.
It effectively reduces the impact of noise and interference on traffic data, makes it easier for the model to learn the essential characteristics of traffic data, and improves the accuracy and robustness of network traffic classification.
Smart Images
Figure CN119341996A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of network security intrusion detection, and in particular relates to a highly robust network traffic classification method resistant to noise pollution. Background Art
[0002] With the continuous development of information technology, various network attack methods and means have emerged one after another. Intrusion detection technology based on network traffic has played an important role in ensuring network security. With the continuous development of deep learning, more and more studies have applied deep learning technology to network traffic classification technology. Compared with traditional methods, deep learning methods have obvious advantages in extracting deep-level features of data, making it easier to mine the characteristics of the data itself and improve the accuracy of network traffic classification.
[0003] At present, the main difficulty faced by the application of deep learning technology in network traffic classification is that the traffic data noise in the training data set is too large, resulting in low classification accuracy of the trained deep learning model in actual application. The main reasons for the traffic data noise are as follows: first, packet loss and delay in network transmission; second, differences in data encoding formats caused by different equipment models; third, label errors, data redundancy, numerical errors, etc. introduced in the process of traffic data preprocessing.
[0004] Therefore, it is very necessary to study a method that can effectively solve the problem of traffic data noise in order to improve the application effect of deep learning technology in network traffic classification. Summary of the invention
[0005] 1. Technical issues to be resolved
[0006] The technical problem to be solved by the present invention is to provide a method that can effectively solve the traffic data noise problem, so as to improve the application effect of deep learning technology in network traffic classification.
[0007] (II) Technical solution
[0008] In order to solve the above technical problems, the present invention provides a highly robust network traffic classification method resistant to noise pollution, comprising the following steps:
[0009] Step 1: Collect flow data and perform data preprocessing on the flow data;
[0010] Step 2: Use clustering algorithm to clean the preprocessed traffic data:
[0011] Step 3: Add Gaussian noise to the cleaned data samples;
[0012] Step 4: Use the autoencoder model to train the data samples, and use the trained autoencoder model to filter the traffic data noise and classify the traffic data.
[0013] The present invention also provides a system for implementing the method.
[0014] (III) Beneficial effects
[0015] The present invention proposes a highly robust network traffic classification method that is resistant to noise pollution. The method first uses a clustering algorithm to achieve preliminary cleaning of traffic data, then adds Gaussian noise to the traffic data, and then uses the anti-interference ability and anomaly detection ability of the autoencoder model to restore the data without adding noise and classify it, which effectively reduces the impact of noise and interference in the real network environment on the traffic data, makes the model easier to learn the essential characteristics of the traffic data, and improves the accuracy and robustness of network traffic classification. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 is a flow chart of the method of the present invention;
[0017] Figure 2 Schematic diagram of the autoencoder model structure designed for the present invention. DETAILED DESCRIPTION
[0018] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.
[0019] The present invention proposes a highly robust network traffic classification method that is resistant to noise pollution. First, clustering and evaluating the pre-processed traffic data is performed through a clustering algorithm, and isolated points and erroneous data far away from the clustered data set are deleted to achieve the purpose of cleaning the traffic data; secondly, Gaussian noise is added to the cleaned traffic data to simulate the traffic data collected in the real scene, so that more essential features of the data can be learned during subsequent model training; finally, the autoencoder model is used for classification, and the autoencoder is used to encode first and then decode. On the one hand, it can filter the noise and reconstruct the sample data, and at the same time, it also enables the autoencoder model to better learn the relatively more critical features in the attack sample data and extract more distribution features of the sample data. The above method can improve the accuracy of the deep learning algorithm for network traffic classification and reduce the impact of network traffic data noise.
[0020] refer to Figure 1 , the method of the present invention specifically comprises the following steps:
[0021] Step 1, data preprocessing:
[0022] Collect traffic data from the latest public data sets or actual network environments, and preprocess the traffic data: uniformly digitize and normalize the acquired data features to obtain binary features, and generate data in 1×n format;
[0023] Step 2: Use clustering algorithm to clean the preprocessed traffic data:
[0024] The kmeans clustering method is used to cluster and evaluate the traffic data to be classified, and the isolated points and erroneous data far away from the cluster data set are removed; specifically, firstly, the sample set of traffic data is x = {x1, x2, ..., x n}, the number of clusters is k, and then the following process is performed:
[0025] a. Randomly select k samples from the sample set x as the centroid vectors of the cluster. The centroid vector set is represented by {y1,y2,…,y k};
[0026] b. For each cluster class Z m To initialize: m is the cluster label, m = 1, 2, 3, ..., k; calculate the sample x i (1≤i≤n) to each centroid vector y m Distance Determine x based on the nearest centroid vector i The cluster label l i , then x i The cluster class Zl i Updated to: Zl i '=Zl i ∪{x i}; for Z m Recalculate the new centroid vector for all samples within: If all k centroid vectors do not change, the clustering process ends. This step b is repeated n times;
[0027] c. Output clustering result Z = {Z1, Z2, ..., Z k};
[0028] d. According to the distance between the sample and the centroid vector of the cluster, each cluster {Z1, Z2, ..., Z k}, the 5% samples farthest from the centroid vector of the cluster are removed. The 5% threshold can be determined according to the data quality of different traffic data sets. The above steps achieve the purpose of initial sample cleaning, that is, to remove isolated points and erroneous data far away from the cluster data set.
[0029] Step 3: Add Gaussian noise to the cleaned data samples:
[0030] By adding Gaussian noise to the data, we can simulate the noise introduced by network traffic data in the real world during data transmission and data labeling, so that the neural network model can distinguish between noise characteristics and data essential characteristics during training, and thus correctly classify the traffic data, rather than taking the noise characteristics as the essential characteristics of the data and causing misclassification. Specifically, we design Figure 2 The autoencoder model shown in FIG. 1 is a Gaussian noise layer, in which a special Gaussian noise layer is set to add Gaussian noise. In the autoencoder model, the sample x is quantized by the Gaussian noise layer (GaussianNoise layer). i Add Gaussian noise. The autoencoder model, the input layer, the first batch normalization layer, and the output layer form a complete neural network model. The autoencoder model includes an encoder, a decoder, and an output layer. The encoder includes a Gaussian noise layer, a first fully connected layer, a second batch normalization layer, and a first activation layer in sequence; the decoder includes a first random dropout layer, a second fully connected layer, a third fully connected layer, a third batch normalization layer, a second activation layer, a second random dropout layer, and a fourth fully connected layer in sequence.
[0031] Step 4: Use the autoencoder model to train the data samples, and use the trained autoencoder model to filter the traffic data noise and classify the traffic data:
[0032] By using the autoencoder model to first encode and then decode, the data without added noise can be restored and the data can be classified.
[0033] First, input sample x i Normalized by the first batch normalization layer, and then input sample x i is sent to the autoencoder model for training. In the autoencoder model, the sample x is conditioned by a Gaussian noise layer. iAdd Gaussian noise. By adding Gaussian noise, the neural network model can learn more essential features. Then, through the first fully connected layer, the learned "distributed feature representation" is mapped to the sample label space, standardized through the second batch normalization layer (Batch Normalization layer), and activated through the first activation layer (Activation layer) to construct the encoder. The encoder converts the traffic data with added noise from 1×n dimensions to 1×m dimensions, where m << n. In this process, because Gaussian noise is added in this method, the encoder can extract the main features of the data and filter the noise when encoding the traffic data. Subsequently, the encoder is processed through the first and second dropout layers (Dropout layer) to alleviate overfitting and the second fully connected layer. Finally, the processing results are sequentially input into the third fully connected layer, the third batch normalization layer, the second activation layer, the second dropout layer, and the fourth fully connected layer for processing to construct the decoder. The decoder converts the traffic data from 1×m dimensions to 1×n dimensions to restore the data without added noise, thus forming the overall structure of the autoencoder model. Through the above-mentioned method of first encoding and then decoding of the autoencoder model, on the one hand, it can filter the noise in the traffic data, and at the same time, it also enables the autoencoder model to better learn the relatively more critical features in the traffic data, extract the distribution features of more sample data, and realize the classification of the traffic data.
[0034] The above is only the preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the technical principle of the present invention, several improvements and deformations can be made, and these improvements and deformations should also be regarded as the protection scope of the present invention.
Claims
1. A highly robust network traffic classification method resistant to noise pollution, characterized in that: The following steps are involved: Step 1: Collect flow data and perform data preprocessing on the flow data; Step 2: Use clustering algorithm to clean the preprocessed traffic data: Step 3: Add Gaussian noise to the cleaned data samples; Step 4: Use the autoencoder model to train the data samples, and use the trained autoencoder model to filter the traffic data noise and classify the traffic data.
2. The method according to claim 1, characterized in that In step 1, the traffic data is preprocessed as follows: the acquired data features are uniformly digitized and normalized to obtain binary features, and data in 1×n format is generated.
3. The method according to claim 2, characterized in that In step 2, the kmeans clustering method is used to perform clustering evaluation on the traffic data to be classified, and isolated points and erroneous data far away from the cluster data set are eliminated.
4. The method according to claim 3, characterized in that In step 2, first, let the sample set of traffic data be x = {x1, x2, ..., x n }, the number of clusters is k, and then the following process is performed: a. Randomly select k samples from the sample set x as the centroid vectors of the cluster. The centroid vector set is represented by {y1, y2, …, y k }; b. For each cluster class Z m To initialize: m is the cluster label, m = 1, 2, 3, ..., k; Calculate sample x i To each centroid vector y m Distance Determine x based on the nearest centroid vector i The cluster label l i , then x i The cluster class Zl i Updated to: Zl i '=Zl i ∪{x i }; for Z m Recalculate the new centroid vector for all samples within: If all k centroid vectors no longer change, the clustering process ends; c. After step b is repeated n times, the clustering result Z = {Z1, Z2, ..., Z k }; d. According to the distance between the sample and the centroid vector of the cluster, each cluster {Z1, Z2, ..., Z k A certain percentage of samples that are farthest from the cluster centroid vector are removed.
5. The method according to claim 3, characterized in that In step 3, by adding Gaussian noise to the data, the noise introduced by network traffic data in the real world during data transmission and data labeling is simulated, so that the neural network model can distinguish between noise characteristics and data essential characteristics during training, thereby achieving correct classification of traffic data.
6. The method according to claim 4, characterized in that In step 3, an autoencoder model is designed, and a Gaussian noise layer is set in it to add Gaussian noise. In the autoencoder model, the Gaussian noise layer is used to add Gaussian noise to the sample x. i Add Gaussian noise; the autoencoder model and the input layer, the first batch normalization layer, and the output layer form a complete neural network model. The autoencoder model includes an encoder, a decoder, and an output layer, wherein the encoder sequentially includes a Gaussian noise layer, a first fully connected layer, a second batch normalization layer, and a first activation layer; the decoder sequentially includes a first random drop layer, a second fully connected layer, a third fully connected layer, a third batch normalization layer, a second activation layer, a second random drop layer, and a fourth fully connected layer.
7. The method according to claim 6, characterized in that In step 4, the data without noise is restored by first encoding and then decoding through the autoencoder model, and the data is classified.
8. The method according to claim 6, characterized in that In step 4, first, input the sample x i Normalize it through the first batch normalization layer, and then input the sample x i into the autoencoder model for training; in the autoencoder model, add Gaussian noise to the sample x i through the Gaussian noise layer; then map the learned distributed feature representation to the sample label space through the first fully connected layer, normalize it through the second batch normalization layer, activate it through the first activation layer, and convert the traffic data with added noise from 1×n dimensions to 1×m dimensions through the encoder, where m << n; subsequently, the encoder is processed through the first and second dropout layers to alleviate overfitting and through the second fully connected layer, and finally the processed result is successively input into the third fully connected layer, the third batch normalization layer, the second activation layer, the second dropout layer, and the fourth fully connected layer for processing, and the traffic data is converted from 1×m dimensions to 1×n dimensions through the decoder to restore the data without added noise.
9. The method according to claim 4, characterized in that The certain percentage is determined according to the data quality of different traffic data sets.
10. A system for implementing the method according to any one of claims 1 to 9.