An Abnormal Traffic Detection Method for Multi-Scale Network Attacks

Abstract

The present invention discloses an abnormal traffic detection method for multi-scale network attacks, which specifically includes: building a network environment of MSA and collecting MSA data in the real network environment; converting traffic data into a data set composed of csv files to generate a network model data set for training the model; performing numerical and normalization operations on the features in the network model data set, and at the same time using an RF-based feature selection method to perform feature selection on the model data set; the preprocessed model data set is constructed into a graph structure with network traffic as node information and traffic similarity as edge information, and the weight of the connecting edge between nodes is the cosine similarity between traffic; the graph structure is input into an abnormal traffic detection model based on GNN for training to extract potential feature information of network traffic and form a low-dimensional feature vector representation; the low-dimensional feature vector is used as the input data of the Sigmoid classifier to obtain the network intrusion detection result.

Description

Technical Field

[0001] The present invention belongs to the technical field of network security intrusion detection, and particularly relates to an abnormal traffic detection method for multi-scale network attacks. Background Art

[0002] With the booming development of China's Internet industry and the popularization of technologies such as 5G, gigabit optical fiber, and IPv6, network information security has become a key research topic. Among them, the attack detection technology based on abnormal network traffic is an effective means to protect systems and networks from malicious activities. This technology uses deep learning (DL) and data mining methods to analyze the data traffic transmitted in the network, identify the sources and destinations of the traffic, and classify them, thereby improving the visibility of malicious activities in the network and ensuring the security and stability of critical systems.

[0003] However, in the design process of existing abnormal traffic detection methods based on machine learning (ML), most use outdated data sets and adopt traditional learning methods, relying heavily on manual or semi-manual acquisition of statistical features. Moreover, most models cannot solve the problem of few sample data and cannot effectively protect the target server.

[0004] At the same time, the abnormal traffic detection method based on DL is difficult to effectively capture the interaction features between traffic, and is limited by the scale of training data. The accuracy of its results highly depends on the quality of training data and can no longer adapt to the current popular multi-scale network attack (MSA) strategy.

[0005] During the attack process of MSA, the ultimate goal is not to defeat the target server. The traffic data generated during the attack process does not deviate significantly from the normal network access traffic, and no abnormal data volume far exceeding the normal network traffic is generated. MSA hides its attack behavior during the attack process, reduces the amount of transmitted data, and thus hides the attack intention in normal network access behavior, which is a more threatening attack method. Attackers usually simulate normal network access behavior as the starting point of their attack process and achieve the consumption of target server resources through highly concealed, multi-stage, and long-span MSA. Summary of the Invention

[0006] The purpose of the present invention is to provide an abnormal traffic detection method for multi-scale network attacks, which solves the problems existing in the prior art. The technical problems to be solved are as follows:

[0007] 1. Solve the problem of using outdated data sets in the model design process;

[0008] 2. Solve the problems of low training efficiency and high complexity of traditional abnormal traffic detection models;

[0009] 3. Solve the problem of small number of training data samples and insufficient features;

[0010] 4. Solve the problem that learning methods rely on manual or semi-manual acquisition of statistical features.

[0011] In order to achieve the above object, the present invention adopts the following technical solutions:

[0012] A method for detecting abnormal traffic against multi-scale network attacks is implemented in the following steps:

[0013] S1, build the network environment of MSA, collect MSA data in the real network environment, and capture the network traffic data transmitted in the network;

[0014] S2, converts the network traffic data captured in S1 into a data set consisting of csv files, removes redundant zero-value features in the data set, and generates a network model data set for training the model;

[0015] S3, digitizes and normalizes the features in the network model data set of the training model generated by S2, and uses the RF-based feature selection method to select features of the network model data set;

[0016] S4, the network model dataset preprocessed by S3 is constructed into a graph structure with network traffic as node information and traffic similarity as edge information. The weight of the edge connecting the nodes is the cosine similarity between the network traffic. The constructed graph structure is subsequently used to capture the correlation between the nodes, so as to better classify the abnormal network traffic.

[0017] S5, the graph structure constructed by S4 is input into the GNN-based abnormal traffic detection model for training. The GNN-based abnormal traffic detection model has three graph convolutional layers GCN, which extracts the potential feature information of network traffic and updates the node representation to form a low-dimensional feature vector representation as the input of the classifier;

[0018] S6, uses the low-dimensional feature vector as the input data of the Sigmoid classifier to obtain the network intrusion detection results. The abnormal traffic detection model extracts abnormal traffic information and divides the network traffic into normal traffic and abnormal traffic.

[0019] Furthermore, the specific method of step S1 is:

[0020] S1.1. The network environment of the MSA consists of two Ubuntu 20.04 nodes acting as attacker nodes and a victim server, and one Windows 10 node acting as a normal user to access the victim server;

[0021] S1.2. During the attack process, the attack node adjusts the attack scale according to the access frequency of the normal user, so that the attack behavior is hidden in the network access behavior of the normal user. At the same time, the TCPDump application captures the network traffic of the communication between the victim server and other hosts, and dumps the captured network traffic into a pcap file.

[0022] Furthermore, the specific approach for step S2 is as follows:

[0023] S2.1. Convert the pcap file dumped in step S1.2. Use the CIC FlowMeter tool to convert the original pcap file into a csv file with network traffic as the basic unit. The csv file stores the characteristics of each network traffic, including ID, source IP, destination IP, source port, and destination port information;

[0024] S2.2. Delete the traffic rows in the csv file that have nan values and all-zero values, reduce the scale of the dataset, eliminate invalid data, and generate a network traffic dataset.

[0025] Furthermore, the specific approach for step S3 is as follows:

[0026] S3.1. Further reduce the network traffic dataset generated in S2.2. First, for symbolic features, use one-hot encoding to map symbolic features to numerical features to ensure that numerical features are used as input data during model training;

[0027] S3.2. Normalize the numerical features. The normalization process uses the Z-score numerical feature normalization method, and its standardization is:

[0028]

[0029] where μ is the sample mean, σ is the sample standard deviation, and x is the original feature data. This method adjusts the mean of the sample to 0 and the standard deviation to 1, centralizes and standardizes the data, and helps the detection model better learn the distribution of the data;

[0030] S3.3. To address the problems of low training efficiency and high complexity in traditional abnormal traffic detection models, a feature selection method based on RF is proposed. This method takes the original dataset as input and assigns an importance score to each feature based on the accuracy of the training results. The feature with the highest score is the one that contributes the most to the classification model. RF consists of several decision trees, and each decision tree randomly selects several features as training features during the training process. Therefore, the accuracy of each decision tree's training results affects the importance of the features. After feature selection, the 78 features in the model dataset are reduced to 18 features, and these 18 features will be used as the input for the abnormal traffic detection model based on GNN.

[0031] Further, the specific approach for step S4 is as follows:

[0032] After preprocessing in S3, the number of features of the traffic in the dataset is 18 for each. Therefore, in the present invention, the features of each network traffic are used as node features. Then, edge features between nodes are constructed based on the similarity of two nodes, and the cosine similarity between two nodes is used as the edge weight. The definition of cosine similarity is

[0033]

[0034] where v represents the node vector v, and w represents the vector w. is the i-th feature of vector v. is the i-th feature of vector w, and n represents the number of features.

[0035] Further, the specific approach for step S5 is as follows: The abnormal traffic detection model based on GNN includes three GCN layers, each with a different number of layers. The input dimension of the first layer is 18, and the output dimension is 128; the input dimension of the second layer is 128, and the output dimension is 64; the input dimension of the third layer is 64, and the output dimension is 32.

[0036] Further, the specific approach for step S6 is as follows: After feature extraction in S5, the low-dimensional feature vector is used as the classification basis, and the network traffic is classified into normal traffic and abnormal traffic through a Sigmoid classifier.

[0037] A computer terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements an abnormal traffic detection method for multi-scale network attacks as described above.

[0038] A computer-readable medium stores a computer program, and when the computer program is executed by a processor, it can implement an abnormal traffic detection method for multi-scale network attacks as described above.

[0039] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0040] 1. Aiming at the problem of using outdated data sets in the model design process, the present invention builds a local area network environment and collects MSA data in the real network environment;

[0041] 2. Aiming at the problems of low training efficiency and high complexity of traditional abnormal traffic detection models, a feature selection method based on RF is proposed;

[0042] 3. Aiming at the problems of small number of training data samples and insufficient features, the present invention designs a graph structure with network traffic as node features and traffic similarity as edge features to extract deeper potential information from limited training data;

[0043] 4. Aiming at the problem that the learning method relies on manual or semi-manual acquisition of statistical features, the present invention designs an abnormal traffic detection model based on GNN. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a schematic flow chart of an abnormal traffic detection method for multi-scale network attacks according to the present invention;

[0045] Figure 2 is a schematic diagram of the experimental network environment of an abnormal traffic detection method for multi-scale network attacks according to the present invention;

[0046] Figure 3 is a schematic diagram of the network model structure of an abnormal traffic detection method for multi-scale network attacks according to the present invention;

[0047] Figure 4 is a schematic diagram of the detection result of an abnormal traffic detection method for multi-scale network attacks deployed in the actual network according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0048] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0049] As Figure 1 shown, an abnormal traffic detection method for multi-scale network attacks is specifically implemented according to the following steps:

[0050] S1. Build the network environment of MSA and collect MSA data in the real network environment. This network environment consists of two Ubuntu 20.04 nodes and one Windows 10 node, and captures the network traffic data in this network through the TCPDump application;

[0051] S1.1. The experimental environment is as Figure 2As shown in the figure, this network environment consists of two Ubuntu 20.04 nodes acting as attacker nodes and a victim server, and one Windows 10 node acting as a normal user to access the victim server.

[0052] S1.2 During the attack, the attack node adjusts the attack scale according to the access frequency of normal users, so that the attack behavior is hidden in the network access behavior of normal users. At the same time, the TCPDump application captures the network traffic of the communication between the victim server and other hosts, and dumps the captured network traffic into a pcap file.

[0053] S2 Convert the network traffic data captured in S1 into a dataset composed of csv files, remove redundant zero-value features in the dataset, and generate a network traffic dataset for training the model;

[0054] S2.1 Convert the pcap file dumped in step S1.2. Use the CIC FlowMeter tool to convert the original pcap file into a csv file with network traffic as the basic unit. The csv file stores the features of each network traffic, including information such as ID, source IP, destination IP, source port, and destination port.

[0055] S2.2 Delete the traffic rows with nan values and all-zero values in the csv file, reduce the scale of the dataset, remove invalid data, and generate a network traffic dataset.

[0056] S3 Numerically normalize the features in the network model dataset generated in S2, and at the same time use the RF-based feature selection method to perform feature selection on the network model dataset;

[0057] S3.1 Further reduce the network traffic dataset generated in S2.2. First, for symbolic features, use one-hot encoding to map symbolic features to numerical features to ensure that numerical features are used as input data during model training;

[0058] S3.2 Normalize the numerical features. The normalization process uses the Z-score numerical feature normalization method, and its standardization is as follows:

[0059]

[0060] where μ is the sample mean, σ is the sample standard deviation, and x is the original feature data. This method adjusts the mean of the sample to 0 and the standard deviation to 1, centralizes and standardizes the data, and helps the detection model better learn the distribution of the data;

[0061] S3.3. To address the problems of low training efficiency and high complexity in traditional abnormal traffic detection models, a feature selection method based on RF is proposed. This method takes the original dataset as input and assigns importance scores to each feature based on the accuracy of the training results. The feature with the highest score is the one that contributes the most to the classification model. RF consists of several decision trees, and each decision tree randomly selects several features as training features during the training process. Therefore, the accuracy of the training results of each decision tree affects the importance of the features. After feature selection, the 78 original features in the model dataset are reduced to 18 features (Flow Duration, Total Fwd Packet, Fwd IAT Total, Fwd IAT Std, Bwd URGFlags, Bwd Packets / s, Packet Length Min, Packet Length Std, SYN Flag Count, RSTFlag Count, PSH Flag Count, CWR Flag Count, ECE Flag Count, Fwd Segment Size Avg, Bwd Bulk Rate Avg, Subflow Bwd Packets, Subflow Bwd Bytes, FWD Init Win Bytes), and these 18 features will be used as the input of the abnormal traffic detection model based on GNN.

[0062] S4. The model dataset preprocessed in S3 is constructed into a graph structure with network traffic as node information and node similarity as edge information. The edges connecting nodes are the cosine similarity between traffic.

[0063] After preprocessing, the number of features of the traffic in the dataset is 18. Therefore, in the present invention, the features of each network traffic are used as node features. Then, edge features between nodes are constructed for the similarity between two nodes, and the cosine similarity between two nodes is used as the edge weight. The definition of cosine similarity is

[0064]

[0065] where v represents the node vector v, and w represents the vector w. is the i-th feature of vector v. is the i-th feature of vector w, and n represents the number of features.

[0066] S5. The constructed graph structure is input into the GNN-based abnormal traffic detection model for training. The GNN-based abnormal traffic detection model has three GCN layers, which are used to extract the potential feature information of network traffic, update the node representation, and form a low-dimensional feature vector representation as the input of the classifier.

[0067] The GNN-based abnormal traffic detection model is as Figure 3 shown. This model contains three GCN layers, each layer having a different number of layers. The input dimension of the first layer is 18, and the output dimension is 128; the input dimension of the second layer is 128, and the output dimension is 64; the input dimension of the third layer is 64, and the output dimension is 32.

[0068] The GNN-based abnormal traffic detection model uses the Relu (Rectified Linear Unit) activation function to solve the gradient vanishing problem, improve the calculation efficiency, and increase the sparsity. The ReLU activation function is simple and effective, which helps to solve some key problems in neural networks, such as the gradient vanishing problem.

[0069] S6. After the feature extraction in S5, the extracted potential features are used as the classification basis, and the low-dimensional feature vector is used as the input data of the Sigmoid classifier to obtain the network intrusion detection result. The abnormal traffic detection model classifies network traffic into normal traffic and abnormal traffic by extracting abnormal traffic information.

[0070] This embodiment also provides a computer terminal, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements an abnormal traffic detection method for multi-scale network attacks as described above.

[0071] This embodiment also provides a computer-readable medium, on which a computer program is stored. When the computer program is executed by the processor, it can implement an abnormal traffic detection method for multi-scale network attacks as described above.

[0072] The present invention combines the experimental environment for collecting network traffic and deploys the generated network model in the actual test environment. The model is deployed on the victim server. After deployment, normal user nodes are used to generate the network access behaviors of normal users, and an attacker initiates an MSA. The abnormal traffic detection model is deployed on the victim server to intercept the attacks initiated by the attacker and filter the intercepted data. The filtered network traffic is compared with the source IP and destination IP in the captured data packets to obtain the accuracy, precision, and recall rate of the abnormal traffic detection model. The present invention initiates three attacks within a 10-minute capture cycle and displays the detection results in the form of a bar chart, as Figure 4 shown.

[0073] After deploying the present invention in a test network environment, the detection rate of network attacks has been greatly improved. Three network attacks were launched in the network model generated by deployment in the actual test environment, and the defense of the model against the three attacks achieved very good results. The accuracy rates of the detection of the three abnormal network traffic were all higher than 96.7%, proving the effectiveness of the present invention against MSA.

[0074] Although the above-described illustrative specific embodiments of the present invention have been described to facilitate the understanding of the present invention by those skilled in the art, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention defined and determined by the appended claims, these changes are obvious, and all inventions made using the inventive concept of the present invention are within the scope of protection.

Claims

1. An abnormal traffic detection method for multi-scale network attacks, characterized in that The specific implementation steps are as follows: S1. Build the network environment of MSA, collect MSA data in the real network environment, and capture the network traffic data transmitted in this network; S2. Convert the network traffic data captured in S1 into a dataset composed of csv files, remove the redundant zero-value features in the dataset, and generate a network model dataset for training the model; S3. Perform numerical and normalization operations on the features in the network model dataset generated in S2. At the same time, use the RF-based feature selection method to perform feature selection on the network model dataset; S4. The network model dataset preprocessed in S3 is constructed into a graph structure with network traffic as node information and traffic similarity as edge information. The weight of the connection edge between nodes is the cosine similarity between network traffics; S5. The graph structure constructed in S4 is input into the anomaly traffic detection model based on GNN for training. The anomaly traffic detection model based on GNN has three graph convolutional layers GCN, extracts the potential feature information of network traffic, and updates the node representation to form a low-dimensional feature vector representation as the input of the classifier; S6. Use the low-dimensional feature vector as the input data of the Sigmoid classifier to obtain the network intrusion detection result. The anomaly traffic detection model classifies network traffic into normal traffic and abnormal traffic by extracting abnormal traffic information.

2. The abnormal traffic detection method for multi-scale network attacks according to claim 1, wherein, The specific approach for step S1 is as follows: S1.

1. The network environment of MSA consists of two Ubuntu 20.04 nodes acting as attacker nodes and a victim server, and one Windows 10 node acting as a normal user accessing the victim server; S1.

2. During the attack process, the attack node adjusts the attack scale according to the access frequency of normal users, so that the attack behavior is hidden in the network access behavior of normal users. At the same time, the TCPDump application captures the network traffic of the communication between the victim server and other hosts, and dumps the captured network traffic into a pcap file.

3. An abnormal traffic detection method for multi-scale network attacks according to claim 2, characterized in that The specific approach for step S2 is as follows: S2.

1. Convert the pcap file dumped in step S1.

2. Use the CIC FlowMeter tool to convert the original pcap file into a csv file with network traffic as the basic unit. The csv file stores the features of each network traffic, including ID, source IP, destination IP, source port, and destination port information; S2.

2. Delete the traffic rows with nan values and all-zero values in the csv file, reduce the scale of the dataset, remove invalid data, and generate a network traffic dataset.

4. An abnormal traffic detection method for multi-scale network attacks according to claim 3, characterized in that, The specific approach for step S3 is as follows: S3.

1. Further reduce the network traffic dataset generated in S2.

2. First, for symbolic features, use one-hot encoding to map symbolic features to numerical features to ensure that numerical features are used as input data during model training; S3.

2. Normalize the numerical features. The normalization process uses the Z-score numerical feature normalization method, and its standardization is: Among them, μ is the sample mean, σ is the sample standard deviation, and x is the original feature data. The mean of the sample is adjusted to 0, and the standard deviation is adjusted to 1 to centralize and standardize the data; S3.

3. The RF-based feature selection method is adopted. This method takes the original data set as the input and assigns an importance score to each feature based on the accuracy of the training result. The feature with the highest score is the one that contributes the most to the classification model. RF consists of several decision trees. Each decision tree randomly selects several features as training features during the training process. Therefore, the accuracy of each decision tree's training result affects the importance of the features. After feature selection, the 78 features in the model data set are reduced to 18 features, and these 18 features will be used as the input of the GNN-based abnormal traffic detection model.

5. The abnormal traffic detection method for multi-scale network attacks according to claim 4, characterized in that The specific implementation of step S4 is as follows: After the preprocessing in S3, the number of features of the traffic in the data set is 18. Therefore, the features of each network traffic are used as node features. Then, the edge features between nodes are constructed based on the similarity of two nodes, and the cosine similarity between two nodes is used as the edge weight. The definition of cosine similarity is Among them, v represents the node vector v, and w represents the vector w. is the i-th feature of the vector v. is the i-th feature of the vector w, and n represents the number of features.

6. An abnormal traffic detection method for multi-scale network attacks according to claim 5, characterized in that, The specific implementation of step S5 is as follows: The GNN-based abnormal traffic detection model contains three GCN layers, each layer has a different number of layers. The input dimension of the first layer is 18, and the output dimension is 128; the input dimension of the second layer is 128, and the output dimension is 64; the input dimension of the third layer is 64, and the output dimension is 32.

7. An abnormal traffic detection method for multi-scale network attacks according to claim 5, characterized in that, The specific implementation of step S6 is as follows: After the feature extraction in S5, the low-dimensional feature vector is used as the classification basis, and the network traffic is classified into normal traffic and abnormal traffic through a Sigmoid classifier.

8. A computer terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements an abnormal traffic detection method for multi-scale network attacks as described in any one of claims 1-7.

9. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it can implement an abnormal traffic detection method for multi-scale network attacks as described in any one of claims 1-7.

Images