Abnormal traffic detection method for multi-scale network attack

Abstract

The invention discloses an abnormal traffic detection method for multi-scale network attacks, and the method specifically comprises the steps: building a network environment of an MSA, and collecting MSA data in a real network environment; converting the traffic data into a data set composed of csv files, and generating a network model data set of a training model; carrying out numeralization and normalization operation on features in the network model data set, and carrying out feature selection on the model data set by using an RF-based feature selection method; the preprocessed model data set is constructed into a graph structure which takes network traffic as node information and traffic similarity as side information, and the weight of a connecting side between nodes is cosine similarity between the traffic; inputting the graph structure into a GNN-based abnormal traffic detection model for training, extracting potential feature information of network traffic, and forming low-dimensional feature vector representation; and taking the low-dimensional feature vector as input data of a Sigmoid classifier to obtain a network intrusion detection result.

Description

Technical Field

[0001] The present invention belongs to the technical field of network security intrusion detection, and in particular relates to an abnormal traffic detection method for multi-scale network attacks. Background Art

[0002] With the vigorous development of my country's Internet industry and the popularization of technologies such as 5G, Gigabit fiber, and IPv6, network information security has become a key research topic. Among them, attack detection technology based on abnormal network traffic is an effective means to protect systems and networks from malicious activities. This technology uses deep learning (DL) and data mining methods to analyze the data traffic transmitted in the network, identify the source and destination of the traffic, and classify it, thereby improving the visibility of malicious activities in the network and ensuring the security and stability of key systems.

[0003] However, most of the existing abnormal traffic detection methods based on machine learning (ML) use outdated data sets in their design process, adopt traditional learning methods, and rely heavily on manual or semi-manual acquisition of statistical features. In addition, most models cannot solve the problem of small sample data and cannot effectively protect the target server.

[0004] At the same time, DL-based abnormal traffic detection methods are difficult to effectively capture the interactive characteristics between traffic, and are limited by the scale of training data. The accuracy of their results is highly dependent on the quality of training data and can no longer adapt to the current popular multi-scale network attack (MSA) strategy.

[0005] The ultimate goal of MSA attack is not to bring down the target server. The traffic data generated during the attack will not deviate too much from the normal network access traffic, and will not generate abnormal data that far exceeds the normal network traffic. MSA hides its attack behavior during the attack and reduces the amount of data transmitted, thereby hiding the attack intention in normal network access behavior. It is a more threatening attack method. Attackers usually simulate normal network access behavior as the starting point of their attack process, and use highly concealed, multi-stage, and long-span MSA to consume the resources of the target server. Summary of the invention

[0006] The purpose of the present invention is to provide an abnormal traffic detection method for multi-scale network attacks, which solves the problems existing in the prior art. The technical problems to be solved are as follows:

[0007] 1. Solve the problem of using outdated data sets in the model design process;

[0008] 2. Solve the problems of low training efficiency and high complexity of traditional abnormal traffic detection models;

[0009] 3. Solve the problem of small number of training data samples and insufficient features;

[0010] 4. Solve the problem that learning methods rely on manual or semi-manual acquisition of statistical features.

[0011] In order to achieve the above object, the present invention adopts the following technical solutions:

[0012] A method for detecting abnormal traffic against multi-scale network attacks is implemented in the following steps:

[0013] S1, build the network environment of MSA, collect MSA data in the real network environment, and capture the network traffic data transmitted in the network;

[0014] S2, converts the network traffic data captured in S1 into a data set consisting of csv files, removes redundant zero-value features in the data set, and generates a network model data set for training the model;

[0015] S3, digitizes and normalizes the features in the network model data set of the training model generated by S2, and uses the RF-based feature selection method to select features of the network model data set;

[0016] S4, the network model dataset preprocessed by S3 is constructed into a graph structure with network traffic as node information and traffic similarity as edge information. The weight of the edge connecting the nodes is the cosine similarity between the network traffic. The constructed graph structure is subsequently used to capture the correlation between the nodes, so as to better classify the abnormal network traffic.

[0017] S5, the graph structure constructed by S4 is input into the GNN-based abnormal traffic detection model for training. The GNN-based abnormal traffic detection model has three graph convolutional layers GCN, which extracts the potential feature information of network traffic and updates the node representation to form a low-dimensional feature vector representation as the input of the classifier;

[0018] S6, uses the low-dimensional feature vector as the input data of the Sigmoid classifier to obtain the network intrusion detection results. The abnormal traffic detection model extracts abnormal traffic information and divides the network traffic into normal traffic and abnormal traffic.

[0019] Furthermore, the specific method of step S1 is:

[0020] S1.1, the network environment of MSA consists of two Ubuntu 20.04 nodes acting as the attacker node and the victim server, and one Windows 10 node acting as a normal user to access the victim server;

[0021] S1.2, during the attack, the attack node adjusts the attack scale according to the access frequency of normal users, so that the attack behavior is hidden in the network access behavior of normal users. At the same time, the TCPDump application captures the network traffic between the victim server and other hosts, and dumps the captured network traffic into a pcap file.

[0022] Furthermore, the specific method of step S2 is:

[0023] S2.1, convert the pcap file dumped in step S1.2, and use the CIC FlowMeter tool to convert the original pcap file into a csv file with network traffic as the basic unit. The csv file stores the characteristics of each network traffic, including ID, source IP, destination IP, source port, and destination port information;

[0024] S2.2, delete the traffic rows with nan values ​​and all zero values ​​in the csv file, reduce the size of the data set, eliminate invalid data, and generate a network traffic data set.

[0025] Furthermore, the specific method of step S3 is:

[0026] S3.1, further reduce the network traffic data set generated by S2.2. First, for the symbolic features, use one-hot encoding to map the symbolic features to numerical features to ensure that the numerical features are used as input data during model training;

[0027] S3.2, normalize the numerical features. The normalization process uses the Z-score numerical feature normalization method, which is standardized as follows:

[0028]

[0029] Among them, μ is the sample mean, σ is the sample standard deviation, and x is the original feature data. This method adjusts the sample mean to 0 and the standard deviation to 1, making the data centralized and standardized, which helps the detection model to better learn the distribution of data;

[0030] S3.3, in order to solve the problems of low training efficiency and high complexity of traditional abnormal traffic detection models, a feature selection method based on RF is proposed. This method takes the original data set as input and assigns an importance score to each feature based on the accuracy of the training results. The feature with the highest score is the feature that contributes the most to the classification model. RF is composed of several decision trees, and each decision tree randomly selects several features as training features during the training process. Therefore, the accuracy of the training results of each decision tree affects the importance of the feature. After feature selection, the 78 features in the model data set are reduced to 18 features, and these 18 features will be used as the input of the GNN-based abnormal traffic detection model.

[0031] Furthermore, the specific method of step S4 is:

[0032] After S3 preprocessing, the number of traffic features in the data set is 18. Therefore, the present invention uses the feature of each network traffic as the node feature, and then constructs the edge feature between the nodes based on the similarity of the two nodes, and uses the cosine similarity between the two nodes as the edge weight. The definition of cosine similarity is:

[0033]

[0034] Among them, v represents the node vector v, w represents the vector w, is the i-th feature of vector v, is the i-th feature of vector w, and n represents the number of features.

[0035] Furthermore, the specific method of step S5 is: the GNN-based abnormal traffic detection model includes three GCN layers, each layer has a different number of layers, the input dimension of the first layer is 18, and the output dimension is 128; the input dimension of the second layer is 128, and the output dimension is 64; the input dimension of the third layer is 64, and the output dimension is 32.

[0036] Furthermore, the specific method of step S6 is: after the feature extraction in S5, the low-dimensional feature vector is used as the classification basis, and the network traffic is divided into normal traffic and abnormal traffic through the Sigmoid classifier.

[0037] A computer terminal comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, an abnormal traffic detection method for multi-scale network attacks as described above is implemented.

[0038] A computer-readable medium stores a computer program, which is executed by a processor to implement the above-mentioned abnormal traffic detection method for multi-scale network attacks.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1. Aiming at the problem of using outdated data sets in the model design process, the present invention builds a local area network environment to collect MSA data in the real network environment;

[0041] 2. Aiming at the problems of low training efficiency and high complexity of traditional abnormal traffic detection models, a feature selection method based on RF is proposed;

[0042] 3. To address the problem of small number of training data samples and insufficient features, the present invention designs a graph structure with network traffic as node features and traffic similarity as edge features to extract deeper potential information from limited training data;

[0043] 4. To address the problem that learning methods rely on manual or semi-manual acquisition of statistical features, the present invention designs an abnormal traffic detection model based on GNN. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flow chart of an abnormal traffic detection method for multi-scale network attacks of the present invention;

[0045] Figure 2 It is a schematic diagram of an experimental network environment of an abnormal traffic detection method for multi-scale network attacks of the present invention;

[0046] Figure 3 It is a schematic diagram of the network model structure of an abnormal traffic detection method for multi-scale network attacks of the present invention;

[0047] Figure 4 It is a schematic diagram of detection results of an abnormal traffic detection method for multi-scale network attacks deployed in an actual network according to the present invention. DETAILED DESCRIPTION

[0048] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0049] like Figure 1 As shown, a method for detecting abnormal traffic for multi-scale network attacks is implemented in the following steps:

[0050] S1, build the MSA network environment and collect MSA data in the real network environment. The network environment consists of two Ubuntu 20.04 nodes and one Windows 10 node, and the network traffic data in the network is captured through the TCPDump application;

[0051] S1.1, experimental environment Figure 2As shown in the figure, the network environment consists of two Ubuntu 20.04 nodes acting as attacker nodes and victim servers, and one Windows 10 node acting as a normal user to access the victim server.

[0052] S1.2,During the attack process, the attack node adjusts the attack scale according to the access frequency of normal users, so that the attack behavior is hidden in the network access behavior of normal users. At the same time, the TCPDump application captures the network traffic between the victim server and other hosts, and dumps the captured network traffic into a pcap file.

[0053] S2, converts the network traffic data captured in S1 into a data set consisting of csv files, removes redundant zero-value features in the data set, and generates a network traffic data set for training the model;

[0054] S2.1, convert the pcap file dumped in step S1.2, and use the CIC FlowMeter tool to convert the original pcap file into a csv file with network traffic as the basic unit. The csv file stores the characteristics of each network traffic, including ID, source IP, destination IP, source port, destination port and other information.

[0055] S2.2, delete the traffic rows with nan values ​​and all zero values ​​in the csv file, reduce the size of the data set, eliminate invalid data, and generate a network traffic data set.

[0056] S3, digitizes and normalizes the features in the network model data set of the training model generated by S2, and uses the RF-based feature selection method to select features of the network model data set;

[0057] S3.1, further reduce the network traffic data set generated by S2.2. First, for the symbolic features, use one-hot encoding to map the symbolic features to numerical features to ensure that the numerical features are used as input data during model training;

[0058] S3.2, normalize the numerical features. The normalization process uses the Z-score numerical feature normalization method, which is standardized as follows:

[0059]

[0060] Among them, μ is the sample mean, σ is the sample standard deviation, and x is the original feature data. This method adjusts the sample mean to 0 and the standard deviation to 1, making the data centralized and standardized, which helps the detection model to better learn the distribution of data;

[0061] S3.3, in order to solve the problems of low training efficiency and high complexity of traditional abnormal traffic detection models, a feature selection method based on RF is proposed. This method takes the original data set as input and assigns an importance score to each feature according to the accuracy of the training results. The feature with the highest score is the feature that contributes the most to the classification model. RF is composed of several decision trees. Each decision tree randomly selects several features as training features during the training process. Therefore, the accuracy of the training results of each decision tree affects the importance of the feature. After feature selection, the 78 original features in the model data set are reduced to 18 features (Flow Duration, Total Fwd Packet, Fwd IAT Total, Fwd IAT Std, Bwd URGFlags, Bwd Packets / s, Packet Length Min, Packet Length Std, SYN Flag Count, RSTFlag Count, PSH Flag Count, CWR Flag Count, ECE Flag Count, Fwd Segment Size Avg, Bwd Bulk Rate Avg, Subflow Bwd Packets, Subflow Bwd Bytes, FWD Init Win Bytes), and these 18 features will be used as the input of the GNN-based abnormal traffic detection model.

[0062] S4, the model dataset after S3 preprocessing is constructed into a graph structure with network traffic as node information, node similarity as edge information, and the edge connecting the nodes is the cosine similarity between traffic;

[0063] After preprocessing, the number of traffic features in the data set is 18. Therefore, the present invention uses the feature of each network traffic as the node feature, and then constructs the edge feature between the nodes based on the similarity of the two nodes, and uses the cosine similarity between the two nodes as the edge weight. The definition of cosine similarity is,

[0064]

[0065] Among them, v represents the node vector v, w represents the vector w, is the i-th feature of vector v, is the i-th feature of vector w, and n represents the number of features.

[0066] S5, the constructed graph structure is input into the GNN-based abnormal traffic detection model for training. The GNN-based abnormal traffic detection model has three GCN layers, which are used to extract the potential feature information of network traffic and update the node representation to form a low-dimensional feature vector representation as the input of the classifier;

[0067] The abnormal traffic detection model based on GNN is as follows Figure 3 As shown in the figure, the model contains three GCN layers, each with a different number of layers. The input dimension of the first layer is 18 and the output dimension is 128; the input dimension of the second layer is 128 and the output dimension is 64; the input dimension of the third layer is 64 and the output dimension is 32;

[0068] The GNN-based abnormal traffic detection model uses the Relu (Rectified Linear Unit) activation function to solve the gradient vanishing problem, while improving computational efficiency and increasing sparsity. The ReLU activation function is simple and effective in design, and helps solve some key problems in neural networks, such as the gradient vanishing problem.

[0069] S6, after feature extraction in S5, the extracted potential features are used as the basis for classification, and the low-dimensional feature vector is used as the input data of the Sigmoid classifier to obtain the network intrusion detection results. The abnormal traffic detection model extracts abnormal traffic information and divides the network traffic into normal traffic and abnormal traffic.

[0070] This embodiment also provides a computer terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for detecting abnormal traffic against multi-scale network attacks as described above is implemented.

[0071] This embodiment also provides a computer-readable medium on which a computer program is stored. The computer program is executed by a processor to implement the above-mentioned abnormal traffic detection method for multi-scale network attacks.

[0072] The present invention combines the experimental environment for collecting network traffic and deploys the generated network model in the actual test environment. The model is deployed on the victim server. After deployment, normal user nodes are used to generate normal users' network access behaviors, and the attacker is used to initiate MSA; an abnormal traffic detection model is deployed on the victim server to intercept the attack initiated by the attacker and filter the intercepted data. The filtered network traffic is compared with the source IP and destination IP in the captured data packet to obtain the accuracy, precision, and recall of the abnormal traffic detection model. The present invention launches three attacks within a 10-minute capture period and displays the detection results in the form of a bar graph, as shown in the figure. Figure 4 shown.

[0073] After the present invention is deployed in the test network environment, the detection rate of network attacks is greatly improved. Three network attacks are launched in the generated network model deployed in the actual test environment. The model's defense against the three attacks has achieved good results. The accuracy of the three abnormal network traffic detections is higher than 96.7%, proving the effectiveness of the present invention for MSA.

[0074] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A method for detecting abnormal traffic against multi-scale network attacks, characterized in that: Follow the steps below to implement it: S1, build the network environment of MSA, collect MSA data in the real network environment, and capture the network traffic data transmitted in the network; S2, converts the network traffic data captured in S1 into a data set consisting of csv files, removes redundant zero-value features in the data set, and generates a network model data set for training the model; S3, digitizes and normalizes the features in the network model data set of the training model generated by S2, and uses the RF-based feature selection method to select features of the network model data set; S4, the network model dataset after S3 preprocessing is constructed into a graph structure with network traffic as node information and traffic similarity as edge information. The weight of the edge connecting the nodes is the cosine similarity between the network traffic; S5, the graph structure constructed by S4 is input into the GNN-based abnormal traffic detection model for training. The GNN-based abnormal traffic detection model has three graph convolutional layers GCN, which extracts the potential feature information of network traffic and updates the node representation to form a low-dimensional feature vector representation as the input of the classifier; S6, uses the low-dimensional feature vector as the input data of the Sigmoid classifier to obtain the network intrusion detection results. The abnormal traffic detection model extracts abnormal traffic information and divides the network traffic into normal traffic and abnormal traffic.

2. According to claim 1, a method for detecting abnormal traffic against multi-scale network attacks is characterized in that: The specific steps of step S1 are: S1.1, the network environment of MSA consists of two Ubuntu 20.04 nodes acting as the attacker node and the victim server, and one Windows 10 node acting as a normal user to access the victim server; S1.2, during the attack, the attack node adjusts the attack scale according to the access frequency of normal users, so that the attack behavior is hidden in the network access behavior of normal users. At the same time, the TCPDump application captures the network traffic between the victim server and other hosts, and dumps the captured network traffic into a pcap file.

3. The method for detecting abnormal traffic against multi-scale network attacks according to claim 2, characterized in that: The specific steps of step S2 are: S2.1, convert the pcap file dumped in step S1.2, and use the CIC FlowMeter tool to convert the original pcap file into a csv file with network traffic as the basic unit. The csv file stores the characteristics of each network traffic, including ID, source IP, destination IP, source port, and destination port information; S2.2, delete the traffic rows with nan values ​​and all zero values ​​in the csv file, reduce the size of the data set, eliminate invalid data, and generate a network traffic data set.

4. The method for detecting abnormal traffic against multi-scale network attacks according to claim 3 is characterized in that: The specific steps of step S3 are: S3.1, further reduce the network traffic data set generated by S2.

2. First, for the symbolic features, use one-hot encoding to map the symbolic features to numerical features to ensure that the numerical features are used as input data during model training; S3.2, normalize the numerical features. The normalization process uses the Z-score numerical feature normalization method, which is standardized as follows: Among them, μ is the sample mean, σ is the sample standard deviation, and x is the original feature data. The sample mean is adjusted to 0 and the standard deviation is adjusted to 1 to make the data centralized and standardized; S3.3, adopts the RF-based feature selection method, which takes the original data set as input and assigns an importance score to each feature through the accuracy of the training results. The feature with the highest score is the feature that contributes the most to the classification model; RF is composed of several decision trees, and each decision tree randomly selects several features as training features during the training process. Therefore, the accuracy of the training results of each decision tree affects the importance of the feature. After feature selection, the 78 features in the model data set are reduced to 18 features, and these 18 features will be used as the input of the GNN-based abnormal traffic detection model.

5. The method for detecting abnormal traffic against multi-scale network attacks according to claim 4 is characterized in that: The specific steps of step S4 are: After S3 preprocessing, the number of traffic features in the data set is 18. Therefore, the feature of each network traffic is used as the node feature. Then, the edge feature between the nodes is constructed based on the similarity of the two nodes, and the cosine similarity between the two nodes is used as the edge weight. The definition of cosine similarity is: Among them, v represents the node vector v, w represents the vector w, is the i-th feature of vector v, is the i-th feature of vector w, and n represents the number of features.

6. The method for detecting abnormal traffic against multi-scale network attacks according to claim 5, characterized in that: The specific method of step S5 is: the GNN-based abnormal traffic detection model contains three GCN layers, each layer has a different number of layers, the input dimension of the first layer is 18, and the output dimension is 128; the input dimension of the second layer is 128, and the output dimension is 64; the input dimension of the third layer is 64, and the output dimension is 32.

7. The method for detecting abnormal traffic against multi-scale network attacks according to claim 5, characterized in that: The specific method of step S6 is: after the feature extraction in S5, the low-dimensional feature vector is used as the classification basis, and the network traffic is divided into normal traffic and abnormal traffic through the Sigmoid classifier.

8. A computer terminal comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, an abnormal traffic detection method for multi-scale network attacks as described in any one of claims 1-7 is implemented.

9. A computer readable medium having a computer program stored thereon, characterized in that: The computer program is executed by a processor to implement an abnormal traffic detection method for multi-scale network attacks as described in any one of claims 1-7.

Images