Multi-task intelligent detection method for typical network threat traffic

Abstract

The application discloses a kind of multi-task intelligent detection methods for typical network threat flow, which comprises: for typical network threat flow, flow is sampled with fixed time period, and the flow space scale feature, time scale feature, host level feature in specific time window are extracted to form the feature set of flow multi-scale;Based on the multi-scale feature of typical network threat flow, a general multi-scale heterogeneous graph representation is constructed, and the graph representation is mapped to a representation matrix through an adjacency matrix;Finally, based on the multi-scale heterogeneous graph representation, the multi-task learning model based on self-attention mechanism is used to detect the typical network threat flow.The application can well detect the typical network threat flow of routing platform by deeply mining the multi-scale features of typical network threat flow and combining the current mainstream multi-task learning model, which is of great significance for maintaining network space security and ensuring network stability.

Description

Technical Field

[0001] This invention belongs to the field of network security technology, specifically a multi-task intelligent detection method for typical network threat traffic. Background Technology

[0002] With the rapid development of the Internet, the number of Internet users and the Internet penetration rate in my country have continued to rise. While the rapid development of network technology has promoted the country's digital transformation, with the increasing number of network devices connected to homes, enterprises and industrial settings, typical cybersecurity threats, represented by DDoS attacks, C&C attacks and ransomware, have become increasingly serious.

[0003] However, related work has demonstrated that while traditional methods based on statistical analysis and machine learning can detect some cyberattacks, the continuous development of cyberattack techniques and the proliferation of attack methods make it difficult to uncover the deep characteristics of cyberattack traffic at multiple scales. Furthermore, deploying built-in security modules in every network device is impractical. Routing platforms, due to their pivotal role connecting private and public networks, and the continuous improvement in the performance of current programmable routing platforms, can effectively reduce the deployment cost of network security protection by implementing network threat detection on their platforms. In addition, the increasing intensity of current cyber threat confrontation makes end-to-end multi-task deep learning-based intelligent threat analysis and detection a promising area of ​​application.

[0004] Therefore, in order to better ensure the stability of cyberspace and further strengthen my country's defense against cyber threats, there is an urgent need for a multi-task intelligent detection solution for typical cyber threat traffic deployed on a routing platform. Summary of the Invention

[0005] The purpose of this invention is to address the problems existing in the prior art by providing a multi-task intelligent detection method for typical network threat traffic. This method extracts packet-level features, statistical features, and host-level features of traffic flowing through a routing platform to form a multi-scale heterogeneous graph representation. At the same time, it combines a multi-task learning model based on a self-attention mechanism to detect typical network threat traffic.

[0006] The technical solution to achieve the purpose of this invention is: a multi-task intelligent detection method for typical network threat traffic, the method comprising the following steps:

[0007] Step 1: For normal traffic and typical network threat traffic flowing through the routing platform, perform preprocessing operations such as sorting, deduplication, and grouping; the grouping operation is to group all traffic within a fixed sampling period T into a traffic group, which contains traffic data packets from multiple sessions.

[0008] Step 2: For each traffic group, take the data packets within the time window Δt as a unit, and extract the data packet level features, statistical level features, and host level features of the traffic within that time window;

[0009] Step 3: Based on the features extracted in Step 2, further processing is performed to form new features; further processing includes normalizing the packet length sequence and window size, and calculating the entropy of source IP, destination IP, and payload information;

[0010] Step 4: Using the flow characteristics processed in Step 3, the multi-scale features within each Δt time window are segmented and filled, and then arranged into a two-dimensional vector. This vector is used as a node in the heterogeneous graph, and multiple nodes are connected to form a multi-scale heterogeneous graph representation.

[0011] Step 5: Map the multi-scale heterogeneous graph representation from Step 4 into the form of a multi-scale representation matrix using the adjacency matrix.

[0012] Step 6: Using the multi-scale representation matrix generated in Step 5, a multi-task learning model based on a self-attention mechanism is used to detect typical network threat traffic.

[0013] Compared with the prior art, the significant advantages of this invention are:

[0014] 1) By conducting in-depth analysis of the traffic from three network threats—DDoS, C&C, and ransomware—accurate analysis was performed at the packet level, traffic statistics level, and host level, resulting in a comprehensive understanding of the multi-scale characteristics of these traffic.

[0015] 2) For the extracted multi-scale features, a multi-scale heterogeneous graph representation is constructed. The constructed graph representation uses the multi-scale features of multiple flows within the time window as nodes, the time difference between nodes as edge weights, and finally maps them in the form of an adjacency matrix. This representation can not only preserve the multi-scale features of the flows well, but also reflect the heterogeneous relationship between multiple flows.

[0016] 3) By combining multi-scale traffic representation with a self-attention mechanism in a multi-task learning model, the detection of typical network threat traffic is achieved. Furthermore, the system is deployed on a programmable routing platform, integrating security mechanisms and routing functions into one.

[0017] In summary, this invention, by deeply exploring the multi-scale features of typical network threat traffic and constructing a multi-scale heterogeneous graph representation based on these features, combined with a multi-task learning model using a self-attention mechanism, can effectively detect typical network threat traffic. This is of great significance for maintaining cyberspace security and ensuring network stability. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of a multi-task intelligent detection method for typical network threat traffic according to the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0020] Combination Figure 1 A multi-task intelligent detection method for typical network threat traffic includes the following steps:

[0021] Step 1: For normal traffic and typical network threat traffic flowing through the routing platform, perform preprocessing operations such as sorting, deduplication, and grouping; the grouping operation is to group all traffic within a fixed sampling period T into a traffic group, which contains traffic data packets from multiple sessions.

[0022] Here, the routing platform refers to a programmable routing platform based on OpenWRT.

[0023] Here, typical cyber threats mainly include three types: DDoS attacks, C&C attacks, and ransomware.

[0024] Step 2: For each traffic group, take the data packets within the time window Δt as a unit, and extract the data packet level features, statistical level features, and host level features of the traffic within that time window;

[0025] Here, packet-level features include packet length sequence, time series, window size, and load information; statistical-level features include packet count, session count, total load, and uplink / downlink packet ratio; and host-level features extract source IP, destination IP, source port, destination port, and IP geographic location information.

[0026] Step 3: Based on the features extracted in Step 2, in order to better describe the characteristics of typical network threat traffic, the features are further processed to form new features.

[0027] Further processing here includes normalizing the packet length sequence and window size, and calculating the entropy of the source IP, destination IP, and payload information. Specifically:

[0028] For each feature vector x that needs to be normalized, the maximum and minimum value normalization method is used to normalize it. The normalized vector is:

[0029]

[0030] For each feature whose entropy value needs to be calculated, the Renyi entropy between features is used as the measure. For feature x, its Renyi entropy H(x) is:

[0031]

[0032] Here, α is a real parameter, and to better capture the characteristics of different distributions, α is set to 2. p(x) represents the probability of the random variable x.

[0033] Step 4: Using the flow characteristics processed in Step 3, the multi-scale features within each Δt time window are segmented and filled, then arranged into a two-dimensional vector. This vector serves as a node in the heterogeneous graph, and multiple nodes are connected to form a multi-scale heterogeneous graph representation.

[0034] Here, after segmentation and padding of multi-scale features, they are arranged into n-dimensional vectors. Multiple n-dimensional vector nodes are interconnected to form a heterogeneous graph representation, specifically including:

[0035] For each node in the heterogeneous graph representation, a multidimensional vector is constructed based on the extracted multi-scale features. This vector can be represented by an N*(M+1) matrix, where N represents the number of data packets in that node (N is set to 20), and M is the number of extracted data packet-level features (8). The last row of the matrix represents the statistical scale features of the traffic. To ensure dimensionality consistency, nodes with more than 20 data packets are split to remove excess data packets, while nodes with fewer than 20 data packets are padded with 0s.

[0036] Here, the interconnection between multiple nodes is achieved by using the time interval between the first and last packets of data between different nodes as the edge weight for each node. Connecting each node forms the final multi-scale heterogeneous graph representation.

[0037] Step 5: Map the multi-scale heterogeneous graph representation from Step 4 into a multi-scale representation matrix using the adjacency matrix.

[0038] Here, the multi-scale heterogeneous graph representation mapping to a multi-scale representation matrix is ​​achieved through an adjacency matrix, specifically as follows:

[0039] The adjacency matrix is ​​used to describe the connection mapping between multi-scale heterogeneous graphs. If there is an edge between node i and node j, then the i-th row and j-th column of the adjacency matrix are τ, where τ represents the edge weight between the nodes, and its value is the time difference between the two nodes. If there is no edge, then it is 0. The multi-scale representation matrix S can then be expressed as:

[0040]

[0041] Where X = {x1, x2, ..., x} nflow} T It is the extracted multi-scale feature matrix, where x i Let be the multidimensional feature vector of the i-th traffic group corresponding to node i.

[0042] Step 6: Using the multi-scale representation matrix generated in Step 5, a multi-task learning model based on a self-attention mechanism is used to detect typical network threat traffic.

[0043] Here, the multi-task learning model based on the self-attention mechanism specifically includes an input layer, a 2-layer CNN layer, a 2-layer LSTM layer, and an output layer.

[0044] This invention extracts multi-scale features from traffic to form a multi-scale heterogeneous graph representation, and combines this with a multi-task learning model based on a self-attention mechanism to detect typical network threat traffic. Furthermore, the detection system is deployed in a programmable routing platform, integrating security mechanisms and routing functions into one, which is of great significance for maintaining cyberspace security and ensuring network stability.

[0045] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A multi-task intelligent detection method for typical network threat traffic, characterized in that, Includes the following steps: Step 1: For normal traffic and typical network threat traffic flowing through the routing platform, perform preprocessing operations such as sorting, deduplication, and packetization. The grouping operation involves grouping all traffic within a fixed sampling period T into a single traffic group, which contains traffic data packets from multiple sessions. Step 2: For each traffic group, take the data packets within the time window ∆t as a unit, and extract the data packet level features, statistical level features, and host level features of the traffic within that time window; the data packet level features include packet length sequence, time sequence, window size, and load information; the statistical level features include the number of data packets, number of sessions, total load, and uplink / downlink packet ratio; the host level features extract the source IP, destination IP, source port, destination port, and IP geographical location information. Step 3: Based on the features extracted in Step 2, further processing is performed to form new features; further processing includes normalizing the packet length sequence and window size, and calculating the entropy of source IP, destination IP, and payload information; Step 4: Using the flow characteristics processed in Step 3, the multi-scale features within each ∆t time window are segmented and filled, and then arranged into a two-dimensional vector. This vector is used as a node in the heterogeneous graph, and multiple nodes are connected to form a multi-scale heterogeneous graph representation. Step 5: Map the multi-scale heterogeneous graph representation from Step 4 into a multi-scale representation matrix using an adjacency matrix. Step 6: Using the multi-scale representation matrix generated in Step 5, a multi-task learning model based on a self-attention mechanism is used to detect typical network threat traffic.

2. The multi-task intelligent detection method for typical network threat traffic according to claim 1, characterized in that, The routing platform in step 1 is a programmable routing platform based on OpenWRT.

3. The multi-task intelligent detection method for typical network threat traffic according to claim 1, characterized in that, The typical cyber threats in step 1 mainly include three types: DDoS attacks, C&C attacks, and ransomware.

4. The multi-task intelligent detection method for typical network threat traffic according to claim 1, characterized in that, The further processing described in step 3 includes normalizing the packet length sequence and window size, and calculating the entropy of the source IP, destination IP, and payload information, specifically: For each feature vector x that needs to be normalized, the maximum and minimum value normalization method is used to normalize it, and the normalized vector is: ； For each feature whose entropy value needs to be calculated, the Renyi entropy between features is used as the measure. For feature x, its Renyi entropy H(x) is: ； Where α is a real parameter, and p(x) represents the probability of the random variable x.

5. The multi-task intelligent detection method for typical network threat traffic according to claim 4, characterized in that, Let α be 2.

6. The multi-task intelligent detection method for typical network threat traffic according to claim 1, characterized in that, In step 4, the multi-scale features are segmented and filled, then arranged into two-dimensional vectors. Multiple two-dimensional vector nodes are interconnected to form a heterogeneous graph representation, specifically including: For each node in the heterogeneous graph representation, a multidimensional vector is constructed based on the extracted multi-scale features and represented by a matrix of size N*(M+1), where N represents the number of data packets in the node, which is set to 20; M is the number of extracted data packet level features, which is set to 8; in addition, the last row of the matrix is ​​the statistical scale feature of the traffic; nodes with more than 20 data packets are split into redundant data packets, and nodes with less than 20 data packets are padded with 0.

7. The multi-task intelligent detection method for typical network threat traffic according to claim 1, characterized in that, The interconnection method between multiple nodes in step 4 is that the edge weight of each node is the time interval between the first and last packets between different nodes. Connect each node to form the final multi-scale heterogeneous graph representation.

8. The multi-task intelligent detection method for typical network threat traffic according to claim 1, characterized in that, The multi-scale heterogeneous graph representation mapping described in step 5 is a multi-scale representation matrix, specifically as follows: The adjacency matrix is ​​used to describe the connection mapping between multi-scale heterogeneous graphs. If there is an edge between node i and node j, then the i-th row and j-th column of the adjacency matrix are τ, where τ represents the edge weight between the nodes, and its value is the time difference between the two nodes. If there is no edge, then it is 0. The multi-scale representation matrix S can then be expressed as: ； Where X = {x1, x2, …, x} nflow } T It is the extracted multi-scale feature matrix, where x i Let be the multidimensional feature vector of the i-th traffic group corresponding to node i.

9. The multi-task intelligent detection method for typical network threat traffic according to claim 1, characterized in that, The multi-task learning model based on the self-attention mechanism described in step 6 specifically includes an input layer, a 2-layer CNN layer, a 2-layer LSTM layer, a self-attention mechanism, and an output layer.

Images