Hidden encryption malicious traffic detection method based on multi-stream pruning graph

Abstract

The invention discloses a hidden encrypted malicious traffic detection method based on a multi-stream pruning graph, and belongs to the technical field of encrypted malicious traffic detection in encrypted traffic analysis. The method comprises the following steps of: 1, visualizing a communication process of a hostile attack into a graph, further measuring the similarity of flow node feature vectors, and pruning an original graph structure into a lightweight graph structure, namely a multi-flow pruning graph, by evaluating the feature similarity; 2, inputting the multi-flow pruning graph into a graph convolutional neural network, and further learning association features among a plurality of flows by using the graph convolutional neural network, including a topological relation among flow nodes and feature information; compared with the prior art, the method has the advantages that the multi-stream pruning graph input into the graph convolutional neural network is subjected to lightweight processing by adopting cosine similarity, and the classifier is constructed by utilizing the GCN to realize hidden encryption malicious traffic detection; meanwhile, the robustness of camouflage malicious traffic detection implemented under the conditions of noise injection and data imbalance is improved.

Description

Technical Field

[0001] The present invention relates to a concealed encrypted malicious traffic detection method based on a multi-stream pruning graph, belonging to the technical field of encrypted malicious traffic detection in encrypted traffic analysis. Background Art

[0002] With the maturity of encryption protocols and the rapid growth of Internet encrypted traffic, threat activities such as botnet denial of service attacks, network scanning attacks, port scanning attacks, and ransomware attacks continue to emerge under the cover of encryption protocols, posing a serious threat to network security. Attackers may use certain technical means to steal sensitive information or launch malicious attacks. Traffic analysis is an important means of network governance, and encrypted traffic classification has become an emerging method for maintaining network security.

[0003] Attackers may hide malicious behaviors through encryption protocols to evade anomaly detection. However, effective encrypted traffic classification methods can respond to potential threats in a timely manner, take defensive measures, and ensure the security and stability of the network. Traditional encrypted malicious traffic detection methods attempt to extract features based on a single malicious traffic. However, malicious attacks usually involve multiple steps and interactions between multiple streams, and the complex stream interaction pattern between attackers and victims is different from the normal stream interaction pattern. For example, in a botnet, multiple infected hosts jointly initiate network communications involving multiple related encrypted sessions to implement malicious threat activities. Therefore, it is difficult for traditional single-stream-based detection methods to capture this behavior, which means that hidden encrypted malicious traffic cannot be effectively detected based on the features of a single stream alone.

[0004] Current encrypted malicious traffic detection methods have limitations when dealing with unbalanced data scenarios. These methods usually rely on known attack patterns and features for identification, which limits their effectiveness in dealing with unbalanced data situations. In practical applications, the number of malicious traffic samples is often less than the number of normal traffic samples. This leads to data imbalance, which makes traditional detection methods based on known patterns challenging. In addition, with the continuous development of adversarial technology, attackers can easily destroy and circumvent existing detection models by exploiting camouflage obfuscation techniques such as injected noise; for example, packets generated by benign applications. In addition, attackers also imitate the behavior of legitimate protocols to disguise malicious traffic and make it look similar to normal protocol communications. These obfuscation techniques make it difficult for traditional encrypted malicious traffic detection methods to distinguish malicious traffic from normal traffic. Therefore, traditional detection methods cannot accurately identify hidden encrypted malicious traffic, which reduces the reliability and robustness of detection.

[0005] Therefore, how to solve the poor detection effect caused by the disguise and confusion of encrypted malicious traffic and data imbalance in the existing technology and achieve accurate detection of hidden encrypted malicious traffic has become a technical problem that needs to be solved urgently. Summary of the invention

[0006] The purpose of the present invention is to improve the technical problem of the robustness of disguised malicious traffic detection implemented under injected noise and data imbalance, and to propose a hidden encrypted malicious traffic detection method based on a multi-stream pruning graph.

[0007] The workflow of the present invention is as follows: The present invention constructs a new traffic representation method, the multi-flow pruning graph, which extracts the multi-flow interaction pattern that is unique to malicious traffic compared to benign traffic. The present invention uses the multi-flow pruning graph as a model input and constructs a deep learning-based classification model, which extracts unique features to achieve hidden encrypted malicious traffic detection.

[0008] The working principle of the present invention is as follows: the present invention discloses a hidden encrypted malicious traffic detection method based on a multi-stream pruning graph. Based on the analysis of the principle of malicious remote control attacks, a multi-stream pruning graph is designed to express the principle of multi-stream interaction. The present invention measures the similarity of the feature vectors of the flow nodes, and prunes the original graph structure into a lightweight graph structure through the evaluation of feature similarity. In addition, the present invention uses a graph convolutional neural network to construct a hidden encrypted malicious traffic detection model to learn the correlation features between multiple flows, including the topological relationship and feature information between flow nodes. The present invention has a higher accuracy rate in identifying attack types that require multiple flows to implement attacks, is more systematic, and can effectively improve the detection accuracy of hidden encrypted malicious traffic in scenarios with data imbalance and disguised and obfuscated traffic.

[0009] The objective of the present invention is achieved through the following technical solutions:

[0010] The present invention discloses a method for detecting hidden encrypted malicious traffic based on a multi-stream pruning graph, which is applied to a disguised and obfuscated hidden encrypted malicious traffic detection scenario, and comprises the following steps:

[0011] Step 1: Utilize the principle of multi-flow interaction of encrypted traffic to realize the traffic representation based on graph structure, visualize the communication process of malicious attack as a graph, further measure the similarity of feature vectors of flow nodes, and prune the original graph structure into a lightweight graph structure, namely, multi-flow pruned graph, through the evaluation of feature similarity;

[0012] Step 1.1: Use a multi-flow graph to characterize traffic; further use the feature vector of a single flow as a vertex of the multi-flow graph; construct edges for vertices with IP association relationships;

[0013] Step 1.1.1: Construct a multi-flow graph using the method shown in formula (1);

[0014] G=(V,E) (1)

[0015] Among them, V and E represent the vertices and edges of the multi-flow graph respectively; the specific vertices are composed of the characteristic vectors v of the flow;

[0016] Step 1.1.2: Based on the TLS handshake flow f, a multi-flow set F is formed within the time window τ.

[0017] {f1,f2,…,f N}; The flow f with the same IP address i With flow j Connect and construct edge e=(f i ,f j );

[0018] Step 1.1.3: Extract the feature vector v of a single flow and construct the feature vector set V corresponding to the multi-flow set F = {v1,v2,…,v N};

[0019] Step 1.2: Use cosine similarity to prune the multi-flow graph to form a multi-flow pruned graph to reduce time overhead; specifically, the similarity of any two flow vertices with a connection relationship is calculated using the method shown in formula (2); when the similarity between the two is lower than the threshold δ, the edge connection relationship is deleted; otherwise, the edge connection relationship is retained;

[0020]

[0021] Among them, f i and f j are any two flow nodes in the multi-flow graph G, and D is the dimension of the feature vector.

[0022] Step 2: Input the multi-flow pruning graph obtained in step 1 into the graph convolutional neural network, and use the graph convolutional neural network to further learn the correlation features between multiple flows, including the topological relationship and feature information between flow nodes;

[0023] Step 2.1: Input the multi-stream pruning graph sample G containing N nodes into the graph convolutional neural network; specifically, the feature matrix of the node and the adjacency matrix representing the connection relationship between nodes is input into the model; where is a real number;

[0024] Step 2.2: In the graph convolution layer of the graph convolutional neural network, extract the features in the multi-stream pruned graph, and use the learned weight parameters to adjust the parameters of the model; generate a global representation vector based on the features of a single node and the graph structure features for classification and recognition in the fully connected layer.

[0025] Step 2.2.1: In the graph convolution layer, the adjacency matrix A and the feature matrix X are subjected to feature learning and layer propagation as shown in formula (3) to obtain the node features of the lth layer:

[0026]

[0027] in, I is the identity matrix, is the feature of the lth layer with node index i, and σ is a nonlinear activation function. for The degree matrix of The input of the first layer of GCN is the original feature matrix X;

[0028] Step 2.2.2: Aggregate the features of all nodes in the last layer, generate a global representation vector using the method shown in formula (4), and input the obtained global representation vector into the fully connected layer;

[0029]

[0030] Step 2.3: Use the cross entropy loss function as shown in formula (5) to improve the consistency between the predicted probability and the true label;

[0031]

[0032] Among them, C is the number of samples for model training, y i is the actual category of the sample, y i =1 for malicious traffic, y i =0 is normal flow, The predicted sample category output by the model;

[0033] Beneficial effects:

[0034] Compared with the prior art, the present invention has the following effects:

[0035] 1. The present invention analyzes the principle of multi-stream interaction and embeds the topological associations between multiple streams into a graph structure, which can reveal the relationship and behavior pattern between the attacker and the victim host. This association information includes the connection relationship, similarity, transmission path, etc. between multiple streams.

[0036] 2. By extracting packet-level features and data stream-level features, the present invention can not only retain packet-level information, but also capture the behavior patterns and properties of the entire stream. Then, by using GCN to build a classifier, the present invention can achieve hidden encrypted malicious traffic detection.

[0037] 3. The present invention greatly reduces the complexity of the graph structure through edge pruning based on cosine similarity, thereby reducing the time overhead of the algorithm model, so that the model can achieve high-precision hidden encrypted malicious traffic detection with low time overhead.

[0038] 4. The present invention can effectively cope with the challenges of frequent and diverse network threats by extracting robust features of encrypted traffic. Even if attackers conceal attack features through disguise and confusion, the present invention can still capture the unique patterns and features of malicious attacks and abnormal behaviors in traffic, thereby achieving accurate detection of hidden malicious attack traffic. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a structural schematic diagram of the present invention;

[0040] Figure 2 It is a schematic diagram of the method flow of the present invention;

[0041] Figure 3 This is a diagram showing the effect of the present invention on detecting hidden encrypted malicious traffic under different obfuscation probabilities. DETAILED DESCRIPTION

[0042] In order to better illustrate the purpose and advantages of the present invention, the present embodiment uses a disguised, obfuscated, concealed, encrypted malicious traffic detection scenario as an example, and the invention is further described below with reference to the accompanying drawings and examples. It should be noted that the implementation of the present invention is not limited to the following embodiments, and any form of modification or change made to the present invention will fall within the protection scope of the present invention.

[0043] like Figure 1 As shown, the present invention is mainly composed of three modules: a multi-stream pruning graph construction module, a GCN-based feature extraction module, and a classification and recognition module. The multi-stream pruning graph construction module performs multi-stream graph representation on the original encrypted traffic data, and mines the correlation between them by analyzing multiple related encrypted sessions. The GCN-based feature extraction module uses GCN to further extract the graph structure features of the multi-stream pruning graph and learn the correlation features between multiple flows, including the topological relationship and feature information between traffic nodes. The classification and recognition module uses the trained GCN model to achieve accurate detection of hidden encrypted malicious traffic.

[0044] Example

[0045] like Figure 2 As shown, a method for detecting hidden encrypted malicious traffic based on a multi-stream pruning graph in this embodiment is specifically implemented in the following steps:

[0046] Step 1: Utilize the principle of multi-flow interaction of encrypted traffic to realize the traffic representation based on graph structure, visualize the communication process of malicious attack as a graph, further measure the similarity of feature vectors of flow nodes, and prune the original graph structure into a lightweight graph structure, namely, multi-flow pruned graph, through the evaluation of feature similarity;

[0047] Step 1.1: Use a multi-flow graph to characterize traffic; further use the feature vector of a single flow as a vertex of the multi-flow graph; construct edges for vertices with IP association relationships;

[0048] Step 1.1.1: Construct a multi-flow graph using the method shown in formula (1);

[0049] G=(V,E) (1)

[0050] Among them, V and E represent the vertices and edges of the multi-flow graph respectively; the specific vertices are composed of the characteristic vectors v of the flow;

[0051] In this embodiment, the vertex is composed of the feature vector of the flow, which includes the statistical features at the flow level, namely the protocol type, the number of packets, the average value of the packet length, and the duration of the flow; the statistical features at the packet level, for the long sequence of packets and the corresponding time window sequence, extract the maximum value, minimum value, average value, median, skewness, peak value, upper and lower quartiles and other statistical features. Based on this, each flow node will maintain the statistical features at the packet level and the data flow level.

[0052] Step 1.1.2: Based on the TLS handshake flow f, a multi-flow set F is formed within the time window τ.

[0053] {f1,f2,…,f N}; The flow f with the same IP address i With flow j Connect and construct edge e=(f i ,f j );

[0054] In this embodiment, if a malicious attacker's server performs a TLS handshake with a victim host, then this handshake can generate a flow f connecting the two endpoints. Therefore, a set of multiple flows within a certain time window τ can be represented by F = {f1, f2, ..., f N} means. If any flow f in a set of multiple flows F i With another stream f j If there is the same source IP address or destination IP address, then an edge e=(f i ,f j ) connections, this is because streams with the same IP address have more similar location information.

[0055] Step 1.1.3: Extract the feature vector v of a single flow and construct the feature vector set V corresponding to the multi-flow set F = {v1,v2,…,v N};

[0056] In this embodiment, for each flow in the multi-flow set, this embodiment extracts its feature vector v, and the feature vector set corresponding to the multi-flow set F can be expressed as V = {v1, v2, ..., v N}, the feature fields and their specific descriptions contained in the vector v are shown in Table 1.

[0057] Table 1 Feature fields of stream node feature vector

[0058]

[0059] Step 1.2: Use cosine similarity to prune the multi-flow graph to form a multi-flow pruned graph to reduce time overhead; specifically, the similarity of any two flow vertices with a connection relationship is calculated using the method shown in formula (2); when the similarity between the two is lower than the threshold δ, the edge connection relationship is deleted; otherwise, the edge connection relationship is retained;

[0060]

[0061] Among them, f i and f j are any two flow nodes in the multi-flow graph G, and D is the dimension of the feature vector.

[0062] Step 2: Input the multi-flow pruning graph obtained in step 1 into the graph convolutional neural network, and use the graph convolutional neural network to further learn the correlation features between multiple flows, including the topological relationship and feature information between flow nodes;

[0063] Step 2.1: Input the multi-stream pruning graph sample G containing N nodes into the graph convolutional neural network; specifically, the feature matrix of the node and the adjacency matrix representing the connection relationship between nodes is input into the model; where is a real number;

[0064] The data set used to train the graph convolutional neural network in this embodiment is a set of multi-stream pruned graphs

[0065] {G1,G2,…,G C} and the corresponding labels {y1,y2,…,y C}, C is the number of samples.

[0066] Step 2.2: In the graph convolution layer of the graph convolutional neural network, extract the features in the multi-stream pruned graph, and use the learned weight parameters to adjust the parameters of the model; generate a global representation vector based on the features of a single node and the graph structure features for classification and recognition in the fully connected layer.

[0067] Step 2.2.1: In the graph convolution layer, the adjacency matrix A and the feature matrix X are subjected to feature learning and layer propagation as shown in formula (3) to obtain the node features of the lth layer:

[0068]

[0069] in, I is the identity matrix, is the feature of the lth layer with node index i, and σ is a nonlinear activation function. for The degree matrix of The input of the first layer of GCN is the original feature matrix X;

[0070] In this embodiment, the traffic features generated by GCN are not only related to the packet size and timestamp in the flow, but also affected by other flows related to it. This traffic feature that takes into account both the data flow itself and the correlation characteristics between data flows provides more information leakage, which is conducive to the accurate identification of the classifier, thereby improving the classification performance.

[0071] Step 2.2.2: Aggregate the features of all nodes in the last layer, generate a global representation vector using the method shown in formula (4), and input the obtained global representation vector into the fully connected layer;

[0072]

[0073] Step 2.3: Use the cross entropy loss function as shown in formula (5) to improve the consistency between the predicted probability and the true label;

[0074]

[0075] Among them, C is the number of samples for model training, y i is the actual category of the sample, y i =1 for malicious traffic, y i =0 is normal flow, The predicted sample category output by the model;

[0076] In this embodiment, the Adam optimizer is used to reduce the loss during the training process so that the model can improve the value of the predicted probability. The purpose of this is to gradually optimize the parameters of the GCN model so that it can better capture the features related to malicious traffic and reduce the impact of other irrelevant features. Through the adjustment of the optimizer, the model can better learn the features related to the target task, thereby improving the accuracy of the classifier.

[0077] In order to verify the superiority and effectiveness of the present invention, the present invention is compared with the existing advanced method FG-Net for experimental evaluation. This embodiment is experimentally evaluated in a scenario simulating an attacker injecting noise. The experimental results are as follows: Figure 3 As shown, specifically, for the hidden encrypted malicious traffic detection model parameters proposed in step 2, the specific settings of this embodiment are: the number of graph aggregation network layers is set to 2, the batch size is set to 16, the training rounds are set to 30, and the learning rate is set to 0.0005. The results show that the present invention can achieve better detection effects in different probability confusion scenarios.

[0078] The specific description above further illustrates the purpose, technical solutions and beneficial effects of the invention in detail. It should be understood that the above is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for detecting hidden encrypted malicious traffic based on a multi-stream pruning graph, characterized in that: The following steps are included: Step 1: Utilize the principle of multi-flow interaction of encrypted traffic to realize the traffic representation based on graph structure, visualize the communication process of malicious attack as a graph, further measure the similarity of feature vectors of flow nodes, and prune the original graph structure into a lightweight graph structure, namely, multi-flow pruned graph, through the evaluation of feature similarity; Step 1.1: Use a multi-flow graph to characterize traffic; further use the feature vector of a single flow as a vertex of the multi-flow graph; construct edges for vertices with IP association relationships; Step 1.1.1: Construct a multi-flow graph using the method shown in formula (1); G=(V,E) (1) Among them, V and E represent the vertices and edges of the multi-flow graph respectively; the specific vertices are composed of the characteristic vectors v of the flow; Step 1.1.2: Based on the TLS handshake flow f, a multi-flow set F = {f1,f2,…,f N }; The flow f with the same IP address i With flow i Connect and construct edge e=(f i ,f j ); Step 1.1.3: Extract the feature vector v of a single flow and construct the feature vector set V corresponding to the multi-flow set F = {v1,v2,…,v N }; Step 1.2: Use cosine similarity to prune the multi-flow graph to form a multi-flow pruned graph to reduce time overhead; specifically, the similarity of any two flow vertices with a connection relationship is calculated using the method shown in formula (2); when the similarity between the two is lower than the threshold δ, the edge connection relationship is deleted; otherwise, the edge connection relationship is retained; Among them, f i and f j are any two flow nodes in the multi-flow graph G, and D is the dimension of the feature vector; Step 2: Input the multi-flow pruning graph obtained in step 1 into the graph convolutional neural network, and use the graph convolutional neural network to further learn the association features between multiple flows, including the topological relationship and feature information between flow nodes.

2. A method for detecting hidden encrypted malicious traffic based on a multi-stream pruning graph as claimed in claim 1, characterized in that: Step 2 is implemented as follows: Step 2.1: Input the multi-stream pruning graph sample G containing N nodes into the graph convolutional neural network; specifically, the feature matrix of the node and the adjacency matrix representing the connection relationship between nodes Input to the model; where is a real number; Step 2.2: In the graph convolution layer of the graph convolutional neural network, extract the features in the multi-stream pruning graph and adjust the parameters of the model using the learned weight parameters; generate a global representation vector based on the features of individual nodes and graph structure features for classification and recognition in the fully connected layer; Step 2.3: Use the cross entropy loss function as shown in formula (5) to improve the consistency between the predicted probability and the true label; Among them, C is the number of samples for model training, y i is the actual category of the sample, y i =1 for malicious traffic, y i =0 is normal flow, The predicted sample category output by the model.

3. A method for detecting hidden encrypted malicious traffic based on a multi-stream pruning graph as claimed in claim 2, characterized in that: Step 2.2 is implemented as follows: Step 2.2.1: In the graph convolution layer, the adjacency matrix A and the feature matrix X are subjected to feature learning and layer propagation as shown in formula (3) to obtain the node features of the lth layer: in, I is the identity matrix, is the feature of the lth layer with node subscript i, σ is the nonlinear activation function; for The degree matrix of The input of the first layer of GCN is the original feature matrix x; Step 2.2.2: Aggregate the features of all nodes in the last layer, generate a global representation vector using the method shown in formula (4), and input the obtained global representation vector into the fully connected layer.

Images