A deep learning-based encrypted malicious traffic detection method

Abstract

The application discloses a kind of encryption malicious traffic detection methods based on deep learning, utilize LSTM model to extract the time sequence characteristics of network traffic data, further extract the semantic features of traffic data by BERT model, finally form comprehensive feature representation;Comprehensive feature representation is clustered using DBSCAN algorithm;According to the defined naming rules, the sample label in each cluster is optimized, and the traffic type with similar attack characteristics is classified as a unified label;Construct a graph structure, connect the edge of the optimized traffic sample label and its comprehensive feature representation, and establish edge connection between different clusters according to cosine similarity;Construct a GraphSAGE graph neural network model, aggregate the information of the nodes and their neighbor nodes of the graph structure to obtain the updated feature vector of each node, and use the updated node features for classification processing to identify whether it is malicious encrypted traffic.The application improves the detection accuracy and robustness.

Description

Technical Field

[0001] This invention belongs to the field of cyberspace security technology, specifically a method for detecting encrypted malicious traffic based on deep learning. Background Technology

[0002] In recent years, with the rapid development of the internet and the continuous improvement of network security technologies, attackers have increasingly turned to using encrypted traffic for malicious activities. Encrypted traffic refers to network traffic transmitted through encryption protocols (such as HTTPS and TLS), primarily intended to protect user privacy and data security. However, attackers can also use encrypted traffic to conceal their malicious activities, making traditional plaintext-based network security measures ineffective in detecting and preventing these attacks. Malicious encrypted traffic can be used to carry out various attack types, such as data theft, distributed denial-of-service (DDoS) attacks, and malware propagation. Furthermore, with the widespread adoption of encryption technology, the proportion of encrypted traffic in internet traffic is constantly increasing, further escalating the challenges to network security monitoring and analysis.

[0003] Traditional malicious traffic detection methods are gradually becoming ineffective against encrypted transmissions. This is primarily because they often rely on plaintext feature extraction and rule matching. When data traffic is encrypted using protocols such as HTTPS or TLS, previously directly parsable information is hidden, making detection methods based on specific keywords or traffic signatures ineffective. Furthermore, simple statistical feature analysis cannot fully express the complex behavioral patterns inherent in encrypted traffic. Malicious actors can use obfuscation techniques to mimic the distribution characteristics of normal traffic, increasing the difficulty of detection. In addition, traditional methods lack the fusion of temporal and semantic features, failing to characterize the inherent relationships and potential semantic information of traffic from multiple perspectives, making it difficult to effectively identify hidden attack behaviors in massive, dynamic, and high-dimensional encrypted traffic. Unlike traditional detection methods based on manual feature extraction, deep learning models can automatically uncover the temporal, semantic, and spatial relationships hidden in high-dimensional data, possessing stronger feature modeling capabilities. In feature modeling of encrypted traffic, deep learning not only demonstrates greater adaptability and flexibility, but also strengthens the similarity relationships between samples through multi-layered representation learning, thereby improving the generalization ability and robustness against unknown malicious encrypted traffic with complex structures and diverse types. However, existing deep learning-based methods for detecting malicious encrypted traffic still face the following problems: First, existing methods often ignore the implicit correlations between traffic samples and cannot fully utilize the spatial and temporal features of traffic samples; second, existing methods have certain limitations in classifying encrypted traffic, especially when faced with encrypted traffic with similar characteristics but belonging to different categories, it is difficult to distinguish these subtle differences. Summary of the Invention

[0004] The purpose of this invention is to propose a method for detecting malicious encrypted traffic based on deep learning, so as to detect encrypted malicious traffic.

[0005] The technical solution for implementing this invention is: a method for detecting encrypted malicious traffic based on deep learning, comprising the following steps:

[0006] Step 1: Use the LSTM model to extract the temporal features of network traffic data, and use the BERT model to further extract the semantic features of traffic data, finally forming a comprehensive feature representation;

[0007] Step 2: Use the DBSCAN algorithm to cluster the comprehensive feature representation. Based on feature similarity, divide the network traffic data samples into different clusters, with each cluster representing a potential type of encrypted traffic.

[0008] Step 3: Optimize the sample labels in each cluster according to the defined naming rules to achieve fuzzy classification of traffic labels and classify traffic types with similar attack characteristics into a unified label;

[0009] Step 4: Construct a graph structure by connecting the optimized traffic sample labels and their comprehensive feature representations with edges, and establishing edge connections between different clusters based on cosine similarity to form a complete graph model.

[0010] Step 5: Construct the GraphSAGE graph neural network model, aggregate information of the nodes and their neighboring nodes in the graph structure to obtain the updated feature vector of each node, and use the updated node features for classification to identify whether it is malicious encrypted traffic.

[0011] Step 6: Train the GraphSAGE graph neural network model and classifier using backpropagation and optimization algorithms, and perform malicious traffic detection on new network traffic samples.

[0012] Further, in step 1, the temporal features of network traffic data are extracted using the LSTM model, and the semantic features of the traffic data are further extracted using the BERT model, ultimately forming a comprehensive feature representation. The specific method is as follows:

[0013] Step 1-1: First, collect raw network traffic data from network monitoring equipment, log systems, or other data sources, including protocol type, packet size, latency, source IP address, destination IP address, and transmission port. Then, preprocess the collected traffic data, including noise removal, missing value filling, and standardization.

[0014] Steps 1-2: Arrange the preprocessed network traffic data in chronological order to form time series data. Use the LSTM model to process these time series data and extract the key temporal features from the traffic data.

[0015] Steps 1-3: Input the temporal feature vector extracted by the LSTM model into the BERT model, and through its self-attention mechanism, analyze the contextual semantic information of the traffic data to further extract the semantic features of the traffic.

[0016] Steps 1-4: Use the feature concatenation method to fuse the temporal features extracted by LSTM with the semantic features extracted by BERT to form a comprehensive feature representation vector.

[0017] Further, in step 2, the DBSCAN algorithm is used to cluster the comprehensive feature representation. Based on feature similarity, the network traffic data samples are divided into different clusters, each cluster representing a potential type of encrypted traffic. The specific method is as follows:

[0018] Step 2-1: Use the DBSCAN algorithm to calculate the neighborhood density of each traffic sample, identify the high-density area as the core point, and determine the density connection samples around the core point according to the preset neighborhood radius and density threshold. Use these density connection samples to classify traffic samples with similar characteristics into the same category.

[0019] Step 2-2: Assign a unique cluster label to each cluster, representing the potential type of encrypted traffic;

[0020] Steps 2-3: Record the cluster label of each traffic sample to generate a dataset containing the traffic sample and its corresponding cluster label.

[0021] Furthermore, in step 3, the sample labels in each cluster are optimized according to the defined naming rules to achieve fuzzy classification of traffic labels, grouping traffic types with similar attack characteristics into a unified label. The specific method is as follows:

[0022] Step 3-1: Establish a unified naming convention, including:

[0023] Denial of Service Attacks: Includes DoS and DDoS attacks.

[0024] Data Exfiltration: including DNS tunneling and HTTP tunneling.

[0025] Malware Traffic: Includes RAT traffic and virus propagation traffic.

[0026] Network Scanning: Includes port scanning and network traffic reconnaissance.

[0027] Phishing Attacks: This includes phishing emails and phishing website traffic.

[0028] Unknown: Used to identify noise points that cannot be classified or traffic samples of unknown type.

[0029] Step 3-2: Map each cluster label to the corresponding unified label according to the defined naming rules.

[0030] Further, in step 4, a graph structure is constructed by connecting the optimized traffic sample labels and their comprehensive feature representations with edges, and establishing edge connections between different clusters based on cosine similarity to form a complete graph model. The specific method is as follows:

[0031] Step 4-1: Summarize the edge information within and across clusters to construct a complete graph model. Nodes in the graph represent network traffic samples, and edges represent the similarity relationships between samples.

[0032] Step 4-2: For each cluster, establish edges between samples within the cluster based on cosine similarity. The specific steps are as follows:

[0033] Traverse each cluster and obtain the feature vectors of all samples in the cluster, including protocol type, packet size, and latency. Calculate the cosine similarity between any two samples in the cluster. If the cosine similarity between two samples is greater than a preset threshold θ, then establish an edge between the two nodes to indicate that they have high feature similarity; otherwise, do not establish an edge.

[0034] Step 4-3: Establish edges between different clusters. The specific steps are as follows:

[0035] Traverse the sample pairs between different clusters and calculate their cosine similarity. If the cosine similarity between two samples from different clusters is greater than the threshold θ, then establish an edge between the two nodes; otherwise, do not establish an edge.

[0036] Further, in step 5, a GraphSAGE graph neural network model is constructed. Information is aggregated from the nodes of the graph structure and their neighboring nodes to obtain an updated feature vector for each node. The updated node features are then used for classification to identify whether the traffic is malicious encryption. The specific method is as follows:

[0037] Step 5-1: Construct the GraphSAGE graph neural network model. The GraphSAGE graph neural network model consists of multiple GraphSAGE layers connected in series. Each GraphSAGE layer includes a neighbor sampling module, a feature aggregation module, and a feature transformation module, wherein:

[0038] The neighbor sampling module, for each target node v, randomly samples a fixed number of neighbor nodes from its neighbor node set N(v) to generate a sampled subset of neighbor features. Output the sampled neighbor feature set N(X) k-1), the sampled neighbor features N(X) k-1 Enter the feature aggregation module;

[0039] The feature aggregation module consists of three sub-modules: mean aggregation, maximum aggregation, and LSTM aggregation. The mean aggregation module calculates the element-wise average of the feature vectors of the sampled neighbors to generate the mean-aggregated feature f. mean The maximum value aggregation module extracts the maximum value element by element from the neighbor features, generating the maximum value aggregation feature f. max The LSTM aggregation module converts neighbor feature sequences. The input is fed into an LSTM network for sequence processing, and the output is a fixed-dimensional LSTM aggregated feature f. LSTM The output of the feature aggregation module is a comprehensive aggregated feature vector [f] mean ,f max ,f LSTM ], the comprehensive aggregated feature vector [f mean ,f max ,f LSTM It is passed to the feature transformation module;

[0040] In the feature transformation module, aggregated features are transformed by a linear transformation matrix W. k A linear transformation is performed, followed by processing with the ReLU activation function σ to generate new node feature representations. The specific update formula is as follows:

[0041]

[0042] Where AGGREGATE represents the selected aggregation operation, and σ selects the ReLU activation function. This represents the feature representation of node v at the k-th layer.

[0043] Output of the feature transformation module X will be used as the input X of the next GraphSAGE layer k The process of neighbor sampling and feature aggregation continues. Through layer stacking, node features are gradually fused with information from a wider range of neighbors to generate the final node feature matrix X. n ;

[0044] Step 5-2: The classifier consists of fully connected layers that receive X. n A series of linear transformations are used to generate a classification output Y, which is used to determine whether each node is malicious encrypted traffic.

[0045] Further, in step 6, the GraphSAGE graph neural network model and classifier are trained using backpropagation and optimization algorithms, and malicious traffic detection is performed on new network traffic samples. The specific method is as follows:

[0046] Labeled training data was used to optimize the parameters of the GraphSAGE model and classifier through backpropagation and the Adam optimizer. During the optimization process, the learning rate of the generator and discriminator was set to 0.001, the momentum parameters β1 and β2 were set to 0.9 and 0.999 respectively, and the batch size was set to 8.

[0047] A deep learning-based encrypted malicious traffic detection system implements the deep learning-based encrypted malicious traffic detection method to achieve deep learning-based encrypted malicious traffic detection, and executes steps 1 to 6 in six modules respectively.

[0048] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the deep learning-based encrypted malicious traffic detection method to achieve deep learning-based encrypted malicious traffic detection.

[0049] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the deep learning-based encrypted malicious traffic detection method is implemented to achieve deep learning-based encrypted malicious traffic detection.

[0050] Compared with the prior art, the significant advantage of this invention is that by constructing a graph structure between traffic samples, it can effectively capture and aggregate the features of neighboring samples, thereby strengthening the mining of similarity relationships between traffic and improving the detection accuracy and robustness of malicious traffic. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating a deep learning-based method for detecting encrypted malicious traffic.

[0052] Figure 2 A schematic diagram of LSTM and BERT extracting combined features.

[0053] Figure 3 A diagram illustrating fuzzy classification.

[0054] Figure 4 Construct a schematic diagram for the graph.

[0055] Figure 5 GraphSAGE structure diagram Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0057] like Figure 1 As shown, a method for detecting encrypted malicious traffic based on deep learning includes the following steps:

[0058] Step 1: Collect network traffic data. First, use the LSTM model to extract time-series features. Then, use the time-series features as input and use the BERT model to further extract semantic features of the traffic data, ultimately forming a comprehensive feature representation.

[0059] Step 1-1: First, collect raw network traffic data from network monitoring equipment, log systems, or other data sources. This data typically contains various information, such as protocol type, packet size, latency, source IP address, destination IP address, and transmission port. To ensure the comprehensiveness and accuracy of the data, the collected traffic data needs to be preprocessed, including noise removal, missing value imputation, and standardization.

[0060] Steps 1-2: As Figure 2 As shown, the preprocessed network traffic data is arranged in chronological order to form time series data. Then, an LSTM (Long Short-Term Memory) model is used to process this time series data. LSTM can effectively capture the temporal dynamics of the data and identify patterns and trends in traffic changes. Specifically, the LSTM model, through its memory units, can remember data dependencies over long time spans, thereby extracting key temporal features from the traffic data. These feature vectors will serve as input for the next step of semantic feature extraction.

[0061] Steps 1-3: Input the temporal feature vectors extracted by the LSTM model into the BERT (Bidirectional Encoder-Representation Transformer) model. Through its self-attention mechanism, the BERT model can analyze the contextual semantic information of traffic data and further extract semantic features of the traffic. This process not only enhances the expressive power of the features but also enables the model to understand the implicit semantic relationships in the traffic data. These semantic feature vectors, combined with the temporal feature vectors, form a comprehensive feature representation.

[0062] Steps 1-4: Finally, the temporal features extracted by LSTM and the semantic features extracted by BERT are fused to form a comprehensive feature representation vector. The specific fusion method uses feature concatenation. This comprehensive feature representation vector will serve as input for subsequent clustering analysis, providing a rich feature foundation for malicious traffic detection.

[0063] Step 2: Use the DBSCAN algorithm to perform cluster analysis on the above comprehensive features, divide the samples into different clusters according to feature similarity, and output the cluster label to which each sample belongs.

[0064] Step 2-1: Perform cluster analysis on the comprehensive feature representation vector using the DBSCAN algorithm. DBSCAN effectively identifies traffic clusters with different feature patterns by evaluating the density similarity between samples. First, set the parameters of the DBSCAN algorithm, including the neighborhood radius (eps) and the minimum number of samples (min_samples). Then, the algorithm scans each sample and determines whether its neighborhood contains at least min_samples samples, thereby identifying core points, boundary points, and noise points.

[0065] Step 2-2: Group samples with similar characteristics into the same cluster and assign a unique label (e.g., 1, 2, 3…) to each cluster. Samples identified as noise points are given a special label (e.g., -1) to indicate that they do not belong to any cluster. In this way, cluster analysis not only groups traffic samples but also lays the foundation for subsequent label optimization and graph structure construction.

[0066] Steps 2-3: Assign clustering results to the original traffic samples to establish a cluster label mapping. This involves recording the cluster label of each traffic sample and generating a dataset containing the traffic samples and their corresponding cluster labels. This step ensures that the clustering information for each sample is accurately recorded, facilitating subsequent label optimization and graph structure construction.

[0067] Step 3: Optimize the sample labels in each cluster according to the defined naming rules to achieve fuzzy classification of traffic labels, that is, group similar traffic samples with similar labels.

[0068] Step 3-1: Establish unified categorization and naming rules, such as Figure 3 As shown, traffic types with similar attack characteristics are categorized into a unified label. Specific rules include:

[0069] 1) Denial of Service Attacks: including DoS (Denial of Service attack) and DDoS (Distributed Denial of Service attack).

[0070] 2) Data Exfiltration: This includes DNS tunneling and HTTP tunneling.

[0071] 3) Malware Traffic: This includes RAT (Remote Access Trojan) traffic and virus propagation traffic.

[0072] 4) Network Scanning: This includes port scanning and network traffic reconnaissance.

[0073] 5) Phishing Attacks: This includes phishing emails and phishing website traffic.

[0074] 6) Unknown: Used to identify noise points that could not be classified or traffic samples of unknown type.

[0075] Step 3-2: Map each cluster label to the corresponding unified label according to the defined naming rules. The specific steps are as follows:

[0076] 1) Iterate through each cluster label and map it to a predefined unified label based on the attack characteristics it represents. 2) For example, if cluster label 0 represents DoS and DDoS attacks, then map cluster label 0 to "Denial of Service Attacks".

[0077] Step 3-3: For noise points marked as -1 in the DBSCAN algorithm, assign the label "Unknown" to indicate that these samples could not be classified into any defined attack type. This ensures that all traffic samples have a corresponding label, facilitating subsequent detection and analysis.

[0078] Step 4: Construct a graph structure, such as... Figure 4 As shown, the optimized traffic sample labels and their comprehensive feature representations are connected by edges, and edge connections are established between different clusters based on cosine similarity, thus forming a complete graph model.

[0079] Step 4-1: Summarize the intra-cluster and inter-cluster edge information to construct a complete graph model. Nodes in the graph represent network traffic samples, and edges represent the similarity relationships between samples.

[0080] Step 4-2: For each cluster, establish edges between samples within the cluster based on cosine similarity. The specific steps are as follows:

[0081] 1) Traverse each cluster and obtain the feature vectors of all samples in that cluster.

[0082] 2) Calculate the cosine similarity between any two samples within a cluster.

[0083] 3) If the cosine similarity between two samples is greater than a preset threshold θ, then an edge is established between these two nodes.

[0084] This indicates that they have a high degree of feature similarity.

[0085] Step 4-3: To further enhance the expressive power of the graph structure, edges are also established between different clusters.

[0086] The specific steps are as follows:

[0087] 1) Traverse the sample pairs between different clusters and calculate their cosine similarity.

[0088] 2) If the cosine similarity between samples from two different clusters is greater than the threshold θ, then an edge is established between these two nodes.

[0089] Step 5: Apply the GraphSAGE graph neural network model. The GraphSAGE graph neural network model consists of multiple GraphSAGE layers connected in series. Each GraphSAGE layer includes a neighbor sampling module, a feature aggregation module (covering mean aggregation, maximum aggregation, and LSTM aggregation), and a feature transformation module, such as... Figure 5 As shown.

[0090] First, the initial node feature vector After sample label optimization in step 3, a new feature matrix X is formed. k-1 The data is then passed to the neighbor sampling module. This module, for each target node v, randomly samples a fixed number of neighbor nodes from its set N(v) to generate a sampled subset of neighbor features. Output the sampled neighbor feature set N(X) k-1 ).

[0091] Next, the sampled neighbor features N(X) k-1 The module then proceeds to the feature aggregation module. This module contains three sub-modules: mean aggregation, maximum aggregation, and LSTM aggregation. The mean aggregation module calculates the element-wise average of the feature vectors of the sampled neighbors to generate the mean-aggregated feature f. mean The maximum value aggregation module extracts the maximum value element by element from the neighbor features, generating the maximum value aggregation feature f. max The LSTM aggregation module converts neighbor feature sequences. The input is fed into an LSTM network for sequence processing, and the output is a fixed-dimensional LSTM aggregated feature f. LSTM The output of the feature aggregation module is a comprehensive aggregated feature vector [f] mean ,f max ,f LSTM ].

[0092] Subsequently, the aggregated feature vector [f] is synthesized. mean ,f max ,f LSTM The features are passed to the feature transformation module. In the feature transformation module, the aggregated features are transformed by the linear transformation matrix W. k A linear transformation is performed, followed by processing with the ReLU activation function σ to generate new node feature representations. The specific update formula is as follows: Where AGGREGATE represents the selected aggregation operation, and σ selects the ReLU activation function. This represents the feature representation of node v at layer k. The output of the feature transformation module. X will be used as the input X of the next GraphSAGE layer k The process of neighbor sampling and feature aggregation continues. Through layer-by-layer stacking, node features are gradually fused with information from a wider range of neighbors, generating the final node feature matrix X. n .

[0093] Finally, the node feature matrix X n It is passed to the classification module. The classification module consists of fully connected layers and receives X. n The model generates a classification output Y through a series of linear transformations to determine whether each node represents malicious encrypted traffic. During model training, labeled training data is used to optimize the parameters of the GraphSAGE model and classifier through backpropagation and the Adam optimizer. During optimization, the learning rate of both the generator and discriminator is set to 0.001, the momentum parameters β1 and β2 are set to 0.9 and 0.999 respectively, and the batch size is set to 8.

[0094] This invention also proposes a deep learning-based encrypted malicious traffic detection system, which implements the deep learning-based encrypted malicious traffic detection method to achieve deep learning-based encrypted malicious traffic detection. Steps 1 to 6 are executed in six modules respectively.

[0095] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the deep learning-based encrypted malicious traffic detection method to achieve deep learning-based encrypted malicious traffic detection.

[0096] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the deep learning-based encrypted malicious traffic detection method is implemented to achieve deep learning-based encrypted malicious traffic detection.

[0097] Example

[0098] Combination Figure 1 This embodiment uses the deep learning-based encrypted malicious traffic detection method proposed in this invention to detect malicious encrypted traffic. The steps are as follows:

[0099] 1. Dataset Selection: In this embodiment, the CIC-IDS2017 dataset is selected as the experimental object. The CIC-IDS2017 dataset contains various types of network traffic, including normal traffic and various known malicious traffic (such as DoS attacks, DDoS attacks, data breaches, malware propagation, etc.). This dataset is widely used in network security research, has high realism and diversity, and is suitable as an evaluation benchmark for malicious traffic detection methods.

[0100] 2. Data Preprocessing: Network traffic records were extracted from the CIC-IDS2017 dataset, including features such as protocol type, packet size, latency, source IP address, destination IP address, and transmission port. Outliers and duplicate records were removed, missing values ​​were imputed using the mean imputation method, and numerical features were standardized using Z-score standardization.

[0101] 3. Feature Extraction and Fusion: First, the preprocessed traffic data is arranged in chronological order to form time series data. Next, a model containing two LSTM layers is constructed to extract temporal features at each time step, generating a temporal feature vector. Then, the temporal feature vector extracted by the LSTM is used as input to a pre-trained BERT model for further semantic feature extraction. Finally, a feature concatenation method is used to fuse the temporal features extracted by the LSTM with the semantic features extracted by the BERT model, forming a comprehensive feature representation vector.

[0102] 4. Cluster Analysis: The neighborhood radius (eps) of DBSCAN is set to 0.5, and the minimum number of samples (min_samples) is set to 10. The DBSCAN algorithm is applied to cluster the comprehensive feature representation vector, dividing the traffic samples into different clusters, and assigning a -1 label to noise points. According to predefined naming rules, each cluster label is mapped to a unified attack category label (such as "Denial of Service Attacks", "Data Exfiltration", etc.), and noise points are marked as "Unknown".

[0103] 5. Cosine Similarity-Based Graph Construction: Each network traffic sample is treated as a node in the graph. The cosine similarity between samples within each cluster is calculated. If the similarity is greater than a threshold θ (set to 0.8), an edge is established between the corresponding nodes. Similarly, the cosine similarity between samples from different clusters is calculated; if the similarity is greater than θ, an edge is established between these nodes. This ultimately forms a graph structure containing nodes and edges, where the edges between nodes reflect the feature similarity between traffic samples.

[0104] 6. GraphSAGE Model Application: Construct a graph neural network model containing two GraphSAGE layers, using mean aggregation as the feature aggregation function. Each GraphSAGE layer samples a fixed number of neighboring nodes from each node's neighbors, aggregates the feature information of the neighboring nodes through the GraphSAGE layer, and updates the node's feature representation. A fully connected layer is connected to the output layer of GraphSAGE to output the classification result (malicious traffic or normal traffic) for each node. The Adam optimizer is used, with a learning rate of 0.001. The cross-entropy loss function is minimized through backpropagation. The GraphSAGE model is trained using labeled training data for 50 epochs.

[0105] 7. Experimental setup: The CIC-IDS2017 dataset was divided into a training set (70%) and a test set (30%). Accuracy, precision, recall, and F1 score were used as the main evaluation metrics.

[0106] 8. Experimental Results: This embodiment demonstrates the superior performance of the deep learning-based encrypted malicious traffic detection method in practical applications through systematic experimental verification, as shown in Table 1. Compared with traditional single feature extraction or traditional machine learning methods, the proposed method achieves significant performance improvements through multi-level feature fusion, effective clustering analysis, and the application of graph neural networks.

[0107] Table 1 Experimental Results

[0108]

[0109] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0110] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for detecting encrypted malicious traffic based on deep learning, characterized in that, Includes the following steps: Step 1: Use the LSTM model to extract the temporal features of network traffic data, and use the BERT model to further extract the semantic features of traffic data, finally forming a comprehensive feature representation; Step 2: Use the DBSCAN algorithm to cluster the comprehensive feature representation. Divide the network traffic data samples into different clusters based on feature similarity. Each cluster represents a potential encrypted traffic type. Assign a unique cluster label to each cluster to represent the potential encrypted traffic type. Step 3: Optimize the sample labels in each cluster according to the defined naming rules to achieve fuzzy classification of traffic labels and classify traffic types with similar attack characteristics into a unified label; Step 4: Construct a graph structure by connecting the optimized traffic sample labels and their comprehensive feature representations with edges, and establishing edge connections between different clusters based on cosine similarity to form a complete graph model. Step 5: Construct the GraphSAGE graph neural network model, aggregate information of the nodes and their neighboring nodes in the graph structure to obtain the updated feature vector of each node, and use the updated node features for classification to identify whether it is malicious encrypted traffic. Step 6: Train the GraphSAGE graph neural network model and classifier using backpropagation and optimization algorithms, and perform malicious traffic detection on new network traffic samples. in, Step 4: Construct a graph structure by connecting the optimized traffic sample labels and their comprehensive feature representations with edges, and establishing edge connections between different clusters based on cosine similarity to form a complete graph model. The specific method is as follows: Step 4-1: Summarize the edge information within and across clusters to construct a complete graph model. Nodes in the graph represent network traffic samples, and edges represent the similarity relationships between samples. Step 4-2: For each cluster, establish edges between samples within the cluster based on cosine similarity. The specific steps are as follows: Traverse each cluster and obtain the feature vectors of all samples in the cluster, including protocol type, packet size, and latency. Calculate the cosine similarity between any two samples in the cluster. If the cosine similarity between two samples is greater than a preset threshold θ, then establish an edge between the two nodes to indicate that they have high feature similarity; otherwise, do not establish an edge. Step 4-3: Establish edges between different clusters. The specific steps are as follows: Traverse the sample pairs between different clusters and calculate their cosine similarity. If the cosine similarity between two samples from different clusters is greater than the threshold θ, then establish an edge between the two nodes; otherwise, do not establish an edge. Step 5: Construct the GraphSAGE graph neural network model. Aggregate information from the nodes and their neighboring nodes in the graph structure to obtain the updated feature vector of each node. Use the updated node features for classification to identify whether it is malicious encrypted traffic. The specific method is as follows: Step 5-1: Construct the GraphSAGE graph neural network model. The GraphSAGE graph neural network model consists of multiple GraphSAGE layers connected in series. Each GraphSAGE layer includes a neighbor sampling module, a feature aggregation module, and a feature transformation module, wherein: The neighbor sampling module, for each target node v, randomly samples a fixed number of neighbor nodes from its neighbor node set N(v) to generate a sampled subset of neighbor features. Output the sampled neighbor feature set N(X) k-1 ), the sampled neighbor features N(X) k-1 Enter the feature aggregation module; The feature aggregation module consists of three sub-modules: mean aggregation, maximum aggregation, and LSTM aggregation. The mean aggregation module calculates the element-wise average of the feature vectors of the sampled neighbors to generate the mean-aggregated feature f. mean The maximum value aggregation module extracts the maximum value element by element from the neighbor features, generating the maximum value aggregation feature f. max The LSTM aggregation module converts neighbor feature sequences. The input is fed into an LSTM network for sequence processing, and the output is a fixed-dimensional LSTM aggregated feature f. LSTM The output of the feature aggregation module is a comprehensive aggregated feature vector [f] mean ,f max ,f LSTM ], the comprehensive aggregated feature vector [f mean ,f max ,f LSTM It is passed to the feature transformation module; In the feature transformation module, aggregated features are transformed by a linear transformation matrix W. k A linear transformation is performed, followed by processing with the ReLU activation function σ to generate new node feature representations. The specific update formula is as follows: Where AGGREGATE represents the selected aggregation operation, and σ selects the ReLU activation function. This represents the feature representation of node v at the k-th layer. Output of the feature transformation module X will be used as the input X of the next GraphSAGE layer k The process of neighbor sampling and feature aggregation continues. Through layer stacking, node features are gradually fused with information from a wider range of neighbors to generate the final node feature matrix X. n ; Step 5-2: The classifier consists of fully connected layers that receive X. n A series of linear transformations are used to generate a classification output Y, which is used to determine whether each node is malicious encrypted traffic.

2. The method for detecting encrypted malicious traffic based on deep learning according to claim 1, characterized in that, Step 1: Extract the temporal features of network traffic data using the LSTM model, and further extract the semantic features of the traffic data using the BERT model to finally form a comprehensive feature representation. The specific method is as follows: Step 1-1: First, collect raw network traffic data from network monitoring equipment, log systems, or other data sources, including protocol type, packet size, latency, source IP address, destination IP address, and transmission port. Then, preprocess the collected traffic data, including noise removal, missing value filling, and standardization. Steps 1-2: Arrange the preprocessed network traffic data in chronological order to form time series data. Use the LSTM model to process these time series data and extract the key temporal features from the traffic data. Steps 1-3: Input the temporal feature vector extracted by the LSTM model into the BERT model, and through its self-attention mechanism, analyze the contextual semantic information of the traffic data to further extract the semantic features of the traffic. Steps 1-4: Use the feature concatenation method to fuse the temporal features extracted by LSTM with the semantic features extracted by BERT to form a comprehensive feature representation vector.

3. The method for detecting encrypted malicious traffic based on deep learning according to claim 1, characterized in that, Step 2: Cluster the comprehensive feature representation using the DBSCAN algorithm. Based on feature similarity, divide the network traffic data samples into different clusters, each cluster representing a potential type of encrypted traffic. The specific method is as follows: Step 2-1: Use the DBSCAN algorithm to calculate the neighborhood density of each traffic sample, identify the high-density area as the core point, and determine the density connection samples around the core point according to the preset neighborhood radius and density threshold. Use these density connection samples to classify traffic samples with similar characteristics into the same category. Step 2-2: Assign a unique cluster label to each cluster, representing the potential type of encrypted traffic; Steps 2-3: Record the cluster label of each traffic sample to generate a dataset containing the traffic sample and its corresponding cluster label.

4. The method for detecting encrypted malicious traffic based on deep learning according to claim 1, characterized in that, Step 3: Optimize the sample labels in each cluster according to the defined naming rules to achieve fuzzy classification of traffic labels, grouping traffic types with similar attack characteristics into a unified label. The specific method is as follows: Step 3-1: Establish a unified naming convention, including: Denial of Service Attacks: including DoS and DDoS attacks; Data Exfiltration: including DNS tunneling and HTTP tunneling; Malware Traffic: Includes RAT traffic and virus propagation traffic; Network Scanning: Includes port scanning and network traffic reconnaissance; Phishing Attacks: Including phishing emails and phishing website traffic; Unknown: Used to identify noise points that cannot be classified or traffic samples of unknown type; Step 3-2: Map each cluster label to the corresponding unified label according to the defined naming rules.

5. The method for detecting encrypted malicious traffic based on deep learning according to claim 1, characterized in that, Step 6: Train the GraphSAGE graph neural network model and classifier using backpropagation and optimization algorithms, and perform malicious traffic detection on new network traffic samples. The specific method is as follows: Labeled training data was used to optimize the parameters of the GraphSAGE model and classifier through backpropagation and the Adam optimizer. During the optimization process, the learning rate of the generator and discriminator was set to 0.001, the momentum parameters β1 and β2 were set to 0.9 and 0.999 respectively, and the batch size was set to 8.

6. A deep learning-based encrypted malicious traffic detection system, characterized in that, The method for detecting encrypted malicious traffic based on deep learning, as described in any one of claims 1-5, is implemented to achieve the detection of encrypted malicious traffic based on deep learning. Steps 1 to 6 are executed in six modules respectively.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the deep learning-based encrypted malicious traffic detection method according to any one of claims 1-5, thereby realizing deep learning-based encrypted malicious traffic detection.

8. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the deep learning-based encrypted malicious traffic detection method according to any one of claims 1-5, thereby realizing deep learning-based encrypted malicious traffic detection.

Images