Multi-modal tunnel encrypted traffic fine-grained sensing method based on pre-training model and graph neural network

Abstract

A multi-modal tunnel encrypted traffic fine-grained sensing method based on a pre-training model and a graph neural network belongs to the crossing field of tunnel encrypted traffic and artificial intelligence, is used for improving traffic classification accuracy, and is technically characterized by cleaning encrypted traffic data and cutting the data into sessions; performing data enhancement by using a sliding window; preprocessing the data to generate a multi-modal data set; training a perception model according to the data set; through the perception model, extracting encrypted tunnel traffic global semantic features, encrypted tunnel traffic time sequence features and encrypted tunnel traffic non-Euclidean features; fusing the traffic global semantic feature, the traffic non-Euclidean feature and the traffic time sequence feature through the perception model; and the perception model classifies the traffic according to the fusion features.

Description

Technical Field

[0001] The present invention relates to the cross-technical field of tunnel encryption traffic and artificial intelligence, and in particular to a multimodal tunnel encryption traffic fine-grained perception method based on a pre-trained model and a graph neural network. Background Art

[0002] With the increasing awareness of protecting personal privacy information and the rapid development of traffic encryption technology, more and more netizens choose to use encryption technologies such as VPN and Tor to meet their own Internet needs. According to Google data, as of February 2024, 96% of Chrome web pages have enabled SSL / TLS encryption. However, while effectively protecting user privacy, traffic encryption technology encrypts the payload part of the original data packet, resulting in the inability of traffic classification methods based on deep packet inspection to analyze patterns and keywords in the encrypted payload, thus bringing greater technical challenges to network management and behavior analysis for operating service providers and security management departments. Therefore, there is an urgent need for an effective method to correctly classify encrypted traffic, so as to help network administrators better manage communication behaviors in different categories.

[0003] In order to solve the above problems, people have tried many different methods. From the initial port-based identification, to experts manually performing feature engineering based on statistical features in network traffic (such as packet length, packet arrival time, etc.), and using machine learning classifiers (such as FlowPrint, AppScanner, etc.) to obtain traffic classification results. However, feature engineering requires reliance on professional domain knowledge and is difficult to generalize, so machine learning methods are relatively challenging.

[0004] Deep learning is a hot topic in the current research of encrypted traffic classification because it can automatically learn the implicit feature relationships in data without feature engineering. Existing deep learning-based methods can be summarized into several categories: image-based, text-based, pre-training-based, graph neural network-based, and multimodal-based methods.

[0005] First, for CNN-based classifiers, Wang et al. first proposed the idea of ​​converting network traffic into grayscale images, and further used 1DCNN to classify encrypted traffic. However, in the process of converting traffic into images, there is no spatial feature for the encrypted traffic bytes corresponding to the upper and lower adjacent pixels of the image. Secondly, for RNN-based classifiers, FS-Net uses recurrent neural networks (RNNs) to treat network traffic as time series, but recurrent neural networks themselves have the problem of long-term memory failure that is difficult to avoid. In addition, the implementation of the above two supervised methods requires a large amount of labeled data for training, and performs poorly when faced with insufficient data.

[0006] In order to alleviate the problem of insufficient data, methods based on pre-training came into being. PERT first introduced dynamic word embedding technology into pre-training. Lin et al. proposed ET-BERT, and first proposed two pre-training tasks: homologous BURST prediction and masked BURST. YaTC first introduced the idea of ​​MAE into encrypted traffic classification, and learned representation by reconstructing encrypted traffic images. However, existing pre-training methods simply treat the header and payload of the data packet equally, or only use the encrypted payload, which ignores the actual meaning of the header and payload. Fourth, for graph neural networks, Shen et al. proposed the GraphDApp model, which constructs a traffic interaction graph by utilizing BURST in encrypted traffic, but there are problems such as long training convergence time, and the traffic interaction graph only focuses on packet-level features and fails to utilize the overall features of BURST.

[0007] It can be observed that most deep learning methods only focus on a single modal feature and fail to fully learn the underlying interaction patterns of encrypted traffic. Encrypted network traffic essentially contains information of multiple modalities, which can be generally divided into two categories. First, features in Euclidean space: side channel information containing traffic statistical features, global semantic information extracted through pre-trained text models, spatial information converted into images, and temporal information regarded as time series. Second, due to the complex topological structure and dynamic characteristics of modern computer networks, encrypted traffic also contains typical non-Euclidean information.

[0008] Therefore, multimodal methods have become a research hotspot in recent years. ATVITSC converts data packets in encrypted traffic into images, and uses VIT and spatiotemporal feature extraction modules based on CNN and BiLSTM to extract semantic features and spatiotemporal features at the traffic packet level for classification. MTCM uses the original bytes of the packet as the node features of GNN, and embeds the flow-level features extracted by BERT into the graph nodes on the original graph neural network classification network that combines SGC and GAT. It is easy to cause the encrypted traffic flow-level features to confuse the packet drop features, resulting in slow convergence and low reproduction accuracy. Although multimodal features have been used in existing research, the choice of modality only stays within the Euclidean features, and the effect of using Euclidean features and non-Euclidean features at the same time is not good. In addition, pre-training is required again, and the convergence speed and training cost are high. Summary of the invention

[0009] In order to solve the above problems and improve the accuracy of traffic classification, in a first aspect, a multimodal tunnel encrypted traffic fine-grained perception method based on a pre-trained model and a graph neural network in some embodiments of the present application includes the following steps:

[0010] Clean the encrypted traffic data and cut it into sessions;

[0011] Use sliding windows for data augmentation;

[0012] Preprocess the data to generate a multimodal dataset;

[0013] training a perception model based on the data set;

[0014] Through the perception model, extract the global semantic features of the encrypted tunnel traffic, the time series features of the encrypted tunnel traffic, and the non-Euclidean features of the encrypted tunnel traffic;

[0015] The traffic global semantic features, traffic non-Euclidean features and traffic time series features are integrated through the perception model;

[0016] The perception model classifies traffic based on the fused features.

[0017] According to the multimodal tunnel encrypted traffic fine-grained perception method based on a pre-trained model and a graph neural network in some embodiments of the present application, cleaning the encrypted traffic data and cutting it into sessions includes:

[0018] Step 1.1. Clean the original Pcap data, including:

[0019] Use tshark to convert Pcapng files into Pcap files;

[0020] Each Pcap file contains a maximum of 100,000 Pcapts, and removes network query messages, leaving only IP messages;

[0021] Copy the cleaned Pcap files to folders representing different categories in App and Service according to the granularity;

[0022] Step 1.2. Divide into sessions, including:

[0023] Use SplitCap to split the original Pcap traffic into two-way conversation flows;

[0024] Among them, the bidirectional session flow is a pair of unidirectional flows with interchangeable sources and destinations, and the unidirectional flow is a set of source-to-destination data packets identified by a five-tuple.

[0025] According to the multimodal tunnel encrypted traffic fine-grained perception method based on pre-trained model and graph neural network in some embodiments of the present application, wherein data enhancement is performed using a sliding window, including

[0026] Use EditCap to split the data into sliding windows of size 25 for data enhancement;

[0027] There are no more than 500 samples in each category.

[0028] According to a multimodal tunnel encrypted traffic fine-grained perception method based on a pre-trained model and a graph neural network in some embodiments of the present application, data is pre-processed to generate a multimodal data set, including:

[0029] Step 2.1. Construct a burst relationship graph BRG (BURST Relationship Graph), where BURST is defined as a group of consecutive data packets in the same direction within a short period of time;

[0030] Step 2.2. Convert the encrypted traffic payload into BERT Token; form a tuple of the burst relationship graph BRG, Token and Label to obtain a processed multimodal dataset;

[0031] Step 2.3. Split the processed multimodal dataset into training set and test set;

[0032] Wherein, the burst relationship graph BRG of the undirected graph in step 2.1 includes nodes and edges;

[0033] The nodes include nodes representing packets in the BURST and nodes representing the entire BURST; for the nodes representing packets in the BURST, each node is associated with the length and arrival time interval of the packet, the arrival time interval is discretized and normalized, and the direction of the BURST is distinguished by the positive and negative values ​​of the packets; for the nodes representing the entire BURST, the nodes are associated with the average value of the length of all packets in the current BURST and the average value of the arrival time interval, representing the overall characteristics of the current BURST;

[0034] Among them, the edges are BURST internal edges and BURST inter-edges, wherein, for BURST internal edges, they connect the consecutive nodes in each BURST in sequence, and connect the start node and the end node of the current BURST to the node representing the overall characteristics of the BURST; wherein, for BURST inter-edges, they are constructed by connecting the nodes representing the overall characteristics of each BURST for different BURSTs in the same session;

[0035] Among them, step 2.2. The encrypted traffic payload is converted into BERT Token, including

[0036] Remove the header of each data packet in the session, leaving only the encrypted payload;

[0037] Encode the encrypted payload in hexadecimal and use the Bi-Gram model to encode the encrypted payload, where each unit consists of two adjacent bytes and each Token unit does not exceed the maximum value of the dictionary, 65535;

[0038] Take the first five packets in each conversation, concatenate the encoded payloads together, and add special tokens [CLS] and [SEP] at the beginning and end;

[0039] Among them, step 2.3. splitting the processed multimodal dataset into a training set and a test set includes:

[0040] The processed multimodal dataset is split into training set and test set in a ratio of 8:2 through Sklearn's train_test_split function, and saved as pt files in turn.

[0041] According to the multimodal tunnel encrypted traffic fine-grained perception method based on pre-trained model and graph neural network in some embodiments of the present application, the perception model (PGM) includes a global semantic feature extraction module, a temporal feature extraction module, a graph neural network module, a gated multi-scale feature fusion mechanism and a classification module;

[0042] The steps include extracting global semantic features of encrypted tunnel traffic, temporal features of encrypted tunnel traffic, and non-Euclidean features of encrypted tunnel traffic through a perception model, and include the following steps:

[0043] Step 3.1. The global semantic feature extraction module extracts the global semantic feature Pooler_Output of the encrypted tunnel traffic;

[0044] The global semantic feature extraction module uses the same pre-trained BERT as ETBERT as the global semantic feature extraction module, which consists of 12 transformer blocks, 12 attention heads in each self-attention layer, hidden_size is set to 768, and the input token length is 512;

[0045] Step 3.2. The timing feature extraction module extracts the timing features of the encrypted tunnel traffic;

[0046] Among them, the time series feature extraction module uses the time convolution module to extract time series features. The sequence_output dimension output by the BERT module is [Batchsize, sequence_len, hidden_size], where the length of sequence_len is 512;

[0047] Consider sequence_len as the length of the time series and hidden_size as the number of feature channels, and input them into the time series feature extraction module;

[0048] Among them, the temporal feature extraction module consists of two dilated convolutional layers DIL, one DIL is followed by a tanh activation function and used as a filter, and the other DIL is followed by a sigmoid activation function and used as a gate to control the information passed from the filter to the next module;

[0049] Step 3.3. The graph neural network module extracts the non-Euclidean features of the encrypted tunnel traffic;

[0050] Among them, the graph neural network module consists of three layers of GIN modules. The message transmission and aggregation mechanism of the GIN module is as follows:

[0051]

[0052] In the formula, is the embedding vector of node v at layer k; MLP (k) is the k-th layer of the multilayer perceptron; ∈ is a learnable parameter, or is set to a fixed constant to adjust the influence of the node's own features; N(v) represents the set of neighbor nodes of node v; is the embedding vector of node v at the k-1th layer; is the embedding vector of node u at the k-1th layer; node u is the node directly connected to node v;

[0053] The side channel features contained in the input BRG are first converted into graph embedding representations through the embedding layer in the MLP;

[0054] In each layer of the GIN module, the ReadOut operation is used to obtain multi-scale full-image feature representation;

[0055] The full-image feature representation of each layer is combined into a three-channel one-dimensional vector combination, and the channel attention module SEBlock is used to fuse the full-image feature representations of different scales;

[0056] The weighted full-graph features of different scales are added in the channel dimension to obtain the non-Euclidean graph feature representation of the encrypted tunnel traffic, which is expressed by the following formula:

[0057]

[0058] s=σ(W1z+b1)

[0059] s'=σ(W2s+b2

[0060] y=X·Sigmoid(s′)

[0061] Where H represents the characteristic length of the input vector; z cis the global average pooling result of channel c, which represents the global feature of the channel; S is the weight of the channel generated by compressing the feature length dimension using average pooling; s' is the weight of each channel obtained by restoring the number of channels by the excitation operation; y represents the weighted feature of each channel obtained by normalizing it with Sigmoid and multiplying it with the input X; x h,c Represents the eigenvalue of the input feature at position h and channel c; W1 represents the weight matrix of the first fully connected layer in the excitation operation; b1 represents the bias vector of the first fully connected layer: W2 represents the weight matrix of the second fully connected layer in the excitation operation; b2 represents: the bias vector of the second fully connected layer;

[0062] The step of fusing the traffic global features, traffic non-Euclidean features and traffic time series features through a perception model includes the following steps:

[0063] Step 3.4. The gated multi-scale feature fusion mechanism fuses the global semantic features, traffic non-Euclidean features and traffic temporal features;

[0064] Among them, the gated multi-scale feature fusion mechanism firstly passes the global semantic features extracted by the BERT module of the global semantic feature extraction module, the temporal features extracted by the temporal feature extraction module, and the traffic non-Euclidean features extracted by the graph neural network module through the Feature Modify layer, and standardizes the feature dimension to 128;

[0065] In the gating branch of Self-Gating, the 1-dimensional weight coefficient relative to itself is obtained through MLP;

[0066] The weight coefficients of each feature are spliced ​​together and input into softmax to obtain the weight coefficients relative to other features;

[0067] The weight coefficient is multiplied by its own features, and the features of multiple scales are added together to obtain the final features used for classification;

[0068] The perception model in step 1 classifies traffic according to the fusion features, including the following steps:

[0069] Step 3.5. The classification module classifies the traffic according to the fusion features;

[0070] Among them, the classification module is a two-layer fully connected layer.

[0071] According to the multimodal tunnel encrypted traffic fine-grained perception method based on a pre-trained model and a graph neural network in some embodiments of the present application, the classification module classification step includes:

[0072] First increase the dimension to 256 and then reduce the dimension to the number of categories;

[0073] Combined with cross entropy loss function for multi-classification.

[0074] According to the multimodal tunnel encrypted traffic fine-grained perception method based on pre-trained model and graph neural network in some embodiments of the present application, each DIL layer applies a set of 1D causal dilation convolutions with convolution kernel sizes of 2, 3, 6, and 7 to discover time patterns of various ranges;

[0075] Each causal dilation convolution maps the channel to 1 / 4 of the input channel, and takes the feature with the least time dimension among the four dilated causal convolutions as the standard in the time dimension to keep their time dimensions consistent;

[0076] The four multi-scale feature values ​​are concatenated on the channel to restore the same number of channels as the input, and the multi-scale temporal features extracted by the dilated causal convolution are passed through two fully connected layers;

[0077] First increase the dimension and then reduce it to increase the expressive power and restore the same time dimension as the input.

[0078] According to the multimodal tunnel encrypted traffic fine-grained perception method based on pre-trained models and graph neural networks in some embodiments of the present application, the perception model is trained according to the data set, using the AdamW optimizer, the learning rate of the BERT module is 2e-5, the dropout is 0.5, the learning rate of other modules is 5e-4, the batchsize is 32, the seed is 7, the warm-up ratio is 0.1, and a total of 70 epochs are trained.

[0079] In a second aspect, an embodiment of the present application further provides an electronic device, comprising: one or more processors, a memory, and one or more programs; wherein the one or more programs are stored in the memory, and the one or more programs include instructions, which, when executed by the electronic device, enable the electronic device to execute the first aspect and any possible technical solution of the first aspect.

[0080] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium includes a computer program. When the computer program runs on an electronic device, the electronic device executes the first aspect and any possible technical solution of the first aspect.

[0081] Beneficial effects of the present invention:

[0082] In the first aspect, the present invention proposes a new graph neural network mapping method BRG, which can effectively integrate local and overall features within BURST, retain the topological structure of the traffic, and thus integrate the features between different BURSTs, effectively combining the non-Euclidean and side channel features of encrypted traffic, thereby improving the classification accuracy of the model.

[0083] On the second aspect, the model of the present invention utilizes the heterogeneous characteristics of encrypted traffic from different perspectives, and effectively extracts the global semantic features, timing features, side channel features and non-Euclidean features of the encrypted traffic by processing the packet header and encrypted payload separately, so that the features of different modes between the packet header and the payload can complement each other, enhance the information exchange between the features, and thus improve the effect of multimodal learning and classification performance.

[0084] On the third aspect, the present invention proposes a gated multi-scale feature fusion mechanism, which enables the model to automatically adjust parameters, strengthen useful features, and suppress less useful features. It can also automatically learn the feature combination that best reflects the current tunnel encryption traffic according to the contribution of features of different scales, achieving a higher convergence speed and improving the classification accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 This is an overall flow chart of the fine-grained perception method of multimodal tunnel encrypted traffic based on a pre-trained model and a graph neural network described in an embodiment of the present invention.

[0086] Figure 2 This is an example diagram of the data preprocessing described in the embodiment of the present invention.

[0087] Figure 3 This is a model architecture diagram (PGM) of the multimodal tunnel encrypted traffic fine-grained perception method based on a pre-trained model and a graph neural network described in an embodiment of the present invention.

[0088] Figure 4 for Figure 3 Architecture diagram of the MLGFBlock module.

[0089] Figure 5 for Figure 3 Architecture diagram of the Dilated Inception Layer module in Figure 2. DETAILED DESCRIPTION

[0090] The embodiments of the present invention are implemented on the premise of the technical solution of the present invention, and detailed implementation methods and specific operation processes are given, but the protection scope of the present invention is not limited to the following embodiments. The embodiments of the present application are described in detail below with reference to the accompanying drawings, and examples of the embodiments are shown in the accompanying drawings. The present application provides a method, an electronic device, and a computer-readable storage medium. Among them, the method, the device, and the computer-readable storage medium are based on the same technical concept. Since the principles of solving the problem by the method and the device are similar, the implementation of the device and the method can refer to each other, and the repeated parts will not be repeated.

[0091] In the first embodiment, in order to solve the shortcomings of the existing multimodal technology, the present invention utilizes the heterogeneous characteristics of encrypted traffic from different perspectives as much as possible, so that the features between different modalities can complement each other and enhance the information exchange between features, thereby improving the effect of multimodal learning and classification performance.

[0092] Based on this, the present invention proposes a multimodal tunnel encrypted traffic fine-grained perception method based on a pre-trained model and graph neural network, which is a new graph neural network graph construction method BRG (BURST Relationship Graph), which can effectively integrate local and overall features within BURST, and effectively combine the non-Euclidean and side channel features of encrypted traffic.

[0093] In addition, this method also utilizes the global semantic features and temporal features of encrypted traffic, and proposes a feature fusion mechanism, which enables the model to learn and automatically fuse the global, multi-scale, and multi-modal features of tunnel encrypted traffic, thereby improving the classification accuracy of the model.

[0094] Specifically, the multimodal tunnel encrypted traffic fine-grained perception method based on a pre-trained model and a graph neural network of the present invention includes the following steps:

[0095] Step 1: Clean the original Pcap dataset and divide it into Service and App by granularity.

[0096] Furthermore, the step 1 specifically includes:

[0097] Step 1.1: Clean the original Pcap data, use tshark to convert the Pcapng file to a Pcap file, cut each Pcap file to contain a maximum of 100,000 Pcakets, and remove network query messages (ICMP, DNS and other useless protocols), and only retain IP messages, because network query messages are usually caused by domain name resolution, statistical analysis and other requests that are not related to encrypted traffic classification. Retaining only IP messages can eliminate interference factors in the training and testing process and further improve efficiency. And copy the cleaned Pcap files to the folders representing different categories in App and Service according to the division granularity in Table 1.

[0098] Table 1 Division of ISCXVPN2016

[0099]

[0100]

[0101] Step 1.2: Use SplitCap to split the original Pcap traffic into bidirectional conversation flows. A unidirectional flow is a collection of source-to-destination data packets identified by a five-tuple (source IP address, source port number, destination IP address, destination port number, communication protocol). A bidirectional conversation flow is a paired unidirectional flow with interchangeable source and destination.

[0102] Step 1.3: For the larger files in the divided conversation stream, use EditCap to split them again according to the sliding window data of size 25, and perform data enhancement. Each category does not exceed 500 samples.

[0103] Step 2: Construct a multimodal dataset and divide it into training set and test set.

[0104] Furthermore, the step 2 specifically includes:

[0105] Step 2.1: Construct a burst relationship graph (BRG). A BURST is defined as a group of consecutive packets in the same direction within a short period of time. The construction process of the undirected graph BRG is as follows:

[0106] Node. BRG contains two types of nodes. One is the node representing the data packets in the BURST. Each node is associated with the length and arrival time interval of the data packet. At the same time, we distinguish the direction of the BURST by making the data packets have positive and negative values, and discretize and normalize the arrival time interval. The other is the node representing the entire BURST. The node is related to the average value of the length of all data packets in the current BURST and the average value of the arrival time interval, thus representing the overall characteristics of the current BURST.

[0107] Edges. There are also two types of edges in BRG: intra-BURST edges and inter-BURST edges. Intra-BURST edges connect the consecutive nodes (data packets) in each BURST from beginning to end, and connect the start and end nodes of the current BURST to the nodes representing the overall characteristics of the BURST. For different BURSTs in the same session, we connect the nodes representing the overall characteristics of each BURST.

[0108] Step 2.2: Convert the encrypted traffic payload into BERT Token; form a tuple of the burst relationship graph BRG, Token, and Label to obtain a processed multimodal dataset;

[0109] First, the header (Ethernet header, IP header, and transport layer header) of each data packet in the session is deleted, leaving only the encrypted payload. The encrypted payload is then encoded in hexadecimal and encoded using the Bi-Gram model, where each unit consists of two adjacent bytes and each Token unit does not exceed the maximum value of the dictionary, 65535.

[0110] Take the first five packets in each conversation, concatenate the encoded payloads together in sequence, and control the total length to 512 by truncation or padding with 0, and add special tokens [CLS] and [SEP] at the beginning and end.

[0111] Step 2.3: Split the processed multimodal dataset into training set and test set. Use the train_test_split function of Sklearn to split the processed multimodal dataset into training set and test set in a ratio of 8:2, and save them as pt files in turn.

[0112] Step 3: Build a multimodal tunnel encrypted traffic fine-grained perception model (PGM) based on pre-trained models and graph neural networks. PGM includes global semantic feature extraction module, temporal feature extraction module, graph neural network module, gated multi-scale feature fusion mechanism, and classification module.

[0113] Furthermore, the step 3 specifically includes:

[0114] Step 3.1: Global semantic feature extraction module. The same pre-trained BERT as ETBERT is used as the global semantic feature extraction module. It consists of 12 transformer blocks, each self-attention layer has 12 attention heads, hidden_size is set to 768, and the input token length is 512.

[0115] Step 3.2: Temporal feature extraction module. Because the sequence_output dimension of the BERT module output is [Batchsize, sequence_len, hidden_size], where the length of sequence_len is 512, the problem of memory loss cannot be avoided for traditional RNN or bidirectional RNN such as Bi-GRU. Therefore, the temporal convolution module is used to extract temporal features. We regard sequence_len as the length of the time series and hidden_size as the number of feature channels, which are input into this module. This module consists of two dilated convolutional layers (DIL), one DIL is followed by a tanh activation function, which is used as a filter. The other layer is followed by a sigmoid activation function, which acts as a gate to control the information that the filter can pass to the next module. In addition, each DIL layer applies a set of 1D causal dilated convolutions with kernel sizes of 2, 3, 6, and 7 to discover temporal patterns of various ranges.

[0116] Step 3.3: Graph neural network module. In order to extract local and global features for full-graph classification in BRG, the graph neural network module consists of three layers of GIN modules. GIN's ability to distinguish different graph structures can reach the highest level of GNN network's discrimination ability in WL graph isomorphism test. GIN's message passing and aggregation mechanism is as follows: is the embedding vector of node v in the kth layer. MLP is the multilayer perceptron in the kth layer. ∈ is a learnable parameter, or set to a fixed constant, used to adjust the influence of the node's own features. N(v) represents the set of neighbor nodes of node v, is the embedding vector of node v at the k-1th layer; is the embedding vector of node u at the k-1th layer; node u is the node directly connected to node v;

[0117]

[0118] The side channel features (signed packet length and packet inter-arrival time) contained in the input BRG are first converted into graph embedding representations through the embedded layer in the MLP. The ReadOut operation is used in each layer of GIN to obtain multi-scale full-graph feature representations. The full-graph feature representation of each layer is combined into a three-channel one-dimensional vector combination, and the channel attention module SEBlock is used to fuse the full-graph feature representations of different scales. Finally, the weighted full-graph features of different scales are added in the channel dimension to obtain the non-Euclidean graph feature representation of the encrypted tunnel traffic.

[0119] Step 3.4: Gated multi-scale feature fusion mechanism.

[0120] This module is inspired by the Self-Gating self-gating module. Self-Gating is widely used because it can automatically select which features should be retained based on the input features through learnable parameters, thereby enhancing the model's ability to capture key information, while suppressing noise or irrelevant information, and preventing information overload or gradient disappearance.

[0121] First, the global semantic feature Pooler_Output output by the BERT module, the features extracted by the time series feature extraction module, and the features extracted by the graph neural network are uniformly passed through the Feature Modify layer, and the feature dimension is standardized to 128. Secondly, in the gated branch of Self-Gating, the 1-dimensional weight coefficient relative to itself is obtained through MLP, and then the parameters of each feature are spliced ​​together and input into softmax to obtain the weight coefficient relative to other features. Finally, the weight coefficient is multiplied by its own features, and the features of multiple scales are added to obtain the final features used for classification.

[0122] As mentioned above, the existing models extract less than two of the above features. The reason is that too much feature extraction and fusion will lead to slow model convergence (almost reaching more than 100 Epochs, mostly concentrated at around 150 Epochs), which is difficult to train. The reason why the present invention can extract four features to improve model accuracy and has a higher convergence speed is that the present invention adopts the above-mentioned unique gated multi-scale feature fusion mechanism for feature fusion. Based on the effect of the above fusion, it can be achieved in the case of multi-feature fusion. The convergence speed can be improved (a total of 70 Epochs are trained). Step 3.5: Classification module. The classification module is a two-layer fully connected layer, which is first upgraded to 256 and then reduced to the number of classification categories. And combined with the cross entropy loss function for multi-classification.

[0123] Step 4: Use the multimodal tunnel encrypted traffic dataset obtained in step 2 to train and test the PGM model constructed in step 3. Use the AdamW optimizer, the learning rate of the BERT module is 2e-5, the dropout is 0.5, the learning rate of other modules is 5e-4, the batch size is 32, the seed is 7, the warm-up ratio is 0.1, and a total of 70 epochs are trained.

[0124] In the second embodiment, this embodiment uses the Ubuntu system as the development environment, Pycharm as the development platform, Python as the development language, Pytorch as the development framework, uses A100 for training, and adopts the multimodal tunnel encrypted traffic fine-grained perception method based on the pre-trained model and graph neural network of the present invention to complete the App-level and Service-level traffic classification tasks for the public data set ISCXVPN2016.

[0125] In the second embodiment, the present invention uses the ISCXVPN2016 dataset as input data to conduct experiments at the App level and the Service level. The overall process is as follows: Figure 1 As shown, the details are as follows:

[0126] Step 1: Clean the original Pcap dataset and divide it into Service and App by granularity.

[0127] Furthermore, the step 1 specifically includes:

[0128] Step 1.1: Clean the original Pcap data, use tshark to convert the Pcapng file to a Pcap file, cut each Pcap file to contain a maximum of 100,000 Pcakets, and remove network query messages (ICMP, DNS and other useless protocols), and only retain IP messages, because network query messages are usually caused by domain name resolution, statistical analysis and other requests that are not related to encrypted traffic classification. Retaining only IP messages can eliminate interference factors in the training and testing process and further improve efficiency. And copy the cleaned Pcap files to the folders representing different categories in App and Service according to the division granularity in Table 1.

[0129] Step 1.2: Figure 2 As shown in (1), SplitCap is used to split the original Pcap traffic into bidirectional conversation flows. A unidirectional flow is a set of source-to-destination data packets identified by a five-tuple (source IP address, source port number, destination IP address, destination port number, communication protocol). A bidirectional conversation flow is a paired unidirectional flow with interchangeable source and destination.

[0130] Step 1.3: Bipartite the larger files in the split session stream, such as Figure 2 As shown in , EditCap is used to segment the data according to a sliding window of size 25 for data enhancement, with a maximum of 500 samples per category.

[0131] Step 2: Construct a multimodal dataset and divide it into training set and test set.

[0132] Furthermore, the step 2 specifically includes:

[0133] Step 2.1: Construct a burst relationship graph (BRG). A BURST is defined as a group of consecutive packets in the same direction within a short period of time. The construction process of the undirected graph BRG is as follows: Figure 2 As shown in (2):

[0134] Node. BRG contains two types of nodes. One is the node representing the data packets in the BURST. Each node is associated with the length and arrival time interval of the data packet. At the same time, we distinguish the direction of the BURST by making the data packets have positive and negative values, and discretize and normalize the arrival time interval. The other is the node representing the entire BURST. The node is related to the average value of the length of all data packets in the current BURST and the average value of the arrival time interval, thus representing the overall characteristics of the current BURST.

[0135] Edge. There are also two types of edges in BRG: intra-BURST edges and inter-BURST edges. Intra-BURST edges connect the consecutive nodes (data packets) in each BURST in sequence, and connect the start and end nodes of the current BURST to the nodes representing the overall characteristics of the BURST. For different BURSTs in the same session, such as Figure 2 As shown in ①②, we construct the edges between BURSTs by connecting the nodes representing the overall characteristics of each BURST.

[0136] Step 2.2: Convert the encrypted traffic payload into BERT tokens.

[0137] First, delete the header (Ethernet header, IP header, and transport layer header) of each data packet in the session, leaving only the encrypted payload. Then take the first 5 data packets in each session, encode the encrypted payload in hexadecimal, and encode the encrypted payload using the Bi-Gram model, where each unit consists of two adjacent bytes, and the single element of each Token unit does not exceed the maximum value of the dictionary, 65535. The total length of the Token is controlled to 512 by truncation or padding with 0, and special tokens [CLS] and [SEP] are added at the beginning and end. The final result is as follows: Figure 2 As shown in (3).

[0138] Step 2.3: Split the training set and test set. Use the train_test_split function of Sklearn to split the processed multimodal dataset into a training set and a test set in a ratio of 8:2, and save them as pt files in turn.

[0139] Step 3: Build a multimodal tunnel encrypted traffic fine-grained perception model (PGM) based on pre-trained models and graph neural networks. PGM includes a global semantic feature extraction module, a temporal feature extraction module, a graph neural network module, a gated multi-scale feature fusion mechanism, and a classification module. The specific model structure is as follows: Figure 3 shown.

[0140] Furthermore, the step 3 specifically includes:

[0141] Step 3.1: Global semantic feature extraction module. The same pre-trained BERT as ETBERT is used as the global semantic feature extraction module. It consists of 12 transformer blocks, each self-attention layer has 12 attention heads, hidden_size is set to 768, and the input token length is 512.

[0142] Step 3.2: Time series feature extraction module. Because the sequence_output dimension of the BERT module output is [Batchsize, sequence_len, hidden_size], where the length of sequence_len is 512, the problem of memory loss cannot be avoided for traditional RNN or bidirectional RNN such as Bi-GRU. Therefore, the time convolution module is used to extract time series features. We regard sequence_len as the length of the time series and hidden_size as the number of feature channels, which are input into this module. This module consists of two dilated convolutional layers (DIL), one DIL is followed by a tanh activation function, which is used as a filter. The other layer is followed by a sigmoid activation function, which acts as a gate to control the information that the filter can pass to the next module.

[0143] In addition, for each DIL layer such as Figure 5 As shown in the figure, a set of unpadding 1D causal dilated convolutions with kernel sizes of 2, 3, 6, and 7 are applied to discover temporal patterns of various ranges. At the same time, each causal dilated convolution maps the channel to 1 / 4 of the input channel. The feature with the least temporal dimension among the four dilated causal convolutions is taken as the standard in the temporal dimension to keep their temporal dimensions consistent. Finally, the four multi-scale feature values ​​are concatenated on the channel to restore the same number of channels as the input. The multi-scale temporal features extracted by the dilated causal convolution are passed through two fully connected layers, first increasing the dimension and then reducing the dimension to increase the expression ability and restore the same temporal dimension as the input.

[0144] Step 3.3: Graph neural network module. In order to extract local and global features for full-graph classification in BRG, the graph neural network module consists of three layers of GIN modules. GIN's ability to distinguish different graph structures can reach the highest level of GNN network's discrimination ability in WL graph isomorphism test. GIN's message passing and aggregation mechanism is as follows: is the embedding vector of node v in the kth layer. MLP is the multilayer perceptron in the kth layer. ∈ is a learnable parameter, or set to a fixed constant, used to adjust the influence of the node's own features. N(v) represents the set of neighbor nodes of node v, is the embedding vector of node v at the k-1th layer; is the embedding vector of node u in the k-1th layer; node u is the node directly connected to node v.

[0145]

[0146] The side channel features (signed packet length and packet inter-arrival time) contained in the input BRG are first converted into graph embedding representations through the embedded layer in the MLP. The ReadOut operation is used in each layer of GIN to obtain multi-scale full-graph feature representations. The full-graph feature representation of each layer is combined into a three-channel one-dimensional vector combination, and the channel attention module SEBlock is used to fuse the full-graph feature representations of different scales. Finally, the weighted full-graph features of different scales are added in the channel dimension to obtain the non-Euclidean graph feature representation of the encrypted tunnel traffic.

[0147] Because it is a three-channel one-dimensional vector, SEBlock is deformed here, where H represents the characteristic length of the input vector, z c is the global average pooling result of channel c, which represents the global feature of the channel. S is the weight of the channel generated by compressing the feature length dimension using average pooling. s' is the weight of each channel obtained by restoring the number of channels through the excitation operation (through a two-layer fully connected network). Finally, it is normalized by Sigmoid and multiplied by the input X to obtain the weighted feature of each channel. The formula is as follows:

[0148]

[0149] s=σ(W1z+b1)

[0150] s'=σ(W2s+b2

[0151] y=X·Sigmoid(s′)

[0152] In the formula, x h,c Represents the eigenvalue of the input feature at position h and channel c; W1 represents the weight matrix of the first fully connected layer in the excitation operation; b1 represents the bias vector of the first fully connected layer: W2 represents the weight matrix of the second fully connected layer in the excitation operation; b2 represents: the bias vector of the second fully connected layer.

[0153] Step 3.4: Gated multi-scale feature fusion mechanism.

[0154] like Figure 4As shown in the figure, this module is inspired by the Self-Gating self-gating module. Self-Gating can automatically select which features should be retained based on the input features through learnable parameters, thereby enhancing the model's ability to capture key information, while suppressing noise or irrelevant information, and preventing information overload or gradient disappearance problems. Therefore, it is widely used.

[0155] First, the global semantic feature Pooler_Output output by the BERT module, the features extracted by the temporal feature extraction module, and the features extracted by the graph neural network are uniformly passed through the Feature Modify layer, and the feature dimension is standardized to 128. Secondly, in the gated branch of Self-Gating, the 1-dimensional channel descriptor relative to itself is obtained through MLP, and then the three global channel feature descriptors of different scales are concatenated in the second dimension through the Concat operation of Torch to obtain X_=R C ×3 , and apply Softmax to the second dimension (i.e. the dimension where 3 is located) to obtain the weight coefficient of each scale feature relative to other features. Finally, the weight coefficient is multiplied element by element with the input feature itself, and the features of multiple scales are added to obtain the multi-scale features that are finally used for classification.

[0156] As mentioned above, the existing model extracts less than two of the above features, because too many feature extractions and fusions will lead to slow model convergence (almost reaching more than 100 Epochs, mostly concentrated around 150 Epochs), making it difficult to train. The reason why the present invention can extract four features to improve model accuracy and has a higher convergence speed is that the present invention adopts the above unique gated multi-scale feature fusion mechanism for feature fusion. Based on the above fusion effect, it can be achieved in the case of multi-feature fusion, and the convergence speed can be improved (a total of 70 Epochs are trained).

[0157] Step 3.5: Classification module. The classification module is a two-layer fully connected layer, which first increases the dimension to 256 and then reduces the dimension to the number of categories. It also combines the cross entropy loss function for multi-classification.

[0158] Step 4: Use the multimodal tunnel encrypted traffic dataset obtained in step 2 to train and test the PGM model constructed in step 3. Use the AdamW optimizer, the learning rate of the BERT module is 2e-5, the dropout is 0.5, the learning rate of other modules is 5e-4, the batch size is 32, the seed is 7, the warm-up ratio is 0.1, and a total of 70 epochs are trained.

[0159] Experimental example: The method of the present invention uses six deep learning models from 2022 to 2024 as baselines, and conducts multi-classification comparative experiments on the App and Service classification granularity of ISCXVPN2016. The ratio of training set to validation set is 8:2, and the experimental results of the baseline model all use the data in the corresponding paper. The evaluation index is accuracy, and the formula is as follows:

[0160]

[0161] The accuracy of the model proposed in the present invention is compared with the accuracy of the results of six models, namely ET-BERT, GraphDApp, YaTC, ATVTSC, TFE-GNN and SAFE, to prove the improvement of model performance. ET-BERT is a method proposed in 2022 that uses BERT for pre-training and fine-tuning, GraphDApp is a method proposed in 2021 that uses traffic interaction graphs and combines graph neural networks for classification, YaTC is a method proposed in 2024 that converts encrypted traffic into images and uses MAE for pre-training and fine-tuning, ATVTSC is a multimodal classification method using VIT, CNN and LSTM proposed in 2024, TFE-GNN is a classification method proposed in 2023 that uses a graph neural network to extract features from packet headers and encrypted payloads respectively, and SAFE is a self-supervised method proposed in 2024 that uses contrastive learning to classify encrypted traffic. The experimental results are shown in Table 2.

[0162] Table 2 Performance comparison of various methods on the ISCXVPN2016 dataset

[0163]

[0164] As can be seen from Table 2, the accuracy of the method proposed in the present invention has been significantly improved at the App (17 categories) and Service (12 categories) granularity of ISCXVPN2016. It can be shown that the present invention can effectively learn the topological structure, global semantic features of side channel features, and temporal features implied in tunnel encrypted traffic, and can automatically integrate the features of tunnel encrypted traffic from a global, multi-scale, and multi-modal perspective, greatly improving the accuracy of multi-classification.

[0165] In summary, only specific embodiments of the present invention are provided to describe the basic principles, advantages and purposes of the present invention. It should be clearly understood by those skilled in the art that the present invention is not limited to the above embodiments, and more changes and substitutions can be imagined based on the above description and without departing from the accuracy and scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A multimodal tunnel encrypted traffic fine-grained perception method based on pre-trained model and graph neural network, characterized in that: The steps include: Clean the encrypted traffic data and cut it into sessions; Use sliding windows for data augmentation; Preprocess the data to generate a multimodal dataset; training a perception model based on the data set; Through the perception model, extract the global semantic features of the encrypted tunnel traffic, the time series features of the encrypted tunnel traffic, and the non-Euclidean features of the encrypted tunnel traffic; The traffic global semantic features, traffic non-Euclidean features and traffic time series features are integrated through the perception model; The perception model classifies traffic based on the fused features.

2. According to claim 1, the multimodal tunnel encrypted traffic fine-grained perception method based on pre-training model and graph neural network is characterized in that: in, Cleaning encrypted traffic data and cutting it into sessions includes: Step 1.

1. Clean the original Pcap data, including: Use tshark to convert Pcapng files into Pcap files; Each Pcap file contains a maximum of 100,000 Pcapts, and removes network query messages, leaving only IP messages; Copy the cleaned Pcap files to folders representing different categories in App and Service according to the granularity; Step 1.

2. Divide into sessions, including: Use SplitCap to split the original Pcap traffic into two-way conversation flows; Among them, the bidirectional session flow is a pair of unidirectional flows with interchangeable sources and destinations, and the unidirectional flow is a set of source-to-destination data packets identified by a five-tuple.

3. According to claim 1, the multimodal tunnel encrypted traffic fine-grained perception method based on pre-training model and graph neural network is characterized in that: in, Use sliding windows for data augmentation, including Use EditCap to split the data into sliding windows of size 25 for data enhancement; There are no more than 500 samples in each category.

4. According to claim 1, the multimodal tunnel encrypted traffic fine-grained perception method based on pre-training model and graph neural network is characterized in that: in, The data is preprocessed to generate a multimodal dataset, including: Step 2.

1. Construct a burst relationship graph BRG (BURST Relationship Graph), where BURST is defined as a group of consecutive data packets in the same direction within a short period of time; Step 2.

2. Convert the encrypted traffic payload into BERT Token; form a tuple of the burst relationship graph BRG, Token and Label to obtain a processed multimodal dataset; Step 2.

3. Split the processed multimodal dataset into training set and test set; Wherein, the burst relationship graph BRG of the undirected graph in step 2.1 includes nodes and edges; The nodes include nodes representing packets in the BURST and nodes representing the entire BURST; for the nodes representing packets in the BURST, each node is associated with the length and arrival time interval of the packet, the arrival time interval is discretized and normalized, and the direction of the BURST is distinguished by the positive and negative values ​​of the packets; for the nodes representing the entire BURST, the nodes are associated with the average value of the length of all packets in the current BURST and the average value of the arrival time interval, representing the overall characteristics of the current BURST; Among them, the edges are BURST internal edges and BURST inter-edges, wherein, for BURST internal edges, they connect the consecutive nodes in each BURST in sequence, and connect the start node and the end node of the current BURST to the node representing the overall characteristics of the BURST; wherein, for BURST inter-edges, they are constructed by connecting the nodes representing the overall characteristics of each BURST for different BURSTs in the same session; Among them, step 2.

2. converts the encrypted traffic payload into BERT token, including Remove the header of each data packet in the session, leaving only the encrypted payload; Encode the encrypted payload in hexadecimal and use the Bi-Gram model to encode the encrypted payload, where each unit consists of two adjacent bytes and each Token unit does not exceed the maximum value of the dictionary, 65535; Take the first five packets in each conversation, concatenate the encoded payloads together, and control the total length to 512 by truncation or padding with 0. Add special tokens [CLS] and [SEP] at the beginning and end. Among them, step 2.

3. splitting the processed multimodal dataset into a training set and a test set includes: The processed multimodal dataset is split into training set and test set in a ratio of 8:2 through Sklearn's train_test_split function, and saved as pt files in turn.

5. According to claim 1, the multimodal tunnel encrypted traffic fine-grained perception method based on pre-training model and graph neural network is characterized in that: in, The perception model (PGM) includes a global semantic feature extraction module, a temporal feature extraction module, a graph neural network module, a gated multi-scale feature fusion mechanism, and a classification module; The steps include extracting global semantic features of encrypted tunnel traffic, temporal features of encrypted tunnel traffic, and non-Euclidean features of encrypted tunnel traffic through a perception model, and include the following steps: Step 3.

1. The global semantic feature extraction module extracts the global semantic feature Pooler_Output of the encrypted tunnel traffic; The global semantic feature extraction module uses the same pre-trained BERT as ETBERT as the global semantic feature extraction module, which consists of 12 transformer blocks, 12 attention heads in each self-attention layer, hidden_size is set to 768, and the input token length is 512; Step 3.

2. The timing feature extraction module extracts the timing features of the encrypted tunnel traffic; Among them, the time series feature extraction module uses the time convolution module to extract time series features. The sequence_output dimension output by the BERT module is [Batchsize, sequence_len, hidden_size], where the length of sequence_len is 512; Consider sequence_len as the length of the time series and hidden_size as the number of feature channels, and input them into the time series feature extraction module; Among them, the temporal feature extraction module consists of two dilated convolutional layers DIL, one DIL is followed by a tanh activation function and used as a filter, and the other DIL is followed by a sigmoid activation function and used as a gate to control the information passed from the filter to the next module; Step 3.

3. The graph neural network module extracts the non-Euclidean features of the encrypted tunnel traffic; Among them, the graph neural network module consists of three layers of GIN modules. The message transmission and aggregation mechanism of the GIN module is as follows: In the formula, is the embedding vector of node v at layer k; MLP k is the k-th layer of the multilayer perceptron; ∈ is a learnable parameter, or is set to a fixed constant to adjust the influence of the node's own features; N(v) represents the set of neighbor nodes of node v; is the embedding vector of node v at the k-1th layer; is the embedding vector of node u at the k-1th layer; node u is the node directly connected to node v; The side channel features contained in the input BRG are first converted into graph embedding representations through the embedding layer in the MLP; In each layer of the GIN module, the ReadOut operation is used to obtain multi-scale full-image feature representation; The full-image feature representation of each layer is combined into a three-channel one-dimensional vector combination, and the channel attention module SEBlock is used to fuse the full-image feature representations of different scales; The weighted full-graph features of different scales are added in the channel dimension to obtain the non-Euclidean graph feature representation of the encrypted tunnel traffic, which is expressed by the following formula: s=σ(w1z+b1) s′=σ(W2s+b2) y=X·Sigmoid(s′) Where H represents the characteristic length of the input vector; z c is the global average pooling result of channel c, which represents the global feature of the channel; S is the weight of the channel generated by compressing the feature length dimension using average pooling; s' is the weight of each channel obtained by restoring the number of channels by the excitation operation; y represents the weighted feature of each channel obtained by normalizing it with Sigmoid and multiplying it with the input X; x h,c Represents the eigenvalue of the input feature at position h and channel c; W1 represents the weight matrix of the first fully connected layer in the excitation operation; b1 represents the bias vector of the first fully connected layer: W2 represents the weight matrix of the second fully connected layer in the excitation operation; b2 represents: the bias vector of the second fully connected layer; The step of fusing the traffic global features, traffic non-Euclidean features and traffic time series features through a perception model includes the following steps: Step 3.

4. The gated multi-scale feature fusion mechanism fuses the global semantic features, traffic non-Euclidean features and traffic temporal features; Among them, the gated multi-scale feature fusion mechanism firstly passes the global semantic features extracted by the BERT module of the global semantic feature extraction module, the temporal features extracted by the temporal feature extraction module, and the traffic non-Euclidean features extracted by the graph neural network module through the FeatureModify layer, and standardizes the feature dimension to 128; In the gating branch of Self-Gating, the 1-dimensional weight coefficient relative to itself is obtained through MLP; The weight coefficients of each feature are spliced ​​together and input into softmax to obtain the weight coefficients relative to other features; The weight coefficient is multiplied by its own features, and the features of multiple scales are added together to obtain the final features used for classification; The perception model in step 1 classifies traffic according to the fusion features, including the following steps: Step 3.

5. The classification module classifies the traffic according to the fusion features; Among them, the classification module is a two-layer fully connected layer.

6. According to claim 5, the multimodal tunnel encrypted traffic fine-grained perception method based on pre-trained model and graph neural network is characterized in that: The steps of classification module classification include: First increase the dimension to 256 and then reduce the dimension to the number of categories; Combined with cross entropy loss function for multi-classification.

7. According to claim 5, the multimodal tunnel encrypted traffic fine-grained perception method based on pre-training model and graph neural network is characterized in that: in, Each DIL layer applies a set of 1D causal dilated convolutions with kernel sizes of 2, 3, 6, and 7 to discover temporal patterns of various ranges; Each causal dilation convolution maps the channel to 1 / 4 of the input channel, and takes the feature with the least time dimension among the four dilated causal convolutions as the standard in the time dimension to keep their time dimensions consistent; The four multi-scale feature values ​​are concatenated on the channel to restore the same number of channels as the input, and the multi-scale temporal features extracted by the dilated causal convolution are passed through two fully connected layers; First increase the dimension and then reduce it to increase the expressive power and restore the same time dimension as the input.

8. According to claim 7, the multimodal tunnel encrypted traffic fine-grained perception method based on pre-training model and graph neural network is characterized in that: The perception model was trained according to the dataset, using the AdamW optimizer, with a learning rate of 2e-5 and a dropout of 0.5 for the BERT module, a learning rate of 5e-4 for other modules, a batch size of 32, a seed of 7, a warm-up ratio of 0.1, and a total of 70 epochs.

9. An electronic device, comprising: One or more processors, a memory, and one or more programs; wherein the one or more programs are stored in the memory, and the one or more programs include instructions, which, when executed by the electronic device, enable the electronic device to perform the method described in any one of claims 1-8.

10. A computer-readable storage medium, comprising a computer program, and when the computer program is executed on an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 8.

Images