Encryption traffic classification method and device based on SwinT-CNN model

Abstract

This invention discloses a method and apparatus for classifying encrypted traffic based on the SwinT-CNN model, comprising the following steps: S1, converting the encrypted traffic data to be classified into a two-dimensional matrix and generating a grayscale image, using the two-dimensional matrix as the input vector V of the SwinT-CNN model. input S2. Input the two-dimensional matrix into the SwinT-CNN model. The SwinT-CNN model includes an improved CNN module and a Swin T module. First, the improved CNN module extracts the local spatial features of the data and outputs the processed data V. cnn Then V cnn The Swin T module is input to perform attention mechanism calculations on local spatial features to obtain the global spatial features of the data, and outputs a weighted combined feature vector V. swinT S3. Finally, a fully connected layer activated by Softmax is used to make predictions, and the multi-class cross-entropy loss is calculated to generate classification results. This invention combines an improved CNN module with a Swing module extracted from the Swing Transformer, which can more effectively capture local and global features in the data and improve the accuracy of encrypted traffic classification.

Description

Technical Field

[0001] This invention relates to the field of network encrypted traffic classification technology, and in particular to an encrypted traffic classification method and apparatus based on the SwinT-CNN model. Background Technology

[0002] In recent years, encrypted traffic has accounted for a steadily increasing proportion of network transmissions. Encrypted traffic has become an indispensable part of modern internet communication. With the increasing prevalence and use of encrypted traffic, encrypted traffic classification technology has attracted widespread attention from academia and industry.

[0003] With the emergence of various encryption technologies, many attackers exploit encryption protocols to hide malicious content and evade anomaly detection. Encrypted traffic classification, as a fundamental aspect of network security defense, is not only used for implementing Quality of Service (QoS), traffic engineering, and law enforcement, but also helps detect security threats such as malware. Encrypted traffic classification provides a useful tool for gaining deeper insights into information leakage within network connections, enabling the design and implementation of corresponding defensive measures, and thus possesses significant research value and practical implications. Effectively classifying encrypted traffic by type allows for better responses to the security threats posed by encrypted traffic, ensuring network security and reliability. Therefore, accurately and effectively identifying the categories of encrypted traffic is crucial.

[0004] Traditional traffic classification methods are based on port numbers and deep packet inspection (DPI). However, the widespread use of port obfuscation and port hopping techniques in modern traffic has significantly reduced the accuracy of traditional port-based traffic identification. DPI-based traffic identification and classification methods typically rely on valuable information from plaintext payloads, which are data encapsulated within frames in network traffic. However, the encryption of the payload further hinders the effectiveness of DPI-based methods. Machine learning algorithms, such as random forests, can be used to classify encrypted traffic, but these methods require manual feature extraction, resulting in low accuracy and significant labor costs. Gil et al. (Draper-Gil, G., Lashkari, AH, Mamun, MSI, et al. Characterization of encrypted and VPN traffic using time-related [C] / / Proceedings of the 2nd international conference on information systems security and privacy (ICISSP). 2016:407-414.) used time-dependent features, such as flow duration, bytes per second, forward arrival time, and backward arrival time, and employed the C4.5 decision tree algorithm to describe network traffic. They used the C4.5 algorithm to describe six main types of encrypted traffic, including web browsing, email, chat, and VoIP, achieving a recall rate of nearly 88%. However, machine learning-based methods require manual feature extraction, which is time-consuming, labor-intensive, and yields less than ideal results.

[0005] Currently popular methods for classifying encrypted traffic based on deep learning algorithms include those using Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Long Short-Term Memory (LSTMs). While these deep learning-based encrypted traffic classification algorithms avoid the drawbacks of traditional classification methods and the complexity of manual feature extraction by not using port numbers and payload information as classification criteria, the port obfuscation or port switching techniques commonly used in these algorithms result in low accuracy for traffic classification methods based on port number identification.

[0006] Wang et al. (Wang, W., Zhu, M., Wang, J., et al. End-to-end encrypted traffic classification with one-dimensional convolution neural networks [C] / / 2017 IEEE international conference on intelligence and security informatics (ISI). IEEE, 2017: 43-48.) proposed an end-to-end encrypted traffic classification method using a one-dimensional convolutional neural network. This method integrates feature extraction, feature selection, and classifier into a unified end-to-end framework, aiming to automatically learn the non-linear relationship between the original input and the desired output. Experiments demonstrate the effectiveness of convolutional neural networks for encrypted traffic classification, but without model tuning or consideration of dataset balance.

[0007] Zhuang et al. (Zou, Z., Ge, J., Zheng, H., et al. Encrypted traffic classification with a convolutional long short-term memory neural network [C] / / 2018 IEEE 20th International Conference on High Performance Computing and Communications; IEEE 16th International Conference on Smart City; IEEE 4th International Conference on Data Science and Systems (HPCC / SmartCity / DSS). IEEE, 2018: 329-334.) proposed a novel deep neural network combining convolutional and recurrent neural networks to improve classification accuracy. They also demonstrated the effectiveness of hidden temporal features in traffic data packets for encrypted traffic classification, but the accuracy improvement was not significant.

[0008] Chen Lei (Chen Lei. Design and Implementation of Encrypted Traffic Classification System Based on Transfer Learning [D]. Beijing University of Posts and Telecommunications, 2021) proposed a method for classifying encrypted traffic based on the Transformer model, which is also based on the attention mechanism. It has good results and proves the effectiveness of the attention mechanism, but the accuracy still needs to be improved.

[0009] To identify the increasingly diverse types of encrypted traffic and address the issues of traditional traffic classification methods requiring manual feature extraction and the low accuracy of existing deep learning-based traffic classification methods, this invention constructs a novel encrypted traffic classification method and apparatus based on the SwinT-CNN model. Summary of the Invention

[0010] The purpose of this invention is to propose an encrypted traffic classification method and apparatus based on the SwinT-CNN model. By combining an improved CNN module with a SwinT module extracted from Swin Transformer, it can capture the spatial local features of traffic data as well as the correlation between global data attributes. This can more effectively capture complex relationships and global features in the data, improve the accuracy of encrypted traffic classification, and identify multiple categories of encrypted traffic.

[0011] To achieve the above objectives, the present invention provides the following technical solution:

[0012] This invention provides a method for classifying encrypted traffic based on the SwinT-CNN model, comprising the following steps:

[0013] S1. Convert the encrypted traffic data to be classified into a two-dimensional matrix and generate a grayscale image. Use the two-dimensional matrix as the input vector V of the SwinT-CNN model. input Each element of the matrix represents a pixel in the grayscale image, which is one byte of traffic data.

[0014] S2. Input the two-dimensional matrix into the SwinT-CNN model. The SwinT-CNN model includes an improved CNN module and a Swin T module. First, the improved CNN module extracts the local spatial features of the data and outputs the processed data V. cnn Then V cnn The Swin T module is input to perform attention mechanism calculations on local spatial features to obtain the global spatial features of the data, and outputs a weighted combined feature vector V. swinT ;

[0015] S3. Finally, a fully connected layer activated by Softmax is used to make predictions, and the multi-class cross-entropy loss is calculated to produce classification results.

[0016] Furthermore, the improved CNN module in step S2 contains 6 convolutional layers and 6 pooling layers, with a stride of 1 for all convolutional layers and a stride of 2 for all pooling layers, all using a padding method.

[0017] Furthermore, the improved CNN module in step S2 extracts features from the traffic data using the following formula:

[0018] Vcnn =Max(Contact(σ1(V)) input ), ..., σ2(V input )))

[0019] Where V cnn This represents the output feature vector, where σ represents the ReLU activation function, and σ1(V input ), ..., σ2(V input ) represents four different convolution operations that result in four vectors, Contact represents the concatenation of the four vectors along the channel dimension, and Max is the pooling operation.

[0020] Further, the Swin T module in step S2 is linearly connected between a first Swin T module and a second Swin T module. The first Swin T module contains only one Swin Transformer Block, and the second Swin T module contains a Patchmerging mechanism and a Swin Transformer Block. The Patchmerging mechanism is used for downsampling and increasing the number of channels. The Swin Transformer Block contains a multi-head self-attention mechanism for windows, a multilayer perceptron, a normalization layer, and residual connections. The Swin Transformer Block performs regular window partitioning on the feature vectors obtained by the CNN module and performs an attention mechanism on the window feature maps. The weighted combined feature vector V is obtained by calculating the attention score. swinT .

[0021] Furthermore, the dimension of the feature map vector input to Patch merging is changed from [H,W,C] to [H / 2,W / 2,2C], where H, W, and C represent the width, height, and number of channels of the feature map vector, respectively.

[0022] Furthermore, the calculation formulas for the inputs and outputs in the Swing Transformer Block are as follows:

[0023] V t =W_MSA(LN(V cnn ))+V cnn

[0024] V t+1 =MLP(LN(V t ))+V t

[0025] V t+2 =SW_MSA(LN(V t+1 ))+V t+1

[0026] v = MLP(LN(V) t+2 ))+V t+2

[0027] Here, LN is the normalization operation, which standardizes the feature map vector. MLP extracts the multiple feature vectors output by the attention mechanism, resulting in a weighted combined feature vector V. t+1 The module outputs v; W_MSA operates an attention mechanism on the non-overlapping window feature map vectors to obtain a weighted combined feature vector V. t SN_MSA is used to offset and cut the window, and the cut window is restored to the window shape of N_MSA. Then, an attention mechanism is applied to each window to obtain a weighted combined feature vector V. t+2 v represents the output vector of a SwinT module. The feature vector v output by the first SwinT module will be fed into the second SwinT module, and after downsampling, the calculation formulas for the input and output in the Swin Transformer Block will be executed again.

[0028] Furthermore, the formula for the fully connected layer in step S3 is expressed as follows:

[0029] y = softmax(Wv + b)

[0030] Where V is the vector of multiple feature combinations output by the SwinT module. swinT y is the output classification label, which is an n-dimensional vector where n represents the number of categories, and the value of each dimension represents the probability of the corresponding category. The sum of these probabilities equals 1. W and b are the parameters that the connection layer needs to learn.

[0031] Furthermore, step S3 uses the multivariate cross-entropy loss function to handle multi-class classification problems. The calculation formula is as follows:

[0032]

[0033] Where p = [p0, ..., p] C-1 ] is a probability distribution, where each element p i This represents the probability that a sample belongs to the i-th class; the output is the probability that it belongs to the label y, where y = [y0, ..., y]. C-1 ] is the one-hot union of sample labels, representing the value of y when the sample belongs to class i. i =1, otherwise y i =0; C is the number of sample labels.

[0034] Furthermore, the construction process of the SwinT-CNN-based model is as follows:

[0035] (1) The ISCX VPN-nonVPN dataset was selected. There are a total of 12 traffic types in the ISCX VPN-nonVPN dataset, namely: Email, VPN-Email, Streaming, VPN-Streaming, File, VPN-File, VOIP, VPN-VOIP, P2P, VPN-P2P, Chat and VPN-Chat;

[0036] (2) Preprocess the dataset to generate the processed dataset. pre The preprocessing process includes data segmentation, data cleaning, data extraction, and dimension transformation, which are used to transform encrypted traffic data into a 28x28 dimension traffic matrix vector and generate a 28x28 grayscale image.

[0037] (3) Use the SMOTE algorithm to equalize the preprocessed dataset. pre Generate a new dataset. new The formula for the SMOTE algorithm is as follows:

[0038]

[0039] Where, x new Here, x represents the newly generated sample point, and x represents the currently scanned sample point. For a certain neighboring sample point, calculate the difference between each feature of the random neighboring point and the observed sample point, multiply the difference by a random factor of [0, 1], and add it to the observed point to complete the synthesis of a feature.

[0040] (4) Generate input vectors: Dataset new The traffic data was transformed into a 28×28 two-dimensional matrix. 20% of the data was randomly selected as the training set, and the remaining 80% was used as the test set. The two-dimensional matrix was then used as the input vector V. input ;

[0041] (5) Use the training data to train the model, obtain the trained model and save it, and then use the model to classify the test data to obtain the classification results.

[0042] On the other hand, the present invention also provides an encrypted traffic classification device based on the SwinT-CNN model, comprising the following modules to implement the method described in any of the above:

[0043] The data preprocessing module is used to convert encrypted traffic data into a two-dimensional matrix and generate a grayscale image;

[0044] An improved CNN module is used to perform convolution and pooling operations on the input data, extract local spatial features from the input data, and output the processed data V. cnn ;

[0045] The SwinT module will transfer data V cnn As input, the global spatial features of the data are calculated through an attention mechanism, and a weighted combined feature vector V is output. swinT ;

[0046] The fully connected layer will weight and combine the feature vector V. swinT As input, it is used to calculate and output the classification result.

[0047] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0048] 1. This invention is the first to introduce the Swing Transformer model into the field of encrypted traffic classification. Drawing on the ideas of this model, it extracts and encapsulates its core components to form a general SwingT module. As the core module of SwingT-CNN, the SwingT module extracts global features from encrypted traffic data through a window attention mechanism, demonstrating excellent performance in encrypted traffic classification tasks. Furthermore, the window attention mechanism has lower computational complexity than traditional attention mechanisms, reducing the computational load on the model.

[0049] 2. This invention draws inspiration from the Inception model design and improves upon it by replacing the original two-dimensional convolution with one-dimensional convolution and adding pooling layers to obtain an improved CNN module. This improved CNN module can extract local spatial features of encrypted traffic data at a deeper level, playing a significant auxiliary role in encrypted traffic tasks and helping to improve the model's accuracy.

[0050] 3. The encrypted traffic classification method based on the SwinT-CNN model proposed in this invention combines an improved CNN module and a Swin Transformer to form the SwinT-CNN model, which is used to calculate the probability of whether encrypted traffic belongs to a certain category of traffic. The SwinT-CNN model extracts local spatial features of the data based on the improved CNN module, and calculates the attention mechanism of the feature vectors based on the proposed SwinT module to obtain the attention scores between data attributes, thus obtaining the global features of the data. The SwinT-CNN model can capture more correlation features of traffic data, and finally outputs the classification result through a fully connected layer, improving the accuracy of encrypted traffic classification.

[0051] 4. This invention uses the ISCX VPN-nonVPN dataset, commonly used in encrypted traffic classification, to train the SwinT-CNN model. The SMOTE balancing algorithm addresses the data imbalance problem in the encrypted traffic dataset, further improving the accuracy of encrypted traffic classification. The SwinT-CNN model improves the accuracy of encrypted traffic classification to 96.7%, 3% higher than the best-performing Transformer model. The recall reaches 97.3%, the F1 score reaches 96.2%, and the precision reaches 95.7%. These results demonstrate that the overall performance of this method is significantly improved compared to previous deep learning-based encrypted traffic classification methods, and it also performs well in categories with fewer data samples.

[0052] 5. The method proposed in this invention is based on the SwinT-CNN deep learning neural network model, which classifies traffic by extracting features from the original traffic session data. The SwinT-CNN model does not rely on port number information and plaintext information of the payload as the basis for judgment, and through self-learning features, it can significantly improve the accuracy of encrypted traffic classification, solving the problem of low accuracy of traditional port-based traffic classification and deep packet inspection traffic classification methods, while avoiding the cumbersome manual feature extraction of machine learning.

[0053] In summary, this method improves the accuracy of encrypted traffic classification. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0055] Figure 1 A flowchart of an encrypted traffic classification method based on the SwinT-CNN model provided in an embodiment of the present invention.

[0056] Figure 2 Grayscale images of different types of traffic provided in embodiments of the present invention.

[0057] Figure 3 This is a functional module structure diagram of the encrypted traffic classification device based on the SwinT-CNN model provided in an embodiment of the present invention.

[0058] Figure 4 This is a diagram illustrating the improved architecture of the CNN module and the SwinT module provided in an embodiment of the present invention. Detailed Implementation

[0059] To better understand this technical solution, the technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described examples are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of the present invention.

[0060] This invention proposes a method for classifying encrypted traffic based on the SwinT-CNN model, the process of which is as follows: Figure 1 As shown, the original dataset is first processed, feature selection is performed, and then it is divided into training and test sets. The model is then trained using the training data, and the trained model is saved. The model is then used to classify the test data, and the classification results are analyzed. This method identifies a total of 12 traffic categories, as shown in Table 1.

[0061] Table 1 Flow Types

[0062]

[0063] The encrypted traffic classification method designed in this invention, compared to other traffic classification methods, processes traffic into sequences and uses it in a one-dimensional dimension. To facilitate intuitive analysis of the differences between different categories, this method converts encrypted traffic data into grayscale images, with each byte of session data corresponding to a pixel value in the grayscale image, characterizing the spatial features of different traffic types. Then, an improved convolutional neural network is used to extract local spatial features of the data, and the attention mechanism of the Swin Transformer is used to extract global spatial features, thereby improving the accuracy of encrypted traffic classification.

[0064] The following will detail the steps involved in building the SwinT-CNN model:

[0065] (1) Dataset Selection: This method uses the ISCX VPN-nonVPN dataset. This dataset is a public and free dataset containing packet capture data of encrypted traffic in VPN and non-VPN networks, stored in Pcap and Pcapng formats. It aims to support research on network traffic classification and encrypted traffic detection. The dataset contains 28GB of data, with 7 different application types. Each application type is further divided into VPN tunnel traffic and regular traffic, resulting in a total of 14 encrypted traffic types. After studying the 14 application types in the ISCX VPN-nonVPN dataset, it was found that the labeling of the Browser and VPN-Browser application type traffic is controversial. Therefore, this paper temporarily abandons the classification of this traffic and removes it from the dataset.

[0066] (2) Data preprocessing: In order to better train the SwinT-CNN model, the data in the dataset needs to be processed according to certain rules. The processed dataset is called Dataset. pre The processed training set contains 125,973 data points. Preprocessing mainly includes data splitting, data cleaning, data extraction, and dimensionality transformation, as follows:

[0067] a. Data Segmentation. ISCX VPN-nonVPN datasets centrally store files in Pcap and Pcapng formats. A Pcap file contains multiple data streams; packets with the same 5-tuple (source IP, destination IP, source port, destination port, traffic protocol) belong to the same data stream. Data streams with identical or interchangeable source and destination addresses belong to the same session. First, the Scapy library is used to convert the Pcapng file to Pcap format. Then, the SplitCap tool is used to split the Pcap file into session-based traffic data.

[0068] b. Data Cleaning: Filtering the segmented session traffic data. This primarily filters out useless information such as IP and MAC addresses; discarding packets, such as DNS packets and ACK, SYN, or FIN packets in TCP sessions. These packets exist to implement or maintain some basic functions used by most network services and are unrelated to the traffic classification task.

[0069] c. Data Extraction: To meet the model's input requirements, the traffic data needs to be processed into a uniform input size, i.e., extracting the first n fixed bytes of each session. This paper selects 784 bytes as the fixed length of the session stream. For traffic data with a session length less than this length, 0x00 is padded to the end; for traffic data with a length greater than this length, the excess portion is discarded. The fixed length is determined based on the characteristics of the traffic data; the initial part of the session generally contains connection establishment and content data, which better reflects the traffic characteristics.

[0070] d. Dimensional Transformation: The uniform 784-byte traffic data is transformed into a 28×28 traffic matrix vector, which is used as input to the model. This generates a 28×28 grayscale image. It can be seen that there are significant differences between the grayscale images of different categories. (The text then repeats itself, so the translation stops.) Figure 2 As shown.

[0071] (3) Data Balancing: To avoid the drawbacks of imbalanced datasets, the problem of sample imbalance needs to be addressed. This invention uses the SMOTE (Synthetic Minority Over-sampling Technique) algorithm to balance the dataset. The SMOTE algorithm is used to balance the dataset.pre After obtaining the dataset, generate a new dataset. new This dataset contains 336,715 data points. The SMOTE algorithm is an oversampling method, and as the most widely applicable processing method, it can directly and effectively enrich minority samples. Specifically, the SMOTE algorithm increases the number of minority class samples by inserting newly generated samples between minority class samples to achieve sample balance. The SMOTE algorithm process is as follows: First, a minority class sample is selected, and then a sample is selected from its nearest neighbor. A new sample is generated by drawing a line segment between the two samples and randomly selecting a point on the line segment. This can effectively increase the number of minority class samples and improve the balance of the dataset. The SMOTE algorithm has three parameters: (a) sampling ratio, which is generally a multiple of 100, 200, 300, etc., corresponding to the number of samples generated being a multiple of 1, 2, 3, etc. of the original number of samples; (b) the total number of samples in the training set; (c) the number of nearest neighbors sampled: this method uses a default value of 5 and uses the Euclidean distance function to determine the nearest neighbor. The formula for generating new samples by this method is as follows:

[0072]

[0073] Where, x new Here, x represents the newly generated sample point, and x represents the currently scanned sample point. For a given neighboring sample point, this function calculates the difference between each feature of the random neighboring point and the observed sample point, multiplies the difference by a random factor of [0, 1], and adds it to the observed point to complete the synthesis of a feature.

[0074] (4) Generate input vectors: We will use the dataset Dataset new The traffic data was transformed into a 28×28 two-dimensional matrix. 20% of the data was randomly selected as the training set, and the remaining 80% as the test set. This two-dimensional matrix was then used as the input vector V. input V input It contains data stream information of encrypted traffic.

[0075] (5) Training the SwinT-CNN model: The encrypted traffic classification process based on the SwinT-CNN model constructed in this invention is as follows: Figure 4 As shown. This model is a combination of CNN and Swin Transformer core modules, designed for multi-class classification tasks. The input is a 28×28 two-dimensional matrix V. inputEach element of the matrix represents a pixel in the grayscale image, i.e., one byte of data. A CNN module is used to extract local spatial features from the data. Then, the core module of the Swin Transformer performs attention calculations on these features to obtain global features, which are then fed into a fully connected layer for prediction. We will use V... input The data processed by the CNN module is represented as V. cnn After that, V cnn After being passed into the SwinT module, it processes the data and generates the output V. swinT SwinT's output V swinT Finally, a fully connected layer activated by Softmax is passed through, and then the multi-class cross-entropy loss is calculated to produce the classification result.

[0076] The following section introduces the model from the perspectives of the CNN module and the SwinT module:

[0077] (1) Improved CNN module:

[0078] The improved CNN module of this invention can perform convolution and pooling operations on the input data to extract important features, which helps to capture spatial and temporal information related to classification tasks. The architecture connecting the CNN module and the SwinT module is as follows: Figure 4 As shown in the figure, the numbers represent the kernel parameter sizes used in the respective layers. This module borrows from the Inception model design and improves upon it by replacing the original two-dimensional convolutions with one-dimensional convolutions, while adding pooling layers to remove redundant information and prevent overfitting while maintaining feature invariance. The structure contains 6 convolutional layers and 6 pooling layers, with a stride of 1 for all convolutional layers and a stride of 2 for all pooling layers, all using padding. This module preprocesses the data to output a 28×28 graph feature vector V. input Convert into a graph feature vector V of size 14×14. cnn This serves as the input to the SwinT module. The formula for extracting features from traffic data by this module is as follows:

[0079] V cnn =Max(Contact(σ1(V)) input ), ..., σ2(V input (2)

[0080] Where V cnn This indicates that the output is a 14×14 feature vector, where σ represents the ReLU activation function, and σ1(V input ), ..., σ2(V input ) represents four different convolution operations that result in four vectors, Contact represents the concatenation of the four vectors along the channel dimension, and Max is the pooling operation.

[0081] (2) Extracted SwinT module:

[0082] In encrypted traffic classification tasks, while there may be correlations between labels and extracted traffic information, there may also be attributes that are unrelated or indirectly related. Attention mechanisms assign different weights to different parts of the input, thereby better identifying the prominent key information between them. Therefore, this invention uses the attention mechanism in the Swin Transformer to extract correlation features between data attributes. Compared with traditional self-attention mechanisms, the window attention mechanism in the Swin Transformer has lower time and space complexity, thus improving the efficiency of encrypted traffic classification.

[0083] To suit encrypted traffic classification tasks, this invention encapsulates the core modules of the Swin Transformer model: a linearly connected Patch merging module and a Swin Transformer Block (abbreviated as Block), named the SwinT module. The Block includes a window multi-head self-attention mechanism (W_MSA, SW_MSA), a multilayer perceptron (MLP), a normalization layer (Layers Norm), and residual connections. This method uses two linearly connected SwinT modules. The first SwinT module contains only one Block, and the second SwinT module contains one Patch merging module and one Block.

[0084] a. Patch merging: Used to downsample and increase the number of channels. The dimension of the input feature map vector changes from [H, W, C] to [H / 2, W / 2, 2C]. Here, H, W, and C represent the width, height, and number of channels of the feature map vector, respectively.

[0085] b. Block divides the 14x14 feature vector obtained from the CNN module into a 7x7 window matrix (each window is 2x2), and applies an attention mechanism to the window feature map. A weighted combined feature vector is obtained by calculating the attention score. The calculation formulas for input and output in Block are as follows:

[0086] V t =W_MSA(LN(V cnn ))+V cnn (3)

[0087] V t+1 =MLP(LN(V t ))+V t (4)

[0088] V t+2 =SW_MSA(LN(V t+1 ))+V t+1 (5)

[0089] v = MLP(LN(V) t+2 ))+V t+2 (6)

[0090] LN is the normalization operation, which standardizes the feature map vector. MLP mainly extracts the multiple feature vectors output by the attention mechanism, obtaining the weighted combined feature vector V. t+1 The module outputs v. W_MSA operates an attention mechanism on the non-overlapping window feature map vectors to obtain a weighted combined feature vector V. t This operation significantly reduces the computational cost of the attention mechanism in traditional Transformers. SW_MSA performs offset window cutting and restores the cut window to the shape of W_MSA. Then, the attention mechanism is applied to each window to obtain a weighted combined feature vector V. t+2 This operation bridges the information gap between windows without increasing the computational complexity of the model. v represents the output vector of a SwinT module. The feature vector v output by the first module will continue to be passed into the next SwinT module, and after downsampling, the operations in formulas (3) to (6) will be performed again.

[0091] V cnn After passing through two SwinT modules, the weighted combined feature vector V is finally obtained. swinT , as input to the fully connected layer.

[0092] (3) Fully Connected Layer: The connected layer serves as the output layer and corresponds to the number of labels in the multi-class classification. The formula for the fully connected layer is expressed as:

[0093] y = softmax(Wv + b) (7)

[0094] Where v is the vector V of multiple feature combinations output by the SwinT module. swinT y represents the output classification label, which is an n-dimensional vector. Here, n represents the number of categories, which is 12 in this paper. The value of each dimension represents the probability of the corresponding category, and the sum of these probabilities equals 1. W and b are the parameters that this layer needs to learn.

[0095] a. Output layer function: softmax activation function, used to normalize the output vector into a probability distribution.

[0096] b. Loss Function: Multivariate Cross-Entropy Function. This method uses the multivariate cross-entropy loss function to handle multi-class classification problems. Its calculation formula is as follows:

[0097]

[0098] Where p = [p0, ..., p] C-1 ] is a probability distribution, where each element p i This represents the probability that a sample belongs to the i-th class. Output the probability of a sample belonging to the label y. y = [y0, ..., y...] C-1 ] is the one-hot union of sample labels, representing the value of y when the sample belongs to class i. i =1, otherwise y i =0. C is the number of sample labels.

[0099] c. Optimizer: This model uses Adam as the optimization algorithm. Adam is a first-order optimization algorithm that can replace the traditional stochastic gradient descent process. It can iteratively update the neural network weights based on the training data.

[0100] Corresponding to the above method, the present invention also provides an encrypted traffic classification device based on the SwinT-CNN model, such as... Figure 3 As shown, the following modules are included to implement the method described in any of the above:

[0101] The data preprocessing module is used to convert encrypted traffic data into a two-dimensional matrix and generate a grayscale image;

[0102] The CNN module performs convolution and pooling operations on the input data, extracts local spatial features from the input data, and outputs processed data V. cnn ;

[0103] The SwinT module will transfer data V cnn As input, the global spatial features of the data are calculated through an attention mechanism, and a weighted combined feature vector V is output. swinT ;

[0104] The fully connected layer will weight and combine the feature vector V. swinT As input, it is used to calculate and output the classification result.

[0105] Corresponding to the apparatus provided in the above embodiments of the present invention, the present invention also provides an electronic device, including: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0106] Memory, used to store computer programs;

[0107] The processor, when executing a program stored in memory, implements the system flow provided in the above embodiments of the present invention.

[0108] The communication bus mentioned in the control equipment above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0109] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0110] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0111] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0112] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0113] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments, electronic device embodiments, computer-readable storage medium embodiments, and computer program product embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0114] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for encrypted traffic classification based on SwinT-CNN model, characterized in that, The method comprises the following steps: S1, the encrypted traffic data to be classified is converted into a two-dimensional matrix, and a grayscale image is generated, and the two-dimensional matrix is taken as an input vector V of the SwinT-CNN model input , each element of the matrix represents a pixel point of the grayscale image, that is, one byte of traffic data; S2, input the two-dimensional matrix into the SwinT-CNN model, the SwinT-CNN model includes an improved CNN module and a Swin T module, first extract the local spatial features of the data through the improved CNN module, output the processed data V cnn , then input V cnn into the Swin T module, perform attention mechanism calculation on the local spatial features to obtain the global spatial features of the data, and output a weighted combination feature vector V swinT ; The Swin T module is linearly connected with a first Swin T module and a second Swin T module, the first Swin T module only contains one Swin Transformer Block, and the second Swin T module contains one Patch merging and one Swin Transformer Block, wherein the Patch merging is used for downsampling and increasing the number of channels, and the Swin Transformer Block contains a window multi-head self-attention mechanism, a multi-layer perception, a normalization layer and a residual connection; the Swin Transformer Block performs regular window division on the image feature vector obtained by the CNN module, and performs attention mechanism on the window feature map, so as to obtain a weighted combination feature vector V through calculation of an attention score swinT ​ The dimension size of the feature map vector of the patch merging input is changed from [H, W, C] to [H / 2, W / 2, 2C], wherein H, W and C represent the width, height and channel number of the feature map vector respectively; The calculation formula of the input and output in the Swin Transformer Block is as follows: V t = W_MSA(LN(V cnn ))+ V cnn V t+1 = MLP(LN(V t ))+V t V t+2 = SW_MSA(LN(V t+1 ))+ V t+1 v = MLP(LN(v t+2 ))+ v t+2 wherein LN is a normalization operation, and the MLP extracts the multi-feature vectors output by the attention mechanism to obtain a weighted combination feature vector V t+1 and the output v of the module; W_MSA operates the attention mechanism on the non-overlapping window feature map vectors to obtain a weighted combination feature vector V t , SW_MSA performs offset cutting windows and restores the cut windows to the window shape of W_MSA, and then operates the attention mechanism on each window to obtain a weighted combination feature vector V t+2 , v represents an output vector of a SwinT module, and the feature vector v output by the first SwinT module is continuously transmitted into the second SwinT module, and after down-sampling, the calculation formula of the input and output in the Swin Transformer Block is executed again. S3, finally, a fully connected layer activated by Softmax is used to make a prediction, and a multi-class cross-entropy loss is calculated to generate a classification result.

2. The method of claim 1, wherein the SwinT-CNN model is a Swin Transformer (SwinT) model. The improved CNN module in step S2 comprises 6 convolutional layers and 6 pooling layers, wherein the step of the convolutional layers is 1, the step of the pooling layers is 2, and the padding method is used.

3. The method of claim 1, wherein the SwinT-CNN model is a Swin Transformer (SwinT) model. The formula for the improved CNN module in step S2 to extract features from the traffic data is as follows: V cnn = Max(Contact(σ1(V input ),…,σ2(V input ))) where V cnn represents the output feature vector, σ represents the ReLU activation function, σ1(V input ),…,σ2(V input ) represent four different convolution operations that result in four vectors, Contact represents concatenation of the four vectors in the channel dimension, and Max is the pooling operation.

4. The method of claim 1, wherein the SwinT-CNN model is based on a Swin Transformer (Swin-T) and a convolutional neural network (CNN). The formula of the fully connected layer in step S3 is represented as: y = softmax(Wv + b) where v is the vector V of multi-feature combinations output by the SwinT module swinT y is the output classification label, and the label is an n-dimensional vector, where n represents the number of classifications, and the value of each dimension represents the probability of the corresponding classification, and the sum of the probabilities is equal to 1, and W and b are parameters to be learned by the connection layer.

5. The method of claim 1, wherein the SwinT-CNN model-based encrypted traffic classification method is characterized by, The formula for the improved CNN module in step S2 to extract features from the traffic data is as follows: where p = [p0,..., p C-1 ] is the probability distribution, each element p i represents the probability that the sample belongs to the i-th class; y = [y0,..., y C-1 ] is the one-hot of the sample label and represents y i = 1 when the sample belongs to the i-th class, otherwise y i = 0; C is the number of sample labels.

6. The method of claim 1, wherein the SwinT-CNN model-based encrypted traffic classification method is characterized by, The formula of the fully connected layer in step S3 is represented as: y = softmax(Wv + b) (2) Preprocessing the data set to generate a processed data set Dataset pre The preprocessing process includes data segmentation, data cleaning, data extraction and dimension conversion, which is used to convert the encrypted traffic data into a traffic matrix vector with dimensions of 28x28, and generate a grayscale image with a size of 28x28; (3) The preprocessed dataset Dataset is balanced using the SMOTE algorithm pre , and a new dataset Dataset new is generated The formula of the SMOTE algorithm is as follows: where x new is a newly generated sample point, x is the current scanned sample point, is a certain neighboring sample point, the difference between the random neighboring point and each feature of the observed sample point is calculated, multiplied by a [0, 1] random factor, and then added to the observed point to complete the synthesis of a feature. (4) Generating input vectors: converting the flow data of the dataset Dataset new into a two-dimensional matrix of 28x28 dimensions, randomly selecting 20% of the data as the training set, and the remaining 80% as the test set, and taking the two-dimensional matrix as the input vector V input ; The formula for the improved CNN module in step S2 to extract features from the traffic data is as follows:

7. A device for classifying encrypted traffic based on a SwinT-CNN model, characterized in that, The construction process of the SwinT-CNN model is as follows: (1) select the ISCX VPN-nonVPN dataset, the ISCX VPN-nonVPN dataset has a total of 12 types of traffic, including Email, VPN-Email, Streaming, VPN-Streaming, File, VPN-File, VOIP, VPN-VOIP, P2P, VPN-P2P, Chat and VPN-Chat; (5) use the training data to train the model, obtain the trained model and save it, and then use the model to classify the test data to obtain the classification result. The method comprises the following modules: A data preprocessing module is configured to convert the encrypted traffic data into a two-dimensional matrix and generate a grayscale image. An improved CNN module is used to perform convolution and pooling operations on the input data, extract local spatial features of the data from the input data, and output processed data V cnn ; SwinT module, data V cnn As input, the global spatial features of the data are calculated by the attention mechanism, and the weighted combination feature vector V swinT ; a fully connected layer that weights and combines the feature vector V swinT as input and outputs a classification result by computation.

Images