Encrypted traffic classification method and system based on word meaning aggregation and semantic separation technology

Abstract

The invention discloses an encrypted traffic classification method and system based on word meaning aggregation and semantic separation technologies, belongs to the field of cyberspace security, and aims to solve the problem that encrypted traffic data cannot be effectively understood in a current method. And abstract features are decoupled in different category spaces through semantic separation, so that the feature distinguishability is enhanced, and a more accurate encrypted traffic classification effect is achieved.

Description

Technical Field

[0001] The present invention belongs to the field of cyberspace security, and in particular relates to an encrypted traffic classification method and system based on word sense aggregation and semantic separation technology. Background Art

[0002] Encrypted traffic analysis refers to the monitoring and analysis of encrypted network traffic without decrypting the actual content. It is an important research area in network security and network management. Understanding encrypted traffic is very important because it can provide valuable insights for network operators to perform network operation and management activities. For example, by analyzing anomalies in encrypted traffic patterns to detect malicious activities and intrusions, network security can be guaranteed. Some common applications include encryption application classification, attack detection, VPN (Virtual Private Networks) classification, etc.

[0003] Network traffic often exists in the form of data packets in cyberspace, containing rich semantic information. Data packets can usually be divided into two parts. The header part mainly includes the source IP address, destination IP address, source port, destination port, protocol type, etc. The payload part refers to the actual data being transmitted, which is usually encrypted. Nowadays, the development trend of encryption protocols and the rapid development of network traffic have made traditional traffic classification methods obsolete. These traditional methods, such as port matching, server name indication matching, deep packet inspection, etc., mainly perform rule matching based on plaintext fields or payloads. However, they cannot perform further feature extraction and analysis on the payload in encrypted traffic.

[0004] In recent years, with the development of deep learning and pre-training language models, more and more researchers have focused on using natural language processing methods to learn directly from raw traffic data, perform automatic feature extraction, and treat encrypted data as text sequences for semantic understanding. By treating the traffic data analysis problem as a language modeling task, deep learning models such as long short-term memory networks and transformers can be applied to sequential modeling and semantic understanding of network traffic. This allows learning the patterns of encrypted traffic and performing analysis tasks without decrypting the payload, such as traffic classification and anomaly detection. At present, the classification of encrypted traffic data can mainly include byte-based methods and methods based on pre-trained language models.

[0005] Byte-based models process raw traffic bytes directly and learn traffic feature representations using techniques such as convolutional neural networks (CNNs) and long short-term memory networks. They can capture complex patterns in encrypted packets without feature engineering. Depending on how these methods understand the input traffic, they can be classified as image-based or text-based classification methods. Specifically, image-based models treat traffic data as 2D images by converting raw bytes into grayscale intensity values ​​or RGB colors. Then, they apply off-the-shelf image classification CNN architectures such as ResNet or Inception for traffic classification. Text-based models, such as recurrent neural networks and long short-term memory networks, process inputs sequentially as text sequences. They can analyze long-term example dependencies in raw traffic packets. Self-attention models like Transformer are also widely used to learn contextual relationships in encrypted traffic data.

[0006] Recently, more and more researchers have used pre-trained language models for encrypted traffic classification. Following the self-supervised pre-training paradigm, which has been shown to be very effective for natural language processing tasks. The model is first trained on a large amount of unlabeled network traffic data in a self-supervised manner to learn a general traffic representation. This is achieved through predefined tasks such as masked token prediction, homologous BURST prediction, etc., which allow the model to capture raw bytes and mine the intrinsic characteristics of the traffic. ET-BERT obtains contextual data representations by pre-training from large-scale traffic data. NetGPT provides a generative pre-trained model for traffic understanding and generation tasks. However, this type of approach requires a lot of data and computing resources to retrain large language models. In this invention, the proposed method effectively utilizes the general natural language knowledge learned in the pre-training phase of BERT without re-pretraining. At the same time, through the semantic separation technique, independent feature learning is explicitly performed in the semantic space of different categories, thereby enhancing the distinguishability of features.

[0007] With the changes in the network environment and the rapid development of encryption protocols, traditional rule-based methods (such as port matching, deep packet inspection technology, etc.) are no longer applicable to encrypted traffic classification tasks. From the perspective of word meaning, the understanding of traffic data is inconsistent with the understanding of words in the text, especially when the traffic is encrypted. The existing traffic analysis methods based on pre-trained language models, such as BERT, mainly understand the information at the natural language level, and may not be suitable for directly processing traffic information. From a semantic perspective, traffic data can have multiple semantics. However, most of the current methods implicitly classify through the abstract semantic features learned at the top level, without further decoupling the independent semantic features under different categories or tasks, resulting in the feature distinguishability of the samples is not obvious. It can be seen that the current methods based on pre-trained language models either directly use BERT to process traffic data or re-pre-train the BERT model, although both can improve the classification effect to a certain extent. However, directly using BERT to process traffic data will destroy the complete word meaning in the traffic data, resulting in feature failure. Although re-pre-training the BERT model can ensure the integrity of the word meaning, it requires a lot of computing resources and data, and does not consider the separation of top-level abstract semantics. Summary of the invention

[0008] The purpose of the present invention is to propose an encrypted traffic classification method and system based on word sense aggregation and semantic separation technology, which retains the complete word meaning through word sense aggregation and decouples abstract features in different category spaces through semantic separation, enhances feature distinguishability, and improves the classification effect of encrypted traffic.

[0009] The technical solution adopted by the present invention to achieve the above-mentioned purpose is:

[0010] An encrypted traffic classification method based on word sense aggregation and semantic separation technology includes the following steps:

[0011] The network traffic data is segmented through the BERT tokenizer to obtain subwords;

[0012] Encode the semantic features of the subwords through the flow encoder to obtain the hidden state representation;

[0013] The hidden state representation is semantically aggregated through a semantic aggregator to obtain package-level semantic features;

[0014] Design a prompt template with classification prompts. Under the guidance of the prompt template, the package-level semantic features are semantically separated through a semantic separator to obtain the category-level semantic features.

[0015] Input the category-level semantic features into the classifier for classification prediction;

[0016] The training data with real category labels are processed through the above steps to train the traffic encoder, semantic separator and classifier. After the training is completed, the encrypted traffic is processed according to the above steps to obtain the classification result of the encrypted traffic.

[0017] Furthermore, the BERT tokenizer uses the WordPiece tokenizer.

[0018] Furthermore, the traffic encoder uses a pre-trained BERT encoder.

[0019] Furthermore, the sense aggregator utilizes the subword length information retained by the BERT tokenizer to aggregate the senses represented by the hidden states.

[0020] Furthermore, the prompt template consists of a data category query and a category sequence.

[0021] Furthermore, the step of performing semantic separation of the package-level semantic features through a semantic separator under the guidance of the prompt template includes:

[0022] Use another flow encoder with the same architecture as the above flow encoder to encode the semantic features of the prompt and obtain the hidden state representation of the prompt;

[0023] Activate the package-level semantic features and the hidden state representation of the prompt together, and separate the package-level semantic features into independent hidden states corresponding to each category;

[0024] The package-level semantic features and the corresponding independent hidden states are average-pooled to obtain the category-level semantic features.

[0025] Furthermore, the formula for activating the package-level semantic features and the hidden state representation of the prompt together is as follows:

[0026] score = softmax(H·H′ T )

[0027] H″ c =(score·H′)W c +b c

[0028] Among them, H is the semantic feature of the package level, H′ is the hidden state representation of the prompt, and H″ c is an independent hidden state, softmax represents the activation function, W c ,b c is a learnable parameter and c represents the category.

[0029] Furthermore, the residual structure is used to combine the package-level semantic features and the corresponding independent hidden states, and then average pooling is performed. The formula is as follows:

[0030] l c =average-pooling(H+αH″ c )

[0031] Among them, l c is the semantic feature at the category level, H is the semantic feature at the package level, and H″ c is an independent hidden state, α is a weight hyperparameter, and average-pooling represents average pooling.

[0032] Furthermore, the classifier performs classification prediction based on a multilayer perceptron and a sigmoid function, and the formula is as follows:

[0033] p c =sigmoid(MLP(l c ))

[0034] Among them, p c represents the category prediction probability, sigmoid represents the activation function that maps the variable to 0 to 1, MLP represents the multi-layer perceptron, l c is the semantic feature at the category level.

[0035] Furthermore, the following cross entropy loss function is used to optimize the model during training:

[0036]

[0037] Among them, L1 represents the cross entropy loss, N represents the number of samples in the training data set, y represents the true category label corresponding to the sample, and p c Represents the class prediction probability.

[0038] An encrypted traffic classification system based on word sense aggregation and semantic separation technology, comprising:

[0039] BERT tokenizer, used to tokenize network traffic data and obtain subwords;

[0040] Traffic encoder, used to encode the semantic features of subwords to obtain hidden state representation;

[0041] Semantic aggregator, used to perform semantic aggregation on hidden state representations to obtain package-level semantic features;

[0042] A semantic separator, which is used to perform semantic separation on the package-level semantic features under the guidance of the prompt template to obtain the category-level semantic features;

[0043] The classifier is used to make classification predictions based on the semantic features at the category level to obtain the encrypted traffic classification results.

[0044] Compared with the prior art, the present invention has the following advantages:

[0045] The method proposed in the present invention aims to solve the problem that current methods cannot effectively understand encrypted traffic data, and achieves more accurate encrypted traffic classification through word sense aggregation and semantic separation technology. By adopting word sense aggregation of the units after word segmentation of the pre-trained language model, and combining the prompt language learning technology to perform explicit semantic separation, the model's understanding of traffic data is enhanced, thereby improving the model's effect on classification tasks. From the two perspectives of word meaning and semantics, the present invention can decouple abstract features into different category spaces while retaining the complete word meaning, thereby enhancing the distinguishability of the features and improving the classification effect. Further improvements have been made in the four types of tasks on two public data sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a flow chart of an encrypted traffic classification method based on word sense aggregation and semantic separation technology of the present invention.

[0047] Figure 2 It is a schematic diagram of a word sense aggregator.

[0048] Figure 3 It is a schematic diagram of the category-constrained semantic separator. DETAILED DESCRIPTION

[0049] In order to make the various technical features and advantages or technical effects in the above technical solutions of the present invention more obvious and easy to understand, they are described in detail below with reference to the accompanying drawings.

[0050] The embodiment of the present invention specifically proposes an encrypted traffic classification method and system based on word meaning aggregation and semantic separation technology, such as Figure 1 As shown, the system includes models such as BERT segmenter, traffic encoder, word sense aggregator, category-constrained semantic separator and classifier. The method realizes the classification of encrypted traffic through these modules of the system. Among them, the word sense aggregator and the category-constrained semantic separator are core modules. The word sense aggregator can aggregate subwords after word segmentation to obtain complete word meanings without adding additional model parameters, which enables BERT to quickly adapt to understanding raw traffic data; the category-constrained semantic separator uses prompt templates to allow independent feature learning in semantic spaces of different categories, thereby enhancing the distinguishability of features. The technical solution of the present invention is described in detail as follows.

[0051] 1. Training Data

[0052] The captured network traffic data pcap file is parsed by the Scapy tool to obtain multiple data packets, each of which is presented in the form of a string. Formally, the data packet is recorded as X = {x1, x2, ..., xN}, contains N words and the corresponding true category label y as training data. For each word x i It can be divided into several subwords, recorded as: Where k represents the kth subword in the i-th word. This method outputs the true category label corresponding to the given data packet input, thereby classifying encrypted traffic.

[0053] 2. BERT tokenizer

[0054] The BERT tokenizer splits the input traffic data text into subwords that can be processed by the BERT encoder. It plays a vital role in preparing the input data for the model. Generally, the BERT tokenizer uses the WordPiece tokenizer, which splits a complete word into subwords. For a given word like "playing", the tokenizer splits it into "play" and "##ing". The prefix "##" indicates that this token is a subword. If a word does not exist in BERT's vocabulary, the tokenizer splits it into separate characters or replaces it with the [UNK] special character. Specifically, the tokenizer splits the traffic data X={x1,x2,…,x N} is divided into subwords, and the formula is as follows:

[0055]

[0056] The number of subwords each word is divided into is also recorded, which can help the subsequent word meaning aggregator to re-aggregate the complete word meaning.

[0057] 3. Flow encoder

[0058] The purpose of the traffic encoder is to generate a feature vector representation of network traffic data. After the BERT tokenizer finishes splitting the traffic data into subwords, the traffic encoder encodes these subwords to capture semantic-level traffic features and outputs a hidden state H sep The present invention uses a pre-trained BERT encoder as the flow encoder of the present invention, and the formula is as follows:

[0059]

[0060] Among them, Traffic Encoder represents the BERT encoder, X sep Symbols are used to represent the divided subwords; Representing subwords Hidden vector representation after passing through the encoder.

[0061] 4. Word Sense Aggregator

[0062] Since the BERT encoder is pre-trained on a natural language corpus, it lacks representation of common payload bytes (such as "\x17" and "x0f") found in network traffic data. Figure 2 As shown, in order to make full use of the general linguistic knowledge learned in the BERT pre-training stage and retain the complete word meaning information of the encrypted payload, the present invention proposes a simple and parameter-free word meaning aggregator. To achieve this, the present invention uses the subword length information retained by the BERT word segmenter to aggregate the word meaning after word segmentation. Specifically, the separated hidden states are aggregated, and the packet-level semantic feature information H is obtained by addition operation. The formula is as follows:

[0063] H=(h1,…,h i ,…,h N )

[0064]

[0065] where H∈R N×d , d is the hidden size of the encoder, and N is the number of words. Aggregating means aggregation operation. In this invention, the representation of these subwords is added together to obtain the representation of each word, i.e., h i represents the representation of the i-th word, and k represents h i The kth subword in the word.

[0066] 5. Category-constrained semantic separator

[0067] From a semantic point of view, traffic data has multiple types of semantics. For example, traffic in Gmail tends to be more text-based, while traffic in YouTube contains more video and audio semantics. In order to obtain better feature distinguishability, the present invention designs a category-constrained semantic separator, such as Figure 3 As shown, using the prompt template, the abstract semantic features are further explicitly separated in different category spaces. Prompt learning can be used as a stimulus and prompt to the model to explicitly guide the pre-trained model to complete human needs and tasks. For example, in the application classification task, the prompt P (i.e., Prompt) can be "Which category of application does the following data packet belong to? YouTube, Spotify, Skype, Gmail...". This prompt explicitly provides a natural way to inject bias into the semantic separator to guide the separation of semantic features, which helps to identify the feature semantics of different categories in traffic data.

[0068] To this end, we first encode the prompt P like traffic data using another traffic encoder (i.e., BERT encoder) with the same architecture as the traffic encoder to obtain the hidden state representation H of the prompt ′, the formula is as follows:

[0069] H′=Prompt Encoder(P)=(h′1,…,h′ j ,…,h′ M )

[0070] where H′∈R M×d , M represents the total number of subwords in the prompt, and j represents the jth subword of the prompt.

[0071] Then, an activation operation is performed to separate the packet-level semantics H into independent hidden states H corresponding to each category. ′ ' c , to obtain better distinguishability. The formula is as follows:

[0072] score = softmax(H·H′ T )

[0073] H″ c =(score·H′)W c +b c

[0074] where score∈R N×M , W c ∈R d×d , H″ c ∈R N×d .W c , b c represents learnable parameters, which can be randomly initialized at the beginning and optimized after training. c represents the category, and different categories have different parameters.

[0075] After the activation operation, in order to preserve the original information, the residual structure is directly used to combine the original and updated hidden states. Then average pooling is used to obtain the final category-level semantic features l for each class. c , the formula is as follows:

[0076] l c =average-pooling(H+αH″ c )

[0077] Among them l c ∈R d , α is a hyperparameter that weighs the weight between the two hidden states.

[0078] 6. Classifier

[0079] The goal of the classifier is to separate the semantic features of the category level l cDifferent tasks contain different categories (see the data set description in the experimental test section below for details). After the semantic separator calculates, each category has its own corresponding category-level semantic features. The classifier uses the sigmoid function to obtain the final class probability of each sample. The formula is as follows:

[0080] p c =sigmoid(MLP(l c ))

[0081] where p c represents the probability of belonging to the cth class, and MLP represents multi-layer perceptron.

[0082] 7. Loss function calculation

[0083] The present invention requires training of the traffic encoder, semantic separator and classifier. The training data uses a public traffic data set, where the input is a traffic data packet, denoted as X = {x1, x2, ..., x N}, contains N words; the output is the corresponding true category label y, such as YouTube, Facebook, Twitter, etc. in the application classification. The training data is input into the BERT word segmenter, and then passes through the traffic encoder, word sense aggregator, semantic separator and classifier to classify the input data packets. Specifically, the present invention uses cross entropy loss to optimize the model, and the formula is as follows:

[0084]

[0085] Where N represents the number of training set samples, and y represents the true category label corresponding to the sample.

[0086] Experimental testing

[0087] The method of the present invention is verified on the data sets ISCXVPN and USTC-TFC. ISCXVPN contains two tasks: VPN classification (task1), which determines whether the traffic packet belongs to VPN traffic; and application classification (task2), such as YouTube, Facebook, Twitter, etc. The USTC-TFC data set also contains two tasks: malware / benign software classification (task1), which determines whether the traffic packet belongs to malicious traffic; and application classification (task2), in which malware contains 10 categories, such as Cridex, Geodo, Htbot, Tinba, etc.; benign software contains 10 categories, such as BitTorrent, MySQL, Gmail, Weibo, etc.

[0088] In order to effectively evaluate the performance, the evaluation indicators consider marco-F1 and micro-F1. Specifically, marco-F1 calculates the F1 score of each category independently, takes the average, and then weights each category equally, which helps to evaluate the performance of the model on imbalanced problems. In contrast, micro-F1 directly calculates the F1 score of all classes without considering the difference in the number of categories.

[0089] The specific calculation of marco-F1 is as follows:

[0090] 1) Calculate the precision P and recall R for each category;

[0091] 2) Calculate F1 = 2*P*R / (P+R) for each category;

[0092] 3) The F1 values ​​of all categories are averaged to obtain the marco-F1.

[0093] 4) The specific calculation of micro-F1 is as follows:

[0094] 5) Aggregate the prediction results and the true category labels;

[0095] 6) Calculate the overall precision P and recall R;

[0096] 7) micro-F1 = 2*P*R / (P+R).

[0097] The precision and recall are calculated as follows: in the prediction results, the number of samples in a certain category is TP (true positive), the number of false positives is FP (false positive), and the number of missed reports is FN (false negative).

[0098] P=TP / (TP+FP)

[0099] R=TP / (TP+FN)

[0100] Among them, TP true positive refers to the number of categories that the classifier correctly classifies, FP false positive refers to the number of other categories that the classifier mistakenly classifies as this category, and FN false negative refers to the number of categories that the classifier fails to correctly classify as this category.

[0101] The main results are shown in Table 1. The proposed method consistently outperforms previous methods and baseline models on the ISCXVPN and USTC-TFC datasets. In the application classification task of ISCXVPN (task2), the proposed method achieves a 2.76% improvement on marco-F1 and a 1.61% improvement on micro-F1. In the application classification task of USTC-TFC (task2), the proposed method achieves a 1.56% improvement on marco-F1 and a 1.63% improvement on micro-F1. In addition, it is found that the effect of the proposed method on VPN classification and malicious / benign classification on both datasets has reached 100%. The possible reason is that the 2-classification task is relatively simple for the proposed method, so given sufficient dataset training, the proposed method can simply analyze the results. This experiment illustrates the effectiveness of the proposed method and shows the importance of learning complete word meanings and explicit independent features.

[0102] Table 1 Effect scores of models on ISCXVPN and USTC-TFC datasets

[0103]

[0104]

[0105] In addition, in order to verify the effectiveness of the word sense aggregator and category-constrained semantic separator proposed in the present invention, the present invention conducted ablation experiments on four tasks of two datasets. As shown in Table 2, removing any module will lead to performance degradation, which shows the degree of their contribution to the overall method and proves the effectiveness of the method of the present invention.

[0106] Table 2 Ablation experiment results on ISCXVPN and USTC-TFC datasets

[0107]

[0108] Although the present invention has been disclosed as above by way of embodiments, it is not intended to limit the present invention. Appropriate modifications or equivalent substitutions of the technical solutions of the present invention made by ordinary technicians in the field should all be included in the protection scope of the present invention. The protection scope of the present invention shall be based on what is defined in the claims.

Claims

1. An encrypted traffic classification method based on word sense aggregation and semantic separation technology, characterized in that: The following steps are involved: The network traffic data is segmented through the BERT tokenizer to obtain subwords; Encode the semantic features of the subwords through the flow encoder to obtain the hidden state representation; The hidden state representation is semantically aggregated through a semantic aggregator to obtain package-level semantic features; Design a prompt template with classification prompts. Under the guidance of the prompt template, the package-level semantic features are semantically separated through a semantic separator to obtain the category-level semantic features. Input the category-level semantic features into the classifier for classification prediction; The training data with real category labels are processed through the above steps to train the traffic encoder, semantic separator and classifier. After the training is completed, the encrypted traffic is processed according to the above steps to obtain the classification result of the encrypted traffic.

2. The method according to claim 1, characterized in that The BERT tokenizer uses the WordPiece tokenizer.

3. The method according to claim 1, characterized in that The traffic encoder uses the pre-trained BERT encoder.

4. The method according to claim 1, characterized in that The sense aggregator utilizes the subword length information retained by the BERT tokenizer to aggregate the senses represented by the hidden state.

5. The method according to claim 1, characterized in that The steps of semantically separating the package-level semantic features through the semantic separator under the guidance of the prompt template include: Use another flow encoder with the same architecture as the above flow encoder to encode the semantic features of the prompt and obtain the hidden state representation of the prompt; Activate the package-level semantic features and the hidden state representation of the prompt together, and separate the package-level semantic features into independent hidden states corresponding to each category; The package-level semantic features and the corresponding independent hidden states are average-pooled to obtain the category-level semantic features.

6. The method according to claim 5, characterized in that The formula for activating the package-level semantic features and the hidden state representation of the prompt together is as follows: score=softmax(H·H′ T ) H″ c =(score·H″)W c +b c Among them, H is the semantic feature of the package level, H" is the hidden state representation of the prompt, and H″ c is an independent hidden state, softmax represents the activation function, W c , b c is a learnable parameter and c represents the category.

7. The method according to claim 5, characterized in that Then, the residual structure is used to combine the package-level semantic features and the corresponding independent hidden states, and then average pooling is performed. The formula is as follows: l c =average-pooling(H+aH″ c ) Among them, l c is the semantic feature at the category level, H is the semantic feature at the package level, and H″ c is an independent hidden state, α is a weight hyperparameter, and average-pooling represents average pooling.

8. The method according to claim 1, characterized in that The classifier performs classification prediction based on multi-layer perceptron and sigmoid function. The formula is as follows: p c =sigmoid(MLP(l c )) Among them, p c represents the category prediction probability, sigmoid represents the activation function that maps the variable to 0 to 1, MLP represents the multi-layer perceptron, l c is the semantic feature at the category level.

9. The method according to claim 1 or 8, characterized in that The following cross entropy loss function is used to optimize the model during training: Among them, L1 represents the cross entropy loss, N represents the number of samples in the training data set, y represents the true category label corresponding to the sample, and p c Represents the class prediction probability.

10. An encrypted traffic classification system based on word sense aggregation and semantic separation technology, characterized in that: include: BERT tokenizer, used to tokenize network traffic data and obtain subwords; Traffic encoder, used to encode the semantic features of subwords to obtain hidden state representation; Semantic aggregator, used to perform semantic aggregation on hidden state representations to obtain package-level semantic features; A semantic separator, which is used to perform semantic separation on the package-level semantic features under the guidance of the prompt template to obtain the category-level semantic features; The classifier is used to make classification predictions based on the semantic features at the category level to obtain the encrypted traffic classification results.

Images