Encrypted network traffic classification model based on comparative learning and training method thereof

Abstract

The invention provides an encrypted network traffic classification model based on comparative learning and a training method thereof, the model comprises a lightweight network module, a classifier and a feature extractor, and the feature extractor comprises an embedded layer and a coding module; the method comprises the steps that a first training set is acquired, the first training set comprises positive samples and negative samples, the positive samples comprise time features and spatial features extracted from the same encrypted traffic, and the negative samples comprise time features and spatial features respectively extracted from two different encrypted traffic; the first training set is utilized to pre-train a feature extractor to learn the relation between the time features and the space features through comparative learning, a pre-trained feature extractor is obtained, a second training set is obtained, and each original sample of the second training set comprises the time features and the space features extracted from the same encrypted traffic; and freezing parameters of a coding module of the pre-trained feature extractor, training the model by using the second training set, and carrying out traffic classification according to the time features and the space features of the encrypted traffic to obtain a trained model.

Description

Technical Field

[0001] The present invention relates to the field of network security, and more specifically to the field of encrypted network traffic identification and deep learning, and more specifically to an encrypted network traffic classification model based on contrastive learning and a training method thereof. Background Art

[0002] With the rapid advancement of network technology, network applications continue to emerge, and network security issues have become increasingly prominent. Among them, network traffic identification, especially the distinction between encrypted traffic (covering protocols, applications, services and malicious traffic), has become a key issue. However, in order to strengthen data transmission and user privacy protection, encryption technology has been widely used, resulting in a sharp increase in encrypted traffic in the network. Although this enhances user privacy protection, it also provides opportunities for malicious traffic designers. Therefore, in the face of the challenges brought by traffic encryption, it is particularly important to develop a more robust encrypted network traffic identification technology.

[0003] There are various network traffic classification technologies, among which methods based on port numbers and deep packet inspection are both rule-driven methods. After traffic encryption, some traditional rule-based traffic identification methods are limited in practicality due to the difficulty of port hiding and feature extraction, such as references [1] and [2]. At the same time, the clear principles of these methods also provide convenience for attackers to design more realistic malicious traffic. Some methods try to identify traffic through the 1.2 version handshake information of the Transport Layer Security (TLS) protocol, but with the popularization of the TLS 1.3 version encryption protocol, some handshake information is no longer visible.

[0004] In recent years, the research focus has shifted to exploring other features that can be used after traffic encryption for network traffic classification. Given that deep learning can directly extract and identify features from raw network data, it can provide a high-precision end-to-end solution. As a result, there are more and more encrypted traffic identification methods based on deep learning in existing methods, such as references [3], [4], [5], and [6]. In addition, there are some other deep learning methods as follows:

[0005] For example, in reference [7], the Deep Packet method proposed by Lotfollahi M et al. uses deep learning to process network traffic classification, including traffic feature extraction of major categories (such as FTP (File Transfer Protocol) and P2P (peer-to-peer)), and identification of end-user applications (such as BitTorrent and Skype). This method can not only identify encrypted traffic, but also distinguish VPN and non-VPN network traffic, but it performs poorly in Tor traffic classification, indicating that although Deep Packet uses a deep learning method, the model design is relatively simple and there is still much room for improvement.

[0006] For example, in reference [8], the FS-Net method proposed by C. Liu et al. is an end-to-end encrypted traffic identification network that automatically extracts and outputs features using packet length sequences as input. This method promotes feature learning through a reconstruction mechanism, so that the generated features contain more identification information to improve the recognition ability. However, FS-Net only relies on the single feature of the packet length sequence of the data packet, which is easily affected by traffic camouflage techniques such as packet loss and disorder, and lacks robustness.

[0007] For example, in reference [9], T. Shapira and Y. Shavitt proposed the FlowPic method, which uses the time-related and size-related characteristics of the flow to convert basic traffic data into intuitive FlowPic images, and then uses the known image classification deep learning technology CNN to identify traffic categories (such as browsing, chatting, video, etc.). However, this method is also susceptible to camouflage techniques such as packet loss and disorder due to its single feature, and has weak anti-interference ability.

[0008] Considering that metadata features such as packet length sequence and packet arrival time interval sequence may be affected by traffic camouflage, some studies have found that although the plaintext information of network flows is encrypted, there are still differences, such as reference

[10] , the plaintext information of network flows contains semantic information that can be used for classification. Therefore, there are also methods that use natural language processing models to learn the semantic information of the encrypted payload to identify encrypted traffic.

[0009] At the same time, in deep learning, the quality and quantity of input samples are also crucial to the model effect. However, models built by supervised learning methods such as CNN and RNN are highly dependent on labeled data. However, labeled data is not only small in size and complex to manually annotate, but may also have problems such as annotation errors. As a result, although these supervised learning methods can achieve good results on the training set, they are not well deployed in real network scenarios. To this end, combined with the characteristics of the encrypted payload information mentioned above, such as the reference

[11] proposed by Lin X et al., which is an encrypted traffic identification method based on the natural language processing model BERT, ET-BERT uses natural language processing methods to process and learn encrypted payload information. Based on the unsupervised pre-training method, a large number of unlabeled data sets are used to learn the representation method of traffic, and then the model is fine-tuned based on a small amount of labeled data. However, encrypted traffic is very different from natural language and does not contain clear semantic information. ET-BERT only uses the encrypted payload byte information, and the learned semantic features are weak. On the one hand, the mask prediction task mainly used by ET-BERT relies on the context information of the encrypted payload. Once packet loss and disorder occur, its context information will be affected to a certain extent, which in turn affects the learning effect of its mask prediction task.

[0010] Therefore, when classifying encrypted network traffic, existing methods, on the one hand, use unsupervised learning and fine-tuning with a small amount of labeled data to obtain weak semantic features of the encrypted payload byte information, which is easily affected by the contextual information of the encrypted payload, resulting in reduced prediction effect; on the other hand, although the supervised learning method can achieve good results on the training set, the labeled data not only has a small amount of data, complex manual labeling, and possible labeling errors, but also has the problem of reduced prediction effect in real network scenarios.

[0011] It should be noted that this background technology is only used to introduce the relevant information of the present invention to help understand the technical solution of the present invention, but it does not mean that the relevant information is necessarily the prior art. The relevant information is submitted and disclosed together with the present invention solution. If there is no evidence that the relevant information has been disclosed before the application date of the present invention, the relevant information shall not be regarded as the prior art.

[0012] References are as follows:

[0013] [1]V. F. Taylor, R. Spolaor, M. Conti, and I. Martinovic, ”Robustsmartphone app identification via encrypted network traffic analysis,” IEEETransactions on Information Forensics and Security, vol. 13, no. 1, Jan.2018, pp. 63-78.

[0014] [2] Y. Chen, T. Zang, Y. Zhang, Y. Zhou, and Y. Wang, “Rethinkingencrypted traffic classification: A multi-attribute associated fingerprintapproach,” in 27th IEEE International Conference on Network Protocols, ICNP2019, Chicago, IL, USA, October 8-10, 2019. IEEE, 2019, pp. 1–11.

[22] J. Che

[0015] [3] T. Ongun, T. Sakharaov, S. Boboila, A. Oprea, and T. Eliassi-Rad,“On designing machine learning models for malicious network trafficclassification,” CoRR, vol. abs / 1907.04846, 2019.

[0016] [4] Rezaei S, Liu X. Deep Learning for Encrypted TrafficClassification: An Overview[J]. IEEE Communications Magazine, 2019, 57(5):76-81.

[0017] [5] Lan J , Liu X , Li B , et al. DarknetSec: A novel self-attentivedeep learning method for darknet traffic classification and applicationidentification[J]. Computers & Security, 2022, 116(2):102663.

[0018] [6]J. Guo et al., "Global-Aware Prototypical Network for Few-ShotEncrypted Traffic Classification," 2022 IFIP Networking Conference (IFIPNetworking), Catania, Italy, 2022, pp. 1-9, doi: 10.23919 / IFIPNetworking55013.2022.9829771.

[0019] [7] Liu C, He L, Xiong G, et al. Fs-net: A flow sequence network forencrypted traffic classification[C] / / IEEE INFOCOM 2019-IEEE Conference OnComputer Communications. IEEE, 2019: 1171-1179.

[0020] [8] Lotfollahi M, Zade R S H, Siavoshani M J, et al. Deep Packet: ANovel Approach For Encrypted Traffic Classification Using Deep Learning[J].Soft Computing, 2017.

[0021] [9] T. Shapira and Y. Shavitt, "FlowPic: A Generic Representation for Encrypted Traffic Classification and Applications Identification," in IEEETransactions on Network and Service Management, vol. 18, no. 2, pp. 1218-1232, June 2021, doi: 10.1109 / TNSM.2021.3071441.

[0022]

[10] Xinjie Lin, Gang Xiong, Gaopeng Gou, Zhen Li, Junzheng Shi, JingYu: ETBERT: A Contextualized Datagram Representation with PretrainingTransformers for Encrypted Traffic Classification. WWW 2022.

[0023]

[11] Lin X, Xiong G, Gou G, et al. Et-bert: A contextualized datagram representation with pre-training transformers for encrypted traffic classification[C] / / Proceedings of the ACM Web Conference 2022. 2022: 633-642. Summary of the invention

[0024] Therefore, the purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide an encrypted network traffic classification model based on contrastive learning and a training method thereof.

[0025] The objective of the present invention is achieved through the following technical solutions:

[0026] According to a first aspect of the present invention, a pre-training method for an encrypted traffic feature extractor based on contrastive learning is provided, wherein the feature extractor is constructed using a neural network and includes an embedding layer and an encoding module, and the method includes: obtaining a first training set, including positive samples and negative samples, the positive samples including time features and spatial features extracted from the same encrypted traffic, and the negative samples including time features and spatial features extracted from two different encrypted traffics, respectively; using the first training set to pre-train the feature extractor through contrastive learning to obtain a pre-trained feature extractor, including: processing the time features of the input positive and negative samples together through the embedding layer and the encoding module to obtain the time codes of the positive and negative samples; processing the spatial features of the input positive and negative samples through the encoding module to obtain the spatial codes of the positive and negative samples; and updating the parameters of the feature extractor with the optimization goal of maximizing the similarity between the time codes and spatial codes of the positive samples and minimizing the similarity between the time codes and spatial codes of the negative samples.

[0027] In some embodiments of the present invention, the encoding module includes a time encoding unit and a space encoding unit, wherein the embedding layer is used to process the input time features to obtain time embedding, and the time encoding unit processes the input time embedding to obtain time coding; the space encoding unit is used to process the input space features to obtain space coding.

[0028] In some embodiments of the present invention, each encrypted traffic is encrypted session traffic, which includes a number of data packets generated during the interaction between the communicating parties. The method for extracting the time characteristics and spatial characteristics includes: extracting the multimodal characteristics of the encrypted session traffic according to the data packets of each encrypted session traffic, which includes metadata characteristics, encrypted payload byte characteristics and burst transmission characteristics; using the metadata characteristics of an encrypted session traffic as the time characteristics, splicing the encrypted payload byte characteristics and burst transmission characteristics of the encrypted session traffic, and obtaining the spatial characteristics of the traffic.

[0029] According to a second aspect of the present invention, a training method for an encrypted network traffic classification model based on contrastive learning is provided, the training method comprising: S1, obtaining an initial model, which comprises a lightweight network module, a classifier and a pre-trained feature extractor obtained based on the method described in the first aspect of the present invention, wherein the feature extractor comprises an embedding layer and an encoding module; S2, obtaining a second training set, comprising a plurality of original samples corresponding to a plurality of categories required to be identified by the model, the original samples comprising original data and category labels, the original data comprising time features and spatial features extracted from the same encrypted traffic, and the category labels indicating the category of the encrypted traffic to which the original data belongs; S3, training the initial model using the second training set to obtain a trained model, the training method comprising: S31, processing the spatial features of the input original samples using the encoding module to obtain spatial coding, and using the lightweight network module based on a neural network to obtain spatial coding. Process the spatial coding to obtain spatial information; S32, use the embedding layer to generate a time prompt for the time feature of the original sample based on the input spatial information according to the introduced soft prompt method, and process the time feature of the input original sample based on the time prompt and the input multiple category labels to obtain the time embedding corresponding to each of the multiple categories; S33, use the encoding module to process the time embedding corresponding to each of the multiple categories input to obtain the time coding corresponding to each of the multiple categories; S34, use the classifier to predict the classification result of the original sample based on the calculated similarity between the time coding corresponding to each of the multiple categories and the spatial coding, which includes the probability value of the original sample belonging to each category; S35, freeze the parameters of the encoding module of the pre-trained feature extractor, and update the parameters of the embedding layer, the lightweight network module and the classifier with the optimization goal of minimizing the difference between the label and the classification result of the original sample.

[0030] In some embodiments of the present invention, in S3, the training method includes multiple rounds of iterative training, and each round of iteration is performed according to the process from S31 to S35, wherein in S32, the method of generating time prompts includes: obtaining spatial information corresponding to the original sample of the current round, and the method of obtaining the spatial information includes: inputting the spatial features of the original sample into the encoding module for processing to obtain the spatial code; inputting the spatial code into the lightweight network module of the current round to obtain the spatial information, wherein the first round uses the initialized lightweight network module, and each round after the first round uses the lightweight network module updated in the previous round; obtaining the initial time prompt of the current round, and using the embedding layer of the current round to fuse the input spatial information with the initial time prompt to generate a time prompt, wherein the first round uses the embedding layer of the pre-trained feature extractor, and each round after the first round uses the embedding layer updated in the previous round.

[0031] In some embodiments of the present invention, the initial time prompt is trainable, wherein when generating time prompts in the first round, a randomly initialized initial time prompt is used, and when generating time prompts in each round after the first round, the initial time prompt updated in the previous round is used, and the updating method of the initial time prompt in each round includes: updating the initial time prompt with the optimization goal of minimizing the difference between the label and classification result of the original sample of the round.

[0032] In some embodiments of the present invention, in S32, the method for obtaining the time embeddings corresponding to each of the multiple categories includes: splicing the attributes of multiple category labels with the time prompts respectively to obtain multiple time prompts containing different category attributes; copying the time features of the original sample according to the number of all categories to obtain multiple time features, and splicing the multiple time features with the multiple time prompts containing different category attributes one-to-one to obtain multiple time features containing different category attributes and time prompts; inputting the multiple time features containing different category attributes and time prompts into the embedding layer for processing to obtain the time embeddings corresponding to the multiple categories, and when the embedding layer is processing, for each category, inputting a time feature containing the category attribute into the embedding layer to obtain the time embedding corresponding to the category.

[0033] According to a third aspect of the present invention, there is provided a method for classifying encrypted network traffic, the classification method comprising: obtaining time features and spatial features of encrypted traffic to be classified, and obtaining a trained model according to the method described in the second aspect of the present invention; processing the spatial features of the encrypted traffic to be classified using the encoding module of the model to obtain spatial coding, and processing the spatial coding using the lightweight network module of the model to obtain spatial information; generating time prompts for time features based on the spatial information of the encrypted traffic to be classified according to the introduced soft prompt method using the embedding layer of the model, processing the time features of the encrypted traffic to be classified multiple times based on the time prompts and multiple categories required to be identified by the model to obtain time embeddings corresponding to the multiple categories; processing the time embeddings corresponding to the multiple categories using the encoding module of the model to obtain time coding corresponding to the multiple categories; and obtaining the probability value of the encrypted traffic to be classified belonging to each category using the classifier of the model according to the calculated similarity between the time coding and the spatial coding corresponding to the multiple categories, and taking the category with the maximum probability value as the category of the encrypted traffic.

[0034] According to a fourth aspect of the present invention, there is provided an electronic device comprising: one or more processors; and a memory, wherein the memory is used to store executable instructions; the one or more processors are configured to implement the steps of the method described in the first aspect, the second aspect and the third aspect of the present invention by executing the executable instructions.

[0035] Compared with the prior art, the advantages of the present invention are:

[0036] When pre-training the feature extractor, the present invention uses a first training set in which the positive samples include time features and space features extracted from the same encrypted traffic, and the negative samples include time features and space features extracted from two different encrypted traffics, respectively. The constructed positive and negative samples include encrypted traffic features of multiple dimensions in time and space. The training set with more semantic features enables the model of the present invention to learn more semantic feature information of encrypted traffic, thereby improving the prediction effect of the model. Self-supervised learning technology is used, and massive unlabeled positive and negative sample data is used to pre-train the feature extractor through comparative learning to learn the intrinsic correlation between the time features and space features of positive samples, and the intrinsic correlation between the time features and space features of negative samples, so as to learn the wider distribution of feature data, which can be applied to a variety of downstream tasks, making the feature extractor more widely used in real network scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The embodiments of the present invention are further described below with reference to the accompanying drawings, in which:

[0038] Figure 1 1. It is a schematic diagram of a flow chart of a pre-training method for a feature extractor according to an embodiment of the present invention;

[0039] Figure 2 Schematic diagram of the pre-training process principle of the feature extractor according to an embodiment of the present invention;

[0040] Figure 3 A flowchart of a training method for an encrypted network traffic classification model according to an embodiment of the present invention;

[0041] Figure 4 Schematic diagram of the structure of the initial model and the principle process of its training method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0043] As mentioned in the background technology section, when classifying encrypted network traffic, existing methods, on the one hand, use unsupervised learning and fine-tuning with a small amount of labeled data to obtain weak semantic features of the encrypted payload byte information, which is easily affected by the contextual information of the encrypted payload, resulting in reduced prediction effect; on the other hand, although the supervised learning method can achieve good results on the training set, the labeled data not only has a small amount of data, complex manual labeling, and possible labeling errors, but also has the problem of reduced prediction effect in real network scenarios.

[0044] Based on the above problems, the present invention provides an encrypted network traffic classification model based on contrastive learning and a training method thereof. The model includes an encrypted traffic feature extractor, a lightweight network module and a classifier, and the training method includes: using a first training set to pre-train the feature extractor to obtain a pre-trained feature extractor, and then using a second training set to train downstream tasks based on the model as a whole, fine-tune some parameters of the model, and obtain a trained model. Among them, with respect to the above-mentioned first aspect of the problem, the inventor analyzed and found that the prior art did not deeply consider the difference between traffic data and natural language data, and could not design more suitable traffic training data and training methods. Therefore, in the first training set used by the present invention when pre-training the feature extractor, the positive sample includes the time features and spatial features extracted from the same encrypted traffic, and the negative sample includes the time features and spatial features extracted from two different encrypted traffics, respectively. The constructed positive and negative samples include encrypted traffic features of multiple dimensions of time and space, and the training set with more semantic features enables the model of the present invention to learn more semantic feature information of encrypted traffic, thereby improving the model prediction effect. In response to the above-mentioned problem on the other hand, the present invention utilizes self-supervised learning technology, that is, when both positive and negative samples are unlabeled, massive unlabeled positive and negative sample data are used to pre-train the feature extractor through comparative learning to learn the intrinsic correlation between the temporal features and spatial features of positive samples, and the intrinsic correlation between the temporal features and spatial features of negative samples, so as to learn a wider distribution of feature data, which can be applied to a variety of downstream tasks, making the feature extractor more widely used in real network scenarios, and also making the model containing the feature extractor have a more robust and generalized understanding of the encrypted network traffic pattern, thereby improving the prediction accuracy.

[0045] In order to better understand the present invention, the following detailed description is given from the perspectives of data set construction, feature extractor structure and pre-training process, model structure and training process, and model application in combination with specific embodiments.

[0046] 1. Dataset Construction Method

[0047] According to one embodiment of the present invention, a method for constructing a data set includes the following steps 1-5:

[0048] In step 1, encrypted traffic data is collected from the network according to the multiple categories required to be identified in the traffic classification task, and a traffic data file is obtained, which includes multiple original PCAP files corresponding to each of the multiple categories for recording encrypted traffic data.

[0049] According to one embodiment of the present invention, the traffic classification task is an application traffic identification task, a business traffic identification task or a malicious traffic identification task, etc., and the present invention is not limited to this. Among them, the multiple categories that need to be identified in the application traffic identification task include traffic categories corresponding to multiple applications, the multiple categories that need to be identified in the business traffic identification task include traffic categories corresponding to multiple businesses, and the multiple categories that need to be identified in the malicious traffic identification task include malicious traffic categories and normal traffic categories. Schematically, the traffic categories corresponding to multiple applications may include WeChat application traffic categories, Taobao application traffic categories, Douyin application traffic categories, Meituan application traffic categories, etc.; the traffic categories corresponding to multiple businesses may include live broadcast traffic categories, on-demand traffic categories, voice traffic categories, download traffic categories, game traffic categories, music traffic categories, etc.

[0050] According to one embodiment of the present invention, taking the task of application traffic identification as an example, tools such as Wireshark or Tshark can be used to capture traffic packets under various applications in the network and save them as original PCAP files. The original PCAP file of the application traffic records the application to which the original PCAP file belongs when the tool is used to capture the packet. This is to distinguish the application traffic category to which the encrypted network traffic recorded in the file belongs.

[0051] In step 2, a number of encrypted session flows are determined based on the flow data file, including determining one or more encrypted session flows contained in each original PCAP file according to the PCAP file, and each encrypted session flow includes a number of data packets generated during the interaction between the communicating parties.

[0052] According to an embodiment of the present invention, in step 2, the method of determining one or more encrypted session flows contained in each original PCAP file includes the following steps 21 and 22:

[0053] Step 21: Extract original information from the original PCAP file, which includes multiple data packets and the packet header field of each data packet.

[0054] According to one embodiment of the present invention, the content of each original PCAP file is read one by one to extract the original information of each original PCAP file. The original information includes all data packets contained in the PCAP file and each data packet information, and each data packet information includes the source IP address, the destination IP address, the source port number, the destination port number, the transport layer protocol, the byte length of the data packet, and the time when the data packet arrives. The packet header field includes a five-tuple information consisting of the source IP address, the destination IP address, the source port number, the destination port number, and the transport layer protocol.

[0055] Step 22: According to the packet header fields of each data packet, determine one or more encrypted session traffic in the form of a session, and divide the multiple data packets into the corresponding encrypted session traffic, wherein the encrypted session traffic in the form of a session includes the bidirectional traffic formed during bidirectional communication between the communicating parties.

[0056] According to an embodiment of the present invention, the encrypted session traffic includes bidirectional traffic, and all data packets of each traffic in the bidirectional traffic have the same five-tuple information, and the encrypted session traffic includes all data packets of the bidirectional traffic. Schematically, if the two communicating parties are Party A and Party B, the bidirectional traffic includes a traffic corresponding to data transmitted from Party A to Party B and a traffic corresponding to data transmitted from Party B to Party A. The technical solution of this embodiment can at least achieve the following beneficial technical effects: because the session traffic includes bidirectional traffic, which contains the interactive information of the communicating parties, more information can be obtained, which is more conducive to traffic classification.

[0057] According to one embodiment of the present invention, the method of dividing multiple data packets into the encrypted session traffic to which they belong includes: determining the IP address and port number of each pair of communicating parties interacting in multiple pairs according to the five-tuple information of each data packet; dividing the data packets related to the IP address and port number of a pair of communicating parties interacting into the corresponding encrypted session traffic according to the five-tuple information of each data packet. That is, the five-tuple information of all data packets in the encrypted session traffic includes the IP address and port number of a pair of communicating parties interacting. And after the source IP address and the destination IP address in the five-tuple information of a data packet in the encrypted session traffic are interchanged, and the source port number and the destination port number are interchanged, the data packet still belongs to the encrypted session traffic.

[0058] In step 3, according to the respective data packets of each encrypted session flow, the multimodal features of the encrypted session flow are extracted, which include metadata features, encrypted payload byte features and burst transmission features.

[0059] According to an embodiment of the present invention, the following describes the extraction methods of metadata features, encrypted payload byte features, and burst transmission features respectively:

[0060] 1) Extraction of metadata features

[0061] According to one embodiment of the present invention, the metadata feature includes multiple time series data, and the method for extracting the multiple time series data includes: obtaining a preset number of data packets in each encrypted session traffic in chronological order; extracting metadata of each data packet in the preset number of data packets obtained, including the byte length and arrival time interval of the data packet, the byte length of each data packet includes the byte length occupied by all data in the packet, and the arrival time interval is determined based on the difference between the arrival time of the data packet and the arrival time of the adjacent data packet at the previous time; obtaining multiple time series data based on the metadata of all the obtained data packets, including splicing the byte length and arrival time interval of each data packet to obtain one time series data.

[0062] According to an embodiment of the present invention, n packets transmitted previously in each encrypted session flow are obtained in chronological order. It should be noted that when extracting the three features of metadata features, encrypted payload byte features, and burst transmission features, the packets used and the total number of packets used are the same, which can be 500 packets transmitted previously, 1000 packets transmitted previously, or 1200 packets transmitted previously, etc. for each encrypted session flow, and the present invention does not limit the value of n.

[0063] According to one embodiment of the present invention, the metadata of each of the n acquired data packets is extracted, including counting the byte length and arrival time interval of the n data packets transmitted before each encrypted session traffic. When n=1000, the byte length of each of the 1000 data packets transmitted before each encrypted session traffic is counted to obtain a byte sequence containing a 1000-byte length, and the arrival time interval of each of the first 1000 data packets of the encrypted session traffic, to obtain a time series containing 1000 arrival time intervals, and the byte sequence and the time series are spliced ​​one by one to form a metadata feature containing 1000 time series data and each time series data dimension is 2. The present invention extracts time series data as metadata features to achieve the analysis of traffic types from a time perspective.

[0064] 2) Extraction of encrypted payload byte features

[0065] According to one embodiment of the present invention, the method for extracting the encrypted payload byte feature in the multimodal feature includes: extracting a preset number of bytes from each encrypted session flow, and all the extracted bytes do not include the bytes of the packet header and the bytes of the packet tail; wherein the partially extracted bytes are encrypted payload bytes, and the encrypted payload bytes refer to the actual transmitted data located in the middle part of the packet after removing the byte information of the packet header and the tail in network communication. The extracted preset number of bytes are constructed into a payload byte image of a preset size, and the byte image is used as the encrypted payload byte feature, wherein the values ​​of the preset number of bytes are used as the values ​​of each pixel in the byte image.

[0066] According to one embodiment of the present invention, the 1000 data packets transmitted before each encrypted session traffic are obtained, and after removing the header and tail bytes of each data packet, bytes are collected from the 1000 data packets from which the header and tail bytes are removed. Take the construction of a payload byte image with a preset size of 256×256×1 (length×width×number of channels, the same below) as an example to illustrate: 65536 bytes are collected from the 1000 data packets from which the header and tail bytes are removed. Among them, if the bytes in the 1000 data packets from which the header and tail bytes are removed are too long, the bytes of the following data packets are trimmed and only the first 65536 bytes are taken; if it is less than 65536 bytes, 0 is added to all bytes of the 1000 data packets from which the header and tail bytes are removed until 65536 bytes are met. 65536 bytes are used, and one byte corresponds to one pixel of the image. An image with a height of 256 and a width of 256 is reconstructed, generating a payload byte image with a size of 256×256×1. Since each byte size is a value of 0-255, the pixels of the payload byte image are values ​​of 0-255. The payload byte image is used as an encrypted payload byte feature. If a payload byte image with a preset size of 128×128×1 is constructed, 16384 bytes need to be collected from the 1000 data packets with the head and tail bytes removed to construct the image according to the above implementation.

[0067] 3) Extraction of Burst Transmission Features

[0068] According to an embodiment of the present invention, a method for extracting a burst transmission feature includes steps e1, e2, e3, e4 and e5:

[0069] Step e1: Obtain a preset number of data packets in each encrypted session flow in chronological order, and count multiple burst transmission processes existing in all the acquired data packets, where each burst transmission process represents a continuous transmission process of multiple data packets in the same communication direction.

[0070] According to an embodiment of the present invention, burst transmission means the continuous transmission of multiple data packets in the same communication direction, and each encrypted session traffic is a bidirectional traffic. For example, in the encrypted session traffic generated when A and B communicate with each other, the 1000 data packets previously transmitted by the encrypted session traffic are obtained, and the transmission from A to B and the transmission from B to A are two directions. If the first to the fifth data packets in the 1000 data packets obtained are all continuously transmitted from A to B, then the process of transmitting the first to the fifth data packets constitutes a burst transmission process. In this way, the number of burst transmission processes existing in the 1000 data packets obtained is counted.

[0071] Step e2: Counting the number of bytes transmitted by each of the multiple burst transmission processes, where the number of bytes transmitted by each burst transmission process is the sum of the number of bytes of all data packets transmitted by the burst transmission process.

[0072] According to an embodiment of the present invention, if a burst transmission process includes 5 data packets transmitted continuously, the number of bytes of the 5 data packets is summed to obtain the number of bytes transmitted by the burst transmission process. When counting, all burst transmission processes are counted in time order to obtain the number of bytes transmitted by each of the multiple burst transmission processes. Schematically, taking 1000 data packets as an example, the process of continuously transmitting the 1st to 8th data packets in the same communication direction is the first burst transmission process, the process of continuously transmitting the 9th to 15th data packets in the same communication direction is the second burst transmission process, and the process of continuously transmitting the 16th to 22nd data packets in the same communication direction is the third burst transmission process, ..., if the number of bytes transmitted by the first, second, and third burst transmission processes is counted in turn as 64, 128, 32, ..., the number of bytes transmitted by each of the multiple burst transmission processes in the 1000 data packets is obtained.

[0073] Step e3: Counting the time intervals corresponding to the multiple burst transmission processes, where the time interval corresponding to each burst transmission process is the difference between the end time of the burst transmission process and the end time of the previous adjacent burst transmission process.

[0074] According to an embodiment of the present invention, schematically, in the first, second, and third burst transmission processes given by the example of the above embodiment, the time interval of the first burst transmission process is recorded as the difference between the end time of the burst transmission process and 0, the time interval of the second burst transmission process is the difference between the end time of the burst transmission process and the end time of the first burst transmission process, and the time interval of the third burst transmission process is the difference between the end time of the burst transmission process and the end time of the second burst transmission process, so as to calculate the time intervals corresponding to the multiple burst transmission processes.

[0075] Step e4: Generate a two-dimensional joint distribution graph according to the number of bytes transmitted in each of the multiple burst transmission processes and the corresponding time intervals, which includes multiple value pairs, each value pair records the number of bytes transmitted in a burst transmission process and the corresponding time interval.

[0076] According to one embodiment of the present invention, the number of bytes transmitted by each of the multiple burst transmission processes is normalized, and the time intervals corresponding to each of the multiple burst transmission processes are normalized to generate a two-dimensional joint distribution picture (two-dimensional graph) of the number of bytes and time intervals of the multiple burst transmission processes. Schematically, the normalization process is to map all values ​​to the interval [0,1], and each value pair in the two-dimensional joint distribution graph records the normalized number of bytes corresponding to a burst transmission process and the corresponding normalized time interval in a two-dimensional form. Schematically, assuming that the horizontal axis of the graph is the x-axis and the vertical axis is the y-axis, then the x-axis refers to a deduplicated ordered sequence formed by the normalized number of bytes of each of the multiple burst transmission processes, and the y-axis refers to a deduplicated ordered sequence formed by the normalized time intervals of each of the multiple burst transmission processes. The de-duplicated ordered sequence means a re-ordered sequence after removing repeated values. For example, the original sequence formed by the normalized number of bytes of multiple burst transmission processes is [0.1, 0.5, 0.2, 0.8, 0.2], and after de-duplication and sorting, it becomes [0.1, 0.2, 0.5, 0.8]. Then any point on this two-dimensional graph corresponds to a two-dimensional value pair. For example, in the value pair (Xi, Yi) corresponding to the i-th point, Xi is the normalized number of bytes of the burst transmission process corresponding to the i-th point, and Yi is the normalized time interval of the burst transmission process corresponding to the i-th point.

[0077] Step e5: Mapping the two-dimensional joint distribution graph into a burst distribution image of a preset size, and using the distribution graph as a burst transmission feature.

[0078] According to one embodiment of the present invention, the two-dimensional joint distribution graph can be proportionally mapped to a height of 256 and a width of 256. Thus, a burst distribution image is obtained, the image size is 256×256×1, 1 is the number of channels, and the distribution graph is used as a burst transmission feature. In addition, the preset sizes of the payload byte image and the burst distribution image are generally set to be the same, which can be 256×256×1, 128×128×1, or 64×64×1, and the present invention is not limited to this. This is to facilitate the subsequent construction of spatial features based on the payload byte features and the burst transmission features.

[0079] In step 4, a data set including multiple samples is constructed according to the multimodal features of several encrypted session flows and the category corresponding to the original PCAP file to which each encrypted session flow belongs, and each sample includes input data constructed according to the multimodal features of each encrypted session flow and a label indicating the category to which the encrypted session flow belongs.

[0080] According to one embodiment of the present invention, the multimodal features of an encrypted session flow are used to construct the input data of a sample, and the construction method includes: using the metadata features of an encrypted session flow as the time features, splicing the encrypted payload byte features and burst transmission features of the encrypted session flow, and obtaining the spatial features of the flow; constructing the input data of the sample, which includes the time features and spatial features of the encrypted session flow.

[0081] According to an embodiment of the present invention, the metadata feature is used as a time feature, the encrypted payload byte feature is used as a channel of the image, and the burst transmission feature is extracted as another channel of the image, and a dual-channel image is jointly generated as a spatial feature. Schematically, the metadata feature of each encrypted session traffic is used as a time feature, and the 1000 data packets transmitted before each encrypted session traffic are obtained as an example. The time feature is a multi-channel time series with a length of 1000 and a channel number of 2. The obtained 256×256×1 payload byte image and 256×256×1 burst distribution image are spliced ​​in the channel dimension, that is, a picture with a height and width of 256 and a channel number of 2 is obtained (the picture dimension is recorded as 256×256×2), which is the spatial feature of the sample input data. Thus, the spatial feature input and time feature input of each sample can be obtained. The technical solution of this embodiment can at least achieve the following beneficial technical effects: the input data of the sample of the present invention includes multimodal traffic feature information, which is conducive to the model learning traffic semantic feature information from multiple dimensions to improve the traffic recognition performance of the model.

[0082] 2. Feature Extractor Structure and Pre-training Process

[0083] According to one embodiment of the present invention, the encrypted traffic feature extractor is constructed using a neural network and includes an embedding layer and a coding module, and the structure of the coding module is based on a multimodal pre-trained neural network (Contrastive Language-ImagePre-Training, referred to as CLIP network), that is, its coding module includes a temporal encoder and a spatial encoder. The coding module can be formed by stacking Transformers or constructed by other deep neural networks, and the present invention is not limited to this.

[0084] According to an embodiment of the present invention, based on the structure of the feature extractor in the above embodiment, a pre-training method for an encrypted traffic feature extractor based on contrastive learning is provided. Figure 1 , which is a flow chart of the pre-training method of the feature extractor of the present invention. The pre-training method includes the following steps a1 and a2, and step a2 includes steps a21, a22 and a23:

[0085] In step a1, a first training set is obtained, including positive samples and negative samples. The positive samples include time features and spatial features extracted from the same encrypted traffic, and the negative samples include time features and spatial features extracted from two different encrypted traffics.

[0086] According to an embodiment of the present invention, each encrypted traffic is an encrypted session traffic determined when the data set is constructed in the above embodiment, which includes several data packets generated during the interaction between the two communicating parties. The extraction method of time features and spatial features adopts the same method as when the data set is constructed in the above embodiment, that is, according to the data packets of each encrypted session traffic, the multimodal features of the encrypted session traffic are extracted, which include metadata features, encrypted payload byte features and burst transmission features; the metadata features of an encrypted session traffic are used as time features, and the encrypted payload byte features and burst transmission features of the encrypted session traffic are spliced ​​to obtain the spatial features of the traffic. When constructing the positive sample of the first training set, the time features and spatial features of the same encrypted session traffic are obtained to obtain the positive sample. When constructing the negative sample, the time features of an encrypted session traffic are obtained, and then the spatial features of another encrypted session traffic are obtained to obtain the negative sample, which includes the time features and spatial features extracted from two different encrypted traffics.

[0087] It should be noted that the pre-training process does not need to consider the specific downstream tasks. Therefore, data sets under multiple different traffic classification tasks can be constructed according to the above data set construction method. When constructing the first training set, multimodal features of multiple encrypted session flows can be obtained from data sets under multiple different traffic classification tasks to achieve the construction of multiple positive samples and multiple negative samples, so that the feature extractor of the present invention can learn a wider distribution of feature data to be applied to multiple downstream tasks, with a wide range of applicability.

[0088] In step a2, the feature extractor is pre-trained by contrastive learning using the first training set to obtain a pre-trained feature extractor, and the pre-training includes: step a21: processing the temporal features of the input positive samples and negative samples together with the embedding layer and the encoding module to obtain the temporal encoding of the positive samples and the negative samples; step a22: processing the spatial features of the input positive samples and negative samples respectively with the encoding module to obtain the spatial encoding of the positive samples and the negative samples respectively; step a23: updating the parameters of the feature extractor with the optimization goal of maximizing the similarity between the temporal encoding and the spatial encoding of the positive samples and minimizing the similarity between the temporal encoding and the spatial encoding of the negative samples.

[0089] According to one embodiment of the present invention, see Figure 2 , which is a schematic diagram of the principle of the pre-training process of the feature extractor. First, the time features and spatial features of each of the positive and negative samples are input into the feature extractor. The time features are metadata features, and the spatial features are the concatenated encrypted payload byte features and burst transmission features. Secondly, the feature extractor uses the embedding layer to process the input time features to obtain time embedding, and the time coding unit processes the input time embedding to obtain time coding; the space coding unit processes the input space features to obtain space coding. Then, the similarity between the time coding and space coding of the positive sample and the similarity between the time coding and space coding of the negative sample are calculated. Among them, the similarity calculation can adopt the cosine similarity calculation method or other similarity calculation methods, which is not limited to the present invention. With Figure 2For example, if n different encrypted session flows are obtained, n positive samples and n*(n-1) negative samples can be constructed. T1, T2, T3, ..., Tn are respectively used as time code 1 obtained by processing the time characteristics of the first encrypted session flow, time code 2 obtained by processing the time characteristics of the second encrypted session flow, time code 3 obtained by processing the time characteristics of the third encrypted session flow, ..., time code n obtained by processing the time characteristics of the nth encrypted session flow, and R1, R2, R3, ..., Rn are respectively used as space code 1 obtained by processing the space characteristics of the first encrypted session flow, space code 2 obtained by processing the space characteristics of the second encrypted session flow, and space code 3 obtained by processing the space characteristics of the third encrypted session flow. , ..., the spatial code 4 obtained by processing the spatial features of the nth encrypted session traffic, the similarities corresponding to the n positive samples and the n*(n-1) negative samples are calculated based on these time codes and space codes, the rose red part of the diagonal is the similarity corresponding to the n positive samples, respectively recorded as R1*T1, R2*T2, R3*T3, ..., Rn*Tn, and the remaining part is the similarity corresponding to the n*(n-1) negative samples, respectively recorded as R1*T2, R1*T3, R1*Tn, R2*T1, R2*T3, R2*Tn, R3*T1, R3*T2, R3*Tn, Rn*T1, Rn*T2, Rn*T3, etc. Finally, based on the calculated similarities corresponding to the positive and negative samples, the parameters of the feature extractor are updated with the optimization goal of maximizing the similarity between the time code and space code of the positive samples and minimizing the similarity between the time code and space code of the negative samples.

[0090] According to one embodiment of the present invention, in step a23, it includes: based on the calculated similarities corresponding to all positive samples and negative samples, using the loss function to calculate the pre-trained loss value loss. Schematically, the positive samples can be marked as 1, and the negative samples can be marked as 0, and the pre-trained loss value loss can be calculated according to the similarities corresponding to the positive samples and the negative samples using the cross entropy loss function. The optimization goal is to maximize the similarity between the temporal encoding and spatial encoding of the positive samples and minimize the similarity between the temporal encoding and spatial encoding of the negative samples, and back-propagate based on the pre-trained loss value loss to update the parameters of the feature extractor. Schematically, based on the loss value loss, back-propagation can be performed according to the chain rule to calculate the gradient of the loss with respect to each parameter, and the parameters of the feature extractor can be updated using the gradient descent method.

[0091] According to one embodiment of the present invention, the feature extractor is pre-trained for multiple rounds of iterations, and each round of iterations includes steps a21-a23. In the process of multiple rounds of iterative pre-training, hyperparameters such as learning rate and batch size are continuously fine-tuned to continue pre-training until the feature extractor converges or reaches the maximum number of pre-training rounds to obtain a pre-trained feature extractor. Convergence means that the accuracy of the verification set reaches the target expected value. Among them, when the accuracy of the feature extractor on the verification set reaches the target expected value, the feature extractor converges, and it is considered that the feature extractor has been fully trained and has strong generalization ability at this time, and the feature extractor at this time is saved for fine-tuning of downstream task scenarios. The present invention uses a pre-training method using self-supervised learning technology, and can use massive unlabeled traffic data to learn the intrinsic relationship between the temporal features and spatial features of the sample through comparative learning to learn a wider distribution of traffic feature data, achieve a robust and generalized understanding of network traffic patterns, and migrate the CLIP network applied to the image field to the network traffic field. The obtained pre-trained feature extractor is called EC-CLIP, which is more suitable for encrypted network traffic data scenarios.

[0092] 3. Model structure and its training process

[0093] After further research and analysis, the inventors found that when the training process is divided into the pre-training stage and the training of the downstream traffic classification task, there is a difference (also called GAP) between the downstream task and the pre-training task, which leads to a small scope of application of the downstream prediction task and reduced prediction accuracy. For example, the ET-BERT in the above-mentioned reference

[11] has such a problem. In response to this problem, the present invention provides a training method for an encrypted network traffic classification model based on contrastive learning, which uses Prompt-Tuning (the core idea of ​​Prompt-Tuning is to convert the target of the downstream task into the form of a pre-training task, and guide the model to generate output that is more in line with expectations by introducing soft prompts) to train the downstream traffic classification task and fine-tune some parameters of the model. That is, the training of the downstream traffic classification task of the present invention introduces soft prompts to construct time features under several categories corresponding to the traffic for the time features and spatial features of each encrypted session traffic, and determines the sample category by the similarity between the time features under several categories and the spatial features of the traffic, which can reduce the difference (GAP) in the pre-training stage of the above-mentioned embodiment. In addition, the present invention adjusts the parameters of the same pre-trained feature extractor for different downstream tasks, and only needs to fine-tune a few parameters of the model, thereby enhancing its applicability to a variety of downstream traffic classification tasks.

[0094] According to one embodiment of the present invention, the encrypted network traffic classification model includes a lightweight network module, a classifier and the feature extractor of the above embodiment. The lightweight network module includes a multi-layer fully connected neural network.

[0095] According to one embodiment of the present invention, see Figure 3 , which is a flow chart of a training method for an encrypted network traffic classification model. The model training method includes the following steps S1, S2 and S3, wherein step S3 includes steps S31, S32, S33, S34 and S35:

[0096] Step S1: Obtain an initial model, which includes a lightweight network module, a classifier, and a pre-trained feature extractor obtained based on the above embodiment method.

[0097] According to one embodiment of the present invention, after obtaining the pre-trained feature extractor obtained in the above embodiment, the output end of the spatial encoding unit of the feature extractor is connected to the input end of the lightweight network module, and the classifier obtains the classification result based on the output prediction of the pre-trained feature extractor.

[0098] Step S2: Obtain a second training set, including multiple original samples corresponding to the multiple categories that the model needs to identify, where the original samples include original data and category labels. The original data includes time features and spatial features extracted from the same encrypted traffic, and the category label indicates the category of the encrypted traffic to which the original data belongs.

[0099] According to one embodiment of the present invention, the second training set is constructed according to the current downstream traffic classification task. That is, the multiple categories that the model needs to identify can be determined according to the downstream traffic classification task, a data set can be constructed according to the construction method of the above data set, and then multiple original samples corresponding to the multiple categories that need to be identified can be constructed based on the data set. Each encrypted traffic is an encrypted session traffic determined when the data set is constructed in the above embodiment. The original data of each original sample refers to the input data of an original sample constructed using the multimodal features of an encrypted session traffic, and the category label of each original sample is the category of the encrypted session traffic to which its original data belongs. Schematically, if the downstream traffic classification task is an application traffic identification task, the constructed second training set includes multiple original samples corresponding to the multiple application traffic categories that the model needs to identify. For example, an original sample is constructed based on the time characteristics and spatial characteristics of an encrypted session traffic corresponding to the Taobao application traffic category.

[0100] Step S3: Use the second training set to train the initial model to obtain a trained model.

[0101] According to one embodiment of the present invention, the training method of the initial model includes multiple rounds of iterative training, during which hyperparameters such as the learning rate and batch size are continuously fine-tuned until the model converges or reaches the maximum number of iterative training rounds, and then the iteration is stopped to obtain a trained model. The model converges when the classification accuracy of the model reaches the target expected value. Each round of iteration is performed according to the following steps S31 to S35:

[0102] In step S31, the spatial features of the input original sample are processed by the encoding module to obtain spatial encoding, and the spatial encoding is processed by the lightweight network module based on the neural network to obtain spatial information.

[0103] According to one embodiment of the present invention, the spatial features are processed by the spatial encoding unit (Spatial Encoder) of the encoding module, and the obtained spatial encoding is recorded as S1. S1 is input into a lightweight network module (Meta-Net) composed of a multi-layer fully connected neural network. The spatial encoding S1 is processed by Meta-Net to obtain spatial information, so that the lightweight network module can further dynamically learn the spatial features. The lightweight network module Meta-Net is connected to the feature extractor EC-CLIP, so the parameters of the lightweight network module of the present invention will be updated with the training of EC-CLIP to realize adaptive learning of the spatial encoding S1.

[0104] In step S32, the embedding layer is used to generate a time prompt for the time feature of the original sample based on the input spatial information according to the introduced soft prompt method, and the time feature of the input original sample is processed based on the time prompt and the input multiple category labels to obtain the time embedding corresponding to each of the multiple categories.

[0105] According to an embodiment of the present invention, in step S32, the method of generating the time prompt includes: obtaining the spatial information corresponding to the original sample of the current round, and the method of obtaining the spatial information includes: inputting the spatial features of the original sample into the encoding module for processing to obtain the spatial code; inputting the spatial code into the lightweight network module of the current round to obtain the spatial information, wherein the first round uses the initialized lightweight network module, and each round after the first round uses the lightweight network module updated in the previous round; obtaining the initial time prompt of the current round, and using the embedding layer of the current round to fuse the input spatial information with the initial time prompt to generate the time prompt, wherein the first round uses the embedding layer of the pre-trained feature extractor, and each round after the first round uses the embedding layer updated in the previous round. The initial time prompt is trainable, wherein when the time prompt is generated in the first round, the randomly initialized initial time prompt is used, and when the time prompt is generated in each round after the first round, the updated initial time prompt of the previous round of each round is used, and the updating method of the initial time prompt of each round includes: updating the initial time prompt with the optimization goal of minimizing the difference between the label of the original sample of the current round and the classification result. The technical solution of this embodiment can at least achieve the following beneficial technical effects: the output spatial information of the lightweight network module is used to fuse with the initial time prompt, to enhance the time prompt, so that the time prompt contains the spatial feature information of the original sample, and the time characteristics of the time prompt are combined to help guide the model to generate output that is more in line with expectations, thereby improving the accuracy of downstream traffic classification tasks.

[0106] According to one embodiment of the present invention, the randomly initialized initial time prompt is a trainable vector or vector sequence, which is used to further enhance the representation ability of the model. The method of fusing the input spatial information with the initial time prompt includes: copying m copies of the spatial information output by the lightweight network module Meta-Net, where m is the number of tokens of the initial time prompt, and adding the m copies of spatial information to each token of the initial time prompt point-to-point to obtain a time prompt, also known as a soft prompt.

[0107] According to an embodiment of the present invention, in step S32, the time embedding corresponding to each of the multiple categories is obtained in the following manner: splicing the attributes of the multiple category labels with the time prompts respectively to obtain multiple time prompts containing different category attributes; duplicating the time features of the original sample according to the number of all categories to obtain multiple time features, and splicing the multiple time features with the multiple time prompts containing different category attributes one-to-one to obtain multiple time features containing different category attributes and time prompts; inputting the multiple time features containing different category attributes and time prompts into the embedding layer for processing to obtain the time embedding corresponding to each of the multiple categories, and when the embedding layer is processing, for each category, inputting a time feature containing the category attribute into the embedding layer to obtain the time embedding corresponding to the category. The present invention uses the time prompt and the category attribute as a template prompt word (Template Prompt) and splices it with the time feature of the sample, and uses this as the input of the feature extractor. The template prompt word refers to the input text or instruction used to guide the model to generate a specific output or perform a specific task. The template prompt words constructed by the present invention are not only used to guide the feature extractor to generate category-related representations, thereby enhancing the model's perception of category information, but also meet the classification requirements of downstream traffic classification tasks.

[0108] According to one embodiment of the present invention, the present invention uses [CLS] to represent the attribute of the category label. For each original sample, the method of obtaining the time embedding corresponding to its multiple categories includes: selecting an attribute of the category label representing its category. Assuming that the downstream task needs to identify n different categories, namely C1, C2, ..., Cn, there is a corresponding [CLS] tag for each category, such as [CLS_C1], [CLS_C2], ..., [CLS_Cn]. The time feature of each original sample is copied n times, and each copy selects the [CLS] tag of the corresponding category and splices it with the time prompt, and then splices it with the time feature and inputs it into the embedding layer for processing to obtain the time embedding corresponding to the category to form the input of the final time coding unit. The following is an example to illustrate the method of obtaining the time embedding corresponding to multiple categories:

[0109] Example 1: Assume that there are two categories, Movie and Music. For a movie category, its time feature is " ”, and the category is “Movie”, you can select [CLS_Movie] as the category attribute and concatenate it with the time feature to form “[CLS_Movie] ”. Next, assuming that the time cue is a vector s, the final time feature input to the embedding layer is “[CLS_Movie] s". For a music category, you can choose [CLS_Music] as the category attribute and concatenate it with the time feature to form "[CLS_Music] [0.3, 0.5, 0.18, 0.9, 0.63]". Similarly, the final time feature input to the embedding layer is "[CLS_Music] s". Replace "[CLS_Movie] s" and "[CLS_Music] s” is input into the embedding layer to obtain the time embedding corresponding to the two categories.

[0110] In step S33, the encoding module is used to process the time embedding corresponding to each of the input multiple categories to obtain the time encoding corresponding to each of the multiple categories.

[0111] According to an embodiment of the present invention, taking the above example 1 as an example, the time corresponding to the two categories is embedded in the time coding unit of the input coding module for processing to obtain the time coding corresponding to the two categories.

[0112] In step S34, a classifier is used to predict the classification results of the original samples according to the calculated similarities between the time codes and space codes corresponding to the various categories, which includes the probability values ​​of the original samples belonging to each category.

[0113] According to one embodiment of the present invention, the classifier includes a SoftMax layer. For each original sample participating in the training, a time code corresponding to each of n categories is obtained (n is the total number of categories). The similarity of the time code of the n categories is calculated with the spatial code S1 of the original sample to obtain the similarity corresponding to each of the n categories. The similarities of the n categories are normalized through the SoftMax layer to obtain n probability values, which represent the probability values ​​of the original sample belonging to each category.

[0114] In step S35, the parameters of the encoding module of the pre-trained feature extractor are frozen, and the parameters of the embedding layer, the lightweight network module and the classifier are updated with the optimization goal of minimizing the difference between the labels of the original samples and the classification results.

[0115] According to one embodiment of the present invention, the cross entropy loss function is used to calculate the loss value of the original sample according to the difference between the predicted classification result and the category label of the original sample. And the loss values ​​of all original samples in the current round are calculated in batches: the loss values ​​of all original samples are accumulated or averaged to obtain the total loss of the current round. According to the total loss of the current round, the chain rule is used to perform back propagation to calculate the gradient of each unfrozen parameter, and the gradient descent method is used to update the parameters of the embedding layer, lightweight network module and classifier.

[0116] In general, according to one embodiment of the present invention, in the training method of the initial model, each round of training process can refer to Figure 4 , which is a schematic diagram of the structure of the initial model and the principle process of its training method. For each original sample, the spatial feature of the original sample is input into the spatial encoding unit of the feature extractor for processing to obtain the spatial encoding R1, and R1 is input into the lightweight network module Meta-Net for processing to obtain the spatial information π. Since the initial time prompt includes m tokens, which are P1, P2, ..., Pm, the spatial information π needs to be copied to generate n spatial information π, and the n spatial information π is added one-to-one with P1, P2, ..., Pm to obtain the time prompt. For n categories, the time feature of each original sample is copied n times, and different category attributes [CLS] are spliced ​​in front of each time feature, and then spliced ​​with the time prompt as the input of the embedding layer to obtain the time embedding corresponding to each of the n categories, and then these time embeddings are input into the time encoding unit for processing to obtain the time codes corresponding to each of the n categories. The time codes corresponding to each of the n categories are recorded as T1, T2, T3, ..., Tn. The similarity between the time encoding and spatial encoding corresponding to each of the n categories is obtained, and the similarities corresponding to each of the n categories are recorded as R1*T1, R1*T2, R1*T3,..., R1*Tn respectively. The classification results of the original samples are predicted according to the similarities corresponding to each of the n categories, and the parameters of the embedding layer, lightweight network module and classifier are updated with the optimization goal of minimizing the difference between the label of the original sample and the classification result.

[0117] The technical solution of performing multiple rounds of iterative training on the model in the above embodiment can at least achieve the following beneficial technical effects: attaching the category attribute of the category label to each time feature to enhance the model's perception of category information. Then splicing with the time prompt ensures that each original sample can generate a corresponding time feature in the preset multiple categories, and input the time coding unit code to obtain the time coding under multiple categories, guide the model to generate more expected output, improve the generalization ability of the model, and use the calculated similarity of time coding and space coding under multiple categories for classification, and combine the time prompt to convert the target of the downstream traffic classification task into a pre-trained form, effectively reducing the inconsistency between the pre-training and the training of the downstream traffic classification task, that is, the difference (CAP), and improving the prediction accuracy.

[0118] IV. Model Application

[0119] According to one embodiment of the present invention, a method for classifying encrypted network traffic is provided, and the classification method includes: obtaining time characteristics and spatial characteristics of encrypted traffic to be classified, and obtaining a trained model according to the method of the above embodiment; using the encoding module of the model to process the spatial characteristics of the encrypted traffic to be classified to obtain spatial coding, and using the lightweight network module of the model to process the spatial coding to obtain spatial information; using the embedding layer of the model to generate time prompts for time characteristics based on the spatial information of the encrypted traffic to be classified according to the introduced soft prompt method, and based on the time prompt and multiple categories required to be identified by the model, the time characteristics of the encrypted traffic to be classified are processed multiple times to obtain time embeddings corresponding to multiple categories; using the encoding module of the model to process the time embeddings corresponding to each of the multiple categories to obtain time coding corresponding to each of the multiple categories; using the classifier of the model, according to the calculated similarity between the time coding and the spatial coding corresponding to each of the multiple categories, the probability value of the encrypted traffic to be classified belonging to each category is obtained, and the category with the maximum probability value is used as the category of the encrypted traffic.

[0120] It should be noted that although the above describes the various steps in a specific order, it does not mean that the various steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order as long as the required functions can be achieved.

[0121] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0122] A computer-readable storage medium may be a tangible device that holds and stores instructions used by an instruction execution device. Computer-readable storage media may include, for example, but are not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof.

[0123] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A pre-training method for an encrypted traffic feature extractor based on contrastive learning, wherein the feature extractor is constructed using a neural network and includes an embedding layer and an encoding module, the method comprising: Obtain a first training set, including positive samples and negative samples, the positive samples including time features and space features extracted from the same encrypted traffic, and the negative samples including time features and space features respectively extracted from two different encrypted traffics; The first training set is used to pre-train the feature extractor through contrastive learning to obtain a pre-trained feature extractor, including: The temporal features of the input positive and negative samples are processed together by the embedding layer and the encoding module to obtain the temporal encoding of the positive and negative samples respectively. The spatial features of the input positive samples and negative samples are processed through the encoding module to obtain the spatial encoding of the positive samples and negative samples respectively; The parameters of the feature extractor are updated with the optimization goal of maximizing the similarity between the temporal and spatial encodings of positive samples and minimizing the similarity between the temporal and spatial encodings of negative samples.

2. The method according to claim 1, characterized in that: The encoding module includes a time encoding unit and a space encoding unit, wherein: The embedding layer is used to process the input time features to obtain time embedding, and the time encoding unit processes the input time embedding to obtain time encoding; The spatial encoding unit is used to process the input spatial features to obtain spatial encoding.

3. The method according to claim 1, characterized in that Each encrypted traffic is encrypted session traffic, which includes a number of data packets generated during the interaction between the two communicating parties. The extraction method of the time feature and the spatial feature includes: Extracting multimodal features of each encrypted session flow according to the data packets of the encrypted session flow, including metadata features, encrypted payload byte features and burst transmission features; The metadata feature of an encrypted session flow is used as the time feature, and the encrypted payload byte feature and burst transmission feature of the encrypted session flow are concatenated to obtain the spatial feature of the flow.

4. A training method for an encrypted network traffic classification model based on contrastive learning, the training method comprising: S1. Obtaining an initial model, which includes a lightweight network module, a classifier, and a pre-trained feature extractor obtained based on the method according to any one of claims 1 to 3, wherein the feature extractor includes an embedding layer and an encoding module; S2. Obtain a second training set, including a plurality of original samples corresponding to the plurality of categories required to be identified by the model, wherein the original samples include original data and category labels, the original data include time features and spatial features extracted from the same encrypted traffic, and the category label indicates the category of the encrypted traffic to which the original data belongs; S3. Using the second training set to train the initial model to obtain a trained model, the training method includes: S31, using the encoding module to process the spatial features of the input original sample to obtain spatial coding, and using the lightweight network module to process the spatial coding based on the neural network to obtain spatial information; S32, using the embedding layer according to the introduced soft prompt method, generating a time prompt for the time feature of the original sample based on the input spatial information, and processing the input time feature of the original sample based on the time prompt and the input multiple category labels to obtain time embeddings corresponding to the multiple categories; S33, using the encoding module to process the time embeddings corresponding to the input multiple categories, to obtain the time encodings corresponding to the multiple categories; S34, using a classifier to predict the classification results of the original sample according to the calculated similarities between the time codes and the space codes corresponding to the multiple categories, including the probability values ​​of the original sample belonging to each category; S35. Freeze the parameters of the encoding module of the pre-trained feature extractor, and update the parameters of the embedding layer, the lightweight network module, and the classifier with the optimization goal of minimizing the difference between the label and the classification result of the original sample.

5. The training method according to claim 4, characterized in that: In S3, the training method includes multiple rounds of iterative training, and each round of iteration is performed according to the process from S31 to S35. In S32, the method of generating the time prompt includes: Obtain the spatial information corresponding to the original sample of the current round. The spatial information is obtained in the following ways: The spatial features of the original samples are input into the encoding module for processing to obtain spatial encoding; Input the spatial code into the lightweight network module of the current round to obtain spatial information, wherein the first round uses the initialized lightweight network module, and each round after the first round uses the lightweight network module updated in the previous round; The initial time prompt of the current round is obtained, and the input spatial information is fused with the initial time prompt using the embedding layer of the current round to generate the time prompt. The first round uses the embedding layer of the pre-trained feature extractor, and each round after the first round uses the updated embedding layer of the previous round.

6. The method according to claim 5, characterized in that The initial time prompt is trainable, wherein, when generating the time prompt in the first round, a randomly initialized initial time prompt is used, and when generating the time prompt in each round after the first round, the updated initial time prompt in the previous round is used, and the updating method of each round of the initial time prompt includes: The initial time prompt is updated with the optimization goal of minimizing the difference between the label and classification result of the original sample of the round.

7. The method according to claim 4, characterized in that In S32, the time embedding corresponding to each of the multiple categories is obtained in the following manner: The attributes of multiple category labels are concatenated with the time prompts to obtain multiple time prompts containing different category attributes; The time features of the original samples are copied according to the number of all categories to obtain multiple time features, and the multiple time features are spliced ​​one-to-one with multiple time prompts containing different category attributes to obtain multiple time features containing different category attributes and time prompts; Multiple time features containing different category attributes and time prompts are input into the embedding layer for processing to obtain the time embeddings corresponding to the multiple categories. When the embedding layer is processing, for each category, a time feature containing the category attributes is input into the embedding layer to obtain the time embedding corresponding to the category.

8. A method for classifying encrypted network traffic, characterized in that: Classification methods include: Obtaining time characteristics and spatial characteristics of the encrypted traffic to be classified, and obtaining a trained model according to a method according to any one of claims 4-7; The encoding module of the model is used to process the spatial characteristics of the encrypted traffic to be classified to obtain spatial coding, and the lightweight network module of the model is used to process the spatial coding to obtain spatial information; The embedding layer of the model is used to generate time hints for time features based on the spatial information of the encrypted traffic to be classified according to the introduced soft hint method. The time features of the encrypted traffic to be classified are processed multiple times based on the time hints and multiple categories that the model needs to identify, and the time embeddings corresponding to the multiple categories are obtained. The encoding module of the model is used to process the time embedding corresponding to each of the multiple categories to obtain the time encoding corresponding to each of the multiple categories; Using the classifier of the model, the probability value of the encrypted traffic to be classified belonging to each category is obtained according to the similarity between the time code and the space code corresponding to the multiple categories, and the category with the maximum probability value is taken as the category of the encrypted traffic.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of the method according to any one of claims 1 to 8.

10. An electronic device, characterized in that: include: one or more processors; as well as A memory, wherein the memory is used to store executable instructions; The one or more processors are configured to implement the steps of the method of any one of claims 1-8 by executing the executable instructions.

Images