A zero-day traffic classification method based on statistical information and payload encoding

Abstract

The invention discloses a zero-day traffic classification method based on statistical information and payload coding, comprising: constructing a hybrid neural network, and pre-training the hybrid neural network with arbitrary unlabeled data, labeled known category traffic and deep metric learning strategy; performing unbiased preprocessing on traffic to be identified and labeled known category traffic to form a statistical information quadruple and a payload; and obtaining formatted samples A and B through standardization processing; obtaining embedding vectors A and B according to the hybrid neural network; clustering the embedding vectors A and B using a k-means clustering algorithm to identify zero-day application traffic; performing unbiased preprocessing and standardization processing on manually labeled zero-day traffic to obtain a formatted sample C, and obtaining an embedding vector C through a hybrid neural network; adding the embedding vectors B and C to a support set, and then classifying the support set and the embedding vector A through a k-NN algorithm; the invention has high classification accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of network traffic classification, and in particular to a zero-day traffic classification method based on statistical information and payload coding. Background Art

[0002] In recent years, traffic classification has become important for network security, service quality, network monitoring and management. These applications cannot work well without understanding network traffic, so traffic classification technology, that is, correctly classifying network traffic into one or more categories according to different task requirements, is actually a prerequisite for these applications.

[0003] Due to the importance of traffic classification, it has been an important research topic since the early days of the Internet. Existing traffic classification technologies can be roughly divided into traditional traffic classification methods and machine learning-based traffic classification methods, but there are still some problems that have not been well solved in both traffic classification problems, including: 1. Input data may be heterogeneous and biased; 2. Multi-classification / multi-label problems; 3. Discovery and identification of zero-day applications. Summary of the invention

[0004] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0005] In view of the above existing problems, the present invention is proposed.

[0006] Therefore, the present invention provides a zero-day traffic classification method based on statistical information and payload encoding, which can solve the problem that the input data is heterogeneous and biased, solve the multi-classification / multi-label problem, and solve the problem of discovering and identifying zero-day applications.

[0007] To solve the above technical problems, the present invention provides the following technical solutions: including, constructing a hybrid neural network, and pre-training the hybrid neural network with any unlabeled data, labeled known category traffic and deep metric learning strategy; performing unbiased preprocessing on the traffic to be identified and the labeled known category traffic participating in the hybrid neural network pre-training to form a statistical information quadruple and a payload part in the data packet; and obtaining formatted samples A and formatted samples B through standardization processing; inputting the formatted samples A and formatted samples B into the hybrid neural network to obtain embedding vectors A and B; clustering the embedding vectors A and B using the k-Means clustering algorithm to identify zero-day application traffic; performing unbiased preprocessing and standardization processing on the manually labeled zero-day traffic to obtain formatted samples C, and inputting the formatted samples C into the hybrid neural network to obtain embedding vectors C; adding the embedding vectors B and C to the support set of the k-NN algorithm, and then classifying the support set and the embedding vector A through the k-NN algorithm.

[0008] As a preferred solution of the zero-day traffic classification method based on statistical information and payload encoding described in the present invention, wherein: the hybrid neural network includes an autoencoder layer, a pre-training head, and a Transformers layer; the autoencoder layer includes: a statistical information encoder-decoder structure composed of a fully connected layer, a convolutional neural network layer, a deconvolutional network layer, a maximum pooling layer, and a payload encoder-decoder structure composed of a ReLU activation layer; the pre-training head and the autoencoder layer share a statistical information decoder and a payload decoder, and have a fully connected network layer with an input dimension of 126 and an output dimension of 2. The pre-training head removes the two dimensions corresponding to the word cap from the output of the Transformers layer and restores it to 126 dimensions. According to the word type at the input of the Transformers layer, the word is input into the statistical information decoder and the payload decoder respectively to be restored to a statistical information quadruple and the original payload to obtain the result of flow reconstruction, and the first word of the matrix is ​​input into a fully connected layer with an input dimension of 126 and an output dimension of 2 to obtain the result of flow integrity prediction.

[0009] As a preferred solution of the zero-day traffic classification method based on statistical information and payload encoding described in the present invention, the Transformer layer splices the outputs of the two encoders of the self-encoder layer into a complete flow matrix in the order of the original traffic; then inserts a 126-dimensional word unit with all zeros at the beginning of the flow matrix, and adds a 2-dimensional word unit cap to each word unit to distinguish the type of the word unit; adds position encoding information to the flow matrix, and converts the position encoding pe pos Appended to each word re in the flow matrix pos On, change it to epos ;

[0010] Among them, in the flow matrix, the position encoding pe of the posth word unit pos Contains 128 dimensions, and its value can be obtained as follows:

[0011]

[0012]

[0013] The pos for:

[0014] e pos =re pos +pe pos

[0015] Among them, 2i,2i+1∈[1,128], i is the i-th position of the vector; pe (pos,2i) For pe pos The 2ith element; is the wavelength of the i∈[1,63]th group; pe (pos,2i+1) For pe pos The 2i+1th element; e pos The posth word element of the front flow matrix before adding the position code; pe pos is the posth word element re of the flow matrix pos Corresponding position code; re pos Add the position encoding to the posth word in the flow matrix.

[0016] As a preferred solution of the zero-day traffic classification method based on statistical information and payload encoding described in the present invention, wherein: pre-training the hybrid neural network with the arbitrary unlabeled data is a pre-training stage; the pre-training stage includes: pre-training the autoencoder layer; and jointly pre-training the autoencoder layer, the Transformers layer and the pre-training head.

[0017] As a preferred solution of the zero-day traffic classification method based on statistical information and payload encoding described in the present invention, the unbiased preprocessing includes collecting a number of data packets from the traffic to be identified and the traffic with labeled known categories participating in the hybrid neural network pre-training; for each data packet, the protocol layer of the header is discarded, and the payload length, TCP window size, arrival time, and direction are retained to form the statistical information quadruple.

[0018] As a preferred solution of the zero-day traffic classification method based on statistical information and payload encoding described in the present invention, the standardization process includes transforming the value h in all statistical information quadruplets into log(1+h), treating each byte p of the payload as a value between [0,255], and transforming it into The value range is [0,1].

[0019] As a preferred solution of the zero-day traffic classification method based on statistical information and payload encoding described in the present invention, wherein: the embedded vector sample A, the embedded vector sample B, and the embedded vector sample C are a 128-dimensional embedded vector.

[0020] As a preferred solution of the zero-day traffic classification method based on statistical information and payload encoding described in the present invention, wherein: the embedding vector A and the embedding vector B are input into the k-Means clustering algorithm to obtain the output cluster, and the cluster must meet two conditions: the number of embedded vectors in the cluster is greater than a threshold, and the threshold is 0 or 1; the cluster does not contain any embedded vector corresponding to any traffic with a known label, so as to identify the zero-day application traffic.

[0021] As a preferred solution of the zero-day traffic classification method based on statistical information and payload encoding described in the present invention, 5% of the traffic in the zero-day application traffic is manually marked to obtain the manually marked zero-day traffic.

[0022] Beneficial effects of the present invention: The present invention retains the information of heterogeneous inputs, effective statistical information, and the intrinsic connection and timing information of the payload, and is 3.5% more accurate than the classification task of the prior art. At the same time, only 5% of the zero-day traffic needs to be manually marked, and the zero-day traffic can be classified with an accuracy of approximately 90%. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0024] Figure 1 A schematic diagram of the structure of a hybrid neural network and a two-stage pre-training process of a zero-day traffic classification method based on statistical information and payload encoding according to the first embodiment of the present invention;

[0025] Figure 2A schematic diagram of correcting feature offset and reducing dimension of a payload encoder of a zero-day traffic classification method based on statistical information and payload encoding according to a first embodiment of the present invention;

[0026] Figure 3 A schematic diagram of the principle and structure of a multi-head attention mechanism for a zero-day traffic classification method based on statistical information and payload encoding according to the first embodiment of the present invention;

[0027] Figure 4 A schematic diagram of the connection mode of the hybrid neural network and the machine learning algorithm of the zero-day traffic classification method based on statistical information and payload encoding described in the first embodiment of the present invention;

[0028] Figure 5 A conceptual diagram of Triplet Margin Loss of deep metric learning of a zero-day traffic classification method based on statistical information and payload encoding according to the first embodiment of the present invention;

[0029] Figure 6 The basic concept of extending deep metric learning to a multi-task multi-label environment for the zero-day traffic classification method based on statistical information and payload encoding described in the first embodiment of the present invention;

[0030] Figure 7 A schematic diagram of the necessary conditions for deep metric learning of the zero-day traffic classification method based on statistical information and payload encoding described in the first embodiment of the present invention to identify unknown classes from training on known classes. DETAILED DESCRIPTION

[0031] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0032] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0033] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0034] The present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0035] At the same time, in the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper, lower, inner and outer" are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0036] In the present invention, unless otherwise clearly specified and limited, the terms "install, connect, connect" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0037] Example 1

[0038] Reference Figure 1 to Figure 7 , which is the first embodiment of the present invention, provides a zero-day traffic classification method based on statistical information and payload encoding, including:

[0039] S1: Build a hybrid neural network and pre-train it with arbitrary unlabeled data, labeled traffic of known categories, and a deep metric learning strategy.

[0040] (1) The hybrid neural network includes an autoencoder layer, a pre-training head, and a Transformers layer;

[0041] Specifically, ① the autoencoder layer includes the following structures: 1. A statistical information (HDR) encoder-decoder structure composed of a fully connected layer, which encodes the input statistical information quadruple into multiple 126-dimensional tokens and restores the 126-dimensional tokens to statistical information quadruple; 2. A payload (PAY) encoder-decoder structure composed of a convolutional neural network layer (CNN), a deconvolutional network layer (Transposed Convolution Neural Network), a maximum pooling layer (MaxPooling) and a ReLU activation layer, which encodes the input payload information into multiple 126-dimensional tokens and restores the 126-dimensional tokens to payload information; combined with Figure 2 , the payload encoder can correct feature offsets and reduce dimensions to a certain extent; compared to the practice of directly inputting bytes in the original traffic into the Transformer layer, the payload encoder composed of a convolutional neural network can encode the most significant local features in the payload into 126-dimensional word vectors, which gives the word vectors a clear meaning; the maximum pooling layer is used in the payload encoder, which allows the payload encoder to handle feature offsets to a certain extent, such as Figure 2 As shown in the figure, the length of the final generated payload word sequence is shortened.

[0042] ② Pre-training head: The pre-training head only participates in the pre-training process. The pre-training head and the autoencoder layer share two decoders and have a fully connected network layer with an input dimension of 126 and an output dimension of 2. The pre-training head removes the two dimensions corresponding to the word cap from the output of the Transformers layer and restores it to 126 dimensions. Then, according to the word type at the input of the Transformers layer, the word is input into the statistics decoder and the payload decoder respectively to restore it to the statistics quadruple and the original payload to obtain the result of flow reconstruction, and the first word of the matrix (the word corresponding to #CLS#) is input into a fully connected network layer with an input dimension of 126 and an output dimension of 2 to obtain the result of flow integrity prediction.

[0043] ③Transformers layer: The Transformers layer first concatenates the outputs of the two encoders of the autoencoder layer into a complete flow matrix in the order of the original flow. The dimension of the matrix is ​​126. Then, a 126-dimensional special word (#CLS#) with all zeros is inserted at the beginning of the matrix. Further, for each word, a 2-dimensional "token hat" is added to it to distinguish their types. #CLS# is represented by (0, 0), the word from the statistics encoder is represented by (1, 0), and the word from the payload encoder is represented by (0, 1). At this time, the dimension of the flow matrix is ​​128 (126+2). Furthermore, position encoding information is added to the flow matrix. For the posth word in the flow matrix W, its position encoding pe pos Contains 128 dimensions, and its value can be obtained as follows:

[0044]

[0045]

[0046] Among them, 2i,2i+1∈[1,128], so each dimension has a different wavelength or phase difference, and each position encoding has a unique texture, which allows the model to understand the position of the word vector from the texture; then as shown in the following formula, the position encoding pe pos Appended to each word re in the flow matrix pos On, change it to e pos :

[0047] e pos =re pos +pe pos

[0048] Among them, e pos The posth word element of the front flow matrix before adding the position encoding, pe pos is the posth word element re of the flow matrix pos Corresponding position code, re pos Add the position encoding to the posth word in the flow matrix.

[0049] Next, the flow matrix is ​​fed into a network structure consisting of multiple (6) stacked Transformers blocks, whose output is also 128-dimensional and has exactly the same size as the input matrix; combined with Figure 3 , illustrating the structure of the Transformers block:

[0050] The input and output of each Transformer block are of the same size and are the output of the previous Transformer block, so more complex relationships and more abstract expressions can be learned. This layer has a total of 6 Transformer blocks, where the output of the nth layer and the input of the n+1th layer are both Where P is the number of words (i.e., sequence length); Figure 1 As shown, each Transformer block contains three main modules: multi-head attention mechanism, add&norm and feed forward layer; Figure 3 The overall structure of the multi-head attention mechanism is shown. Each multi-head attention mechanism contains k = 8 parallel self-attention modules; the input x is divided into k groups [x1, x2, …, x i , …, x k ] and then input them into k self-attention modules respectively; for the i-th self-attention module, input x i Use the parameter W respectively i Q , W i K , W i V Linear transformation to Q i , K i , V i To generate the input of the self-attention mechanism, the purpose of the self-attention mechanism is to find the input x through the similarity function i The intrinsic connection between the words in the text is used to obtain a weight containing the contextual relationship; the self-attention mechanism Attention(Q,K,V) is as follows:

[0051]

[0052] Among them, d k Input x to the self-attention module i Dimensions, Q i , K i , V i are three 128-dimensional vectors, and softmax() is a normalized exponential function. Since the input x dimension of the multi-head attention mechanism is 128, x i has dimension 16 (128 / 8); so the output head of the i-th self-attention block is i can be expressed as:

[0053] head i =Attention(Q i W i Q , K i W iK , V i W i V )

[0054] The output of the multi-head attention mechanism MultiHead(x) is the combined output of each self-attention block using the parameter W O Doing linear transformation, the process can be expressed as:

[0055] MultiHead(x)=Concat(head i ,…,head k )W O

[0056] Among them, W O The multi-head attention mechanism is followed by two layers of add&norm and a Feed Forward network layer. Feed Forward is a network consisting of two fully connected layers with a ReLU activation function between them. Its function is as follows:

[0057] FFN(x)=max(0,xW1+b1)W2+b2

[0058] Among them, W1, b1, W2, b2 are the parameters of two fully connected layers, the size of W1 is 128×384, and the size of W2 is 384×128; the Add&Norm layer is actually a residual connection layer that can be expressed as y=F(x)+x, where F(·) represents a certain operation of multi-head attention mechanism and Feed Forward. Because the model is relatively deep, the residual connection layer can effectively solve the problem of deep network degradation, thereby improving the performance of the model.

[0059] (2) Pre-training the hybrid neural network with arbitrary unlabeled data is the pre-training stage; the training stage includes: pre-training the autoencoder layer; and jointly pre-training the autoencoder layer, the Transformers layer, and the pre-training head.

[0060] Furthermore, the specific steps for pre-training a hybrid neural network using arbitrary unlabeled data are:

[0061] ① In the pre-training stage, the input is unlabeled formatted samples. The source of the samples is not limited, but the more samples there are and the wider the source, the stronger the generalization of the model obtained by pre-training; learn the intrinsic connection between different word units.

[0062] ② Pre-trained autoencoder layer: For the statistical information (HDR) encoder-decoder structure composed of fully connected layers, we input the statistical information quadruple into the encoder, each statistical information quadruple is encoded into a word with a dimension of 126, then the word is input into the decoder, and the encoder and decoder are trained according to the statistical information reconstructed by the decoder and the mean square error (MSE) loss of the input statistical information; similarly, for the payload (PAY) encoder-decoder structure composed of a convolutional neural network layer (CNN), a deconvolution network layer (Transposed Convolution Neural Network), a maximum pooling layer (Max Pooling) and a ReLU activation layer, random noise is added to the payload of each data packet (3% of the values ​​are set to 0, and Gaussian noise in the range of [-0.1, 0.1] is added to each value) and then input into the encoder. The output of the encoder is a sequence of word units with a dimension of 126; all the word units are then input into the decoder, and the encoder and decoder are trained according to the mean square error (MSE) loss of the payload reconstructed by the decoder and the payload without noise added.

[0063] ③ The outputs of the statistics (HDR) encoder and payload (PAY) encoder obtained by pre-training the autoencoder layer may have different distributions, so we need to continue training the autoencoder layer in joint pre-training.

[0064] ④ Jointly pre-train the autoencoder layer, Transformers layer and pre-training head of the hybrid network: The statistics and payload of each packet in the stream are input into the statistics (HDR) encoder and payload (PAY) encoder of the autoencoder layer respectively; for the output of the two encoders, one of the statistics word units and payload word units is replaced with a word unit from other samples with probability p=0.5 to destroy the stream integrity; for each word unit, its value is replaced with a random value with probability 0.04, and its value is replaced with 0 with probability 0.16; then these values ​​are sent to the Transformers layer, and the output of the Transformers layer is sent to the pre-training head; among them, the two decoders shared with the autoencoder layer in the pre-training head reconstruct the payload and statistics of the input autoencoder layer from the output of the Transformers layer, and the stream reconstruction loss is calculated using MSE; a fully connected network layer with an input dimension of 126 and an output dimension of 2 in the pre-training head is responsible for predicting whether the stream integrity is destroyed, and the prediction loss is calculated using cross entropy (CE).

[0065] (3) Pre-training a hybrid neural network using traffic and deep metric learning strategies with known labeled categories.

[0066] ①Construct several triples using labeled samples. Each triple includes an anchor x, a positive sample x belonging to the same class as x + , and a negative sample x of a different class from x - ; Input the three samples in a triple into the hybrid neural network. For each sample, the output of the neural network is a 128-dimensional embedding vector; Use d(·,·) to represent the distance function (which can be the Euclidean distance, i.e., geometric distance, or algorithms such as cosine similarity). The distance d1 between the same-class samples x, x + in the embedding space is d1 = d(f(x), f(x + )) and should be less than the distance d2 between the different-class samples x, x - in the embedding space, which is d2 = d(f(x), f(x - )); To further separate the inter-class distance d2 and the intra-class distance d1, a margin m is introduced, that is, it is required to satisfy the condition d1 + m < d2; Then the loss can be calculated by the following formula:

[0067] L(x, x + , x - ; f) = max(0, d1 - d2 + m)

[0068] where L() is the loss function and f is the mapping function. Since this algorithm is based on triplets, the complexity of the sample space is O(n 3 ), and it is not feasible to enumerate all triplets during the training process. And in most cases, the constructed triplets satisfy the condition d1 + m < d2, so the gradient value is 0; In response to this situation, hard negative mining is required to select the triplets with the most information. The algorithm selected here is Multi SimilarityMiner, and its selection rule is:

[0069] 1. If the distance between a negative sample and the anchor in the embedding space is smaller than the distance between the most difficult positive sample and the anchor in the embedding space minus the margin m, then this negative sample will be selected. This condition can be formally described as: If the following formula is satisfied, the negative sample pair {x i , x j} will be selected;

[0070]

[0071] where, y k , y i are the labels of x k , x i respectively, is the negative sample pair {xi ,x j}Use the mapping function f to map to the distance d(f(x i ),f(x j )),d ik For the sample pair {x i ,x k}Use the mapping function f to map the distance in the embedding space.

[0072] 2. If the distance between a positive sample and the anchor point in the embedding space is smaller than the distance between the most difficult negative sample and the anchor point in the embedding space plus the interval m, then this positive sample will be selected. This condition can be formally described as: If the following equation is satisfied, the positive sample pair {x i ,x j} will be selected.

[0073]

[0074] in For a positive sample pair {x i ,x j}Use the mapping function f to map to the distance d(f(x i ),f(x j )).

[0075] ② In order to further extend the present invention to multi-target (also called multi-label, multi-task) data (i.e., a sample may have three labels at the same time: for example: VPN encryption, VoIP, Skype), for each task, a separate embedding space is used to calculate the loss function and weighted sum (total loss); there are two mappings from the embedding vector output by the hybrid neural network to the embedding space corresponding to each task, namely:

[0076] 1. Without any transformation, the embedding vector is directly used as the embedding space corresponding to each task. Since the loss in each embedding space is calculated separately, there are different intervals m. At this time, the label conflicts of different tasks are resolved by different intervals m in each embedding space.

[0077] 2. Use an MLP consisting of 2 fully connected layers to achieve remapping.

[0078] S2: Perform unbiased preprocessing on the traffic to be identified and the traffic with known labels participating in the hybrid neural network pre-training to form a statistical information quadruple and the payload part of the data packet; and obtain formatted samples A and formatted samples B through standardization.

[0079] (1) Unbiased preprocessing: Collect several packets from the traffic to be identified and the traffic with known labels that participate in the hybrid neural network pre-training; for each packet, discard the protocol layer of the header and retain the four-tuple of statistical information consisting of payload length, TCP window size, arrival time, and direction.

[0080] (2) Normalization: transform the value h in all statistical information quadruplets into log(1+h), treat each byte p of the payload as a value between [0, 255], and transform it into The value range is [0,1].

[0081] S3: Input the formatted sample A and the formatted sample B into the hybrid neural network to obtain a 128-dimensional embedding vector A and a 128-dimensional embedding vector B.

[0082] S4: Use the k-Means clustering algorithm to cluster the embedding vector A and the embedding vector B to identify zero-day application traffic.

[0083] The embedding vectors A and B are input into the k-Means clustering algorithm to obtain the output clusters. The clusters must meet two conditions: the number of embedding vectors in the cluster is greater than the threshold, and the threshold is 0 or 1; the cluster does not contain any embedding vectors corresponding to traffic with known labels to identify zero-day application traffic.

[0084] The specific steps of clustering are:

[0085] 1. Select k samples of embedding vector A and embedding vector B as cluster centers a=[a1,a2,…a k ];

[0086] 2. For each sample x in the data set i , calculate its distance to k cluster centers and divide it into the class corresponding to the smallest cluster center;

[0087] 3. For each category a j , recalculate its cluster center

[0088] 4. Repeat steps 2 and 3 until the cluster center changes less than 1e-4 or the number of repetitions exceeds 300 steps.

[0089] S5: Perform unbiased preprocessing and standardization on the manually labeled zero-day traffic to obtain a formatted sample C, and input the formatted sample C into the hybrid neural network to obtain an embedded vector C.

[0090] The traffic in any 5% of the zero-day application traffic is manually marked to obtain the manually marked zero-day traffic.

[0091] S6: Add the embedding vector B and the embedding vector C to the support set of the k-NN algorithm, and then classify the support set and the embedding vector A through the k-NN algorithm.

[0092] This step can be formally described as: the hybrid neural network is regarded as a mapping function f(·) from input samples to embedding vectors, and the labeled known category samples participating in model training are called dataset T train , the samples to be classified are called data set T test The manually annotated zero-day application samples are called dataset T novel , then the three data sets first satisfy the following formula:

[0093]

[0094] in Is an empty set.

[0095] The support set E is obtained by support :

[0096] E support =f(T train ∪T novel )

[0097] Similarly, the embedding vector E corresponding to the sample to be identified test =f(T test ), then, for the i-th embedding vector e i ∈E test , when the following equation holds, the label y of the corresponding sample i =y j :

[0098]

[0099] Where d() is, e i is the i-th embedding vector, e j is the jth embedding vector, e k is the kth embedding vector.

[0100] Example 2

[0101] In order to verify and illustrate the technical effects adopted in this method, this embodiment selects and performs comparative tests with this method, and compares the test results by means of scientific demonstration to verify the real effects of this method.

[0102] In order to verify and illustrate the technical effects used in this method, this embodiment selects DISTILLER, 1D-CNN, 2-CNN, MLP, HYBRID and adopts this method for comparative testing, and compares the test results by means of scientific demonstration to verify the real effect of this method.

[0103] In order to verify that this method has higher task classification accuracy than DISTILLER, 1D-CNN, 2-CNN, MLP, and HYBRID, in this embodiment, traditional DISTILLER, 1D-CNN, 2-CNN, MLP, HYBRID and this method will be used to perform real-time classification comparison on multi-task traffic.

[0104] We compared the classification effects of different algorithms on a simulation platform. All simulations were implemented on a test platform with python 3.8, Intel i9 CPU, and 2080ti graphics card using the Pytorch 1.8 deep learning framework and the Scikit-Learn library. The classification dataset used was the ISCX VPN 2016 dataset, which was published by the Canadian Cybersecurity Agency. Among them, the data is provided in the form of PCAP files, each PCAP file includes multiple flows, and each PCAP is assigned a three-task label (package category, service category, application category, see Table 1 for details); since each PCAP file in this dataset contains some traffic that is known not to belong to the label, we use the Scapy library to scan all traffic and filter out traffic with known categories and irrelevant to the label, such as traffic with a source IP or destination IP of 255.255.255.255 (local network broadcast), traffic belonging to the Domain Name Resolution Protocol (DNS), traffic belonging to the Link Local Multicast Name Resolution (LLMNR), and traffic belonging to the Network Time Protocol (NTP). Then, unbiased preprocessing is performed according to the method described in the patent to obtain formatted samples. Further, 80% of the samples are divided into the training set and 20% of the samples are divided into the test set; in order to verify the effectiveness of this method, we selected the methods used in the most relevant literature from 2017 to 2021 for comparison, including:

[0105] 1. 1D-CNN (Wang et al.): 1D-CNN proposed by Wang et al. in 2017. This model has a single-modal input and the input only contains the payload.

[0106] 2. HYBRID (Lopez-Martin et al.): A 2D-CNN+LSTM hybrid model proposed by Lopez-Martin et al. in 2017. In the original implementation, the input of this method was statistical information containing biased information. According to the method of the related literature "DISTILLER: Encrypted traffic classification via multimodal multitask deeplearning", the model structure and input were modified to remove the unbiased input.

[0107] 3. 2-CNN (Huang et al.): The model structure proposed by Huang et al. in 2018, with the input being the payload.

[0108] 4.MLP(Sun et al.), MLP(Zhao et al.): Algorithms proposed by Sun et al. and Zhao et al. in 2019 respectively. Both algorithms can implement versions with input as statistics and payload, but due to model structure limitations, neither method can implement simultaneous input of payload and statistics.

[0109] 6. 1D-CNN (Rezaei and Liu): A classification architecture proposed by Rezaei and Liu in 2020. Similar to HYBRID (Lopez-Martin et al.), this method takes as input statistical information containing biased information and modifies the model structure and input according to the method in the related paper DISTILLER: Encrypted traffic classification via multimodal multitask deeplearning to remove unbiased input.

[0110] 7.DISTILLER (Aceto et al.): A multi-modal input multi-task encrypted traffic classification algorithm proposed by Aceto et al. in 2021.

[0111] All methods except this method were trained for 60 rounds. This method first used the unlabeled data of the ISCX Tor2016 dataset after removing the labels for pre-training. After the pre-training, it used the labeled data of the ISCX VPN2016 dataset for 60 rounds of training.

[0112] Since the inputs of the algorithms are not exactly the same, (Both) is used here to indicate that the model has multimodal input (inputting both payload and statistical information at the same time), (HDR) is used to indicate that the model input is statistical information, and (PAY) is used to indicate that the model input is payload.

[0113] The most relevant method here is DISTILLER (Aceto et al.), because it has almost the same input and output as this method. All other methods are also reproduced and compared. However, it is slightly different from this method in that its filtering steps are slightly more relaxed (for example, traffic such as Network Time Protocol NTP is not filtered), which leads to different reproduction results here. In order to make a fair comparison, the reproduction results and the original results will be given together, where the original results are marked with (Ori) and the reproduction results are marked with (Rep).

[0114] The complete results are shown in Table 2.

[0115] Table 1: Description of tasks and categories of the ISCX VPN 2016 dataset

[0116]

[0117] Table 2: Performance comparison of different methods on the ISCX VPN 2016 dataset

[0118]

[0119] The best algorithm for each indicator of each task is marked in bold, and the best algorithm other than this method is marked in italics and boxes, regardless of whether it is a reproducible result. It can be found that this method exceeds the related methods in most indicators of the three tasks, especially in Task2-Traffic Type Classification, with an Accuracy gain of 3.50% and an F1-score gain of 4.09%; if only the reproducible results (Rep) under the same experimental conditions are considered, then this method exceeds the existing algorithms in all indicators of all tasks.

[0120] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A zero-day traffic classification method based on statistical information and payload encoding, characterized by: include, Constructing a hybrid neural network and pre-training the hybrid neural network with arbitrary unlabeled data, labeled known class traffic, and a deep metric learning strategy; The hybrid neural network includes an autoencoder layer, a pre-training head and a Transformers layer; The autoencoder layer includes: a statistical information encoder-decoder structure composed of a fully connected layer, a convolutional neural network layer, a deconvolutional network layer, a maximum pooling layer and a payload encoder-decoder structure composed of a ReLU activation layer; The pre-training head and the autoencoder layer share a statistics decoder and a payload decoder, and have a fully connected network layer with an input dimension of 126 and an output dimension of 2. The pre-training head removes the word cap from the word unit output by the Transformers layer and restores it to 126 dimensions; according to the word unit type at the input of the Transformers layer, the word unit is input into the statistics decoder and the payload decoder respectively to be restored to a statistics quadruple and the original payload to obtain the result of flow reconstruction, and the first word unit of the matrix is ​​input into a fully connected network layer with an input dimension of 126 and an output dimension of 2 to obtain the result of flow integrity prediction; The Transformer layer concatenates the outputs of the two encoders of the autoencoder layer into a complete flow matrix in the order of the original flow; then inserts a 126-dimensional word with all zeros at the beginning of the flow matrix, and adds a 2-dimensional word cap to each word to distinguish the type of the word; Add position encoding information to the convection matrix and convert the position encoding pe pos Appended to each word re in the flow matrix pos On, change it to e pos ; Among them, in the flow matrix, the position encoding pe of the posth word unit is pos Contains 128 dimensions, and its value can be obtained as follows: The pos for: have been pos =re pos + or pos Among them, 2i,2i+1∈[1,128], i is the i-th position of the vector; pe (pos,2i) For pe pos The 2ith element; is the wavelength of the i∈[1,63]th group; pe (pos,2i+1) For pe pos The 2i+1th element; e pos The posth word element of the front flow matrix before adding the position code; pe pos is the posth word element re of the flow matrix pos Corresponding position code; re pos Add the position encoding to the posth word in the flow matrix; The traffic to be identified and the traffic with known labels participating in the pre-training of the hybrid neural network are subjected to unbiased pre-processing to form a statistical information quadruple and the payload part of the data packet; and formatted samples A and B are obtained through standardization processing; Inputting the formatted sample A and the formatted sample B into the hybrid neural network to obtain an embedding vector A and an embedding vector B; Clustering the embedding vector A and the embedding vector B using a k-means clustering algorithm to identify zero-day application traffic; Perform unbiased preprocessing and standardization on the manually labeled zero-day malicious traffic to obtain a formatted sample C, and input the formatted sample C into the hybrid neural network to obtain an embedded vector C; The embedding vector B and the embedding vector C are added to the support set of the k-NN algorithm, and then the support set and the embedding vector A are classified by the k-NN algorithm.

2. The zero-day traffic classification method based on statistical information and payload encoding as claimed in claim 1, characterized in that: Also includes, Pre-training the hybrid neural network with the arbitrary unlabeled data is a pre-training stage; The pre-training stage includes: pre-training the autoencoder layer; Jointly pre-train the autoencoder layers, Transformers layers, and pre-training heads.

3. The zero-day traffic classification method based on statistical information and payload encoding as claimed in claim 2, characterized in that: The unbiased preprocessing includes: Collecting a number of data packets from the traffic to be identified and the traffic with known labels that participate in the hybrid neural network pre-training; For each data packet, the protocol layer of the header is discarded, and the payload length, TCP window size, arrival time, and direction are retained to form the statistical information quadruple; the payload length is the transport layer payload minus the zero padding at the end.

4. The zero-day traffic classification method based on statistical information and payload encoding as claimed in claim 3, characterized in that: The standardization process includes: Transform the values ​​h in all statistical information quadruplets to log(1+h), treat each byte p of the payload as a value between [0,255], and transform it to The value range is [0,1].

5. The zero-day traffic classification method based on statistical information and payload encoding as claimed in claim 4, characterized in that: The embedding vector sample A, the embedding vector sample B, and the embedding vector sample C are a 128-dimensional embedding vector.

6. The zero-day traffic classification method based on statistical information and payload encoding as claimed in claim 5, characterized in that: Also includes, The embedding vector A and the embedding vector B are input into the k-Means clustering algorithm to obtain an output cluster, and the cluster must meet two conditions: the number of embedded vectors in the cluster is greater than a threshold, and the threshold is 0 or 1; the cluster does not contain any embedded vectors corresponding to traffic with known labels to identify the zero-day application traffic.

7. The zero-day traffic classification method based on statistical information and payload coding according to any one of claims 2 to 6, characterized in that: 5% of the malicious traffic in the zero-day application traffic is manually marked to obtain manually marked zero-day malicious traffic.

Images