Encrypted network traffic analysis method and device based on packet length sequence, and electronic equipment

Abstract

The invention discloses a network traffic analysis method, and particularly relates to an encrypted network traffic analysis method and device based on a packet length sequence, and electronic equipment. The method comprises the steps that a to-be-trained PS-LM model is pre-trained based on a first training data set, the PS-LM model is obtained, the PS-LM model comprises a word segmentation embedding module and a PS-LM module, the word segmentation embedding module is used for obtaining embedding of a lexical element sequence corresponding to network flow, the PS-LM module is used for processing embedding of the lexical element sequence, and a second training data set is obtained; obtaining an output vector of each lexical element in the lexical element sequence; supervised fine tuning is carried out on the network traffic analysis model based on the second training data set to obtain a trained network traffic analysis model, and the network traffic analysis model comprises a PS-LM model and a classification head; and analyzing the to-be-processed network flow based on the trained network flow analysis model to obtain an analysis result. The method provided by the invention can realize a better network flow analysis effect.

Description

Technical Field

[0001] The present invention relates to a network traffic analysis method, and in particular to an encrypted network traffic analysis method, device and electronic equipment based on packet length sequence. Background Art

[0002] Application traffic identification technology was originally based on port numbers, but dynamic port numbers and port misuse made this method less and less accurate. Later, deep packet inspection (DPI) achieved traffic identification tasks by identifying signature words in data packets. With the emergence of encrypted traffic, the applicability of deep packet inspection methods based on plaintext information has been greatly reduced. Currently, application traffic identification methods based on machine learning and deep learning are mainstream.

[0003] The packet length sequence-based method mainly relies on the behavior information of encrypted traffic to achieve analysis. The effectiveness of this type of method is not limited by the amount of plaintext information retained in the encryption protocol. Therefore, in the field of encrypted traffic analysis, the analysis of network behavior based on packet length sequence is a more common method.

[0004] In the field of encrypted traffic analysis, there are usually different downstream tasks. Although the existing methods can achieve acceptable accuracy, it is difficult to make adaptive adjustments and focus on optimization for different downstream tasks, and there are problems such as weak robustness and insufficient generalization. Summary of the invention

[0005] The technical problem to be solved by the present invention is that the above-mentioned methods commonly used in the prior art are difficult to adapt and optimize for different downstream tasks, and have problems of weak robustness and insufficient generalization. In order to solve the above-mentioned problems, the present invention provides an encrypted network traffic analysis method, device and electronic device based on packet length sequence.

[0006] The content of the present invention includes: In a first aspect, an embodiment of the present invention provides an encrypted network traffic analysis method based on a packet length sequence, comprising: Pre-training a PS-LM model to be trained based on the first training data set to obtain a PS-LM model, wherein the PS-LM model includes a word segmentation embedding module and a PS-LM module connected in sequence, wherein the word segmentation embedding module is used to process the network flow to obtain the embedding of a word unit sequence corresponding to the network flow, and the PS-LM module is used to process the embedding of the word unit sequence to obtain an output vector of each word unit in the word unit sequence; Performing supervised fine-tuning on the network traffic analysis model based on the second training data set to obtain a trained network traffic analysis model, wherein the network traffic analysis model includes the PS-LM model and a classification head, and the classification head is used to perform classification processing corresponding to the downstream task; The network flow to be processed is analyzed based on the trained network traffic analysis model to obtain an analysis result.

[0007] Optionally, the analyzing the network flow to be processed based on the trained network traffic analysis model to obtain an analysis result includes: Obtaining the embedding of each word in the word-unit sequence corresponding to the network flow to be processed based on the word segmentation embedding module, wherein a special mark is added at the beginning of the word-unit sequence; Input the embedding of each word in the word-gram sequence into the PS-LM module for processing to obtain an output vector of each word in the word-gram sequence, wherein the specially marked output vector is used to represent the word-gram sequence; The output vector of the special mark is input into the classification head to perform classification processing corresponding to the downstream task to obtain the analysis result.

[0008] Optionally, the word segmentation embedding module includes a word segmenter and an embedding layer, and the embedding of each word in the word-unit sequence corresponding to the to-be-processed network flow is obtained based on the word segmentation embedding module, and a special mark is added at the beginning of the word-unit sequence, including: Inputting the network flow to be processed into the word segmenter for word segmentation processing, and adding the special identifier at the beginning of the sequence to obtain the word element sequence based on the packet length value; The word-gram sequence is encoded using the embedding layer to obtain an embedding of each word-gram in the word-gram sequence, wherein the embedding of the word-gram includes position information of the word-gram.

[0009] Optionally, the second training data set includes a first-stage data set and a second-stage data set, and the supervised fine-tuning of the network traffic analysis model based on the second training data set to obtain a trained network traffic analysis model includes: Connecting the classification head after the PS-LM module of the PS-LM model to obtain the network traffic analysis model; Freeze the network parameters of the PS-LM model in the network traffic analysis model, and perform gradient propagation on the classification head based on the first-stage data set to obtain a first-stage training model; Unfreeze the network parameters of the PS-LM model in the first-stage training model, fine-tune all parameters of the first-stage training model based on the second-stage data set, and obtain the trained network traffic analysis model.

[0010] Optionally, the analyzing the network flow to be processed based on the trained network traffic analysis model to obtain an analysis result includes: The sample robustness is calculated based on the probability that the perturbed sample falls into different category areas; Calculate the model robustness based on the distribution of recognition results of the trained network traffic analysis model on the test set under noise disturbance; When the sample robustness and the model robustness meet preset conditions, the network flow to be processed is analyzed based on the trained network traffic analysis model to obtain an analysis result.

[0011] Optionally, calculating sample robustness based on the probability that the perturbed sample falls into different category regions includes: The probability that the perturbed sample falls into the first category area is determined as the first probability, and the probability that the perturbed sample falls into the second category area is determined as the second probability, the first category area is the category area with the largest confidence probability, and the second category area is the category area with the second largest confidence probability; Calculating a lower bound of the first probability and an upper bound of the second probability using a binomial proportion confidence interval; The sample robustness is determined based on a difference between a lower bound of the first probability and an upper bound of the second probability.

[0012] Optionally, the model robustness for: ; in, Determined based on the PA curve: ; in, is the indicator function, For the The labels of samples, For the The labels of samples are determined by the first probability, and , is the number of samples, is the coordinate value in the PA curve, is a logical OR, For the The lower bound of the first probability corresponding to samples, For the The upper bound of the second probability corresponding to samples.

[0013] In a second aspect, an embodiment of the present invention provides an encrypted network traffic analysis model based on packet length sequence, including: A pre-training module, used for pre-training a PS-LM model to be trained based on a first training data set to obtain a PS-LM model, wherein the PS-LM model comprises a word segmentation embedding module and a PS-LM module connected in sequence, wherein the word segmentation embedding module is used for processing a network flow to obtain an embedding of a word unit sequence corresponding to the network flow, and the PS-LM module is used for processing the embedding of the word unit sequence to obtain an output vector of each word unit in the word unit sequence; A fine-tuning module, used to perform supervised fine-tuning on the network traffic analysis model based on the second training data set to obtain a trained network traffic analysis model, wherein the network traffic analysis model includes the PS-LM model and a classification head, and the classification head is used to perform classification processing corresponding to the downstream task; The analysis module is used to analyze the network flow to be processed based on the trained network traffic analysis model to obtain analysis results.

[0014] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a program stored in the memory and executable on the processor; the processor is used to read the program in the memory to implement the steps in the encrypted network traffic analysis method based on packet length sequence as described in the first aspect.

[0015] In a fourth aspect, an embodiment of the present invention provides a readable storage medium for storing a program, which, when executed by a processor, implements the steps in the encrypted network traffic analysis method based on packet length sequence as described in the first aspect.

[0016] In an embodiment of the present application, by pre-training the PS-LM model on massive unlabeled data, the PS-model module can automatically learn the relevant knowledge of the network traffic data. After adding the classification heads corresponding to different downstream tasks to the PS-LM model, the network is trained by fine-tuning to obtain a trained network traffic analysis model for implementing specific analysis tasks. Compared with the existing methods, the method provided in the embodiment of the present invention can not only achieve higher accuracy, but also has stronger robustness under common network flow disturbances. At the same time, the method provided in the embodiment of the present invention can train different classification heads for different downstream tasks in the fine-tuning stage, thereby focusing on optimizing different downstream tasks and obtaining better analysis results. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Attached Figure 1 A flow chart of an encrypted network traffic analysis method based on packet length sequence provided by an embodiment of the present invention; Attached Figure 2a An encrypted traffic analysis framework based on a pre-trained PS-LM model provided in an embodiment of the present invention; Attached Figure 2b For attachment Figure 2a A magnified diagram of the word embedding part; Attached Figure 2c For attachment Figure 2a A magnified schematic diagram of the PS-LM module part; Attached Figure 2d For attachment Figure 2a Schematic diagram of the method for fine-tuning the model; Attached Figure 3 A schematic diagram related to sample robustness provided by an embodiment of the present invention; Attached Figure 4 A schematic diagram related to the robustness of the model provided by an embodiment of the present invention; Attached Figure 5 A schematic diagram of an encrypted network traffic analysis device based on packet length sequence provided by an embodiment of the present invention; Attached Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In the embodiments of the present application, the term "and / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. In the embodiments of the present application, the term "multiple" refers to two or more, and other quantifiers are similar. The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first" and "second" are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0019] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0021] The embodiments of the present application provide a method, device and electronic device for analyzing encrypted network traffic based on packet length sequence, aiming to improve the robustness and generalization of the network traffic analysis model.

[0022] See also Figure 1 , Figure 1 The present invention provides a flow chart of an encrypted network traffic analysis method based on packet length sequence, which specifically includes the following steps: Step 101, pre-training the PS-LM model to be trained based on the first training data set to obtain a PS-LM model, wherein the PS-LM model includes a word segmentation embedding module and a PS-LM module connected in sequence, wherein the word segmentation embedding module is used to process the network flow to obtain the embedding of the word unit sequence corresponding to the network flow, and the PS-LM module is used to process the embedding of the word unit sequence to obtain the output vector of each word unit in the word unit sequence.

[0023] Step 102, supervised fine-tuning the network traffic analysis model based on the second training data set to obtain a trained network traffic analysis model, wherein the network traffic analysis model includes the PS-LM model and a classification head, and the classification head is used to perform classification processing corresponding to downstream tasks.

[0024] Step 103: Analyze the network flow to be processed based on the trained network traffic analysis model to obtain an analysis result.

[0025] See also Figure 2a The embodiment of the present invention also provides an encrypted traffic analysis framework based on a pre-trained PS-LM model, which specifically includes: Figure 2b The word segmentation embedding part shown in Figure 2c The PS-LM module shown in the figure is Figure 2d The model shown is fine-tuned in 3 parts.

[0026] In step 101, the PS-LM model to be trained is first pre-trained. The PS-LM model includes a word segmentation embedding module and a PS-LM module connected in sequence. The PS-LM module is used to process a token sequence to obtain an output vector of each token in the token sequence. A token can also be called a tag. In natural language processing (NLP), a token usually refers to the smallest unit after word segmentation, such as a word, a subword or even a character, and a token sequence is a sequence formed by arranging these tags in a certain order.

[0027] The PS-LM module is used to process word-meta sequences to obtain the output vector of each word-meta in the word-meta sequence, and its specific structure is not limited here. In encrypted traffic analysis tasks, the focus is on understanding data semantic information, and there is less demand for generation tasks. In models based on the Transformer structure, the Encoder-Only type model represented by BERT adopts a complete multi-head attention mechanism, in which each token can pay attention to both the previous and following context information, and is therefore more suitable for understanding tasks. Therefore, the PS-LM module in this embodiment uses Encoder-Only as the basic model architecture. As a specific embodiment, the PS-LM module is stacked by the Encoder structure of the Transformer.

[0028] As an optional implementation, the denoising autoencoder (DAE) method is used for pre-training the PS-LM model to be trained. The basic principle is to mask the words with predictions, and then predict the original value of the masked words based on other non-masked words provided by the context.

[0029] During training, 15% of the tokens in the input sequence need to be masked. Specifically, the masking operation for the selected 15% of tokens also includes a third part: 80% of them are directly replaced with [MASK], 10% are directly replaced with new tokens, and the remaining 10% remain unchanged. The masked language model (MLM) uses [MASK] as noise and can obtain bidirectional semantic context information through self-encoding training. Since the encrypted traffic analysis task does not require tasks such as Question Answering (QA) and Natural Language Inference (NLI), there is no need to add the NSL task (a variant of Next Sentence Prediction (NSP)) during pre-training.

[0030] In this embodiment, the first training data set contains a large amount of training data, so that the pre-training process of the PS-LM model is carried out on the packet length sequence of massive network traffic, and its purpose is to allow the PS-LM model to learn the underlying structure and common patterns of network traffic in a self-supervisory manner, thereby obtaining a universal and meaningful representation of network traffic. The pre-trained PS-LM model cannot solve the problem end-to-end. For specific encrypted traffic analysis tasks, other network layers need to be added after the PS-LM module.

[0031] Optionally, in some embodiments, the second training data set includes a first-stage data set and a second-stage data set, and step 102 includes: Connecting the classification head after the PS-LM module of the PS-LM model to obtain the network traffic analysis model; Freeze the network parameters of the PS-LM model in the network traffic analysis model, and perform gradient propagation on the classification head based on the first-stage data set to obtain a first-stage training model; Unfreeze the network parameters of the PS-LM model in the first-stage training model, fine-tune all parameters of the first-stage training model based on the second-stage data set, and obtain the trained network traffic analysis model.

[0032] like Figure 2d As shown, in this embodiment, a classification header is connected after the PS-LM module, and different downstream tasks have different corresponding classification headers. Optionally, the downstream tasks include encrypted malicious traffic detection, encrypted proxy traffic classification, tunnel website access identification, or application traffic identification.

[0033] Connecting different classification heads to different downstream tasks can make the trained network traffic analysis model more suitable for the downstream task. For downstream classification tasks, the classification head is usually a single-layer multilayer perceptron (MLP) plus a Softmax layer.

[0034] The PS-LM model has been pre-trained on a large-scale second training data set, and its parameters have usually converged and reached a relatively stable state. If all model parameters are adjusted significantly directly on small-scale data, overfitting is likely to occur. In this embodiment, the fine-tuning process is divided into the following two stages: Phase 1: First, freeze the network parameters of the PS-LM model in the network traffic analysis model, and only perform gradient propagation on the classification head based on the first phase data set, so that the parameters of the classification head have preliminary effects and obtain the first phase training model.

[0035] The second stage: Unfreeze the network parameters of the PS-LM model in the first stage training model, and fine-tune all parameters at a lower learning rate based on the second stage data set, so that the model has better results and a trained network traffic analysis model is obtained.

[0036] During fine-tuning, the loss function is: ; in, Used to represent the true label, It is used to express the predicted probability. Used to represent a collection of tags. Used to indicate the number of samples.

[0037] After obtaining the trained network traffic analysis model, the trained network traffic analysis model can be used to perform network flow analysis. In the specific implementation, different network traffic analysis models are trained for different downstream tasks. The trained network traffic analysis models corresponding to different downstream tasks have the same structure, but different model parameters, so that the model can achieve better results on the downstream task.

[0038] Optionally, in some embodiments, step 103 includes: Obtaining the embedding of each word in the word-unit sequence corresponding to the network flow to be processed based on the word segmentation embedding module, wherein a special mark is added at the beginning of the word-unit sequence; Input the embedding of each word in the word-gram sequence into the PS-LM module for processing to obtain an output vector of each word in the word-gram sequence, wherein the specially marked output vector is used to represent the word-gram sequence; The output vector of the special mark is input into the classification head to perform classification processing corresponding to the downstream task to obtain the analysis result.

[0039] See also Figure 2a and Figure 2b . Optionally, in some embodiments, the word segmentation embedding module specifically includes a tokenizer (Tokenizer) and an embedding layer (Embedding) for word segmentation. The word segmentation embedding module is based on obtaining the embedding of each word in the word-unit sequence corresponding to the network flow to be processed, and a special identifier is added at the beginning of the word-unit sequence, including: inputting the network flow to be processed into the tokenizer for word segmentation, and adding the special identifier at the beginning of the sequence to obtain the word-unit sequence based on the packet length value; using the embedding layer to encode the word-unit sequence to obtain the embedding of each word in the word-unit sequence, and the embedding of the word-unit includes the position information of the word-unit.

[0040] It should be understood that the absolute value of the packet length is the payload length of the data packet transport layer, and the sign of the packet length indicates the direction of the data packet. In this embodiment, the upstream packet length is specified to be positive and the downstream packet length is specified to be negative, and the packet length is directly used as a token, thereby realizing the segmentation of the network flow. In the specific implementation, a special mark ([CLS]) is added before the packet length sequence segmentation result, that is, the token sequence (token sequence), to indicate the beginning of the token sequence. The output vector of the specially marked token input is used to characterize the entire token sequence. It should be understood that in order to mark the end of the token sequence, [SEP] is added at the end position of the token sequence.

[0041] In some embodiments, since the PS-LM module requires fixed-length input, a special token, namely [PAD], is used to fill in the word sequence of insufficient length. These filling values ​​will be masked by [MASK] in the subsequent model so that they do not affect the representation results. The specifics are not limited here.

[0042] The purpose of embedding is to map each token to Euclidean space, so as to represent its semantic information in the form of a fixed-length vector for easy machine processing. In this embodiment, the representation of the packet length value Token is achieved directly by adding an embedding layer. In network traffic analysis, the data packet position information also contains information that is helpful for analysis. Data packets with the same length have different effects on network flows at different positions. Therefore, in some embodiments, a position encoding is set for each token in the token sequence in order to add position information to the representation. Specifically, position encoding (PositionEmbedding) is also implemented through an embedding layer with learnable parameters, which is not specifically limited here.

[0043] Specifically, the embedding of each token in the word-unit sequence is input into the PS-LM module of the trained network traffic analysis model for processing to obtain the output vector of each token. The output vector of [CLS] is used to represent the entire sequence. Therefore, the output vector of [CLS] is input into the classification head of the trained network traffic analysis model for classification processing corresponding to the downstream task to obtain the analysis result.

[0044] There are many network noises in the real network, such as packet loss, packet retransmission and packet disorder. These network noises will have a certain impact on the packet length sequence. Therefore, when the network traffic analysis model generates the detection results, it is also necessary to analyze the noise intensity it can resist, that is, robustness. The essence of robustness is whether the model can stably output correct results when the sample is subject to certain disturbances.

[0045] Optionally, in some embodiments, step 103 includes: The sample robustness is calculated based on the probability that the perturbed sample falls into different category areas; Calculate the model robustness based on the distribution of recognition results of the trained network traffic analysis model on the test set under noise disturbance; When the sample robustness and the model robustness meet preset conditions, the network flow to be processed is analyzed based on the trained network traffic analysis model to obtain an analysis result.

[0046] The network traffic analysis model actually divides the sample space into different regions, each corresponding to a category. For a trained network traffic analysis model, the spatial structure of its class region is usually complex. Some samples are far from the boundary of the class region to which they belong, while some samples are far from the boundary of the class region to which they belong. Mathematically speaking, robustness is the size of the sample neighborhood that can stabilize the output. By calculating the distribution of different classes in the sample neighborhood, that is, the probability of the perturbed sample falling into different class regions, the robustness can be measured.

[0047] Optionally, in some embodiments, the calculating sample robustness based on the probability that the perturbed sample falls into different category areas includes: The probability that the perturbed sample falls into the first category area is determined as the first probability, and the probability that the perturbed sample falls into the second category area is determined as the second probability, the first category area is the category area with the largest confidence probability, and the second category area is the category area with the second largest confidence probability; Calculating a lower bound of the first probability and an upper bound of the second probability using a binomial proportion confidence interval; The sample robustness is determined based on a difference between a lower bound of the first probability and an upper bound of the second probability.

[0048] The network traffic analysis model determines that the sample is a category The probability of Among them, the category with the highest probability (i.e. the first category area) is set as , and the category with the second largest confidence probability is (i.e., the second category area), the corresponding confidence probability of the first category area (i.e., the first probability) is , the confidence probability (i.e., the second probability) corresponding to the second category area is .if only , it means that the perturbation in the neighborhood can output the correct result, and and The larger the phase difference, the more stable the output. To measure the robustness of the sample.

[0049] See also Figure 3 In some embodiments, a Monte Carlo method is used to estimate and In order to ensure the reliability of the calculation results, the binomial proportion confidence interval is used to calculate The lower bound of The upper bound of sample robustness It can be used to measure the robustness of the sample under the specified noise. When the preset conditions are met, the sample robustness is considered to meet the requirements.

[0050] As a specific embodiment, under specified noise, the sample robustness is calculated as follows: Set the count vector ; For a certain sample, add the specified distribution noise to the sample , forming a sample containing noise And input the model to get the recognition result ,Right now in When the sample size is If the times, and then for all Perform statistics and obtain and .in, and are the indices of the two largest values ​​in the count vector counts, where The count value of for , The count value of for . Calculated according to the binomial proportion confidence interval algorithm Lower bound on class probability and The upper bound of the category , and finally return , specifically, ; ; For confidence.

[0051] Since different samples have different robustness, the measurement of model robustness should be based on the distribution of sample robustness. Since it is meaningless to consider the robustness of samples that are predicted incorrectly by the model, in this embodiment, the measurement of model robustness also takes into account the accuracy.

[0052] The PA curve can show the distribution of the model's recognition results on the test set under certain noise disturbances. The PA curve equation is as follows: ; in, is an indicator function, which is 1 if the input variable is true, and 0 otherwise. For the The labels of samples, For the The label of the sample is determined by the first probability, that is, category A.

[0053] like Figure 4 As shown in , the PA curve can simultaneously show the accuracy and robustness of the model: the curve is biased to the right, indicating strong robustness; the curve is biased upward, indicating high accuracy. Specifically, the PA curve is used to characterize the concentration of test samples in can be correctly predicted under the conditions The proportion of , the vertical axis is the prediction accuracy (Accuracy(y)).

[0054] Furthermore, in this embodiment, the model robustness PA-aera is proposed to quantitatively analyze the robustness of the model. PA-aera is used to characterize the area enclosed by the PA curve and the coordinate axis, and can be calculated as follows: ; in, is a point on the PA curve, and .

[0055] In this embodiment, when the model robustness PA-aera meets the preset conditions, the model robustness is considered to meet the requirements. Through the above method, the quantitative index of robustness under the influence of network noise is considered, and the robustness is quantitatively analyzed from different angles by measuring the sample robustness and the model robustness, so as to better evaluate the performance of the trained network traffic analysis model. Through the above method, when the performance of the trained network traffic analysis model meets the preset conditions, the trained network traffic analysis model is used to perform network flow analysis, which can further improve the accuracy of the analysis results.

[0056] In an embodiment of the present application, by pre-training the PS-LM model on massive unlabeled data, the PS-LM model can automatically learn the relevant knowledge of network traffic data. After adding the classification heads corresponding to different downstream tasks after the PS-LM model, the network is trained by fine-tuning to obtain a trained network traffic analysis model for implementing specific analysis tasks. Compared with the existing methods, the method provided in the embodiment of the present invention can not only achieve higher accuracy, but also has stronger robustness under common network flow disturbances. In addition, the method provided in the embodiment of the present invention requires less labeled data to achieve the same accuracy, which proves its better generalization.

[0057] See also Figure 5 The embodiment of the present invention further provides an encrypted network traffic analysis model 500 based on packet length sequence, comprising: A pre-training module 501 is used to pre-train the PS-LM model to be trained based on the first training data set to obtain a PS-LM model, wherein the PS-LM model includes a word segmentation embedding module and a PS-LM module connected in sequence, wherein the word segmentation embedding module is used to process the network flow to obtain the embedding of the word unit sequence corresponding to the network flow, and the PS-LM module is used to process the embedding of the word unit sequence to obtain the output vector of each word unit in the word unit sequence; A fine-tuning module 502 is used to perform supervised fine-tuning on the network traffic analysis model based on the second training data set to obtain a trained network traffic analysis model, wherein the network traffic analysis model includes the PS-LM model and a classification head, and the classification head is used to perform classification processing corresponding to the downstream task; The analysis module 503 is used to analyze the network flow to be processed based on the trained network traffic analysis model to obtain analysis results.

[0058] Optionally, the analysis module 503 includes: A word segmentation embedding module, used to obtain the embedding of each word in the word-unit sequence corresponding to the network flow to be processed, and a special mark is added at the beginning of the word-unit sequence; A PS-LM module is used to process the embedding of each word in the word-unit sequence to obtain an output vector of each word in the word-unit sequence, and the specially marked output vector is used to represent the word-unit sequence; The classification head is used to perform classification processing corresponding to the downstream task on the output vector input of the special mark to obtain the analysis result.

[0059] Optionally, the word segmentation embedding module: A word segmenter, used for performing word segmentation processing on the network flow input to be processed, and adding the special mark at the beginning of the sequence to obtain the word element sequence based on the packet length value; The embedding layer is used to encode the word-gram sequence to obtain the embedding of each word-gram in the word-gram sequence, wherein the embedding of the word-gram includes the position information of the word-gram.

[0060] Optionally, the second training data set includes a first-stage data set and a second-stage data set, and the fine-tuning module 502 includes: A construction unit, configured to connect the classification header after the PS-LM module of the PS-LM model to obtain the network traffic analysis model; A first training unit, used for freezing the network parameters of the PS-LM model in the network traffic analysis model, and performing gradient propagation on the classification head based on the first stage data set to obtain a first stage training model; The second training unit is used to unfreeze the network parameters of the PS-LM model in the first-stage training model, and fine-tune all parameters of the first-stage training model based on the second-stage data set to obtain the trained network traffic analysis model.

[0061] Optionally, the analysis module 503 is specifically used for: The sample robustness is calculated based on the probability that the perturbed sample falls into different category areas; Calculate the model robustness based on the distribution of recognition results of the trained network traffic analysis model on the test set under noise disturbance; When the sample robustness and the model robustness meet preset conditions, the network flow to be processed is analyzed based on the trained network traffic analysis model to obtain an analysis result.

[0062] Optionally, calculating sample robustness based on the probability that the perturbed sample falls into different category regions includes: The probability that the perturbed sample falls into the first category area is determined as the first probability, and the probability that the perturbed sample falls into the second category area is determined as the second probability, the first category area is the category area with the largest confidence probability, and the second category area is the category area with the second largest confidence probability; Calculating a lower bound of the first probability and an upper bound of the second probability using a binomial proportion confidence interval; The sample robustness is determined based on a difference between a lower bound of the first probability and an upper bound of the second probability.

[0063] Optionally, the model robustness for: ; in, Determined based on the PA curve: ; in, is the indicator function, For the The labels of samples, For the The labels of samples are determined by the first probability, and , is the number of samples, is the coordinate value in the PA curve, is a logical OR, For the The lower bound of the first probability corresponding to samples, For the The upper bound of the second probability corresponding to samples.

[0064] The encrypted network traffic analysis model 500 based on packet length sequence provided in the embodiment of the present application can execute the above method embodiment, and its implementation principle and technical effects are similar, which will not be repeated in this embodiment.

[0065] It should be noted that the division of units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0066] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0067] like Figure 6 As shown, an embodiment of the present application provides an electronic device 600, comprising: a memory 602, a processor 601, and a program stored in the memory 602 and executable on the processor 601; the processor 601 is used to read the program in the memory 602 to implement the steps in the encrypted network traffic analysis based on packet length sequence as described above.

[0068] An embodiment of the present application also provides a readable storage medium, on which a program is stored. When the program is executed by a processor, the various processes of the above-mentioned encrypted network traffic analysis embodiment based on packet length sequence are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here. Among them, the readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disk, hard disk, magnetic tape, magneto-optical disk (MO), etc.), optical storage (such as compact disk (CD), digital video disc (DVD), Blu-ray Disc (BD), high-definition versatile disc (HVD), etc.), and semiconductor storage (such as read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read only memory (EEPROM), non-volatile memory (NAND FLASH), solid-state drive (Solid State Disk or Solid State Drive, SSD)), etc.

[0069] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0070] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes a number of instructions for a terminal (which can be a mobile phone, computer, server, air conditioner, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0071] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for analyzing encrypted network traffic based on packet length sequence, characterized in that: include: Pre-training a PS-LM model to be trained based on the first training data set to obtain a PS-LM model, wherein the PS-LM model includes a word segmentation embedding module and a PS-LM module connected in sequence, wherein the word segmentation embedding module is used to process the network flow to obtain the embedding of a word unit sequence corresponding to the network flow, and the PS-LM module is used to process the embedding of the word unit sequence to obtain an output vector of each word unit in the word unit sequence; Performing supervised fine-tuning on the network traffic analysis model based on the second training data set to obtain a trained network traffic analysis model, wherein the network traffic analysis model includes the PS-LM model and a classification head, and the classification head is used to perform classification processing corresponding to the downstream task; The network flow to be processed is analyzed based on the trained network traffic analysis model to obtain an analysis result.

2. The method according to claim 1, characterized in that: The analyzing the network flow to be processed based on the trained network traffic analysis model to obtain the analysis result includes: Obtaining the embedding of each word in the word-unit sequence corresponding to the network flow to be processed based on the word segmentation embedding module, wherein a special mark is added at the beginning of the word-unit sequence; Input the embedding of each word in the word-gram sequence into the PS-LM module for processing to obtain an output vector of each word in the word-gram sequence, wherein the specially marked output vector is used to represent the word-gram sequence; The output vector of the special mark is input into the classification head to perform classification processing corresponding to the downstream task to obtain the analysis result.

3. The method according to claim 2, characterized in that: The word segmentation embedding module includes a word segmenter and an embedding layer. The embedding of each word in the word-unit sequence corresponding to the network flow to be processed is obtained based on the word segmentation embedding module, and a special mark is added at the beginning of the word-unit sequence, including: Inputting the network flow to be processed into the word segmenter for word segmentation processing, and adding the special identifier at the beginning of the sequence to obtain the word element sequence based on the packet length value; The word-gram sequence is encoded using the embedding layer to obtain an embedding of each word-gram in the word-gram sequence, wherein the embedding of the word-gram includes position information of the word-gram.

4. The method according to claim 1, characterized in that: The second training data set includes a first-stage data set and a second-stage data set, and the network traffic analysis model is fine-tuned in a supervised manner based on the second training data set to obtain a trained network traffic analysis model, including: Connecting the classification head after the PS-LM module of the PS-LM model to obtain the network traffic analysis model; Freeze the network parameters of the PS-LM model in the network traffic analysis model, and perform gradient propagation on the classification head based on the first-stage data set to obtain a first-stage training model; Unfreeze the network parameters of the PS-LM model in the first-stage training model, fine-tune all parameters of the first-stage training model based on the second-stage data set, and obtain the trained network traffic analysis model.

5. The method according to claim 1, characterized in that: The analyzing the network flow to be processed based on the trained network traffic analysis model to obtain the analysis result includes: The sample robustness is calculated based on the probability that the perturbed sample falls into different category areas; Calculate the model robustness based on the distribution of recognition results of the trained network traffic analysis model on the test set under noise disturbance; When the sample robustness and the model robustness meet preset conditions, the network flow to be processed is analyzed based on the trained network traffic analysis model to obtain an analysis result.

6. The method according to claim 5, characterized in that: The sample robustness is calculated based on the probability that the perturbed sample falls into different category areas, including: The probability that the perturbed sample falls into the first category area is determined as the first probability, and the probability that the perturbed sample falls into the second category area is determined as the second probability, the first category area is the category area with the largest confidence probability, and the second category area is the category area with the second largest confidence probability; Calculating a lower bound of the first probability and an upper bound of the second probability using a binomial proportion confidence interval; The sample robustness is determined based on a difference between a lower bound of the first probability and an upper bound of the second probability.

7. The method according to claim 6, characterized in that: The model robustness for: ； in, Determined based on the PA curve: ； in, is the indicator function, For the The labels of samples, For the The labels of samples are determined by the first probability, and , is the number of samples, is the coordinate value in the PA curve, is a logical OR, For the The lower bound of the first probability corresponding to samples, For the The upper bound of the second probability corresponding to samples.

8. An encrypted network traffic analysis model based on packet length sequence, characterized in that: include: A pre-training module, used for pre-training a PS-LM model to be trained based on a first training data set to obtain a PS-LM model, wherein the PS-LM model comprises a word segmentation embedding module and a PS-LM module connected in sequence, wherein the word segmentation embedding module is used for processing a network flow to obtain an embedding of a word unit sequence corresponding to the network flow, and the PS-LM module is used for processing the embedding of the word unit sequence to obtain an output vector of each word unit in the word unit sequence; A fine-tuning module, used to perform supervised fine-tuning on the network traffic analysis model based on the second training data set to obtain a trained network traffic analysis model, wherein the network traffic analysis model includes the PS-LM model and a classification head, and the classification head is used to perform classification processing corresponding to the downstream task; The analysis module is used to analyze the network flow to be processed based on the trained network traffic analysis model to obtain analysis results.

9. An electronic device, comprising: A memory, a processor, and a program stored in the memory and executable on the processor; wherein the processor is used to read the program in the memory to implement the steps in the encrypted network traffic analysis method based on packet length sequence as described in any one of claims 1 to 7.

10. A readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps in the encrypted network traffic analysis method based on packet length sequence as described in any one of claims 1 to 7 are implemented.

Images