A Classification Method for Encrypted Network Traffic Based on Large Language Model

Abstract

The present invention proposes a method for classifying encrypted network traffic based on a large language model, and belongs to the field of network security technology. The present invention proposes an open source large language model based on BERT, and uses the network traffic channel feature text data (packet length) at the bidirectional flow level as pre-training data to train a base model that can use open domain unlabeled traffic data to learn representations with strong generalization capabilities, and in downstream tasks, a small number of labeled specific classification data are used to complete rapid transfer learning, thereby enhancing the adaptability of the model. Compared with using data packet payloads that do not have readable meaning as training samples, the present invention uses data packet length sequences as channel features, which can learn the behavior patterns of encrypted network traffic; compared with using limited features proposed by experts, the present invention can perfectly characterize the behavior patterns of encrypted network traffic through data packet length.

Description

Technical Field

[0001] The invention relates to an encrypted network traffic classification method based on a large language model, and belongs to the technical field of network security. Background Art

[0002] Network traffic classification refers to the process of classifying and identifying the categories of network traffic through traffic data transmitted through computer networks. Its main purpose is to help network managers monitor, manage and protect the network. Through network traffic classification, network managers can better understand network activities and detect network security threats in a timely manner. The traditional method of network traffic classification is through deep packet inspection, that is, identifying the category of data packets by matching the keywords of the data packet payload. However, with the increase of encrypted content on the network, for data packets with ciphertext payload content, it is not possible to simply analyze their categories by directly identifying keywords, and deep packet inspection is no longer effective. The current method for encrypted traffic classification is mainly to train network traffic machine learning or deep learning models based on channel characteristics, hoping that the model can learn the encryption characteristics and behavior patterns of encrypted traffic through features such as packet length, packet arrival time, and transmission protocol. However, this method is very dependent on the features proposed by experts, and the number of features is limited, so it is difficult to perfectly characterize the behavior of network traffic. In addition, the trained model performs poorly when facing new traffic data or new network traffic environments, and it takes a long time to retrain to adapt. Therefore, the common problems with the above-mentioned technologies are:

[0003] (1) The packet payload in encrypted traffic no longer has readable meaning and cannot guide network traffic classification;

[0004] (2) Machine learning or deep model classification methods that directly use channel features rely too much on limited features proposed by experts and are difficult to perfectly characterize network behavior;

[0005] (3) Encrypted traffic classification technology based on machine learning or deep learning is difficult to migrate from a specific network environment to a new identification environment.

[0006] In summary, there is a need for an encrypted network traffic classification technology that can characterize the behavior patterns of encrypted traffic and quickly adapt to new network environments through transfer learning. Summary of the invention

[0007] A brief overview of the present invention is provided below in order to provide a basic understanding of certain aspects of the present invention. It should be understood that this overview is not an exhaustive overview of the present invention. It is not intended to identify key or important parts of the present invention, nor is it intended to limit the scope of the present invention. Its purpose is merely to present certain concepts in a simplified form as a prelude to a more detailed description discussed later.

[0008] In view of this, in order to solve the technical problems existing in the prior art, the present invention provides an encrypted network traffic classification method based on a large language model.

[0009] Solution 1: A method for classifying encrypted network traffic based on a large language model, comprising:

[0010] S1. Obtain network traffic data, and obtain traffic data of different categories of network traffic with labels according to a specific classification task;

[0011] S2. In the acquired network traffic data, cluster the data packets with the same source IP, destination IP, source port, destination port and protocol quintuple to obtain a bidirectional data packet sequence at the flow level;

[0012] S3. Extract a bidirectional packet length sequence according to the bidirectional packet sequence, and construct a corpus using the flow-level bidirectional packet length sequence of the background traffic; construct a fine-tuning dataset according to the flow-level bidirectional packet length sequence of various types of network traffic;

[0013] S4. Generate an initialization word list, recursively generate a vocabulary list, and generate a final vocabulary list in turn to obtain a traffic data packet length encoding table, perform word segmentation on the bidirectional data packet length sequence text in the corpus and fine-tuning dataset, and convert the word sequence corresponding to the packet length into a corresponding encoding sequence;

[0014] S5. Use the pre-training dataset to pre-train the BERT model. For a specific classification task, use the category-labeled fine-tuning dataset to fine-tune the pre-trained BERT model to obtain an encrypted network traffic classification task model.

[0015] S6. Input the bidirectional data packet sequence into the encrypted network traffic classification task model and output the traffic category.

[0016] Preferably, the bidirectional packet sequence is used to identify the packet flow of a particular network communication.

[0017] Preferably, for each flow, the length l of each data packet is extracted and sorted in the order of arrival time of the bidirectional data packet sequence, any data packet length l is represented by a decimal number, and the lengths of data packets adjacent in time are separated by spaces as separators, and the bidirectional data packet length sequence of length n is l1 l2...l t ; Arrange the bidirectional data packet length sequence in chronological order and store it as the fine-tuning dataset.

[0018] Preferably, S4 specifically comprises the following steps:

[0019] S41. Generate an initialization vocabulary;

[0020] S42. Recursively generate vocabulary v2;

[0021] S43. Generate the final vocabulary v3;

[0022] S44. Generate traffic data packet length encoding table;

[0023] S45. Perform word segmentation on the bidirectional packet length sequence text in the corpus and fine-tuning dataset according to the traffic packet length encoding table δ;

[0024] S46. According to the coding mapping of the traffic data packet length coding table δ, the bidirectional data packet length sequence is converted into a corresponding coding sequence.

[0025] Preferably, the method for generating the initialization vocabulary v1 is: for each bidirectional data packet length sequence text in the corpus, all the single characters that appear are extracted to generate the initialization vocabulary v1; the initialization vocabulary v1 contains all the single characters that appear in the text, and an initialization vocabulary v1 is represented as: v1 = {1, 2, 3, ..., ##9, ##0}

[0026] Among them, ## indicates that the segment is part of a word and is spliced ​​together with the previous segment to form a complete word.

[0027] Preferably, the method for recursively generating vocabulary v2 is: repeatedly performing the following operations to recursively generate vocabulary v2 until the vocabulary size reaches a predetermined size or the merge score is lower than a certain threshold:

[0028] (1) Calculate the frequency of occurrence of each word or each subword x in the dataset in the entire corpus, freq(xy);

[0029] (2) For each pair of adjacent subwords x and y, calculate the frequency score Score(x, y) of the adjacent subwords x and y combined;

[0030]

[0031] (3) Select the candidate with the highest merging score from all merging candidates and merge them into a new subword xy;

[0032] (4) Add the newly generated subword xy to the vocabulary v2; when the stopping condition is reached, the training process ends and the trained vocabulary v2 is obtained.

[0033] Preferably, the method for generating the final vocabulary v3 is: adding PAD, UNK, CLS, SEP, and MASK to the vocabulary v2 to obtain the final vocabulary v3; PAD, UNK, CLS, SEP, and MASK represent filling, unknown vocabulary, sentence start, sentence end, and masking, respectively.

[0034] Preferably, the method for generating the traffic data packet length encoding table is: for each word token in the final vocabulary v3 i Assign a unique integer code index i , get the traffic data packet length encoding table δ.

[0035] Solution 2: An electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of a method for classifying encrypted network traffic based on a large language model described in Solution 1 are implemented.

[0036] Solution three: A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for classifying encrypted network traffic based on a large language model as described in Solution one is implemented.

[0037] The beneficial effects of the present invention are as follows: Based on the open source large language model based on BERT, the present invention uses the bidirectional flow-level network traffic channel feature text data (packet length) as pre-training data to train a base model that can use open domain unlabeled traffic data to learn representations with strong generalization capabilities, and in downstream tasks, a small number of labeled specific classification data are used to complete rapid transfer learning, thereby enhancing the adaptability of the model. Compared with the prior art, the present invention has the following advantages:

[0038] (1) Compared with using data packet payloads that have no readable meaning as training samples, the present invention uses data packet length sequences as channel features, which is able to learn the behavior patterns of encrypted network traffic;

[0039] (2) Compared with the limited features proposed by experts, the present invention can perfectly characterize the behavior pattern of encrypted network traffic through data packet length;

[0040] (3) The present invention is able to learn various network traffic behavior patterns through a large amount of learning with strong generalization ability in the pre-training stage, and is able to continue learning various different classification methods;

[0041] (4) The present invention can quickly complete learning transfer and quickly complete fine-tuning training in downstream tasks with less classification training data, which can save a lot of computing resources and training time. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0043] Figure 1It is a flow chart of a method for classifying encrypted network traffic based on a large language model;

[0044] Figure 2 Schematic diagram of segmented extraction of bidirectional data packet length sequence using stride sliding window method;

[0045] Figure 3 Schematic diagram of word segmentation mask;

[0046] Figure 4 This is a schematic diagram of the Transformer model structure;

[0047] Figure 5 This is a schematic diagram of the Encoder-Decoder structure of Transformer;

[0048] Figure 6 Schematic diagram of the pre-training-fine-tuning task for BERT as a large oracle model;

[0049] Figure 7 Schematic diagram of the BERT model structure when performing BERT classification tasks. DETAILED DESCRIPTION

[0050] In order to make the technical solutions and advantages of the embodiments of the present invention more clearly understood, the exemplary embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0051] Example 1: Reference Figure 1-Figure 7 The present embodiment describes a method for classifying encrypted network traffic based on a large language model, comprising:

[0052] S1. Obtain network traffic data, and obtain traffic data of different categories of network traffic with labels according to a specific classification task;

[0053] Network traffic data includes background traffic and various types of network traffic;

[0054] Background traffic is traffic that is not generated by a specific application or service.

[0055] All types of network traffic are network traffic of a single category;

[0056] Background traffic is used for generalization learning in the pre-training phase, and various types of network traffic are used for transfer learning in the fine-tuning phase.

[0057] The background data used for pre-training is unlabeled traffic data, which needs to be sufficient in quantity and general enough to meet the generalization condition. Specifically, the reason why the background traffic data used for pre-training needs to be sufficient in quantity is that sufficient training data is needed to prevent the model from overfitting; the meaning of the background traffic data used for pre-training needing to be general enough is that the pre-trained model needs to learn the network behavior patterns of various types of traffic from different types of traffic, and at the same time, in order to meet the generalization condition, the traffic must also meet the network behavior patterns in daily life. Among them, "different types of" network traffic refers to both the network traffic containing as many network protocols as possible and the network traffic requiring encrypted / unencrypted traffic at the same time to meet the generalization condition. In the fine-tuning stage, the present invention transfers the base model learned through generalization in the pre-training stage to a specific downstream task, and the classification method of various types of network traffic data may be different according to the specific task. For example, if a specific task is to identify the application category in the encrypted traffic, in order to complete the task, it is necessary to provide the network traffic data of each application category as a data set in the fine-tuning stage. After the fine-tuning is completed, a model that efficiently completes the specific classification task can be obtained. If there is another specific task later, the transfer learning can be completed again from the base model through fine-tuning.

[0058] S2. In the acquired network traffic data, cluster the data packets with the same source IP, destination IP, source port, destination port and protocol quintuple to obtain a bidirectional data packet sequence at the flow level;

[0059] Bidirectional packet sequences are used to identify the packet flow of a particular network communication;

[0060] The source IP, destination IP, source port, destination port and protocol type of the data packets in each flow are the same, and they are called a "connection".

[0061] S3. Extract a bidirectional packet length sequence according to the bidirectional packet sequence, and construct a corpus using the flow-level bidirectional packet length sequence of the background traffic; construct a fine-tuning dataset according to the flow-level bidirectional packet length sequence of various types of network traffic;

[0062] For each flow, extract and sort the length l of each packet in the order of the arrival time of the bidirectional packet sequence. Any packet length l is represented by a decimal number, and the lengths of adjacent packets in time are separated by spaces. The length sequence of bidirectional packets with a length of n is l1 l2...l t ; Arrange the bidirectional data packet length sequence in chronological order and store it as the fine-tuning dataset.

[0063] The corpus is stored in the form of text files;

[0064] The fine-tuning dataset is stored as a text file;

[0065] The bidirectional data packet length sequence shows the logical relationship between the length of the data packet and time within a certain period of time, and describes the behavior pattern of network traffic within a certain time range; the bidirectional data packet length sequence corresponding to a flow is: "5050 50 1443 1443 690";

[0066] Filter the bidirectional packet length sequence by:

[0067] (1) Filter bidirectional data packet length sequences with length less than 4. Sequences with length less than 4 are only example lengths. The filtering length is set according to actual needs.

[0068] (2) For bidirectional data packet length sequences that are too long, the stride sliding window method is used to extract the bidirectional data packet length sequences in segments. The specific window length and moving step size can be selected according to the input dimension of the large language model. Specifically, refer to Figure 2 , a bidirectional packet length sequence of length 192:

[0069] 1 2 3……63 64 65……127 128 129……190 191 192; if the window length is 128 and the moving step is 64, we can get 2 sub-length sequences, specifically: 1 2 3……63 64 65……126 127128; 65 66 67……127 128 129……190 191 192.

[0070] S4. Generate an initialization word list, recursively generate a vocabulary list, and generate a final vocabulary list in turn to obtain a traffic data packet length encoding table, perform word segmentation on the bidirectional data packet length sequence text in the corpus and fine-tuning dataset, and convert the word sequence corresponding to the packet length into a corresponding encoding sequence;

[0071] S41. Generate an initialization vocabulary v1: For each bidirectional data packet length sequence text in the corpus, extract all the single characters that appear and generate an initialization vocabulary v1; the initialization vocabulary v1 contains all the single characters that appear in the text. An initialization vocabulary v1 is represented as: v1 = {1, 2, 3, ..., ##9, ##0}

[0072] Among them, ## indicates that the segment is part of a word and is spliced ​​together with the previous segment to form a complete word;

[0073] S42. Recursively generate vocabulary v2: Repeat the following operations to recursively generate vocabulary v2 until the vocabulary size reaches a predetermined size or the merge score is lower than a certain threshold:

[0074] (1) Calculate the frequency of occurrence of each word or each subword x in the dataset in the entire corpus, freq(xy);

[0075] (2) For each pair of adjacent subwords x and y, calculate the frequency score Score(x,y) of the adjacent subwords x and y combined;

[0076]

[0077] (3) Select the candidate with the highest merging score from all merging candidates and merge them into a new subword xy;

[0078] (4) Add the newly generated subword xy to the vocabulary v2; when the stopping condition is reached, the training process ends and the trained vocabulary v2 is obtained;

[0079] The stopping condition is: the number of words in vocabulary v2 reaches the set upper limit, or the score value in a certain iteration is less than the set threshold;

[0080] The upper limit and threshold are set according to the actual situation;

[0081] S43. Generate final vocabulary v3: Add PAD, UNK, CLS, SEP, and MASK to vocabulary v2 to obtain final vocabulary v3; PAD, UNK, CLS, SEP, and MASK represent padding, unknown vocabulary, sentence start, sentence end, and mask, respectively;

[0082] PAD, UNK, CLS, SEP, and MASK are special marks;

[0083] S44. Generate traffic packet length encoding table: for each word token in the final vocabulary v3 i Assign a unique integer code index i , get the traffic data packet length encoding table δ.

[0084] S45. Perform word segmentation on the bidirectional packet length sequence text in the corpus and fine-tuning dataset according to the traffic packet length encoding table δ;

[0085] It includes vocabulary query, subword segmentation, filling, attention mask and special tag adding operations to split the input text into appropriate subwords or words. It is also necessary to do some special processing on the word sequence according to special needs, such as the operation processing corresponding to the MLM and NSP tasks in the BERT pre-training objective.

[0086] refer to Figure 3, a specific flow corresponds to a bidirectional data packet length sequence "50505014431443690", after word segmentation with a maximum length of 12 and masking, the resulting word sequence is:

[0087] ['[CLS]','[MASK]','[MASK]','50','144','##3','144','##3','69','##0','[SEP]','[PAD]'];

[0088] S46. According to the coding mapping of the traffic packet length coding table δ, the bidirectional packet length sequence is converted into a corresponding coding sequence;

[0089] refer to Figure 3 , the above bidirectional data packet length sequence is further processed by encoding mapping as follows:

[0090] Tokens IDs:[2,4,4,113,65,19,65,19,221,17,3,0],The bidirectional packet length sequence text data in the processed corpus and fine-tuning dataset are converted into the encoding required for the input of the model training.

[0091] Segment IDs should be all 0;

[0092] S5. Use the pre-training dataset to pre-train the BERT large language model, and use the open domain unlabeled traffic data to learn representations with strong generalization capabilities; for specific classification tasks, use the category-labeled fine-tuning dataset to fine-tune the pre-trained BERT model to obtain the encrypted network traffic classification task model;

[0093] The BERT large language model is pre-trained using a pre-training dataset, and open domain unlabeled traffic data is used to learn representations with strong generalization capabilities. Specifically:

[0094] Reference Figure 4 , the structure of the Transformer model consists of two parts: encoder and decoder, refer to Figure 5 , Transformer's encoder encoder is responsible for encoding the input sequence into a high-dimensional vector representation, while the decoder converts the vector into a target sequence. Each encoder consists of multiple stacked identical modules, including a self-attention layer (Self-Attention) and a fully connected feedforward network (FeedForward);

[0095] (1) Self-attention layer: In the self-attention mechanism, each position of the input sequence interacts with other positions in the sequence, allowing the model to take into account the context of the entire sequence when generating representations; the calculation of each attention head involves three intermediate weight matrices Wq , W k , W v , they respectively perform linear transformation on the input vector X to generate three weight vectors Q(query), K(key) and V(value), which are the query vector, key vector and value vector respectively; and transform these three weight vectors to obtain the attention encoding corresponding to the input vector X. Specifically, the calculation steps are as follows:

[0096] ① Input vectors X and W q Multiply to get Q, and do the same to get K and V.

[0097] Q=X·W q

[0098] X is the input vector, with a shape of (n, dmodel), where n is the sequence length and dmodel is the dimension of the input vector; Wq, Wk, Wv are weight matrices, with Wq, Wk shapes of (dmodel, dk) and Wv shapes of (dmodel, dv), which are used to linearly transform the input vector into query, key, and value vectors; Q, K, V are query vectors, key vectors, and value vectors, respectively; Q, K shapes are (n, dk), and V shape is (n, dv);

[0099] ② Multiply Q and K to get the relevance between the words in the input vector X, i.e., the attention score. At the same time, to prevent the value from being too large, divide by The data is scaled and normalized using softmax so that the sum of each row is 1.

[0100]

[0101] d k is the dimension of query vector and key vector; Attention score is the attention score, with shape (n,n); Softmax is the normalization function, which makes the normalized score between 0 and 1, and the sum of all scores in each row is 1;

[0102] ③ Use the attention score matrix to perform weighted summation on V to obtain a weighted representation for each position.

[0103] The weighted sum is expressed as:

[0104]

[0105] V is a value vector with a shape of (n,d v ); Attention(Q,K,V) is the final attention code, with a shape of (n,d v );

[0106] In Transformer, the sub-attention layer includes a multi-head self-attention mechanism, which further enhances the expressiveness of the model by projecting the input sequence into multiple subspaces for attention calculation, and then merging the outputs of multiple subspaces to generate the final representation.

[0107] (2) Fully connected feedforward network: A fully connected feedforward network is used to perform nonlinear transformations on the weighted representations after each head of the attention mechanism, thereby producing the final attention output;

[0108] The feedforward network in the attention mechanism consists of two linear transformation layers and a nonlinear activation function, usually ReLU (Rectified Linear Unit). Each linear transformation layer consists of a learnable weight matrix and a learnable bias vector. There is usually an activation function between the two linear transformation layers to increase the nonlinear ability of the model. The specific operation of the feedforward network is expressed as:

[0109] FFN(x)=ReLU(xW1+b1)W2+b2

[0110] Among them, x is the model input, W i and b i are the weight matrix and bias vector of the i-th layer respectively. ReLU is the activation function used to increase the nonlinear ability of the model.

[0111] In the pre-training stage, the present invention uses unlabeled network traffic background traffic to enable BERT to learn various network traffic behavior patterns from massive encrypted traffic, while improving BERT's generalization ability and reducing the amount of data used for supervised training, so that only a small amount of fine-tuning data is needed to complete transfer learning in subsequent fine-tuning, thereby reducing dependence on a large amount of expensive labeled data.

[0112] For specific classification tasks, the pre-trained BERT model is fine-tuned using a category-labeled fine-tuning dataset to obtain the encrypted network traffic classification task model:

[0113] By fine-tuning BERT, it is possible to transfer the generalized learning of BERT pre-training to specific tasks. The task of the present invention is to perform traffic classification in encrypted network traffic, and according to different downstream tasks, use different fine-tuning data for further training to obtain a model for a specific task, that is, the model has transfer learning capabilities.

[0114] In the last hidden state after the BERT pre-training model, corresponding to the output position of the special tag CLS position of the embedding, an N-layer fully connected neural network is added.

[0115] In the pre-training stage, the BERT base model has learned the behavior patterns of network traffic. Each vector has been processed by BERT's multi-layer Transformer encoder, and self-attention has been performed at each layer to capture the associated information in the input sequence. Therefore, in the fine-tuning stage, after being processed by the base model, the last hidden state can be regarded as an abstraction of the high-level semantic representation of the input sequence. In the output of the last layer, the vector output value at the CLS position is designed to represent the entire sentence, which has been processed by all layers of the BERT model and contains the overall semantic information of the input sequence.

[0116] On this basis, in the fine-tuning stage, the parameters of the base model are not changed. Only N layers of fully connected neural networks are added to the CLS position of the last hidden state of the BERT base model. It is expected that the knowledge that can complete the specific classification task can be learned from the overall semantic information contained in the vector output value of the CLS position through the N layers of fully connected neural networks. Finally, softmax is used to obtain the prediction probability of each category of the fine-tuning model. In this way, the process from the input vector to the output of the BERT base model and then to the final classification category is completed, and the fine-tuning classification task is finally completed.

[0117] In the fine-tuning stage, it is not necessary to learn the parameters of the base model, but only the parameters of the N-layer fully connected neural network added in the fine-tuning stage need to be learned. This N-layer fully connected neural network is relatively simple, and the number of parameters is much smaller than the parameters of the pre-trained model. Therefore, the present invention can use a small amount of labeled data to fine-tune the pre-trained base model in the fine-tuning stage, and use the knowledge learned from the base model to continue training using its intermediate output to obtain a model that can better complete the encrypted network traffic classification task. The fine-tuning process requires less time and training resources than pre-training, so it can be said that the model of the present invention can better adapt to the application of various classification methods in various network traffic environments.

[0118] S6. Input the bidirectional data packet sequence into the BERT large language model and output the traffic category

[0119] Embodiment 2: The computer device of the present invention may be a device including a processor and a memory, such as a single chip microcomputer including a central processing unit, etc. Furthermore, the processor is used to implement the steps of the above-mentioned method for classifying encrypted network traffic based on a large language model when executing a computer program stored in the memory.

[0120] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0121] The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0122] Embodiment 3: Computer readable storage medium embodiment.

[0123] The computer-readable storage medium of the present invention can be any form of storage medium that can be read by a processor of a computer device, including but not limited to non-volatile memory, volatile memory, ferroelectric memory, etc. A computer program is stored on the computer-readable storage medium. When the processor of the computer device reads and executes the computer program stored in the memory, the steps of the above-mentioned method for classifying encrypted network traffic based on a large language model can be implemented.

[0124] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer readable media do not include electric carrier signals and telecommunication signals.

[0125] Although the present invention has been described according to a limited number of embodiments, it will be apparent to those skilled in the art, with the benefit of the above description, that other embodiments may be envisioned within the scope of the invention thus described. In addition, it should be noted that the language used in this specification is selected primarily for readability and teaching purposes, rather than for explaining or defining the subject matter of the present invention. Therefore, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the present invention is illustrative, not restrictive, with respect to the scope of the present invention, which is defined by the appended claims.

Claims

1. A method for classifying encrypted network traffic based on a large language model, characterized in that: include: S1. Obtain network traffic data, and obtain traffic data of different categories of network traffic with labels according to a specific classification task; S2. In the acquired network traffic data, data packets with the same source IP, destination IP, source port, destination port and protocol quintuple are clustered to obtain a flow-level bidirectional data packet sequence, which is used to identify a data packet flow of a specific network communication; S3. Extract a bidirectional packet length sequence according to the bidirectional packet sequence, and construct a corpus using the flow-level bidirectional packet length sequence of the background traffic; construct a fine-tuning dataset according to the flow-level bidirectional packet length sequence of various types of network traffic; For each flow, extract and sort the length l of each packet in the order of the arrival time of the bidirectional packet sequence. Any packet length l is represented by a decimal number, and the lengths of adjacent packets in time are separated by spaces. The length sequence of bidirectional packets with a length of n is l1l2...l t ;Store the bidirectional data packet length sequence in chronological order as a fine-tuning dataset; S4. Generate an initialization word list, recursively generate a vocabulary list, and generate a final vocabulary list in turn to obtain a traffic data packet length encoding table, perform word segmentation on the bidirectional data packet length sequence text in the corpus and fine-tuning dataset, and convert the word sequence corresponding to the packet length into a corresponding encoding sequence; S5. Use the pre-training dataset to pre-train the BERT model. For a specific classification task, use the category-labeled fine-tuning dataset to fine-tune the pre-trained BERT model to obtain an encrypted network traffic classification task model. In the pre-training stage, the BERT base model has learned the behavior pattern of network traffic. Each vector has been processed by BERT's multi-layer Transformer encoder, and self-attention has been performed at each layer to capture the associated information in the input sequence. Therefore, in the fine-tuning stage, after being processed by the base model, the last hidden state can be regarded as an abstraction of the high-level semantic representation of the input sequence; in the output of the last layer, the vector output value at the CLS position is designed to represent the entire sentence, which has been processed by all layers of the BERT model and contains the overall semantic information of the input sequence; On this basis, in the fine-tuning stage, the parameters of the base model are not changed. Only N layers of fully connected neural networks are added to the CLS position of the last hidden state of the BERT base model. It is expected that the knowledge that can complete the specific classification task can be learned from the overall semantic information contained in the vector output value of the CLS position through the N layers of fully connected neural networks. Finally, softmax is used to obtain the prediction probability of each category of the fine-tuning model. In this way, the process from the input vector to the output of the BERT base model and then to the final classification category is completed, and the fine-tuning classification task is finally completed. In the fine-tuning phase, there is no need to learn the parameters of the base model. Only the parameters of the N-layer fully connected neural network added in the fine-tuning phase need to be learned. S6. Input the bidirectional data packet sequence into the encrypted network traffic classification task model and output the traffic category.

2. According to claim 1, a method for classifying encrypted network traffic based on a large language model is characterized in that: S4 specifically includes the following steps: S41. Generate an initialization vocabulary; S42. Recursively generate vocabulary v2; S43. Generate the final vocabulary v3; S44 generates a traffic packet length encoding table δ; S45. Perform word segmentation on the bidirectional packet length sequence text in the corpus and fine-tuning dataset according to the traffic packet length encoding table δ; S46. According to the coding mapping of the traffic data packet length coding table δ, the bidirectional data packet length sequence is converted into a corresponding coding sequence.

3. According to claim 1, a method for classifying encrypted network traffic based on a large language model is characterized in that: The method for generating the initialization vocabulary v1 is as follows: for each bidirectional data packet length sequence text in the corpus, all the single characters that appear are extracted to generate the initialization vocabulary v1; the initialization vocabulary v1 contains all the single characters that appear in the text, and an initialization vocabulary v1 is represented as: v1 = {1, 2, 3, ..., ##9, ##0}; Among them, ## indicates that the segment is part of a word and is spliced ​​together with the previous segment to form a complete word.

4. According to the method for classifying encrypted network traffic based on a large language model in claim 1, it is characterized in that: The method for recursively generating vocabulary v2 is: repeat the following operations to recursively generate vocabulary v2 until the vocabulary size reaches a predetermined size or the merge score is lower than a certain threshold: (1) Calculate the frequency of occurrence of each word or each subword x in the dataset in the entire corpus, freq(xy); (2) For each pair of adjacent subwords x and y, calculate the frequency score Score(x,y) of the adjacent subwords x and y combined; (3) Select the candidate with the highest merging score from all merging candidates and merge them into a new subword xy; (4) Add the newly generated subword xy to the vocabulary v2; when the stopping condition is reached, the training process ends and the trained vocabulary v2 is obtained.

5. According to claim 1, a method for classifying encrypted network traffic based on a large language model is characterized in that: The method for generating the final vocabulary v3 is as follows: add PAD, UNK, CLS, SEP, and MASK to the vocabulary v2 to obtain the final vocabulary v3; PAD, UNK, CLS, SEP, and MASK represent filling, unknown vocabulary, sentence start, sentence end, and masking, respectively.

6. The method for classifying encrypted network traffic based on a large language model according to claim 1, characterized in that: The method for generating the traffic packet length encoding table is as follows: for each word token in the final vocabulary v3 i Assign a unique integer code index i , get the traffic data packet length encoding table δ.

7. An electronic device, characterized in that: It comprises a memory and a processor, the memory stores a computer program, and the processor implements the steps of a method for classifying encrypted network traffic based on a large language model as described in any one of claims 1 to 6 when executing the computer program.

8. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the method for classifying encrypted network traffic based on a large language model as described in any one of claims 1 to 6 is implemented.

Images