Method and device for carrying out encrypted traffic anomaly detection based on large language model

Abstract

The invention provides a method and device for carrying out encrypted traffic anomaly detection based on a large language model, and relates to the technical field of artificial intelligence. The method comprises the following steps: acquiring encrypted traffic data in different modes in different acquisition internet, and performing lexical element processing on the acquired encrypted traffic data to obtain a lexical element list; according to the obtained lexical element list, training a pre-trained large language model through two stage tasks to obtain a trained large language model; constructing instruction fine tuning data; and inputting instruction fine tuning data into the trained large language model for prediction to obtain a prediction result of the traffic label. According to the invention, the flow marking efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and device for detecting anomalies in encrypted traffic based on a large language model. Background Art

[0002] Traffic inspection is a method of monitoring network activity, identifying specific patterns in network traffic, and then extracting valuable information from it to detect whether the traffic is malicious. With the increasing emphasis on network security, SSL and TLS encryption technologies are widely used to encrypt traffic content to protect privacy, making the payload portion of traffic obscured. Because the payload is encrypted, traditional traffic inspection methods based on plaintext payload analysis are no longer applicable. With the proliferation of encrypted traffic on the Internet, encrypted traffic inspection has become a significant research topic.

[0003] Although the data packets are encrypted, traffic generated by abnormal behavior is still different from normal traffic. Machine learning techniques can extract valuable features without directly accessing the raw data, and therefore are widely used in detecting encrypted traffic. Based on the complexity of the model, machine learning methods are categorized as: machine learning-based, deep learning-based, and large language model-based. Machine learning-based methods simply specify and extract features, then apply classical algorithms to process them. However, they also present significant challenges. First, in terms of feature selection, there is no universal set of features that can distinguish the current task. Different features must be designed for different tasks. Furthermore, as the types of malicious traffic in this field continue to evolve, features may need to be continuously added and modified. Deep learning-based methods are essentially shallow feature learning, relying on the results of manual feature extraction. Their generalization ability is insufficient, and their accuracy in identifying malicious patterns is limited. Large language model-based methods have achieved revolutionary progress in the field of natural language processing. These models are pre-trained on large public corpora to acquire a wealth of knowledge and have demonstrated excellent performance in various downstream natural language processing tasks. However, the classification effectiveness of these methods relies on labeled traffic data. However, it is difficult to label traffic data and is difficult to achieve using the above methods. Summary of the Invention

[0004] In order to solve the technical problems in the existing technology of encrypted traffic data detection, the classification effect of the existing methods depends on scarce and expensive labeled traffic data, the generalization ability of small models is insufficient, and there are certain limitations in the accuracy of identifying malicious patterns. The embodiment of the present invention provides a method and device for encrypted traffic anomaly detection based on a large language model. The technical solution is as follows:

[0005] In one aspect, a method for detecting anomalies in encrypted traffic based on a large language model is provided. The method is implemented by a device for detecting anomalies in encrypted traffic based on a large language model, and the method includes:

[0006] S1. Obtain encrypted traffic data in different modes on the Internet; remove the encrypted payload portion of the encrypted traffic data to obtain data of the traffic packet of the first subtask; divide the encrypted traffic data into traffic packets in the same flow and the same time interval according to the traffic quintuple and time interval, remove the encrypted payload portion of the traffic packet, and obtain data of the traffic packet of the second subtask;

[0007] S2. Input the data of the traffic packet of the first subtask into the trained word segmenter for lemmatization to obtain a word-gram list of the first subtask; input the data of the traffic packet of the second subtask into the trained word segmenter for lemmatization to obtain a word-gram list of the second subtask;

[0008] S3. Build a pre-trained large language model. The pre-trained large language model includes: a multi-layer Transformer decoder, a multi-layer perceptron layer, and a softmax layer.

[0009] S4. Input the word list of the first subtask into the pre-trained large language model, calculate it through the multi-layer Transformer decoder to obtain word-word features; input the word-word features into the multi-layer perceptron layer, predict it through the softmax layer, and obtain the probability of the next traffic word in the traffic packet; input the data of the traffic packet of the second subtask into the pre-trained large language model, calculate it through the multi-layer Transformer decoder to obtain word-word features; input the word-word features into the multi-layer perceptron layer, predict it through the softmax layer, and obtain the probability of each traffic word in the next traffic packet; according to the probability of the next traffic word in the traffic packet and the probability of each traffic word in the next traffic packet, calculate the loss function, update the model parameters according to the gradient, and train to obtain a trained large language model;

[0010] S5. Construct instruction fine-tuning data; input the instruction fine-tuning data into the trained word segmenter to obtain a word list; input the word list into the trained large language model for prediction, and output the predicted result of the traffic label.

[0011] Optionally, the step of sequentially inputting the word list of the first subtask into a pre-trained large language model for pre-training includes:

[0012] According to the original position order of the traffic packets, the first k traffic words in the word list of the first subtask are input into the pre-trained large language model; the word list of the first subtask is recorded as When there are less than k tokens, a [pad] symbol is added at the end. When predicting the second token, if there is only one token in front, the input is t1,[pad],[pad]..[pad]. Among them, the [pad] symbol does not participate in the calculation and is a padding symbol. When the front token exceeds k, the redundant part is directly truncated and the k tokens closest to the predicted token are retained.

[0013] Optionally, the loss function is expressed by the following formula (1):

[0014] (1)

[0015] in, Indicates traffic word; Represents the parameters of the pre-trained large language model; k represents the context window size, and the function P represents the pre-trained large language model; Indicates the probability value obtained.

[0016] Optionally, inputting the data of the second subtask traffic packet into a pre-trained large language model includes:

[0017] According to the data of the traffic packet of the second subtask, two temporally adjacent traffic packets in the same flow are taken, and the word list of the previous traffic packet is input into the pre-trained large language model.

[0018] Optionally, the training is performed by calculating a loss function based on the probability of the next traffic word in the traffic packet and the probability of each traffic word in the next traffic packet, and updating model parameters according to the gradient to obtain a trained large language model, including:

[0019] According to the probability of the next traffic word in the traffic packet, the loss function is calculated, and the model parameters are updated according to the gradient for training to obtain the large language model trained for the first subtask; based on the large language model trained for the first subtask, according to the probability of each traffic word in the next traffic packet, the loss function is calculated, and the model parameters are updated according to the gradient for training to obtain the trained large language model.

[0020] Optionally, the step of constructing instruction fine-tuning data in S5; inputting the instruction fine-tuning data into a trained word segmenter to obtain a word list; and inputting the word list into a pre-trained large language model for prediction to obtain a prediction result of a traffic label, including:

[0021] Design prompt words corresponding to the task of detecting malicious encrypted traffic; obtain data of traffic packets for the task of detecting malicious encrypted traffic; wherein the data of the traffic packets is labeled data;

[0022] Combine the prompt word with the data of the traffic package to obtain instruction fine-tuning data;

[0023] The instruction fine-tuning data is input into the pre-trained large language model for instruction fine-tuning, and tokenization is performed through the trained word segmenter to obtain a token list; the token list is input into the multi-layer Transformer decoder for calculation to obtain token features; the token features are input into the multi-layer perceptron layer, predicted through the softmax layer, and the probability distribution of the traffic label is output. The label with the highest probability is selected as the final traffic label.

[0024] Optionally, the cross entropy loss function is used to calculate the difference between the predicted traffic label and the true label based on the output traffic label, wherein the specific process of calculating the difference between the predicted traffic label and the true label is shown by the following formula (2):

[0025] (2)

[0026] in, The i-th instruction input model, if the label output by the model is consistent with the true label, ,otherwise , is the probability that the model predicts the traffic label

[0027] On the other hand, a device for detecting anomalies in encrypted traffic based on a large language model is provided. The device is applied to a method for detecting anomalies in encrypted traffic based on a large language model. The device includes:

[0028] An acquisition unit is configured to acquire encrypted traffic data in different modes on the Internet; remove the encrypted payload portion of the encrypted traffic data to obtain data of traffic packets of the first subtask; divide the encrypted traffic data into traffic packets in the same flow and the same time interval according to the traffic quintuple and the time interval, remove the encrypted payload portion of the traffic packets, and obtain data of traffic packets of the second subtask;

[0029] a processing unit, configured to input the data of the first subtask traffic packet into the trained word segmenter for lemma conversion to obtain a word gram list of the first subtask; and input the data of the second subtask traffic packet into the trained word segmenter for lemma conversion to obtain a word gram list of the second subtask;

[0030] The construction unit is used to build a pre-trained large language model; the pre-trained large language model includes: a multi-layer Transformer decoder, a multi-layer perceptron layer, and a softmax layer;

[0031] The training unit is used to input the word list of the first subtask into the pre-trained large language model, calculate through the multi-layer Transformer decoder to obtain word-word features; input the word-word features into the multi-layer perceptron layer, predict through the softmax layer, and obtain the probability of the next traffic word in the traffic packet; input the data of the traffic packet of the second subtask into the pre-trained large language model, calculate through the multi-layer Transformer decoder to obtain word-word features; input the word-word features into the multi-layer perceptron layer, predict through the softmax layer, and obtain the probability of each traffic word in the next traffic packet; according to the probability of the next traffic word in the traffic packet and the probability of each traffic word in the next traffic packet, the loss function is calculated, and the model parameters are updated according to the gradient to perform training to obtain a trained large language model;

[0032] The prediction unit is used to construct instruction fine-tuning data; input the instruction fine-tuning data into the trained word segmenter to obtain a word list; input the word list into the trained large language model for prediction, and output the predicted result of the traffic label.

[0033] Optionally, the step of sequentially inputting the word list of the first subtask into a pre-trained large language model for pre-training includes:

[0034] According to the original position order of the traffic packets, the first k traffic words in the word list of the first subtask are input into the pre-trained large language model; the word list of the first subtask is recorded as When there are less than k tokens, a [pad] symbol is added at the end. When predicting the second token, if there is only one token in front, the input is t1,[pad],[pad]..[pad]. Among them, the [pad] symbol does not participate in the calculation and is a padding symbol. When the front token exceeds k, the redundant part is directly truncated and the k tokens closest to the predicted token are retained.

[0035] Optionally, the loss function is expressed by the following formula (1):

[0036] (1)

[0037] in, Indicates traffic word; Represents the parameters of the pre-trained large language model; k represents the context window size, and the function P represents the pre-trained large language model; Indicates the probability value obtained.

[0038] Optionally, inputting the data of the second subtask traffic packet into a pre-trained large language model includes:

[0039] According to the data of the traffic packet of the second subtask, two temporally adjacent traffic packets in the same flow are taken, and the word list of the previous traffic packet is input into the pre-trained large language model.

[0040] Optionally, the training is performed by calculating a loss function based on the probability of the next traffic word in the traffic packet and the probability of each traffic word in the next traffic packet, and updating model parameters according to the gradient to obtain a trained large language model, including:

[0041] According to the probability of the next traffic word in the traffic packet, the loss function is calculated, and the model parameters are updated according to the gradient for training to obtain the large language model trained for the first subtask; based on the large language model trained for the first subtask, according to the probability of each traffic word in the next traffic packet, the loss function is calculated, and the model parameters are updated according to the gradient for training to obtain the trained large language model.

[0042] Optionally, the prediction unit is configured to:

[0043] Design prompt words corresponding to the task of detecting malicious encrypted traffic; obtain data of traffic packets for the task of detecting malicious encrypted traffic; wherein the data of the traffic packets is labeled data;

[0044] Combine the prompt word with the data of the traffic package to obtain instruction fine-tuning data;

[0045] The instruction fine-tuning data is input into the pre-trained large language model for instruction fine-tuning, and tokenization is performed through the trained word segmenter to obtain a token list; the token list is input into the multi-layer Transformer decoder for calculation to obtain token features; the token features are input into the multi-layer perceptron layer, predicted through the softmax layer, and the probability distribution of the traffic label is output. The label with the highest probability is selected as the final traffic label.

[0046] Optionally, the cross entropy loss function is used to calculate the difference between the predicted traffic label and the true label based on the output traffic label, wherein the specific process of calculating the difference between the predicted traffic label and the true label is shown by the following formula (2):

[0047] (2)

[0048] in, The i-th instruction input model, if the label output by the model is consistent with the true label, ,otherwise , is the probability that the model predicts the traffic label

[0049] On the other hand, a device for detecting anomalies in encrypted traffic based on a large language model is provided. The device for detecting anomalies in encrypted traffic based on a large language model includes: a processor; a memory, wherein computer-readable instructions are stored in the memory. When the computer-readable instructions are executed by the processor, any one of the methods for detecting anomalies in encrypted traffic based on a large language model is implemented.

[0050] On the other hand, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement any of the above-mentioned methods for detecting anomalies in encrypted traffic based on a large language model.

[0051] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0052] The embodiment of the present invention first obtains encrypted traffic data in different modes on the Internet; removes the encrypted payload part of the encrypted traffic data to obtain the data of the first subtask traffic packet; divides the encrypted traffic data into traffic packets in the same flow and the same time interval according to the traffic quintuple and the time interval, removes the encrypted payload part of the traffic packet, and obtains the data of the second subtask traffic packet; inputs the data of the first subtask traffic packet into a trained word segmenter for word metamerism to obtain a word list of the first subtask; inputs the data of the second subtask traffic packet into a trained word segmenter for word metamerism to obtain a word list of the second subtask; secondly, constructs a pre-trained large language model; the pre-trained large language model includes: a multi-layer Transformer decoder, a multi-layer perceptron layer, and a softmax layer; inputs the word list of the first subtask into the pre-trained large language model, and The decoder calculates and obtains word-unit features; the word-unit features are input into the multi-layer perceptron layer, and predicted through the softmax layer to obtain the probability of the next traffic word in the traffic packet; the data of the second subtask traffic packet is input into the pre-trained large language model, and the word-unit features are obtained through the multi-layer Transformer decoder calculation; the word-unit features are input into the multi-layer perceptron layer, and predicted through the softmax layer to obtain the probability of each traffic word in the next traffic packet; according to the probability of the next traffic word in the traffic packet and the probability of each traffic word in the next traffic packet, the loss function is calculated, and the model parameters are updated according to the gradient for training to obtain a trained large language model; finally, the instruction fine-tuning data is constructed; the instruction fine-tuning data is input into the trained word segmenter to obtain a word-unit list; the word-unit list is input into the trained large language model for prediction, and the prediction result of the traffic label is output.

[0053] The present invention can improve the performance of the large language model for traffic detection by utilizing a large amount of original unlabeled data. When the present invention is used for encrypted traffic detection tasks, the large language model is first pre-trained for traffic domain adaptation, and then fine-tuned for specific tasks. Unlike TrafficLLM, the model that has undergone traffic domain adaptation pre-training tasks has a more powerful ability to handle traffic domain tasks than the native large language model. It will not result in poor classification results due to different traffic patterns, and there is no need to fine-tune different models for different traffic patterns. In specific downstream tasks, only a small amount of labeled data is needed to obtain a model with excellent performance by fine-tuning the model parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0055] Figure 1 This is a flow chart of a method for detecting anomalies in encrypted traffic based on a large language model, provided by an embodiment of the present invention;

[0056] Figure 2 This is a schematic diagram of a structure of a word-unit prediction task provided by an embodiment of the present invention;

[0057] Figure 3 This is a block diagram of a device for detecting anomalies in encrypted traffic based on a large language model, provided by an embodiment of the present invention;

[0058] Figure 4 This is a structural diagram of a device for detecting anomalies in encrypted traffic based on a large language model, provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0060] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0061] In the embodiments of the present invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same. The terms "of," "corresponding," and "corresponding" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same.

[0062] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0063] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0064] The embodiment of the present invention provides a method for detecting anomalies in encrypted traffic based on a large language model. The method can be implemented by a device for detecting anomalies in encrypted traffic based on a large language model. The device for detecting anomalies in encrypted traffic based on a large language model can be a terminal or a server. Figure 1 The flowchart of the method for detecting anomalies in encrypted traffic based on a large language model is shown. The processing flow of the method may include the following steps:

[0065] S1. Obtain encrypted traffic data in different modes on the Internet; remove the encrypted payload part of the encrypted traffic data to obtain the data of the traffic packet of the first subtask; divide the encrypted traffic data into traffic packets in the same flow and the same time interval according to the traffic quintuple and time interval, remove the encrypted payload part of the traffic packet, and obtain the data of the traffic packet of the second subtask.

[0066] S2. Input the data of the first subtask traffic package into the trained word segmenter for word lemma processing to obtain the word lemma list of the first subtask; input the data of the second subtask traffic package into the trained word segmenter for word lemma processing to obtain the word lemma list of the second subtask.

[0067] Among them, the word list of the second subtask is the word list of each traffic packet of the same flow in the second subtask; among them, the first subtask learns the structural features within the traffic packet; among them, in a complete network conversation, the information between the previous and subsequent traffic packets is often related. In the second subtask, the ability of the large language model to understand the relationship and characteristics between traffic packets can be enhanced.

[0068] Among them, such as Figure 2The figure shows a structural diagram of a word-metaphor task; wherein, the data of the second subtask traffic packet is input into the trained word segmenter for word-metaphor processing, and the word-metaphor list of the second subtask is obtained. The process includes: wherein, according to the data of the second subtask traffic packet, two traffic packets that are adjacent in time in the same stream are taken, and the word-metaphor list of the previous traffic packet is input into the pre-trained large language model, wherein the word-metaphor list of the current traffic packet is represented as packet_current_tokenlist; the data of the next traffic packet is input into the trained word segmenter to obtain a word-metaphor list, and each word-metaphor is recorded as token1 to token n When predicting token1, the word list packet_current_tokenlist is input into the pre-trained large language model for calculation; when predicting the probability of token2, the word list packet_current_tokenlist + token1 is input into the pre-trained large language model for calculation. n When inputting the word list packet_current_tokenlist+token1+token2+…+token n-1 , and perform calculations.

[0069] S3. Build a pre-trained large language model; the pre-trained large language model includes: a multi-layer Transformer decoder, a multi-layer perceptron layer, and a softmax layer.

[0070] S4. Input the word list of the first subtask into the pre-trained large language model, and obtain word features through multi-layer Transformer decoder calculations; input the word features into the multi-layer perceptron layer, and predict through the softmax layer to obtain the probability of the next traffic word in the traffic packet; input the data of the traffic packet of the second subtask into the pre-trained large language model, and obtain word features through multi-layer Transformer decoder calculations; input the word features into the multi-layer perceptron layer, and predict through the softmax layer to obtain the probability of each traffic word in the next traffic packet; according to the probability of the next traffic word in the traffic packet and the probability of each traffic word in the next traffic packet, calculate the loss function and gradient update the parameters for training to obtain a trained large language model.

[0071] Optionally, based on the probability of the next traffic word in the traffic packet and the probability of each traffic word in the next traffic packet, a loss function is calculated and model parameters are updated according to the gradient to perform training to obtain a trained large language model, including:

[0072] According to the probability of the next traffic word in the traffic packet, the loss function is calculated, and the model parameters are updated according to the gradient for training to obtain the large language model trained for the first subtask; based on the large language model trained for the first subtask, according to the probability of each traffic word in the next traffic packet, the loss function is calculated, and the model parameters are updated according to the gradient for training to obtain the trained large language model.

[0073] Optionally, the data of the second subtask traffic packet is input into a pre-trained large language model, including:

[0074] According to the data of the traffic packet of the second subtask, two temporally adjacent traffic packets in the same flow are taken, and the word list of the previous traffic packet is input into the pre-trained large language model.

[0075] By learning these two subtasks, the large language model can generate content with a structure similar to the plaintext portion of traffic. Through its powerful learning and pattern mining capabilities, the large language model can learn similar characteristics across different patterns in traffic data. These similar characteristics across different patterns provide the model with relevant knowledge of the traffic domain.

[0076] Among them, the data of a traffic packet will form n-1 word list data with a length of k, which is used to predict word t2 to t n .

[0077] Optionally, the word list of the first subtask is sequentially input into a pre-trained large language model for pre-training, including:

[0078] According to the original position order of the traffic packets, the first k traffic words in the word list of the first subtask are input into the pre-trained large language model; the word list of the first subtask is recorded as When there are less than k tokens, the [pad] symbol is added at the end; when predicting the second token, if there is only one token in front, the input is t1,[pad],[pad]..[pad]; among them, the [pad] symbol does not participate in the calculation and is a filling symbol; when the front token exceeds k, the redundant part is directly truncated and the k tokens closest to the predicted token are retained.

[0079] Optionally, the loss function is expressed by the following formula (1):

[0080] (1)

[0081] in, Indicates traffic word; Represents the parameters of the pre-trained large language model; k represents the context window size, and the function p represents the pre-trained large language model; Indicates the probability value obtained.

[0082] Among them, the AdamW algorithm is used to optimize the loss function and update the model parameters.

[0083] S5. Construct instruction fine-tuning data; input the instruction fine-tuning data into the trained word segmenter to obtain a word list; input the word list into the trained large language model for prediction, and output the predicted result of the traffic label.

[0084] Optionally, build instructions to fine-tune the data, including:

[0085] Design prompt words corresponding to the task of detecting malicious encrypted traffic; obtain data of traffic packets for the task of detecting malicious encrypted traffic; wherein the data of the traffic packets is labeled data;

[0086] Among them, there are labeled data; normal traffic data and malicious traffic data; among them, the label of normal traffic data is normal; among them, the label of malicious traffic data is malicious.

[0087] The prompt word is spliced with the data of the traffic package to obtain the instruction fine-tuning data.

[0088] Among them, the prompt word prompt is Given the following traffic data <packet>thatcontains protocol fields, traffic features, and payloads. Please perform themalicious encrypted traffic detection task. Your output should be benign ormalicious. <packet>.

[0089] in, <packet>The symbol is used to help the model understand that the following data belongs to the traffic packet. Fine-tuning instructions are used to guide the large language model to output the correct traffic category, including: if it is normal traffic, the expected output label is benign, and if it is malicious traffic, the expected output label is malicious.

[0090] Optionally, the specific implementation process of S5 includes:

[0091] Design prompt words corresponding to the task of detecting malicious encrypted traffic; obtain data of traffic packets for the task of detecting malicious encrypted traffic; wherein the data of the traffic packets is labeled data;

[0092] Combine the prompt word with the data of the traffic package to obtain instruction fine-tuning data;

[0093] The instruction fine-tuning data is input into the pre-trained large language model for instruction fine-tuning, and tokenization is performed through the trained word segmenter to obtain a token list; the token list is input into the multi-layer Transformer decoder for calculation to obtain token features; the token features are input into the multi-layer perceptron layer, predicted through the softmax layer, and the probability distribution of the traffic label is output. The label with the highest probability is selected as the final traffic label.

[0094] Among them, the probability distribution of the output traffic label is selected, and the one with the highest probability in the distribution is selected as the output label.

[0095] Optionally, based on the output traffic label, a cross entropy loss function is used to calculate the difference between the predicted traffic label and the true label. The specific process of calculating the difference between the predicted traffic label and the true label is shown by the following formula (2):

[0096] (2)

[0097] in, The i-th instruction input model, if the label output by the model is consistent with the true label, ,otherwise , is the probability that the model predicts the traffic label

[0098] In one feasible implementation, a gradient descent algorithm is used to optimize the cross entropy loss function and update the model parameters. The use of LoRA technology can shorten the training time. The specific implementation process includes:

[0099] The process of updating model parameters during fine-tuning is recorded as , where w is the updated parameter and w0 is the original parameter, is the difference between the original parameters and the updated parameters. So we get The updated parameters can be obtained; among them, only the , the original parameters of the model can be frozen. The above formula can be rewritten as , where B represents the first low-rank matrix; A is the second low-rank matrix. Only the first and second low-rank matrices need to be updated to obtain the updated parameters. Assume that the dimension of w0 is , the dimension of the first low-rank matrix B is , the dimension of the second low-rank matrix A , is a number much smaller than d and k. The number of parameters that need to be updated is , currently only the number of parameters that need to be updated is ,and Very small, so Much smaller than , the amount of model parameters that need to be updated is greatly reduced, which improves the speed of model training, thereby obtaining a large language model that can detect malicious encrypted traffic.

[0100] The embodiment of the present invention first obtains encrypted traffic data in different modes on the Internet; removes the encrypted payload part of the encrypted traffic data to obtain the data of the first subtask traffic packet; divides the encrypted traffic data into traffic packets in the same flow and the same time interval according to the traffic quintuple and the time interval, removes the encrypted payload part of the traffic packet, and obtains the data of the second subtask traffic packet; inputs the data of the first subtask traffic packet into a trained word segmenter for word metamerism to obtain a word list of the first subtask; inputs the data of the second subtask traffic packet into a trained word segmenter for word metamerism to obtain a word list of the second subtask; secondly, constructs a pre-trained large language model; the pre-trained large language model includes: a multi-layer Transformer decoder, a multi-layer perceptron layer, and a softmax layer; inputs the word list of the first subtask into the pre-trained large language model, and The decoder calculates and obtains word-unit features; the word-unit features are input into the multi-layer perceptron layer, and predicted through the softmax layer to obtain the probability of the next traffic word in the traffic packet; the data of the second subtask traffic packet is input into the pre-trained large language model, and the word-unit features are obtained through the multi-layer Transformer decoder calculation; the word-unit features are input into the multi-layer perceptron layer, and predicted through the softmax layer to obtain the probability of each traffic word in the next traffic packet; according to the probability of the next traffic word in the traffic packet and the probability of each traffic word in the next traffic packet, the loss function is calculated, and the model parameters are updated according to the gradient for training to obtain a trained large language model; finally, the instruction fine-tuning data is constructed; the instruction fine-tuning data is input into the trained word segmenter to obtain a word-unit list; the word-unit list is input into the trained large language model for prediction, and the prediction result of the traffic label is output.

[0101] The present invention can improve the performance of the large language model for traffic detection by utilizing a large amount of original unlabeled data. When the present invention is used for encrypted traffic detection tasks, the large language model is first pre-trained for traffic domain adaptation, and then fine-tuned for specific tasks. Unlike TrafficLLM, the model that has undergone traffic domain adaptation pre-training tasks has a more powerful ability to handle traffic domain tasks than the native large language model. It will not result in poor classification results due to different traffic patterns, and there is no need to fine-tune different models for different traffic patterns. In specific downstream tasks, only a small amount of labeled data is needed to obtain a model with excellent performance by fine-tuning the model parameters.

[0102] Figure 3 This is a block diagram of a device for detecting anomalies in encrypted traffic based on a large language model according to an exemplary embodiment. The device is used in a method for detecting anomalies in encrypted traffic based on a large language model. Figure 3 The device includes an acquisition unit 310, a processing unit 320, a construction unit 330, a training unit 340, and a prediction unit 350.

[0103] The acquisition unit 310 is configured to acquire encrypted traffic data in different modes on the Internet; remove the encrypted payload portion of the encrypted traffic data to obtain data of traffic packets of the first subtask; divide the encrypted traffic data into traffic packets in the same flow and the same time interval according to the traffic quintuple and time interval, remove the encrypted payload portion of the traffic packets, and obtain data of traffic packets of the second subtask;

[0104] The processing unit 320 is configured to input the data of the traffic packet of the first subtask into the trained word segmenter for lemmatization to obtain a word-gram list of the first subtask; and input the data of the traffic packet of the second subtask into the trained word segmenter for lemmatization to obtain a word-gram list of the second subtask;

[0105] A construction unit 330 is used to construct a pre-trained large language model; the pre-trained large language model includes: a multi-layer Transformer decoder, a multi-layer perceptron layer, and a softmax layer;

[0106] The training unit 340 is used to input the word list of the first subtask into the pre-trained large language model, calculate through the multi-layer Transformer decoder to obtain word-word features; input the word-word features into the multi-layer perceptron layer, predict through the softmax layer, and obtain the probability of the next traffic word in the traffic packet; input the data of the traffic packet of the second subtask into the pre-trained large language model, calculate through the multi-layer Transformer decoder to obtain word-word features; input the word-word features into the multi-layer perceptron layer, predict through the softmax layer, and obtain the probability of each traffic word in the next traffic packet; based on the probability of the next traffic word in the traffic packet and the probability of each traffic word in the next traffic packet, calculate the loss function, update the model parameters according to the gradient, and perform training to obtain a trained large language model;

[0107] The prediction unit 350 is used to construct instruction fine-tuning data; input the instruction fine-tuning data into the trained word segmenter to obtain a word list; input the word list into the trained large language model for prediction, and output the prediction result of the traffic label.

[0108] Optionally, the step of sequentially inputting the word list of the first subtask into a pre-trained large language model for pre-training includes:

[0109] According to the original position order of the traffic packets, the first k traffic words in the word list of the first subtask are input into the pre-trained large language model; the word list of the first subtask is recorded as When there are less than k tokens, a [pad] symbol is added at the end. When predicting the second token, if there is only one token in front, the input is t1,[pad],[pad]..[pad]. Among them, the [pad] symbol does not participate in the calculation and is a padding symbol. When the front token exceeds k, the redundant part is directly truncated and the k tokens closest to the predicted token are retained.

[0110] Optionally, the loss function is expressed by the following formula (1):

[0111] (1)

[0112] in, Indicates traffic word; Represents the parameters of the pre-trained large language model; k represents the context window size, and the function P represents the pre-trained large language model; Indicates the probability value obtained.

[0113] Optionally, inputting the data of the second subtask traffic packet into a pre-trained large language model includes:

[0114] According to the data of the traffic packet of the second subtask, two temporally adjacent traffic packets in the same flow are taken, and the word list of the previous traffic packet is input into the pre-trained large language model.

[0115] Optionally, the training is performed by calculating a loss function based on the probability of the next traffic word in the traffic packet and the probability of each traffic word in the next traffic packet, and updating model parameters according to the gradient to obtain a trained large language model, including:

[0116] According to the probability of the next traffic word in the traffic packet, the loss function is calculated, and the model parameters are updated according to the gradient for training to obtain the large language model trained for the first subtask; based on the large language model trained for the first subtask, according to the probability of each traffic word in the next traffic packet, the loss function is calculated, and the model parameters are updated according to the gradient for training to obtain the trained large language model.

[0117] Optionally, the prediction unit 350 is configured to:

[0118] Design prompt words corresponding to the task of detecting malicious encrypted traffic; obtain data of traffic packets for the task of detecting malicious encrypted traffic; wherein the data of the traffic packets is labeled data;

[0119] Combine the prompt word with the data of the traffic package to obtain instruction fine-tuning data;

[0120] The instruction fine-tuning data is input into the pre-trained large language model for instruction fine-tuning, and tokenization is performed through the trained word segmenter to obtain a token list; the token list is input into the multi-layer Transformer decoder for calculation to obtain token features; the token features are input into the multi-layer perceptron layer, predicted through the softmax layer, and the probability distribution of the traffic label is output. The label with the highest probability is selected as the final traffic label.

[0121] Optionally, the cross entropy loss function is used to calculate the difference between the predicted traffic label and the true label based on the output traffic label, wherein the specific process of calculating the difference between the predicted traffic label and the true label is shown by the following formula (2):

[0122] (2)

[0123] in, The i-th instruction input model, if the label output by the model is consistent with the true label, ,otherwise , is the probability that the model predicts the traffic label

[0124] The embodiment of the present invention first obtains encrypted traffic data in different modes on the Internet; removes the encrypted payload part of the encrypted traffic data to obtain the data of the first subtask traffic packet; divides the encrypted traffic data into traffic packets in the same flow and the same time interval according to the traffic quintuple and the time interval, removes the encrypted payload part of the traffic packet, and obtains the data of the second subtask traffic packet; inputs the data of the first subtask traffic packet into a trained word segmenter for word metamerism to obtain a word list of the first subtask; inputs the data of the second subtask traffic packet into a trained word segmenter for word metamerism to obtain a word list of the second subtask; secondly, constructs a pre-trained large language model; the pre-trained large language model includes: a multi-layer Transformer decoder, a multi-layer perceptron layer, and a softmax layer; inputs the word list of the first subtask into the pre-trained large language model, and The decoder calculates and obtains word-unit features; the word-unit features are input into the multi-layer perceptron layer, and predicted through the softmax layer to obtain the probability of the next traffic word in the traffic packet; the data of the second subtask traffic packet is input into the pre-trained large language model, and the word-unit features are obtained through the multi-layer Transformer decoder calculation; the word-unit features are input into the multi-layer perceptron layer, and predicted through the softmax layer to obtain the probability of each traffic word in the next traffic packet; according to the probability of the next traffic word in the traffic packet and the probability of each traffic word in the next traffic packet, the loss function is calculated, and the model parameters are updated according to the gradient for training to obtain a trained large language model; finally, the instruction fine-tuning data is constructed; the instruction fine-tuning data is input into the trained word segmenter to obtain a word-unit list; the word-unit list is input into the trained large language model for prediction, and the prediction result of the traffic label is output.

[0125] The present invention can improve the performance of the large language model for traffic detection by utilizing a large amount of original unlabeled data. When the present invention is used for encrypted traffic detection tasks, the large language model is first pre-trained for traffic domain adaptation, and then fine-tuned for specific tasks. Unlike TrafficLLM, the model that has undergone traffic domain adaptation pre-training tasks has a more powerful ability to handle traffic domain tasks than the native large language model. It will not result in poor classification results due to different traffic patterns, and there is no need to fine-tune different models for different traffic patterns. In specific downstream tasks, only a small amount of labeled data is needed to obtain a model with excellent performance by fine-tuning the model parameters.

[0126] Figure 4 is a structural diagram of a device for detecting anomalies in encrypted traffic based on a large language model provided by an embodiment of the present invention. Figure 4 As shown, the device for detecting anomalies in encrypted traffic based on a large language model may include the above Figure 3 The apparatus for detecting anomalies in encrypted traffic based on a large language model is shown. Optionally, the device 410 for detecting anomalies in encrypted traffic based on a large language model may include a first processor 2001 .

[0127] Optionally, the device 410 for performing encrypted traffic anomaly detection based on a large language model may further include a memory 2002 and a transceiver 2003 .

[0128] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0129] The following combination Figure 4 The components of the device 410 for detecting anomalies in encrypted traffic based on a large language model are described in detail.

[0130] The first processor 2001 is the control center of the device 410 for performing encrypted traffic anomaly detection based on a large language model. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), or application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs) or one or more field programmable gate arrays (FPGAs).

[0131] Optionally, the first processor 2001 can perform various functions of the device 410 for detecting anomalies in encrypted traffic based on a large language model by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002.

[0132] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 4 CPU0 and CPU1 are shown in FIG.

[0133] In a specific implementation, as an embodiment, the device 410 for detecting anomalies in encrypted traffic based on a large language model may also include multiple processors, such as Figure 4 1 and 2. The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). A processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0134] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled by the first processor 2001 for execution. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0135] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, a random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including a compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently and be accessed through the interface circuit ( Figure 4 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0136] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0137] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 4 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0138] Optionally, the transceiver 2003 may be integrated with the first processor 2001 or may exist independently and be detected through the interface circuit ( Figure 4 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0139] It should be noted that Figure 4 The structure of the device 410 for detecting anomalies in encrypted traffic based on a large language model shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0140] In addition, the technical effects of the device 410 for detecting anomalies in encrypted traffic based on a large language model can refer to the technical effects of the method for detecting anomalies in encrypted traffic based on a large language model described in the above method embodiment, and will not be repeated here.

[0141] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc.

[0142] It should also be understood that the memory in the embodiments of the present invention may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0143] The above embodiments can be implemented in whole or in part via software, hardware (e.g., circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present invention are fully or partially performed. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired means (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. The available medium can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0144] It should be understood that the term "and / or" in this document simply describes an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. A and B can be singular or plural. Furthermore, the character " / " in this document generally indicates an "or" relationship between the related objects, but it can also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0145] In this disclosure, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, "at least one of a, b, or c" can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0146] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0147] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0148] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0149] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of the device or unit, which can be electrical, mechanical or other forms.

[0150] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0151] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0152] If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical disks.

[0153] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.< / packet> < / packet> < / packet>

Claims

1. A method for detecting anomalies in encrypted traffic based on a large language model, characterized in that: The method comprises: S1. Obtain encrypted traffic data in different modes on the Internet; remove the encrypted payload portion of the encrypted traffic data to obtain data of the traffic packet of the first subtask; divide the encrypted traffic data into traffic packets in the same flow and the same time interval according to the traffic quintuple and time interval, remove the encrypted payload portion of the traffic packet, and obtain data of the traffic packet of the second subtask; S2. Input the data of the traffic packet of the first subtask into the trained word segmenter for lemmatization to obtain a word-gram list of the first subtask; input the data of the traffic packet of the second subtask into the trained word segmenter for lemmatization to obtain a word-gram list of the second subtask; S3. Build a pre-trained large language model. The pre-trained large language model includes: a multi-layer Transformer decoder, a multi-layer perceptron layer, and a softmax layer. S4. Input the word list of the first subtask into the pre-trained large language model, calculate it through the multi-layer Transformer decoder to obtain word-word features; input the word-word features into the multi-layer perceptron layer, predict it through the softmax layer, and obtain the probability of the next traffic word in the traffic packet; input the data of the traffic packet of the second subtask into the pre-trained large language model, calculate it through the multi-layer Transformer decoder to obtain word-word features; input the word-word features into the multi-layer perceptron layer, predict it through the softmax layer, and obtain the probability of each traffic word in the next traffic packet; according to the probability of the next traffic word in the traffic packet and the probability of each traffic word in the next traffic packet, calculate the loss function, update the model parameters according to the gradient, and train to obtain a trained large language model; S5. Construct instruction fine-tuning data; input the instruction fine-tuning data into the trained word segmenter to obtain a word list; input the word list into the trained large language model for prediction, and output the predicted result of the traffic label.

2. The method for detecting anomalies in encrypted traffic based on a large language model according to claim 1, characterized in that: The step of sequentially inputting the word list of the first subtask into the pre-trained large language model for pre-training includes: According to the original position order of the traffic packets, the first k traffic words in the word list of the first subtask are input into the pre-trained large language model; the word list of the first subtask is recorded as When there are less than k tokens, a [pad] symbol is added at the end. When predicting the second token, if there is only one token in front, the input is t1,[pad],[pad]..[pad]. Among them, the [pad] symbol does not participate in the calculation and is a padding symbol. When the front token exceeds k, the redundant part is directly truncated and the k tokens closest to the predicted token are retained.

3. The method for detecting anomalies in encrypted traffic based on a large language model according to claim 1, characterized in that: The loss function is expressed by the following formula (1): （1） in, Indicates traffic word; Represents the parameters of the pre-trained large language model; k represents the context window size, and the function P represents the pre-trained large language model; Indicates the probability value obtained.

4. The method for detecting anomalies in encrypted traffic based on a large language model according to claim 1, characterized in that: Inputting the data of the second subtask traffic packet into the pre-trained large language model includes: Based on the data of the traffic packet of the second subtask, two temporally adjacent traffic packets in the same stream are taken; the word list of the previous traffic packet is input into the pre-trained large language model.

5. The method for detecting anomalies in encrypted traffic based on a large language model according to claim 1, characterized in that: The method of obtaining a trained large language model by calculating the loss function based on the probability of the next traffic word in the traffic packet and the probability of each traffic word in the next traffic packet and updating the model parameters according to the gradient is as follows: According to the probability of the next traffic word in the traffic packet, the loss function is calculated, and the model parameters are updated according to the gradient for training to obtain the large language model trained for the first subtask; based on the large language model trained for the first subtask, according to the probability of each traffic word in the next traffic packet, the loss function is calculated, and the model parameters are updated according to the gradient for training to obtain the trained large language model.

6. The method for detecting anomalies in encrypted traffic based on a large language model according to claim 1, characterized in that: The step S5 constructs instruction fine-tuning data; inputs the instruction fine-tuning data into the trained word segmenter to obtain a word unit list; Input the word list into the pre-trained large language model for prediction to obtain the traffic label prediction results, including: Design prompt words corresponding to the task of detecting malicious encrypted traffic; obtain data of traffic packets for the task of detecting malicious encrypted traffic; wherein the data of the traffic packets is labeled data; Combine the prompt word with the data of the traffic package to obtain the instruction fine-tuning data; The instruction fine-tuning data is input into the pre-trained large language model for instruction fine-tuning, and tokenization is performed through the trained word segmenter to obtain a token list; the token list is input into the multi-layer Transformer decoder for calculation to obtain token features; the token features are input into the multi-layer perceptron layer, predicted through the softmax layer, and the probability distribution of the traffic label is output. The label with the highest probability is selected as the final traffic label.

7. The method for detecting anomalies in encrypted traffic based on a large language model according to claim 6, characterized in that: According to the output traffic label, the cross entropy loss function is used to calculate the difference between the predicted traffic label and the true label. The specific process of calculating the difference between the predicted traffic label and the true label is shown by the following formula (2): （2） in, The i-th instruction input model, if the label output by the model is consistent with the true label, ,otherwise , is the probability of the model predicting the traffic label.

8. A device for detecting anomalies in encrypted traffic based on a large language model, wherein the device is used to implement the method for detecting anomalies in encrypted traffic based on a large language model as claimed in any one of claims 1 to 6, characterized in that: The device comprises: An acquisition unit is configured to acquire encrypted traffic data in different modes on the Internet; remove the encrypted payload portion of the encrypted traffic data to obtain data of traffic packets of the first subtask; divide the encrypted traffic data into traffic packets in the same flow and the same time interval according to the traffic quintuple and the time interval, remove the encrypted payload portion of the traffic packets, and obtain data of traffic packets of the second subtask; a processing unit, configured to input the data of the first subtask traffic packet into the trained word segmenter for lemma conversion to obtain a word gram list of the first subtask; and input the data of the second subtask traffic packet into the trained word segmenter for lemma conversion to obtain a word gram list of the second subtask; The construction unit is used to build a pre-trained large language model; the pre-trained large language model includes: a multi-layer Transformer decoder, a multi-layer perceptron layer, and a softmax layer; The training unit is used to input the word list of the first subtask into the pre-trained large language model, calculate through the multi-layer Transformer decoder to obtain word-word features; input the word-word features into the multi-layer perceptron layer, predict through the softmax layer, and obtain the probability of the next traffic word in the traffic packet; input the data of the traffic packet of the second subtask into the pre-trained large language model, calculate through the multi-layer Transformer decoder to obtain word-word features; input the word-word features into the multi-layer perceptron layer, predict through the softmax layer, and obtain the probability of each traffic word in the next traffic packet; according to the probability of the next traffic word in the traffic packet and the probability of each traffic word in the next traffic packet, the loss function is calculated, and the model parameters are updated according to the gradient to perform training to obtain a trained large language model; The prediction unit is used to construct instruction fine-tuning data; input the instruction fine-tuning data into the trained word segmenter to obtain a word list; input the word list into the trained large language model for prediction, and output the predicted result of the traffic label.

9. A device for detecting anomalies in encrypted traffic based on a large language model, characterized in that: The device for detecting anomalies in encrypted traffic based on a large language model includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images