A lightweight encrypted traffic analysis method and system based on distillation characterization model

Abstract

The present invention relates to a lightweight encrypted traffic analysis method and system based on a distillation representation model, comprising: obtaining an original data set of encrypted network traffic, performing unified processing on the data in the original data set and then dividing it according to a set ratio to generate various sample data sets; inputting pre-trained samples into a BERT model based on representation learning for pre-training to obtain a pre-trained model, inputting fine-tuning samples into the pre-training model for detailed task training to obtain a traffic classification teacher model; using a knowledge distillation method of multi-layer jump fusion learning to extract teacher information of the traffic classification teacher model, and inputting fine-tuning samples into a lightweight traffic analysis model for training; inputting test samples into the lightweight traffic analysis model to realize end-to-end target network encrypted traffic classification tasks; while ensuring classification accuracy, reducing analysis resource requirements, and facilitating the realization of online low-consumption encrypted traffic classification functions in delay-sensitive resource-constrained network scenario nodes.

Description

Technical Field

[0001] The present invention relates to the field of Internet data analysis, and in particular to a lightweight encrypted traffic analysis method and system based on a distillation characterization model. Background Art

[0002] With the vigorous development of computer network technology, the Internet has been integrated into all aspects of production and life. Network traffic is closely connected with hosts, networks, and applications, forming a complex system with multiple factors. With the rapid growth of the Internet, network traffic analysis is of great significance for ensuring QoS, identifying abnormal traffic, and improving the quality of the network environment. However, as encryption technology gradually becomes the mainstream of the times, traffic no longer has the same readability as plain text. Traditional technologies based on port numbers, fingerprint signatures, and deep packet analysis have gradually lost their effectiveness, affecting the effectiveness and performance of traditional network traffic detection methods. Researchers have begun to turn to the research of encrypted traffic analysis technology based on deep learning.

[0003] As an effective quality optimization technology, network encrypted traffic analysis can use network encrypted traffic to determine the communication protocols and application types used by both parties in communication, enhance the understanding of the operation status of its own main network and service network, and provide response decisions for relevant managers. At present, researchers at home and abroad have conducted systematic and in-depth research on network encrypted traffic analysis, but the existing research work has certain defects and problems: 1. Traditional encrypted traffic analysis is based on pre-configured rules to identify the target to be tested by analyzing the characteristics of the traffic. However, in the context of the rapid update and iteration of encryption technology, rule-based recognition can achieve higher detection accuracy and stability, but it cannot effectively cope with emerging encryption protocol technologies. 2. Some research works based on deep learning mostly rely on large-scale labeled data sets. At the same time, the label distribution of the data set will greatly affect the functionality and effectiveness of the analysis system, greatly reducing the application of such methods in current real-world scenarios. 3. With the continuous improvement of hacker technology, some network attack initiators have begun to interfere with and circumvent existing encrypted traffic analysis methods by injecting noise into encrypted traffic (such as adding messages generated in benign applications). 4. Some encrypted traffic analysis methods based on deep learning have certain task orientation and scenario limitations. As network difficulties emerge in an endless stream and the WEB environment becomes increasingly complex, users have different system functions and structures to choose from when facing different functional requirements, which poses a great challenge to the operation and maintenance capabilities of network managers. 5. Although the existing encrypted traffic analysis methods based on deep learning can effectively identify traffic information, they are not suitable for actual network nodes with limited resources due to high processing overhead and high computing space resource requirements. As a result, most deep learning-based methods can only be deployed offline and cannot perform online analysis, which has great application limitations. 6. Most of the existing encrypted traffic analysis methods can only detect and identify past traffic conditions, and there is a great lack of research on the detection of real-time arriving traffic. In actual scenarios, reports on online traffic analysis can assist administrators or users in making forward-looking predictions and grasping network quality in real time.

[0004] In summary, how to achieve online real-time, highly robust, and highly precise encrypted traffic analysis in an actual resource-constrained network environment is an urgent problem to be solved. Summary of the invention

[0005] In view of the technical problems existing in the prior art, the present invention provides a lightweight encrypted traffic analysis method based on a distillation characterization model to solve the problem that the existing Internet encrypted traffic analysis method cannot perform encrypted traffic analysis online in real time with high precision and high robustness in actual resource-constrained network scenarios.

[0006] According to a first aspect of the present invention, a lightweight encrypted traffic analysis method based on a distillation characterization model is provided, comprising: step 1, obtaining an original data set of encrypted network traffic, performing unified processing on the data in the original data set, and then dividing the data according to a set ratio to generate various sample data sets, wherein each sample includes: an initial pre-training sample, an initial fine-tuning sample, and an initial test sample;

[0007] Step 2: inputting the initial pre-training sample into a BERT model based on representation learning to perform pre-training to obtain a pre-training model, wherein the pre-training process includes: tokenizing the initial pre-training sample, embedding the mark information, and performing multi-level coding learning;

[0008] Step 3, inputting the initial fine-tuning sample into the pre-training model for detailed task training to obtain a traffic classification teacher model;

[0009] Step 4, using a knowledge distillation method of multi-layer jump fusion learning to extract teacher information of the traffic classification teacher model, using the traffic classification teacher model as the teacher model to construct a student model as a lightweight traffic analysis model; combining the teacher information, inputting the initial fine-tuning sample into the lightweight traffic analysis model for training;

[0010] Step 5: Input the initial test sample into the trained lightweight traffic analysis model to achieve an end-to-end target network encrypted traffic classification task.

[0011] Based on the above technical solution, the present invention can also make the following improvements.

[0012] Optionally, the step 1 includes:

[0013] Step 101, using a network sniffing tool to access accessible IP, URL and domain name to capture real-time traffic, and process to obtain a binary PCAP file original data set;

[0014] Step 102, classifying the PCAP file into various dialog flow sets according to whether the five-tuple corresponds, and dividing the data in each dialog flow set into various flow data segments according to the time sequence and the five-tuple standard; the five-tuple is {IP source address, IP destination address, source port, destination port, protocol}, and in the process of classifying and judging whether the five-tuple corresponds, the IP source address and the IP destination address can be interchanged, and the source port and the destination port can be interchanged;

[0015] Step 103: Divide the stream data segment according to a set ratio to generate various sample data sets.

[0016] Optionally, the data in the initial pre-training sample is a hexadecimal stream data segment, and the process of tokenizing the stream data segment in step 2 includes:

[0017] Step 201, the stream data segment is cut and decomposed into adjacent byte pairs to obtain a binary token data stream.

[0018] Optionally, the process of embedding the logo information in step 2 includes:

[0019] Step 202, embed the flag [CLS] into the header of each segment of the token data stream; unify the length of all the token data streams, and align them with the flag [PAD]; randomly select M tokens and replace them with the flag [MASK];

[0020] Step 203, position embedding the token data stream after the mark is embedded, and assigning a position number to each token.

[0021] Optionally, the process of performing multi-level coding learning in step 2 includes:

[0022] Step 204: Input the data segment X after the marker information is embedded into the first encoder for linear transformation to obtain vectors K, Q and V, where K = W K X, Q = W Q X, V = W V X, W K , W Q and W V are all linear parameters;

[0023] Step 205, input the linearly transformed vectors K, Q and V into the multi-head self-attention network, and the calculation result of the multi-head self-attention network is: ATT=att1+att2+L+att H ; d is the column dimension of the input vector, H is the number of multi-head attention heads;

[0024] Step 206: input the calculation result ATT into the fully connected layer network to obtain an output result h 1 =max(0,W1ATT+b1)W2+b2, where W1, b1, W2 and b2 are the fully connected layer parameters, and max(0,x) is the RELU activation function;

[0025] Step 207: h 1 Input into the second encoder, and execute steps 204 to 206;

[0026] Step 208, repeat step 207 Z times, where Z is the number of remaining encoders, and output the final vector hZ .

[0027] Optionally, the pre-training process in step 2 further includes:

[0028] For the final vector h Z Performing Mask task training to obtain the pre-trained model based on representation learning;

[0029] The loss function of the Mask task training is: θ is the parameter set of the multi-level encoder, MASK i Indicates that the i-th position of the token data stream is marked as [MASK], and P indicates that in h Z With the parameter θ, the conditional probability that the i-th position of the token data stream is the flag [MASK].

[0030] Optionally, step 4 includes:

[0031] Step 401, learning the encoder output information of the traffic analysis teacher network every two layers, and calculating the jump loss function Where N is the number of initial fine-tuning samples, M is the number of encoder layers in the student model, i and j are the serial numbers of the initial fine-tuning samples and the encoder layers in the student model, respectively. PT is the set of layers for jump learning in the teacher network, the superscript s represents the lightweight traffic analysis model, and the superscript t represents the teacher model;

[0032] Step 402, calculate the cross entropy loss function Among them, C is the category label set, c is the category label, θ s is the parameter of the lightweight traffic analysis model, P s Indicates that in x i With θ s Under the parameters, the output y of the student model i = conditional probability of c;

[0033] Step 404, calculate the learning loss function P t Indicates that in x i With θ t Under the parameters, the output y of the teacher model i = the conditional probability of c.

[0034] Step 403: Add the jump loss function, the cross entropy loss function and the learning loss function according to a set ratio to obtain a target loss function L target =(1-α)L CE +αL D +βL JT , α and β are the set proportional coefficients respectively;

[0035] Step 404: Use the target loss function as the loss function of the lightweight traffic analysis model, and input the initial fine-tuning sample into the lightweight traffic analysis model for training.

[0036] According to a second aspect of the present invention, there is provided a lightweight encrypted traffic analysis system based on a distillation representation model, comprising: a network traffic collection module, a network traffic data set construction module, a pre-training module based on representation learning, a traffic analysis teacher network module and a lightweight traffic analysis module;

[0037] The network traffic collection module is used to obtain the encrypted network traffic original data set;

[0038] The network traffic data set construction module is used to generate various sample data sets after unifying the data in the original data set and dividing them according to a set ratio. Each sample includes: an initial pre-training sample, an initial fine-tuning sample and an initial test sample, and labels are added to the data in the initial fine-tuning sample and the initial test sample;

[0039] The pre-training module based on representation learning is used to input the initial pre-training sample into the BERT model based on representation learning for pre-training to obtain a pre-training model, wherein the pre-training process includes: tokenizing the initial pre-training sample, embedding the mark information, and multi-level coding learning;

[0040] The traffic analysis teacher network module is used to input the initial fine-tuning sample into the pre-training model for detailed task training to obtain a traffic classification teacher model;

[0041] The lightweight traffic analysis module is used to extract the teacher information of the traffic classification teacher model by adopting the knowledge distillation method of multi-layer jump fusion learning, and construct a student model as a lightweight traffic analysis model with the traffic classification teacher model as the teacher model; in combination with the teacher information, the initial fine-tuning sample is input into the lightweight traffic analysis model for training; the initial test sample is input into the trained lightweight traffic analysis model to achieve end-to-end target network encrypted traffic classification task.

[0042] According to a third aspect of the present invention, there is provided an electronic device comprising a memory and a processor, wherein the processor is used to implement the steps of a lightweight encrypted traffic analysis method based on a distillation characterization model when executing a computer management program stored in the memory.

[0043] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer management program is stored. When the computer management program is executed by a processor, the steps of a lightweight encrypted traffic analysis method based on a distillation characterization model are implemented.

[0044] The present invention provides a lightweight encrypted traffic analysis method, system, electronic device and storage medium based on a distillation characterization model, and its beneficial effects include:

[0045] (1) The present invention adopts an end-to-end method to analyze network encrypted traffic and uses the BERT representation learning model to describe the traffic, which enhances the representation capability of traffic data and has low feature redundancy, thereby improving the accuracy and throughput of model detection prediction effects.

[0046] (2) The present invention uses a large-scale unlabeled data set for system pre-training, and then uses a small amount of labeled data for specific task fine-tuning, avoiding the problem of offset caused by the label distribution of the data set and the difficulty in identifying new protocols, and also alleviating the additional operations and complex engineering of operation and maintenance personnel in constructing large-scale labels. It is in line with the current trend of systematization and integration of network traffic engineering, and is practical and universal.

[0047] (3) The target object of the characterization learning of encrypted traffic in the present invention is the flow-level message. The extracted and learned features represent the overall wide-area information of the data stream sequence and are not easily disturbed by the injected noise messages, thereby realizing the robust detection and prediction of the model and having robustness.

[0048] (4) The present invention uses a pre-training-fine-tuning model to adaptively generalize a model in different task scenarios, reducing the use of redundant models, which is consistent with the industry's trend of streamlining and integration.

[0049] (5) The present invention combines the knowledge distillation method of multi-layer jump fusion learning to reduce and eliminate redundant system parameters, retain important information to realize the analysis function, and greatly reduce resource requirements and space occupancy while ensuring high accuracy, thereby improving the processing and analysis speed, which meets the vision requirements of high standards and high efficiency in industrial production.

[0050] (6) The present invention uses a lightweight model to implement the online analysis function of encrypted traffic, which can assist administrators or users in making forward-looking predictions and grasping network quality in real time, conforming to the needs of the production and manufacturing industry to prepare for the present era. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A flowchart of a lightweight encrypted traffic analysis based on a distillation characterization model provided by the present invention;

[0052] Figure 2 A structural block diagram of a lightweight encrypted traffic analysis based on a distillation characterization model provided by the present invention;

[0053] Figure 3A schematic diagram of the hardware structure of a possible electronic device provided by the present invention;

[0054] Figure 4 A schematic diagram of the hardware structure of a possible computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION

[0055] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0056] Figure 1 A flowchart of an embodiment of a lightweight encrypted traffic analysis method based on a distillation characterization model provided by the present invention is shown in FIG. Figure 1 As shown, the analysis method includes:

[0057] Step 1: obtain the original data set of encrypted network traffic, unify the data in the original data set, and divide it according to the set ratio to generate various sample data sets. Each sample includes: initial pre-training sample, initial fine-tuning sample and initial test sample.

[0058] Step 2: Input the initial pre-training samples into the BERT (Bidirectional Encoder Representation from Transformers) model based on representation learning for pre-training to obtain a pre-training model. The pre-training process includes: tokenizing the initial pre-training samples, embedding the sign information, and multi-level encoding learning.

[0059] Step 3: Input the initial fine-tuning samples into the pre-training model for detailed task training to obtain the traffic classification teacher model.

[0060] Step 4: Use the knowledge distillation method of multi-layer jump fusion learning to extract the teacher information of the traffic classification teacher model, and use the traffic classification teacher model as the teacher model to build a student model as a lightweight traffic analysis model; combined with the teacher information, input the initial fine-tuning sample into the lightweight traffic analysis model for training.

[0061] Step 5: Input the initial test sample into the trained lightweight traffic analysis model to achieve the end-to-end target network encrypted traffic classification task.

[0062] The present invention provides a lightweight encrypted traffic analysis method based on a distillation characterization model, which reduces the demand for analysis resources while ensuring classification accuracy, helps to realize online low-consumption encrypted traffic classification functions in delay-sensitive resource-constrained network scenario nodes, and solves the problem that existing Internet encrypted traffic analysis methods cannot perform online, real-time, high-precision, and high-robust encrypted traffic analysis in actual resource-constrained network scenarios.

[0063] Example 1

[0064] Embodiment 1 provided by the present invention is an embodiment of a lightweight encrypted traffic analysis method based on a distillation characterization model provided by the present invention, combined with Figure 1 It can be seen that the embodiments include:

[0065] Step 1: obtain the original data set of encrypted network traffic, unify the data in the original data set, and divide it according to the set ratio to generate various sample data sets. Each sample includes: initial pre-training sample, initial fine-tuning sample and initial test sample.

[0066] In a possible implementation manner, step 1 includes:

[0067] Step 101, using a network sniffing tool to access the reachable IP, URL (Uniform Resource Locator) and domain name to capture real-time traffic, and process it to obtain a binary PCAP (Packet Capture) file original data set.

[0068] Step 102, classify the PCAP file into various dialog flow sets according to whether the five-tuple corresponds, and divide the data in each dialog flow set into various flow data segments according to the time sequence and its five-tuple standard; the five-tuple is {IP source address, IP destination address, source port, destination port, protocol}, and in the process of classifying and judging whether the five-tuple corresponds, the IP source address and the IP destination address can be interchanged, and the source port and the destination port can be interchanged.

[0069] Step 103, dividing the stream data segment according to a set ratio to generate various sample data sets.

[0070] In a specific implementation, the stream data segments can be divided into initial pre-training samples, initial fine-tuning samples and initial test samples in a ratio of 7:2:1, wherein labels are added to the data in the initial fine-tuning samples and the initial test samples, and the initial pre-training samples are not processed.

[0071] Step 2: Input the initial pre-training samples into the BERT model based on representation learning for pre-training to obtain a pre-training model. The pre-training process includes: tokenizing the initial pre-training samples, embedding the landmark information, and learning multi-level encoding.

[0072] In a possible implementation manner, the data in the initial pre-training sample is a hexadecimal stream data segment, and the process of tokenizing the stream data segment in step 2 includes:

[0073] Step 201, the stream data segment is cut and decomposed into adjacent byte pairs to obtain a binary token data stream.

[0074] In a possible implementation manner, the process of embedding the mark information in step 2 includes:

[0075] Step 202, embed the flag [CLS] into the header of each token data stream; unify the length of all token data streams and align them with the flag [PAD]; randomly select M tokens and replace them with the flag [MASK].

[0076] Step 203, position embedding is performed on the token data stream after the logo is embedded, and a position number is assigned to each token to facilitate the model to learn its timing information. The number is a sequence of continuous positive integers starting from 1.

[0077] In a possible implementation manner, the process of performing multi-level coding learning in step 2 includes:

[0078] Step 204: Input the data segment X after the marker information is embedded into the first encoder for linear transformation to obtain vectors K, Q and V, where K = W K X, Q = W Q X, V = W V X, W K , W Q and W V are all linear parameters.

[0079] Step 205: Input the linearly transformed vectors K, Q and V into the multi-head self-attention network. The calculation result of the multi-head self-attention network is: ATT = att1 + att2 + L + att H ; d is the column dimension of the input vector, and H is the number of multi-head attention heads.

[0080] Step 206: Input the calculation result ATT into the fully connected layer network to obtain the output result h 1 =max(0,W1ATT+b1)W2+b2, where W1, b1, W2 and b2 are the fully connected layer parameters, and max(0,x) is the RELU activation function.

[0081] Step 207: h 1 Input into the second encoder and execute steps 204 to 206.

[0082] Step 208, repeat step 207 Z times, where Z is the number of remaining encoders, and output the final vector h Z .

[0083] In a possible implementation manner, the pre-training process in step 2 further includes:

[0084] For the final vector h Z Mask task training is performed to obtain a pre-trained model based on representation learning.

[0085] The loss function of Mask task training is θ is the parameter set of the multi-level encoder, MASK i Indicates that the i-th position of the token data stream is marked as [MASK], and P indicates that in h Z With the parameter θ, the conditional probability that the i-th position of the token data stream is the flag [MASK].

[0086] Step 3: Input the initial fine-tuning samples into the pre-training model for detailed task training to obtain the traffic classification teacher model.

[0087] In a possible implementation method, step 4 uses a knowledge distillation method of multi-layer jump fusion learning to extract teacher information of the traffic classification teacher model, and uses the traffic classification teacher model as the teacher model to construct a student model as a lightweight traffic analysis model; combined with the teacher information, the initial fine-tuning sample is input into the lightweight traffic analysis model for training.

[0088] Step 4 includes:

[0089] Step 401, every two layers of learning flow analysis, the encoder output information of the teacher network is calculated to calculate the jump loss function Among them, N is the number of initial fine-tuning samples, and M is the number of encoder layers in the student model, that is, i and j are the serial numbers of the encoder layers in the initial fine-tuning sample and the student model, respectively. PT is the set of layers for jump learning in the teacher network, the superscript s represents the lightweight traffic analysis model, and the superscript t represents the teacher model.

[0090] Step 402, calculate the cross entropy loss function Among them, C is the category label set, c is the category label, θ s is the parameter of the lightweight traffic analysis model, P s Indicates that in x i With θ s Under the parameters, the output y of the student model i = conditional probability of c (the superscript s represents the student model - student).

[0091] Step 404, calculate the learning loss function P t Indicates that in x i With θ t Under the parameters, the output y of the teacher model i = conditional probability of c (the superscript t represents the teacher model - teacher).

[0092] Step 403: Add the jump loss function, the cross entropy loss function and the learning loss function according to the set ratio to obtain the target loss function L target =(1-α)L CE +αL D +βL JT , α and β are the set proportional coefficients respectively.

[0093] Step 404, using the target loss function as the loss function of the lightweight traffic analysis model, and inputting the initial fine-tuning sample into the lightweight traffic analysis model for training.

[0094] Step 5: Input the initial test sample into the trained lightweight traffic analysis model to achieve the end-to-end target network encrypted traffic classification task.

[0095] Example 2

[0096] Embodiment 2 provided by the present invention is an embodiment of a lightweight encrypted traffic analysis system based on a distillation characterization model provided by the present invention. Figure 2 A structural diagram of an embodiment of a lightweight encrypted traffic analysis system based on a distillation characterization model provided by an embodiment of the present invention, combined with Figure 2 It can be seen that an embodiment of the analysis system includes: a network traffic collection module, a network traffic data set construction module, a representation learning-based pre-training module, a traffic analysis teacher network module and a lightweight traffic analysis module.

[0097] The network traffic collection module is used to obtain the original data set of encrypted network traffic.

[0098] The network traffic data set construction module is used to unify the data in the original data set and divide it according to the set ratio to generate various sample data sets. Each sample includes: initial pre-training sample, initial fine-tuning sample and initial test sample.

[0099] The pre-training module based on representation learning is used to input the initial pre-training samples into the BERT model based on representation learning for pre-training to obtain a pre-training model. The pre-training process includes: tokenizing the initial pre-training samples, embedding the logo information and multi-level encoding learning.

[0100] The traffic analysis teacher network module is used to input the initial fine-tuning samples into the pre-trained model for detailed task training to obtain the traffic classification teacher model.

[0101] The lightweight traffic analysis module is used to extract the teacher information of the traffic classification teacher model using the knowledge distillation method of multi-layer jump fusion learning, and construct a student model as a lightweight traffic analysis model with the traffic classification teacher model as the teacher model; combined with the teacher information, the initial fine-tuning samples are input into the lightweight traffic analysis model for training; the initial test samples are input into the trained lightweight traffic analysis model to achieve end-to-end target network encrypted traffic classification tasks.

[0102] It can be understood that the lightweight encrypted traffic analysis system based on the distillation characterization model provided by the present invention corresponds to the lightweight encrypted traffic analysis method based on the distillation characterization model provided in the aforementioned embodiments. The relevant technical features of the lightweight encrypted traffic analysis system based on the distillation characterization model can refer to the relevant technical features of the lightweight encrypted traffic analysis method based on the distillation characterization model, which will not be repeated here.

[0103] See also Figure 3 , Figure 3 Schematic diagram of an electronic device provided by an embodiment of the present invention. Figure 3 As shown, an embodiment of the present invention provides an electronic device, including a memory 1310, a processor 1320, and a computer program 1311 stored in the memory 1310 and executable on the processor 1320. When the processor 1320 executes the computer program 1311, the following steps are implemented: obtaining an original data set of encrypted network traffic, performing unified processing on the data in the original data set and then dividing the data according to a set ratio to generate various sample data sets, wherein each sample includes: an initial pre-training sample, an initial fine-tuning sample, and an initial test sample; inputting the initial pre-training sample into a BERT model based on representation learning for pre-training to obtain a pre-training model, The pre-training process includes: tokenizing the initial pre-training samples, embedding the logo information and learning multi-level encoding; inputting the initial fine-tuning samples into the pre-training model for detailed task training to obtain the traffic classification teacher model; using the knowledge distillation method of multi-layer jump fusion learning to extract the teacher information of the traffic classification teacher model, and using the traffic classification teacher model as the teacher model to construct the student model as a lightweight traffic analysis model; combining the teacher information, inputting the initial fine-tuning samples into the lightweight traffic analysis model for training; inputting the initial test samples into the trained lightweight traffic analysis model to achieve end-to-end target network encrypted traffic classification tasks.

[0104] See also Figure 4 , Figure 4 Schematic diagram of an embodiment of a computer-readable storage medium provided by the present invention. Figure 4As shown, this embodiment provides a computer-readable storage medium 1400, on which a computer program 1411 is stored. When the computer program 1411 is executed by the processor, the following steps are implemented: obtaining an encrypted network traffic original data set, unifying the data in the original data set and dividing it according to a set ratio to generate each sample data set, each sample includes: an initial pre-training sample, an initial fine-tuning sample and an initial test sample; inputting the initial pre-training sample into a BERT model based on representation learning for pre-training to obtain a pre-training model, and the pre-training process includes: tokenizing the initial pre-training sample, embedding the logo information and multi-level encoding learning; inputting the initial fine-tuning sample into the pre-training model for detailed task training to obtain a traffic classification teacher model; using a knowledge distillation method of multi-layer jump fusion learning to extract the teacher information of the traffic classification teacher model, and using the traffic classification teacher model as the teacher model to construct a student model as a lightweight traffic analysis model; combining the teacher information, inputting the initial fine-tuning sample into the lightweight traffic analysis model for training; inputting the initial test sample into the trained lightweight traffic analysis model to achieve an end-to-end target network encrypted traffic classification task.

[0105] The embodiment of the present invention provides a lightweight encrypted traffic analysis method, system, electronic device and storage medium based on a distillation representation model, which adopts an end-to-end method to analyze network encrypted traffic and uses a BERT representation learning model to describe traffic, thereby enhancing the representation capability of traffic data and having low feature redundancy, thereby improving the accuracy and throughput of model detection prediction effects. Using a large-scale unlabeled data set for system pre-training and then using a small amount of labeled data for specific task fine-tuning avoids the problems of offset and difficulty in identifying new protocols caused by the distribution of data set labels, and also alleviates the additional operations and complex engineering of operation and maintenance personnel in constructing large-scale labels, which is in line with the current trend of systematization and integration of network traffic engineering, and is practical and universal. The target object of the representation learning of encrypted traffic is the flow-level message, and the extracted and learned features represent the overall wide-area information of the data stream sequence, which is not easily disturbed by the injection of noise messages, thereby realizing the robust detection and prediction of the model and having robustness. Through the pre-training-fine-tuning mode, a model is adaptively generalized in different task scenarios, reducing the use of redundant models, which is consistent with the trend of streamlining and integration in the industry. The knowledge distillation method that combines multi-layer jump fusion learning is used to trim and eliminate redundant system parameters, retain important information to achieve analysis functions, and while ensuring high accuracy, it greatly reduces resource requirements and space occupancy, improves processing and analysis speed, and meets the vision requirements of high standards and high efficiency in industrial production. The use of lightweight models to implement the online analysis function of encrypted traffic can provide assistance for administrators or users to make forward-looking predictions and grasp network quality in real time, in line with the needs of the era of production and manufacturing to prepare for a rainy day.

[0106] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0107] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0108] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0109] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0110] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0111] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0112] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A lightweight encrypted traffic analysis method based on a distillation characterization model, characterized in that: The analysis method comprises: Step 1, obtaining an original data set of encrypted network traffic, performing unified processing on the data in the original data set and dividing it according to a set ratio to generate various sample data sets, each sample including: an initial pre-training sample, an initial fine-tuning sample and an initial test sample, adding labels to the data in the initial fine-tuning sample and the initial test sample; Step 2: inputting the initial pre-training sample into a BERT model based on representation learning to perform pre-training to obtain a pre-training model, wherein the pre-training process includes: tokenizing the initial pre-training sample, embedding the mark information, and performing multi-level coding learning; Step 3, inputting the initial fine-tuning sample into the pre-training model for training to obtain a traffic classification teacher model; Step 4, using a knowledge distillation method of multi-layer jump fusion learning to extract teacher information of the traffic classification teacher model, using the traffic classification teacher model as the teacher model to construct a student model as a lightweight traffic analysis model; combining the teacher information, inputting the initial fine-tuning sample into the lightweight traffic analysis model for training; Step 5, inputting the initial test sample into the trained lightweight traffic analysis model to achieve an end-to-end target network encrypted traffic classification task; The data in the initial pre-training sample is a hexadecimal stream data segment, and the process of tokenizing the stream data segment in step 2 includes: Step 201, cutting and decomposing the stream data segment into adjacent byte pairs to obtain a binary token data stream; The process of embedding the logo information in step 2 includes: Step 202, embed the flag [CLS] into the header of each segment of the token data stream; unify the length of all the token data streams, and align them with the flag [PAD]; randomly select M tokens and replace them with the flag [MASK]; Step 203, position embedding the token data stream after the logo is embedded, and assigning a position number to each token; The process of performing multi-level coding learning in step 2 includes: Step 204: Input the data segment X after the marker information is embedded into the first encoder for linear transformation to obtain vectors K, Q and V, where K = W K X, Q = W Q X, V = W V X, W K , W Q and W V are all linear parameters; Step 205, input the linearly transformed vectors K, Q and V into the multi-head self-attention network, and the calculation result of the multi-head self-attention network is: ATT=att1+att2+L+att H ; d is the column dimension of the input vector, H is the number of multi-head attention heads; Step 206: input the calculation result ATT into the fully connected layer network to obtain an output result h 1 =max(0,W1ATT+b1)W2+b2, where W1, b1, W2 and b2 are the fully connected layer parameters, and max(0,x) is the RELU activation function; Step 207: h 1 Input into the second encoder, and execute steps 204 to 206; Step 208, repeat step 207 Z times, where Z is the number of remaining encoders, and output the final vector h Z ; The step 4 comprises: Step 401, every two layers of learning flow analysis, the encoder output information of the teacher network is calculated to calculate the jump loss function Where N is the number of initial fine-tuning samples, M is the number of encoder layers in the student model, i and j are the serial numbers of the initial fine-tuning samples and the encoder layers in the student model, respectively. PT is the set of layers for jump learning in the teacher network, the superscript s represents the lightweight traffic analysis model, and the superscript t represents the teacher model; Step 402, calculate the cross entropy loss function Among them, C is the category label set, c is the category label, θ s is the parameter of the lightweight traffic analysis model, P s Indicates that in x i With θ s Under the parameters, the output y of the student model i = conditional probability of c; Step 404, calculate the learning loss function P t Indicates that in x i With θ t Under the parameters, the output y of the teacher model i = conditional probability of c; Step 403: Add the jump loss function, the cross entropy loss function and the learning loss function according to a set ratio to obtain a target loss function L target =(1-α)L CE +αL D +βL JT , α and β are the set proportional coefficients respectively; Step 404: Use the target loss function as the loss function of the lightweight traffic analysis model, and input the initial fine-tuning sample into the lightweight traffic analysis model for training.

2. The analysis method according to claim 1, characterized in that The step 1 comprises: Step 101, using a network sniffing tool to access accessible IP, URL and domain name to capture real-time traffic, and process to obtain a binary PCAP file original data set; Step 102, classifying the PCAP file into various dialog flow sets according to whether the five-tuple corresponds, and dividing the data in each dialog flow set into various flow data segments according to the time sequence and the five-tuple standard; the five-tuple is {IP source address, IP destination address, source port, destination port, protocol}, and in the process of classifying and judging whether the five-tuple corresponds, the IP source address and the IP destination address can be interchanged, and the source port and the destination port can be interchanged; Step 103: Divide the stream data segment according to a set ratio to generate various sample data sets.

3. The analysis method according to claim 1, characterized in that The pre-training process in step 2 further includes: For the final vector h Z Performing Mask task training to obtain the pre-trained model based on representation learning; The loss function of the Mask task training is: θ is the parameter set of the multi-level encoder, MASK i Indicates that the i-th position of the token data stream is marked as [MASK], and P indicates that in h Z With the parameter θ, the conditional probability that the i-th position of the token data stream is the flag [MASK].

4. A lightweight encrypted traffic analysis system based on a distillation characterization model, characterized in that: include: Network traffic collection module, network traffic dataset construction module, representation learning-based pre-training module, traffic analysis teacher network module and lightweight traffic analysis module; The network traffic collection module is used to obtain the encrypted network traffic original data set; The network traffic data set construction module is used to generate various sample data sets after unifying the data in the original data set and dividing them according to a set ratio. Each sample includes: an initial pre-training sample, an initial fine-tuning sample and an initial test sample, and labels are added to the data in the initial fine-tuning sample and the initial test sample; The pre-training module based on representation learning is used to input the initial pre-training sample into the BERT model based on representation learning for pre-training to obtain a pre-training model, wherein the pre-training process includes: tokenizing the initial pre-training sample, embedding the mark information, and multi-level coding learning; The traffic analysis teacher network module is used to input the initial fine-tuning sample into the pre-training model for training to obtain a traffic classification teacher model; The lightweight traffic analysis module is used to extract the teacher information of the traffic classification teacher model by adopting the knowledge distillation method of multi-layer jump fusion learning, and construct a student model as a lightweight traffic analysis model with the traffic classification teacher model as the teacher model; input the initial fine-tuning sample into the lightweight traffic analysis model for training in combination with the teacher information; input the initial test sample into the trained lightweight traffic analysis model to achieve an end-to-end target network encrypted traffic classification task; The data in the initial pre-training sample is a hexadecimal stream data segment, and the process of tokenizing the stream data segment by the pre-training module based on representation learning includes: Step 201, cutting and decomposing the stream data segment into adjacent byte pairs to obtain a binary token data stream; The process of embedding the sign information by the pre-training module based on representation learning includes: Step 202, embed the flag [CLS] into the header of each segment of the token data stream; unify the length of all the token data streams, and align them with the flag [PAD]; randomly select M tokens and replace them with the flag [MASK]; Step 203, position embedding the token data stream after the logo is embedded, and assigning a position number to each token; The process of performing multi-level encoding learning based on the pre-training module of representation learning includes: Step 204: Input the data segment X after the marker information is embedded into the first encoder for linear transformation to obtain vectors K, Q and V, where K = W K X, Q = W Q X, V = W V X, W K , W Q and W V are all linear parameters; Step 205, input the linearly transformed vectors K, Q and V into the multi-head self-attention network, and the calculation result of the multi-head self-attention network is: ATT=att1+att2+L+att H ; d is the column dimension of the input vector, H is the number of multi-head attention heads; Step 206: input the calculation result ATT into the fully connected layer network to obtain an output result h 1 =max(0,W1ATT+b1)W2+b2, where W1, b1, W2 and b2 are the fully connected layer parameters, and max(0,x) is the RELU activation function; Step 207: h 1 Input into the second encoder, and execute steps 204 to 206; Step 208, repeat step 207 Z times, where Z is the number of remaining encoders, and output the final vector h Z ; The execution process of the lightweight traffic analysis module includes: Step 401, every two layers of learning flow analysis, the encoder output information of the teacher network is calculated to calculate the jump loss function Where N is the number of initial fine-tuning samples, M is the number of encoder layers in the student model, i and j are the serial numbers of the initial fine-tuning samples and the encoder layers in the student model, respectively. PT is the set of layers for jump learning in the teacher network, the superscript s represents the lightweight traffic analysis model, and the superscript t represents the teacher model; Step 402, calculate the cross entropy loss function Among them, C is the category label set, c is the category label, θ s is the parameter of the lightweight traffic analysis model, P s Indicates that in x i With θ s Under the parameters, the output y of the student model i = conditional probability of c; Step 404, calculate the learning loss function P t Indicates that in x i With θ t Under the parameters, the output y of the teacher model i = conditional probability of c; Step 403: Add the jump loss function, the cross entropy loss function and the learning loss function according to a set ratio to obtain a target loss function L target =(1-α)L CE +αL D +βL JT , α and β are the set proportional coefficients respectively; Step 404: Use the target loss function as the loss function of the lightweight traffic analysis model, and input the initial fine-tuning sample into the lightweight traffic analysis model for training.

5. An electronic device, characterized in that: It includes a memory and a processor, and the processor is used to implement the steps of the lightweight encrypted traffic analysis method based on the distillation characterization model as described in any one of claims 1 to 3 when executing the computer management program stored in the memory.

6. A computer-readable storage medium, characterized in that: A computer management program is stored thereon, and when the computer management program is executed by a processor, the steps of the lightweight encrypted traffic analysis method based on the distillation characterization model as described in any one of claims 1 to 3 are implemented.

Images