Incremental network traffic classification method and system based on pre-training representation

Through the incremental network traffic classification method based on pre-training representation, the decision-making boundary of the Internet traffic classification model in the prior art when facing incremental data and categories is solved using the combination of base class training, incremental training and knowledge fusion stages. The problems of inconsistency and increased resource demand have achieved efficient and accurate incremental learning and classification of Internet traffic, providing low-cost expansion capabilities.

CN119030934APending Publication Date: 2024-11-26BEIJING UNIV OF TECH
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411108302.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-13
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

When the existing Internet traffic classification model faces the constantly updated network traffic and increasing categories, the decision-making boundaries are inconsistent, the computing volume and resource requirements have increased significantly, making it difficult to achieve effective incremental learning and low-cost expansion.

Method used

Using the incremental network traffic classification method based on pre-training characterization, through the base class training, incremental training and knowledge fusion stages, the in-package byte splicing of data packets in the data stream is used as features to design the feature extractor of the attention mechanism and A fully connected incremental classifier, combined with knowledge fusion network, realizes incremental learning and classification of Internet traffic.

Benefits of technology

Achieve high accuracy and efficient incremental learning classification of Internet traffic, reduce storage and computing resource overhead, provide low-cost sustainable scaling capabilities, significantly reduce catastrophic forgetting, and improve the accuracy of classification models and portability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119030934A_ABST
    Figure CN119030934A_ABST
Patent Text Reader

Abstract

The invention discloses an incremental network traffic classification method and system based on pre-training representation. The method can be divided into three stages: a base class training stage, an incremental training stage and a knowledge fusion stage. The base class training stage comprises the following steps: preprocessing a base class Internet traffic sample; the incremental training stage comprises the following steps: preprocessing an internet traffic sample of a current incremental time step; the knowledge fusion stage comprises the following steps: loading a feature extractor model, and initializing the feature extractor model by using stored parameters; all Internet traffic samples pass through the feature extractor and then pass through an activation vector obtained by the penultimate layer of the increment classifier as input of a fusion network, and the fusion network is used for fusing the Internet traffic samples; according to the method, three stages are constructed, so that the defects of a traditional deep learning model in the incremental learning field are overcome, and incremental learning of Internet traffic classification is realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a method and system for classifying incremental network traffic based on pre-training representation, and belongs to the technical field of Internet traffic classification. Background Art

[0002] In today's era, the Internet is constantly developing and Internet technology is constantly being updated and iterated. With the popularization of emerging technologies such as artificial intelligence and big data, Internet communication technology has developed rapidly, and the number of Internet users is increasing day by day. Therefore, the field of network traffic classification is very important in computer fields such as network management and network security. Network traffic classification not only helps to optimize network resource allocation and improve network performance, but also plays a key role in identifying and defending against network attacks and maintaining data privacy and security, such as identifying malicious traffic, detecting abnormal behavior, etc., thereby protecting network facilities from potential threats. In Internet service providers (ISPs) and enterprise networks, traffic classification can achieve traffic shaping, load balancing, and quality of service (QoS) management to ensure that users get a better network experience.

[0003] In recent years, the continuous emergence of network applications has significantly increased the volume of Internet information transmission, which has posed certain challenges to the classification speed and accuracy of classification models deployed in large network environments. At the same time, the number of network traffic classification tasks has increased significantly, and the number of traffic categories to be classified in each classification task has also increased significantly, which requires the classification model to meet the needs of both task increment and category increment.

[0004] In order to solve the incremental learning problem in the current field of Internet traffic classification, the existing mainstream solution is to build a separate decision branch for each new network traffic category. In the classification stage, the independent decision branches of all categories are merged at the decision level to produce the classification result. However, the above method faces two major limitations: inconsistent or even contradictory decision boundaries; the computational and resource demand overhead increases significantly with the application time of model deployment.

[0005] Therefore, in view of the current situation of network traffic classification that is constantly updated and iterated, the present invention considers the method of using pre-training and constructing features to perform multivariate fusion network, thereby realizing incremental learning of network traffic classification model. In addition, the present method realizes effective relief at the resource demand level, thereby constructing a sustainable incremental network traffic classification model that can be expanded at low cost. Summary of the invention

[0006] The purpose of the present invention is to design and implement a method and system for incremental network traffic classification based on pre-trained representation. The feature extractor trained on a large amount of Internet traffic data can form a robust feature expression in the incremental learning stage, and the knowledge fusion network can be used to fuse the features of classifiers at different time steps to achieve highly accurate and efficient incremental learning classification of Internet traffic. To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0007] A method and system for incremental network traffic classification based on pre-trained representation, comprising: a base class training phase, an incremental training phase, and a knowledge fusion phase;

[0008] The base class training phase includes the following steps:

[0009] 1) Taking the original Internet traffic sample bound with the label as input, an Internet traffic preprocessing operation is performed on it to obtain a byte sequence of the Internet traffic sample with the label.

[0010] 2) Taking the byte sequence of the labeled Internet traffic sample obtained in step 1) as input, a feature extraction operation is performed on the input of this step to obtain the byte sequence features of the labeled Internet traffic sample.

[0011] 3) Using the labeled byte sequence features of the Internet traffic sample obtained in step 2) as input, use the base class classifier to classify the input features of this step, obtain the prediction results of the Internet traffic sample, compare them with the known labels, calculate the loss function to obtain the loss value, and save the feature extractor.

[0012] The incremental training phase includes the following steps:

[0013] 4) Taking the Internet traffic sample to be classified at the current time step bound with the label as input, perform Internet traffic preprocessing operation on it to obtain the byte sequence of the Internet traffic sample with label at the current time step.

[0014] 5) Using the byte sequence of the labeled Internet traffic sample obtained in step 4) as input, load the model and parameters of the feature extractor obtained in step 3), perform feature extraction on the input of this step, and obtain the byte sequence features of the labeled Internet traffic sample at the current time step.

[0015] 6) Taking the byte sequence features of the labeled Internet traffic sample at the current time step obtained in step 5) as input, using the incremental classifier of the current time step, classify the input features of this step, obtain the prediction result of the Internet traffic sample at the current time step, compare it with the known label, calculate the loss function to obtain the loss value, and save the incremental classifier of the current time step.

[0016] 7) When the Internet traffic sample of each incremental time step arrives, repeat steps 4) to 6) to obtain the incremental classifier corresponding to each time step.

[0017] The knowledge fusion stage includes the following steps:

[0018] 8) Take the activation vector of the penultimate layer of the incremental classifier of the byte sequence features of all categories of Internet traffic samples as input, use the knowledge fusion network to perform a fusion operation on the input of this step, output the prediction results of the Internet traffic samples, and compare them with the known labels, calculate the loss function to get the loss value, and realize the incremental learning of the Internet traffic classification model.

[0019] A pre-trained representation-based incremental network traffic classification system, characterized by comprising an Internet traffic pre-processing module, a feature extractor construction module, and a base class classifier construction module for a base class training phase; an Internet traffic pre-processing module and an incremental classifier construction module for an incremental training phase; a knowledge fusion network construction module for a knowledge fusion phase; wherein the Internet traffic pre-processing module is responsible for extracting a stream from an input raw Internet traffic sample, and then extracting the number of packets in the stream and the number of bytes in the packet; the feature extractor construction module is responsible for forming an embedded vector expression form of the byte sequence based on the Internet traffic sample byte sequence, and extracting features from the packet level and the data stream level, and then generating Internet traffic sample features with good properties for use in a subsequent classification phase; the base class classifier The classifier construction module is responsible for adjusting the network parameters in the base class classifier based on the characteristics of Internet traffic samples with known labels, and then generating a base class classifier with good classification properties for use in the base class training stage; the incremental classifier construction module is responsible for adjusting the network parameters in the incremental classifier based on the characteristics of Internet traffic samples with known labels, and then generating an incremental classifier with good classification properties for use in the incremental training stage; the knowledge fusion network construction module is responsible for fusing the feature dimensions of the incremental classifiers of each incremental time step based on the byte sequences of Internet traffic samples in the base class training stage and the incremental training stage, and then generating a robust knowledge fusion network structure, so as to realize the classification of the arriving Internet traffic to be classified as the time step continues to increase, and finally output the category to which the Internet traffic belongs;

[0020] The key technical points of the present invention are:

[0021] 1. Use the bytes in each data packet in the data stream to splice into a byte sequence as the Internet traffic feature;

[0022] 2. Designed a feature extractor based on the attention mechanism. This feature extractor can flexibly extract effective Internet traffic features from the embedded vector encoded by the original input data, which plays a key role in the large amount of Internet traffic category data that increases over time;

[0023] 3. A fully connected incremental classifier is designed. This incremental classifier builds a model for each incremental Internet data sample at each incremental time step, effectively retaining the knowledge of the Internet traffic data at the current incremental time step, avoiding confusion with the knowledge of the existing time step and disrupting the feature space.

[0024] 4. Designed a fusion network structure. This fusion network fuses the incremental classifiers of different incremental time steps at the feature level, thereby forming a low-cost amplified incremental learning model structure for Internet traffic classification, effectively classifying the Internet traffic data of the current time step and the existing time step, and realizing incremental learning of Internet traffic.

[0025] By utilizing the technology of the present invention, it is possible to achieve incremental and accurate classification of Internet traffic in a real environment. Compared with the known related technologies, it has the following advantages:

[0026] 1. The present invention uses the bytes in the data packets in the data stream and splices them into byte sequences as the characteristics of Internet traffic, which can learn the Internet information transmission mode more comprehensively.

[0027] 2. The present invention uses an Internet traffic classification model feature extractor based on the attention mechanism, which can learn efficient features for classification from base class data, and is more helpful in forming a classification decision boundary in the incremental process.

[0028] 3. The present invention constructs an incremental classifier separately for the Internet traffic data of each incremental time step, effectively preserving the effective knowledge information of the Internet traffic data of the current incremental time step, which is beneficial to significantly reduce catastrophic forgetting in the process of incremental learning.

[0029] 4. The present invention uses a knowledge fusion network structure to perform feature-level information fusion on the knowledge of the current incremental time step and the existing time step. The penultimate layer activation vector of the incremental classifier is used as the input of the knowledge fusion stage to further fuse the categories, which enables the classification model to perform excellent classification effects on both the Internet traffic data of the current incremental time step and the Internet traffic data of the existing time step.

[0030] 5. The use of an incremental learning framework for Internet traffic classification based on the base class training phase, incremental training phase, and knowledge fusion phase can effectively reduce storage overhead and computing resource overhead, and has strong portability. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Flowchart of the base class training phase.

[0032] Figure 2 Flowchart of the incremental training phase.

[0033] Figure 3 Flowchart of knowledge fusion phase.

[0034] Figure 4 System architecture diagram.

[0035] Figure 5 Diagram of the feature extractor structure.

[0036] Figure 6 Knowledge fusion network structure diagram. DETAILED DESCRIPTION

[0037] The workflow of the present invention can be divided into three stages: base class training stage, incremental training stage and knowledge fusion stage.

[0038] In the base class training stage, the learnable parameters of the feature extractor and the base class classifier are trained by supervised learning according to the Internet traffic of known categories, thereby obtaining the feature extractor and the base class classifier. In the base class training stage, the real Internet mobile application category traffic data obtained in the real environment is first preprocessed, and several groups of parameter combinations of the number of packets and the number of bytes are set. After the preprocessing is completed, the obtained byte sequence is used as input, and the feature extractor is used to extract the features of the byte sequence to obtain the byte sequence features of the Internet traffic sample with labels. The byte sequence features with labels are input into the base class classifier, the classification prediction results are output, and the prediction results are compared with the labels, the loss function is calculated, the model parameters are updated in reverse, the training of the base class classifier is completed, and the feature extractor is saved for subsequent experiments. In the incremental training stage, the present invention constructs a separate incremental classifier for each Internet traffic sample of the incremental time step, which is used to save the Internet traffic data knowledge of the current incremental time step. The byte sequence obtained after preprocessing the incremental Internet traffic bound to the label is used as input to train the incremental classifier and save it. In the knowledge fusion stage, the byte sequence obtained by preprocessing the Internet traffic data samples of all time steps is used as input, and the knowledge fusion network is used to fuse the incremental classifiers of the current incremental time step and the existing time steps at the feature level, so that the model can classify the Internet traffic data of the current incremental time step while classifying the Internet traffic data of the existing time steps.

[0039] The classification model of the Internet traffic incremental learning of the present invention is constructed based on the training method of the deep neural network model, and its specific implementation steps are as follows:

[0040] 1. First, describe the problem scenario: Assume that at time step t-1, the Internet traffic to be classified is y t-1 = {Cls1, Cls2, ..., Cls α Then, at time step t, β categories are added to the Internet network environment, that is, the current Internet newly added traffic to be classified is The present invention uses a set of original Internet traffic samples bound to labels after preprocessing operations. Among them, x i represents the ith Internet flow, y i Represents the corresponding Internet traffic category label. The goal of this invention is to enable the model to accurately classify Internet traffic categories At the same time, for Internet traffic category y t-1 After accurate classification, the total Internet traffic category at the current time step is expressed as As time goes by, repeat the above process.

[0041] 2. The preprocessing steps are described in detail as follows: First, extract the Internet flow. For the original Internet traffic, the present invention reassembles the Internet flow according to the five-tuple (source IP, destination IP, source port, destination port, TCP / UDP), and then extracts the Internet flow. Subsequently, extract the flow data packets and the byte sequences in the packets. For each Internet flow, the present invention extracts the first K data packets, represented as {packet1, packet2, ..., packet K}, extract the first B bytes from each data packet, expressed as packet j = {byte1,byte2,…,byte B}, splicing to form the byte sequence of the Internet flow. If the flow has more than K data packets, the first K data packets of the flow are retained; if the flow has less than K data packets, a padding operation is performed. For the number of bytes in the data packet, if the number of bytes is more than B, the first B bytes of the data packet are retained; if the number of bytes in the data packet is less than B, a padding operation is performed. Since 256 does not carry field information related to information communication, 256 is filled to expand the byte sequence in the data packet of the flow. After length adjustment, the length of the byte sequence extracted from the Internet traffic remains consistent.

[0042] 3. Before starting the deep learning model training, the model parameters need to be initialized. In the present invention, the feature extractor and base class classifier in the base class training phase, the incremental classifier in the incremental training phase, and the trainable parameters in the knowledge fusion network in the knowledge fusion phase need to be assigned initial values ​​using random initialization. After completing the training of the feature extractor in the base class training phase, the saved feature extractor model and parameters need to be loaded and initialized in the incremental training phase; similarly, after completing the training of the incremental classifier for each incremental time step, the model and parameters of the incremental classifier for the current incremental time step and the existing time step need to be loaded and initialized in the knowledge fusion phase. The model with parameter initialization can effectively avoid problems such as gradient vanishing or gradient exploding during the training process, and is beneficial to improving the convergence speed and performance of the model.

[0043] The flowchart of the base class training phase is as follows Figure 1 As shown, the present invention obtains the feature extractor model and the base class classifier model at this stage. First, for the labeled base class data set, denoted as D base ={(x i ,y i )|y i ∈{Cls base}}, preprocessing is performed to obtain a byte sequence set of the base class data set. Next, the byte sequence set is used to train the feature extractor of the base class training phase, denoted as FE(·;Ω), and the base class classifier, denoted as CF base (·;Φ).

[0044] The structure of the feature extractor is as follows Figure 5 As shown in Figure 1, it consists of two parts: a feature extraction module at the packet level and a feature extraction module at the data stream level. The preprocessed byte sequence is represented as:

[0045]

[0046] For a byte sequence x' i ={packet1,packet2,…,packet K}, the packet-level feature extraction module focuses on each packet j , j∈[1,K], thereby extracting the features at the packet level and obtaining the packet feature sequence E p ={E p1 ,E p2 ,…,E pK}. Then, the data stream-level feature extraction module focuses on the entire data packet feature sequence and extracts the data stream-level feature sequence from it. The data stream-level feature extraction module uses the self-attention encoder as the main structure and uses three matrices W K,W Q ,W V The input byte sequence E p Converted into Key, Query, Value, used to calculate the attention score between sequence elements and obtain the output sequence H p , specifically expressed as:

[0047] Key=E p W K ,Value=E p W V ,Query=E p W Q

[0048]

[0049] Among them, d k Indicates the dimension of Key. After the L-layer packet-level feature extraction module in the base class training phase, the output feature sequence is obtained Next, an averaging operation is performed on the time dimension to obtain data stream level features, that is, the byte sequence features of the Internet traffic samples.

[0050] The present invention utilizes the byte sequence features of Internet traffic samples and uses a base classifier for classification. Wherein, Ω and Φ represent the parameters of the feature extractor and the base classifier respectively, and the loss function is specifically expressed as:

[0051]

[0052] in, σ represents the Softmax function.

[0053] 4. In the incremental training phase, the labeled incremental data of the current time step is preprocessed, and a set of byte sequences with consistent lengths is obtained after completion. In the incremental training phase, the operation of preprocessing Internet traffic samples is the same as the preprocessing operation in the base class training phase.

[0054] 5. In the incremental training stage, the present invention obtains the incremental classifier of each incremental time step t, represented by CF t (·;Φ t ). The preprocessed labeled incremental data with consistent length is expressed as Get a set of byte sequences with the same length. Load the feature extractor model obtained in the base class training phase and initialize it using the saved feature extractor parameters, expressed as FE(·;Ω *), and keep its parameters frozen in the following training process. Use the feature extractor to extract the byte sequence features. The specific extraction process is the same as the feature extraction process in the base class training stage.

[0055] 6. Build the incremental classifier model for the current time step and initialize the incremental classifier, represented as CF t (·;Φ t ). The incremental classifier is used to classify the byte sequence features. The specific classification operation process is the same as the feature classification process in the base class training phase.

[0056] 7. When the Internet traffic sample of each incremental time step arrives, repeat steps 4 to 6 to obtain the incremental classifier corresponding to each time step, and save the model and parameters of each incremental classifier.

[0057] 8. In the knowledge fusion stage, the present invention fuses the incremental classifiers of the current incremental time step and the existing time step, which is represented by {CF base ,CF1,CF2,…,CF t}, and a unified feature distribution is obtained. In the present invention, all Internet traffic data samples are represented as D fusion ={(X i ,y i )|y i ∈y t = {Cls base ,Cls1,…,Cls α+β}}. The knowledge fusion network structure is as follows Figure 6 As shown. Get the base classifier and incremental classifier CF i The activation vector of the penultimate layer of (i∈base∪[1,t]) is expressed as As the input of the knowledge fusion network. For the incremental classifier CF of the current time step i , generating a parameter matrix Thus, the knowledge from other classifiers is in the form of probability distribution Fuse into the current incremental classifier. The process of knowledge fusion can be expressed as:

[0058]

[0059] v i =z i +Δz i

[0060] At this point, the knowledge fusion stage uses the fused probability distribution {v base ,v1,v2,…,v t}, and get the final probability distribution It is expressed as:

[0061]

[0062] in, Represents a concatenate operation.

[0063] In practical applications, the present invention can be applied to a variety of deep learning models as incremental learning of Internet traffic, wherein, as the deployment time increases, incremental category data samples can be added to base category data samples to form a larger set of base category data samples.

[0064] In the verification experiment, the present invention conducts example experiments on Internet traffic data sets in three different fields. In Internet data set A, traffic is collected from IoT devices in a real network environment, with a total of more than 100,000 items, of which each category contains 100 to 200 flows. The present invention selects 18 traffic categories with more than 500 flows in total, which are divided into three incremental time steps; in Internet data set B, the total number exceeds 70,000, and the present invention selects 12 traffic categories with more than 500 flows in total, which are divided into four incremental time steps; in Internet data set C, traffic is collected from Internet mobile applications, including news, games, social media, video and audio and other popular mobile application categories, with a total of more than 200,000 items. The present invention selects 50 traffic categories with more than 500 items, which are divided into five incremental time steps. The Internet category names and IDs of the three data sets are shown in Table 1.

[0065] Table 1: Category name and ID number of each incremental time step of dataset A, B, and C

[0066]

[0067]

[0068] First, in order to verify the effectiveness of the present invention, the present invention defines the experimental analysis indicators as follows:

[0069] (1) True Yang Value (TP x ): This value is the number of samples in a set, each of which is reported by the Internet traffic classifier of the present invention as belonging to category x and in fact belongs to the corresponding category x.

[0070] (2) True Yin value (TN x ): This value is the number of samples in a set, each of which is reported by the Internet traffic classifier of the present invention as not belonging to category x, and in fact does not belong to the corresponding category x.

[0071] (3) False Positive Value (FP) x): This value is the number of samples in a group, each of which is reported by the Internet traffic classifier of the present invention as belonging to category x, but in fact does not belong to the corresponding category x.

[0072] (4) False Negative Value (FN) x ): This value is the number of samples in a set, each of which is reported by the Internet traffic classifier of the present invention as not belonging to category x, but in fact belongs to the corresponding category x.

[0073] For the convenience of expression, the present invention uses true positive value, true negative value, false positive value, and false negative value to express, and defines Precision, Recall, and F1-measure to evaluate the ability of the classifier. The specific definitions are as follows:

[0074]

[0075] An incremental network traffic classification method and system based on pre-trained representation, the experimental results of the base class training stage are shown in the following table. The present invention sets nine groups of parameters, including the number of data packets K∈{4,8,12} and the number of bytes in the packet B∈{64,128,256}, and the number of base classes is 259. The experimental results are shown in Table 2.

[0076] Table 2: Parameter experimental results of the number of packets K and the number of bytes in the packet B

[0077]

[0078]

[0079] After experiments, this method selects K=8, B=256 as experimental parameters.

[0080] An incremental network traffic classification method and system based on pre-trained representation, the final experimental results of incremental learning experiments on scenarios A, B and C are shown in Table 3 below.

[0081] Table 3: Incremental network traffic classification method and system based on pre-trained representation

[0082]

[0083] As shown in the table, on dataset A, the Precision, Reccall and F1-measure of the first time step are 95.56%, 95.46% and 95.47% respectively; the Precision, Reccall and F1-measure of the second time step are 95.63%, 95.54% and 95.55% respectively; the Precision, Reccall and F1-measure of the third time step are 95.63%, 95.53% and 95.54% respectively.

[0084] On dataset B, the Precision, Reccall and F1-measure of the first time step are 95.43%, 95.33% and 95.34% respectively; the Precision, Reccall and F1-measure of the second time step are 95.56%, 95.46% and 95.47% respectively; the Precision, Reccall and F1-measure of the third time step are 95.50%, 95.40% and 95.41% respectively; the Precision, Reccall and F1-measure of the fourth time step are 95.51%, 95.42% and 95.42% respectively.

[0085] On dataset C, the Precision, Reccall and F1-measure of the first time step are 95.39%, 95.30% and 95.31% respectively; the Precision, Reccall and F1-measure of the second time step are 95.36%, 95.26% and 95.26% respectively; the Precision, Reccall and F1-measure of the third time step are 95.27%, 95.17% and 95.17% respectively; the Precision, Reccall and F1-measure of the fourth time step are 95.00%, 94.84% and 94.86% respectively; the Precision, Reccall and F1-measure of the fifth time step are 94.88%, 94.71% and 94.72% respectively.

[0086] The above examples are only used to illustrate the technical solutions of the present invention rather than to limit the same. A person skilled in the art may modify or make equivalent substitutions for the technical solutions of the present invention without departing from the spirit and scope of the present invention. The protection scope of the present invention shall be subject to the claims.

Claims

1. A method for incremental network traffic classification based on pre-trained representation, characterized in that: It includes three stages: base class training stage, incremental training stage, and knowledge fusion stage; The base class training phase includes the following steps: 1) Taking the original Internet traffic sample bound with the label as input, performing Internet traffic preprocessing operation on it to obtain the byte sequence of the Internet traffic sample with the label; 2) taking the byte sequence of the labeled Internet traffic sample obtained in step 1) as input, performing a feature extraction operation on the input of this step to obtain the byte sequence features of the labeled Internet traffic sample; 3) Taking the byte sequence features of the labeled Internet traffic sample obtained in step 2) as input, using the base class classifier to classify the input features of this step, obtaining the prediction results of the Internet traffic sample, comparing them with the known labels, calculating the loss function to obtain the loss value, and saving the feature extractor; The incremental training phase includes the following steps: 4) taking the Internet traffic sample to be classified at the current time step bound with the label as input, performing Internet traffic preprocessing operation on it, and obtaining a byte sequence of the Internet traffic sample with label at the current time step; 5) using the byte sequence of the labeled Internet traffic sample obtained in step 4) as input, loading the model and parameters of the feature extractor obtained in step 3), performing a feature extraction operation on the input of this step, and obtaining the byte sequence features of the labeled Internet traffic sample at the current time step; 6) Taking the byte sequence features of the labeled Internet traffic sample at the current time step obtained in step 5) as input, using the incremental classifier of the current time step to classify the input features of this step, obtain the prediction result of the Internet traffic sample at the current time step, compare it with the known label, calculate the loss function to obtain the loss value, and save the incremental classifier of the current time step; 7) When the Internet traffic sample of each incremental time step arrives, repeat steps 4) to 6) to obtain the incremental classifier corresponding to each time step; The knowledge fusion stage includes the following steps: 8) Take the activation vector of the penultimate layer of the incremental classifier of the byte sequence features of all categories of Internet traffic samples as input, use the knowledge fusion network to perform a fusion operation on the input of this step, output the prediction results of the Internet traffic samples, and compare them with the known labels, calculate the loss function to get the loss value, and realize the incremental learning of the Internet traffic classification model.

2. The incremental network traffic classification method based on pre-trained representation according to claim 1 is characterized in that: The specific operation method of performing Internet traffic preprocessing in step 1) is: 1-1) Reassemble the input original Internet traffic to the Internet flow; 1-2) using the Internet flow obtained in step 1-1) as input, performing a flow sequence extraction operation according to the number of data packets and the number of bytes in the packets, and obtaining an Internet flow byte sequence; 1-3) using the Internet traffic byte sequence obtained in step 1-2) as input, performing a length adjustment operation on the Internet traffic byte sequence to obtain a byte sequence of Internet traffic samples of uniform length; The specific operation method of extracting features from the byte sequence of the Internet traffic sample in step 2) is: 2-1) Construct a feature extractor based on the self-attention mechanism and randomly initialize the trainable parameters of the feature extractor; 2-2) using the byte sequence of the Internet traffic sample obtained in step 1-3) as input, and using the feature extractor in step 2-1) to perform a feature extraction operation to obtain byte sequence features of the Internet traffic sample; The specific operation method of classifying the byte sequence features of the Internet traffic sample in step 3) is: 3-1) Construct a base class classifier based on full connection, and randomly initialize the trainable parameters of the base class classifier; 3-2) using the byte sequence features of the Internet traffic sample obtained in step 2-2) as input, and using the base classifier in step 3-1) to perform classification, and obtaining probability features of the byte sequence features of the Internet traffic sample belonging to each category; 3-3) Taking the probability features of the byte sequence features of the Internet traffic sample obtained in step 3-2) belonging to each category as input, using the Softmax function, the probability of each Internet traffic sample belonging to each category is obtained, and the predicted category of the Internet traffic sample is obtained by comparison; 3-4) Taking the predicted category of the Internet traffic sample obtained in step 3-3) as input, the loss value is obtained by calculating the loss function with the known label; and the trained feature extractor and base class classifier are saved.

3. The incremental network traffic classification method based on pre-trained representation according to claim 1 is characterized in that: The specific operation method of performing Internet traffic preprocessing at the current time step described in step 4) is the same as step 1-1) to step 1-3), and a byte sequence of Internet traffic samples of uniform length at the current time step is obtained.

4. The incremental network traffic classification method based on pre-trained representation according to claim 1 is characterized in that: The specific operation method of step 5) for extracting features from the byte sequence of the Internet traffic sample at the current time step is: 5-1) loading the model of the feature extractor saved in step 3-4) and initializing it using the saved parameters; 5-2) Using the byte sequence of the Internet traffic sample at the current time step as input, perform feature extraction using the feature extractor of step 5-1) to obtain the byte sequence features of the Internet traffic sample at the current time step. The specific operation method is the same as the specific operation method for extracting base class features described in step 2-2).

5. The incremental network traffic classification method based on pre-trained representation according to claim 1, characterized in that: The specific operation method of classifying the byte sequence features of the Internet traffic sample at the current time step in step 6 is: 6-1) Construct an incremental classifier based on full connection and randomly initialize the trainable parameters of the incremental classifier; 6-2) using the byte sequence features of the Internet traffic sample at the current time step obtained in step 5-2) as input, and using the incremental classifier of step 6-1) to perform classification, and obtaining probability features of the byte sequence features of the Internet traffic sample at the current time step belonging to each category; 6-3) Taking the probability features of the byte sequence features of the Internet traffic sample at the current time step obtained in step 6-2) belonging to each category as input, using the Softmax function, obtain the probability that each Internet traffic sample at the current time step belongs to each category at the current time step, and obtain the predicted category of the Internet traffic sample at the current time step by comparison; 6-4) Taking the predicted category of the Internet traffic sample at the current time step obtained in step 6-3) as input, the loss value is obtained by calculating the loss function with the known label; and saving the incremental classifier of the current time step obtained by training.

6. The incremental network traffic classification method based on pre-trained representation according to claim 1 is characterized in that: The specific operation method of obtaining the incremental classifier corresponding to each time step in step 7 is: 7-1) When the Internet traffic sample of each incremental time step arrives, determine whether the maximum iteration period has been reached; 7-2) If the maximum iteration period is reached, stop training and save the incremental classifier corresponding to this time step; 7-3) If the maximum iteration period has not been reached, continue training until the maximum iteration period is met.

7. The incremental network traffic classification method based on pre-trained representation according to claim 1 is characterized in that: The specific operation method of the knowledge network fusion in step 8) is: 8-1) Construct a knowledge fusion network based on full connection, and randomly initialize the trainable parameters of the knowledge fusion network; 8-2) The byte sequence features of the Internet traffic samples obtained in step 2-2) and step 5-2) are respectively processed in step 3-4) The obtained base class classifier and the penultimate activation vector of the incremental classifier obtained in step 6-4) are used as the input of the knowledge fusion network, the input of this step is fused at the feature level, and the predicted category of the knowledge fusion network is output; 8-3) Taking the predicted category of the Internet traffic sample obtained in step 8-2) as input, the loss value is obtained by calculating the loss function with the known label.

8. The incremental network traffic classification method based on pre-trained representation according to claim 2, characterized in that: The specific operation method of performing the reorganization operation to obtain the Internet flow in step 1-1) is: Taking the original Internet traffic as input, the Internet traffic will be reassembled according to the five-tuple, namely source IP, destination IP, source port, destination port, TCP / UDP, and then the Internet flow will be extracted.

9. The incremental network traffic classification method based on pre-trained representation according to claim 2, characterized in that: The specific operation method of extracting the stream sequence in step 1-2) is: 1-2-1) For each Internet flow, divide it according to the data packets in the flow; 1-2-2) Divide the data packet in step 1-2-1) according to the bytes in the packet; The specific operation method of adjusting the length of the Internet byte sequence in step 1-3) is: 1-3-1) Use truncation or padding operations to adjust the length of the data packet to the specified length; 1-3-2) Using truncation or padding operations, the length of the number of bytes in the data packet is adjusted to a specified length, thereby obtaining a byte sequence of the Internet traffic sample of uniform length; The specific operation method of constructing the feature extractor in step 2-1) is: 2-1-1) Construct a packet-level feature extraction module based on embedding layer, position encoding, and self-attention encoding, and randomly initialize the trainable parameters therein; 2-1-2) Construct a stream-level feature extraction module based on position encoding and self-attention encoding, and randomly initialize the trainable parameters therein; The specific operation method of training the feature extractor in step 2-2) is: 2-2-1) using the byte sequence of the Internet traffic sample obtained in step 1-3-2) as input, and using the packet-level feature extraction module in step 2-1-1) to extract packet-level features to obtain packet-level features; 2-2-2) using the packet-level features obtained in step 2-2-1) as input, and using the flow-level feature extraction module in step 2-1-2) to extract flow-level features to obtain flow-level features; The specific operation method of constructing the base classifier in step 3-1) is: 3-1-1) Building a base class classifier based on the fully connected layer, where the number of neurons in the fully connected layer is set to other integers according to the actual application scenario and the final result; 3-1-2) Randomly initialize the trainable parameters of the base class classifier obtained in step 3-1-1); The specific operation method of calculating the loss value in step 3-4) is: 3-4-1) Take the classification prediction result obtained in step 3-3-1) as input, compare it with the known label, calculate the loss function to get the loss value, which is used to indicate the quality of the current classification model; 3-4-2) Save the models and parameters of the trained feature extractor and base class classifier; The specific operation method of initializing the knowledge fusion network in step 8-1) is: 8-1-1) Construct a knowledge fusion network based on full connection, where the number of neurons in the fully connected layer can be set to other integers according to the actual application scenario and the final result; 8-1-2) Randomly initialize the trainable parameters in the knowledge fusion network obtained in step 8-1-1); The specific operation method of fusing the byte sequence features of the Internet traffic samples to obtain the predicted category in step 8-2) is: 8-2-1) Using the byte sequence features of the Internet traffic sample, the base classifier obtained in step 3-4-2) and the penultimate layer activation vector of the incremental classifier of each time step obtained in step 6-4) are used as inputs, and as inputs to the knowledge fusion network, deep fusion at the feature level is performed; the fused features combine the features of each time step, which can effectively improve the classification effect and improve the performance of the model; 8-2-2) The fused network predicts the category using the fused features output from step 8-2-1).

10. An incremental network traffic classification system based on pre-trained representations implementing the method according to any one of claims 1 to 9, characterized in that: It includes an Internet traffic preprocessing module, a feature extractor building module, and a base classifier building module for the base class training phase; an Internet traffic preprocessing module and an incremental classifier building module for the incremental training phase; The knowledge fusion network building module used in the knowledge fusion stage; 1) The Internet traffic preprocessing module is responsible for extracting the flow of the input raw Internet traffic sample, and then extracting the number of packets in the flow and the number of bytes in the packet; the feature extractor construction module is responsible for forming the byte sequence into an embedded vector expression based on the byte sequence of the Internet traffic sample, and extracting features from the data packet level and the data stream level, and then generating Internet traffic sample features with good properties for use in the subsequent classification stage; the base class classifier construction module is responsible for adjusting the network parameters in the base class classifier based on the Internet traffic sample features with known labels, and then generating the base class classifier for use in the base class training stage; 2) The incremental classifier construction module is responsible for adjusting the network parameters in the incremental classifier based on the characteristics of Internet traffic samples with known labels, thereby generating an incremental classifier with good classification properties for use in the incremental training phase; 3) The knowledge fusion network construction module is based on the byte sequences of Internet traffic samples in the base class training phase and the incremental training phase. It is responsible for fusing the feature dimensions of the incremental classifiers of each incremental time step, and then generating a robust knowledge fusion network structure. It can classify the arriving Internet traffic to be classified as the time step continues to increase, and finally output the category to which the Internet traffic belongs.

Citation Information

Cited By

  • Intrusion detection method for interpretable fine-grained industrial control network

    CN119995979A

  • An interpretable fine-grained industrial control network intrusion detection method

    CN119995979B