A method and system for detecting malicious encrypted traffic based on incremental learning

Abstract

The present invention discloses a malicious encrypted traffic detection method and system based on incremental learning, which belongs to the field of network security technology. A double-layer nested data organization structure is used to process the original traffic into a standardized feature sequence to construct a sample set; the sample set is used to pre-train the malicious traffic detection model; the pre-trained malicious traffic detection model is optimized by incremental learning. In the incremental learning process, an experience buffer for storing historical training samples and their model output results and sample labels is provided, and it is continuously updated with the model parameter optimization process; combined with dynamic experience playback, stability constraints and a double-layer meta-update mechanism, the catastrophic forgetting problem of the model in the continuous learning process is solved; finally, the trained model is used to detect the data to be detected. The present invention can effectively improve the detection accuracy of malicious encrypted traffic, and maintain the continuous adaptability of the model through incremental learning, and has good practical value.

Description

Technical Field

[0001] The present invention belongs to the technical field of network security, and in particular relates to a malicious encrypted traffic detection method and system based on incremental learning. Background Art

[0002] With the rapid development of the Internet and the widespread popularity of network applications, network security issues are becoming increasingly severe, especially when facing malicious encrypted traffic. Traditional content-based detection methods are gradually becoming ineffective. Malicious traffic not only hides its true content through encryption technology, but the popularity of encryption protocols also makes traffic analysis more difficult. Common malicious encrypted traffic includes network attacks, malware communications, data leaks, etc. These attacks bypass traditional traffic monitoring and protection mechanisms through encrypted channels, posing great challenges to network security.

[0003] Traditional malicious traffic detection methods mostly rely on feature extraction and rule-based analysis, which often ignore the variability and concealment of encrypted traffic during data transmission. In encrypted traffic, attackers often evade detection by changing the size, frequency, time distribution and other characteristics of data packets, resulting in static feature-based detection methods no longer being effective. On the other hand, with the advancement of encryption technology, the visibility of network traffic has been further reduced, and traditional detection technology based on plaintext data analysis has gradually become unable to meet the security needs of modern network environments.

[0004] In recent years, machine learning-based methods have been gradually applied to the field of malicious traffic detection. By learning the complex patterns of network traffic, potential attack behaviors can be effectively identified. However, traditional machine learning methods usually require a large amount of labeled data for training, and the model has poor adaptability to changes in data distribution during training. Faced with ever-changing network traffic and attack patterns, traditional models are often difficult to update continuously and effectively, resulting in a gradual decline in the detection performance of the model.

[0005] How to design an efficient, real-time and highly adaptable malicious encrypted traffic detection method is a difficult problem that needs to be solved urgently in the current network security field. Summary of the invention

[0006] In order to solve the problems in the prior art, the present invention proposes a malicious encrypted traffic detection method and system based on incremental learning.

[0007] The technical solution of the present invention is as follows:

[0008] In a first aspect, the present invention discloses a malicious encrypted traffic detection method based on incremental learning, comprising:

[0009] (1) Using a double-layer nested data organization structure to process the original traffic into a standardized feature sequence, the original traffic includes normal encrypted traffic and malicious encrypted traffic, and construct a sample set;

[0010] (2) Use the sample set to pre-train the malicious traffic detection model;

[0011] (3) The pre-trained malicious traffic detection model is optimized using an incremental learning method. During the incremental learning process, an experience buffer is provided for storing historical training samples and their model output results and sample labels. The experience buffer is continuously updated as the model parameter optimization process progresses;

[0012] The incremental learning method is used to optimize the model parameters, including: based on the model classification loss, using a stability constraint mechanism to limit the fluctuation of the model output results of the historical training samples and calculating the stability constraint loss, using an experience replay mechanism to adaptively adjust the number of samples from the experience buffer and calculate the experience replay loss; combining the model classification loss, stability constraint loss and experience replay loss, using a double-layer meta-update mechanism to iteratively optimize the intra-batch model parameters and the inter-batch model parameters;

[0013] (4) Use the malicious traffic detection model with optimized parameters to detect whether the encrypted traffic is malicious encrypted traffic.

[0014] Furthermore, the double-layer nested data organization structure includes an outer network flow object and an inner data burst object. The characteristics of the network flow object include session identification information, time series information, statistical metering information and a burst sequence container for storing all data burst objects contained in the original traffic; the characteristics of the data burst object include direction identification information, timestamp information, and the number of all data packets contained in the data burst object and the cumulative byte length.

[0015] Furthermore, the session identification information includes the source IP address, source port number, target IP address and target port number, the time series information includes the start timestamp and last activity timestamp of the original traffic session, the statistical metering information includes the number of data packets, the direction identification information represents the data transmission direction, and the timestamp information includes the start and end timestamps of the data burst object.

[0016] Furthermore, the double-layer nested data organization structure is used to process the original traffic into a standardized feature sequence, including:

[0017] (1.1) Read the original traffic file and parse the data packet, verify the legitimacy of the data packet protocol type, and remove illegal data packets;

[0018] (1.2) Extract the total length, source IP address, destination IP address, source port number and destination port number of each data packet, and construct the session identifier of the data packet based on the source port number and destination port number to represent the data transmission direction;

[0019] (1.3) Determine whether the current data packet belongs to an existing network flow object based on the source IP address, target IP address, source port number, and target port number of the current data packet:

[0020] If so, update the characteristics of the network flow object, and maintain the characteristics of the data burst object according to the session identifier of the current data packet: if the session identifier of the current data packet is consistent with the direction identification information of the current data burst object, update the characteristics of the data burst object, otherwise create a new data burst object and initialize the characteristics;

[0021] If not, create a network stream object and initialize the first data burst object;

[0022] (1.4) Processing the original traffic into several network flow objects through steps (1.1) to (1.3), each network flow object contains several data burst objects, integrating the characteristics of the network flow objects and the characteristics of the data burst objects to obtain a standardized feature sequence of the original traffic.

[0023] Furthermore, the malicious traffic detection model adopts a Transformer encoder.

[0024] Furthermore, the process of continuously updating the experience buffer zone as the model parameter optimization process progresses includes:

[0025] After the malicious traffic detection model completes pre-training, a preset number of historical training samples are randomly selected from the sample set, and the selected historical training samples and their model output results and sample labels are added to the experience buffer to complete the initialization of the experience buffer;

[0026] In the incremental learning process, after a round of model parameter optimization is completed, a preset number of historical training samples are randomly selected from the last batch of training samples in that round of optimization, and the selected historical training samples and their model output results and sample labels are added to the experience buffer.

[0027] Furthermore, the combination of the model classification loss, stability constraint loss and experience replay loss, and the use of a double-layer meta-update mechanism to iteratively optimize the intra-batch model parameters and the inter-batch model parameters include:

[0028] (3.1) Divide the sample set into training batches and preset the number of training rounds;

[0029] (3.2) Record the model parameter A at the beginning of the current training round and the model parameter B at the beginning of the current training batch in the current training round;

[0030] (3.3) For the current training batch in the current training round, calculate the model classification loss using the model parameters B;

[0031] (3.4) randomly sampling a number of historical training samples from the experience buffer to calculate the stability constraint loss under the current model parameters, wherein the stability constraint loss refers to the loss between the model output results of the sampled historical training samples under the current model parameters and the model output results recorded in the experience buffer;

[0032] (3.5) Adaptively adjusting the preset number of samples from the experience buffer according to the stability constraint loss, randomly sampling the preset number of historical training samples to calculate the experience replay loss under the current model parameters, wherein the experience replay loss refers to the loss between the model output results and the sample labels of the sampled historical training samples under the current model parameters;

[0033] (3.6) Update the current model parameters by combining the model classification loss, stability constraint loss and experience replay loss;

[0034] (3.7) Update the model parameter B according to the current model parameter, return to step (3.3) to start the in-batch model parameter optimization of the next training batch, traverse all training batches in the current training round, and execute step (3.8);

[0035] (3.8) Update the model parameter A according to the current model parameter, return to step (3.2) to start the next training round of batch model parameter optimization, until the preset training round is reached, and execute step (3.9);

[0036] (3.9) The malicious traffic detection model after final parameter optimization is obtained according to the model parameter A.

[0037] Furthermore, the step of adaptively adjusting the preset number of samples from the experience buffer according to the stability constraint loss includes:

[0038] Calculate the dynamic playback factor :

[0039]

[0040] in, represents the adjustable temperature coefficient, represents the stability constraint loss, exp represents the natural exponential function;

[0041] According to the dynamic playback factor , sample a preset number of samples from the experience buffer of historical training samples; among them, represents the training batch size, Indicates the floor symbol.

[0042] Furthermore, when updating model parameters A and model parameters B according to the current model parameters, the intra-batch element update rate and the inter-batch element update rate are set, and the product of the difference between the current model parameters and model parameters B and the intra-batch element update rate is used as the update amount of model parameter B, and the sum of model parameter B before the update and the update amount is used as the updated model parameter B; the product of the difference between the current model parameters and model parameters A and the inter-batch element update rate is used as the update amount of model parameter A, and the sum of model parameter A before the update and the update amount is used as the updated model parameter A.

[0043] In a second aspect, the present invention discloses a malicious encrypted traffic detection system based on incremental learning, which is used to implement the above-mentioned malicious encrypted traffic detection method based on incremental learning.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] First, the present invention processes the original traffic into a standardized feature sequence through a double-layer nested data organization structure design, thereby achieving accurate analysis and feature extraction of encrypted network traffic and improving the accuracy and completeness of the original traffic feature expression.

[0046] Second, the present invention designs a collaborative working scheme of dynamic experience playback, stability constraints and two-layer meta-update mechanism, which effectively solves the catastrophic forgetting problem of the malicious traffic detection model in the incremental learning process, so that the model can effectively maintain the memory of historical knowledge while maintaining the ability to learn new knowledge, thereby achieving continuous optimization of model performance.

[0047] Third, the present invention has good scalability and adaptability, and can be flexibly adjusted and optimized according to specific application scenarios, providing strong technical support for malicious encrypted traffic detection in the field of network security; the method can continuously update and optimize detection capabilities with the emergence of new malicious traffic, and always maintain a high detection accuracy, which has important practical value and promotion significance. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a schematic diagram of the malicious encrypted traffic detection system based on incremental learning proposed by the present invention.

[0049] Figure 2 This is the data processing flow chart of the original traffic processing module.

[0050] Figure 3 Data processing flow chart for the pre-training module.

[0051] Figure 4 Data processing flow chart of the incremental learning module.

[0052] Figure 5 Data processing flow chart of the malicious encrypted traffic detection module. DETAILED DESCRIPTION

[0053] The present invention is further described and illustrated below in conjunction with specific embodiments. The embodiments are merely exemplary of the present disclosure and do not define the scope of limitation. The technical features of each embodiment of the present invention may be combined accordingly without conflicting with each other.

[0054] The accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. Some of the block diagrams shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0055] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the steps. For example, some steps may be decomposed, while some steps may be combined or partially combined, so the actual execution order may change according to the actual situation.

[0056] like Figure 1 As shown, the malicious encrypted traffic detection system based on incremental learning proposed by the present invention includes an original traffic processing module, a pre-training module, an incremental learning module and a malicious encrypted traffic detection module. The following is a detailed introduction of each module as follows:

[0057] 1. Original traffic processing module

[0058] The original traffic processing module obtains necessary system operation parameters through a standardized command execution interaction mechanism. Specifically, the system operation parameters include: an input directory path parameter for specifying the storage location of the original network data, an output file path parameter for specifying the storage location of the processing results, and a type identification parameter for filtering the target file. The original traffic processing module sets a global counter in the initialization phase, which is used to record the total number of files processed and the number of abnormal situations encountered during the processing. This counting mechanism is used to achieve accurate monitoring of the processing process.

[0059] The original traffic processing module of the present invention designs a double-layer nested data organization structure, including an outer network flow object and an inner data burst object. The network flow object is used to characterize a complete network session, which includes the following features:

[0060] (1) Session identification information: A four-tuple consisting of the source IP address, source port number, target IP address, and target port number, which is used to uniquely identify a network session.

[0061] (2) Time series information: including the start timestamp and the last activity timestamp of the session, which is used to record the timing characteristics of the session.

[0062] (3) Statistical metering information: Multiple statistics, including packet counters, are used to record the scale characteristics of the session.

[0063] (4) Burst sequence container: used to store and manage all data burst objects in the global set in a session. All data burst objects form a burst sequence based on timestamps.

[0064] The data burst object is used to characterize a continuous sequence of unidirectional data transmissions, and includes the following features:

[0065] (1) Direction identification information: used to indicate the direction of data transmission, using a binary encoding scheme.

[0066] (2) Timestamp information: includes the start and end timestamps of the burst object, which is used to record the continuous characteristics of the burst transmission.

[0067] (3) Packet metering information: including the number of packets and the cumulative byte length (the sum of the total length of each packet), which is used to characterize the scale characteristics of burst transmission.

[0068] like Figure 2 As shown, the data processing flow of the original traffic processing module specifically includes the following steps:

[0069] The first step is basic file processing.

[0070] The system first traverses all target files that match the type identification in the specified directory according to the input directory path parameters and type identification parameters, and reads the file contents one by one through professional network packet processing tools. For each read file, the system establishes an independent processing context to ensure the isolation of file processing and the controllability of exceptions.

[0071] The second step is data packet analysis.

[0072] The system parses the data packet at the link layer, and first tries the standard Ethernet frame parsing solution; if the parsing fails, the system uses the global counter to update the exception count and record detailed error information, while trying alternative link layer parsing solutions. For data packets with exception counts exceeding the threshold, further consideration can be given to eliminating them. After confirming that the parsing is successful, the system further verifies the legitimacy of the upper layer protocol type of the data packet, and illegal data packets do not need to proceed to the next step.

[0073] The third step is protocol analysis.

[0074] The system extracts the basic information elements of the data packet at the network layer, including the total length of the data packet, the source IP address, and the destination IP address. The IP address information is standardized and converted into a string format to facilitate the subsequent construction of the session identifier. The system then identifies the protocol type at the transport layer, extracts the port number information related to the TCP protocol, including the source port number and the destination port number, and ensures the protocol consistency of the data through the protocol feature screening mechanism.

[0075] The fourth step is direction identification.

[0076] The system identifies the data transmission direction based on the port number information of the transport layer, including:

[0077] (1) When the destination port number of a data packet is the HTTPS standard port A, the system identifies it as inbound and constructs a session identifier based on the source information (source IP address and source port number).

[0078] (2) When the source port of the data packet is HTTPS standard port A, the system identifies it as outbound and constructs a session identifier based on the target end information (target end IP address and target port number).

[0079] (3) For data packets that do not meet the above conditions, the system filters them out of the processing flow.

[0080] Step 5: Session management.

[0081] The system maintains a global collection of network flow objects and data burst objects:

[0082] (1) If the current data packet belongs to an existing network flow, update the last activity timestamp and data packet count information of the network flow object, and maintain the data burst object according to the direction identifier of the data packet, where the last activity timestamp is the timestamp of the current data packet. Specifically, if the direction of the current data packet is consistent with the direction of the current data burst object, update the data packet metering information and the end timestamp of the burst sequence of the data burst object, and the end timestamp of the burst sequence is updated to the timestamp of the current data packet; if the direction of the current data packet is inconsistent with the direction of the current data burst object, it means that the direction of the current data packet in the data flow has changed. At this time, create a new data burst object and initialize the direction identification information, data packet metering information, the start timestamp of the burst sequence, and the end timestamp of the burst sequence, where the direction identification information is the direction of the current data packet, the start timestamp of the burst sequence is the timestamp of the current data packet, and the direction identification information and the start timestamp of the burst sequence do not need to be updated later; the end timestamp of the burst sequence is the timestamp of the current data packet.

[0083] (2) If the current data packet belongs to a new network flow, a network flow object is created and the first data burst object is initialized and added to the global collection. The method of initializing the first data burst object is the same as the method of creating a new data burst object.

[0084] Step 6: Generate results.

[0085] The above steps process the original traffic into several network flow objects, each of which contains several data burst objects. The features of the processed network flow and the features of the data burst objects are integrated and converted into a standardized data structure, that is, a standardized feature sequence reflecting the network behavior pattern is extracted. As a training sample, corresponding label information needs to be generated. The label information includes normal encrypted traffic and malicious encrypted traffic. Finally, the standardized feature sequence and label information are written into the specified output file according to the output file path parameter to ensure the persistent storage of data.

[0086] The original traffic processing module of the present invention realizes accurate analysis and feature extraction of encrypted network traffic through the above technical scheme. The present invention processes network flow objects and data burst objects in layers. The layered data structure design provides flexible feature expression capabilities, the direction identification mechanism ensures accurate capture of traffic features, and the dynamic maintenance of burst sequences realizes fine-grained behavioral feature extraction. Through the exception handling mechanism of systematic protocol type legitimacy verification and efficient data organization, the present invention ensures the reliability and efficiency of data processing, laying a solid technical foundation for subsequent malicious encrypted traffic detection.

[0087] 2. Pre-training module

[0088] like Figure 3 As shown in FIG. 1 , the data processing flow of the pre-training module specifically includes the following steps:

[0089] The first step is to build the model structure.

[0090] In this embodiment, a Transformer-based malicious traffic detection model is constructed. The Transformer-based deep learning architecture is adopted, and feature extraction is performed through a multi-head self-attention mechanism and a feedforward neural network, combined with an adaptive hyperparameter optimization mechanism, which can significantly improve the model's ability to detect malicious encrypted traffic and enhance the model's expressiveness and generalization performance.

[0091] First, a fully connected layer is used to map the normalized feature sequence of the original traffic to dimensional feature space. The fully connected layer uses a linear transformation matrix and the bias vector Implement feature mapping, where is the input feature dimension, It is the preset model dimension parameter.

[0092] The above transformation process can be expressed as:

[0093]

[0094] in, is the input standardized feature sequence, is the feature representation after mapping.

[0095] Next, the mapped feature representation is input into the Transformer encoder layer for processing. Each Transformer encoder layer contains two core sublayers: multi-head self-attention mechanism and feedforward neural network. In the multi-head self-attention mechanism, the mapped feature The query vectors are obtained by linear transformation. , key vector Sum value vector :

[0096]

[0097]

[0098]

[0099] in, , , is the corresponding weight matrix. Then, the attention weight is obtained by scaling the dot product attention:

[0100]

[0101] In the formula, is the dimension of the key vector, Used to scale the attention score. This embodiment uses nhead parallel attention heads, each of which independently calculates the output features, and finally obtains the output of the multi-head attention through connection operations and linear transformations.

[0102] The second step is data preprocessing and loading.

[0103] The data preprocessing process of the present invention first reads the characteristic sequence data generated by the original traffic processing module from the preset file path, including the normal traffic sample set and malicious traffic sample sets , each sample contains m feature dimensions. For each sample , perform the following standardization:

[0104]

[0105] in, is the feature mean vector, is the feature standard deviation vector. The standardized samples are assigned corresponding labels , where the normal traffic sample label is 0 and the malicious traffic sample label is 1.

[0106] To achieve efficient batch training, the present invention uses the DataLoader component of the PyTorch framework for data loading. Specifically, batch_size=B is set, where B is the preset batch size parameter. At the beginning of each training cycle, a small batch of data is constructed by random sampling. .

[0107] The third step is the model training process.

[0108] The present invention adopts an iterative training method based on the Adam optimizer. In each training step, the input batch data is first forward propagated through the model structure described in the first step. For each sample in the batch, the model outputs a two-dimensional probability vector ,in Indicates the probability that the sample belongs to normal traffic, Indicates the probability that the sample is malicious traffic.

[0109] The model uses cross entropy as the loss function to calculate the prediction error:

[0110]

[0111] Where B is the batch size, is the true label of the sample. After the loss calculation is completed, the gradient of the parameters of each layer of the model is calculated by the back propagation algorithm, and the parameters are updated using the Adam optimizer:

[0112]

[0113] in, represents the model parameters at step t, is the learning rate, and are the first-order momentum and the second-order momentum, is the smoothing parameter.

[0114] The fourth step is model evaluation and storage.

[0115] At the end of each training cycle, the present invention monitors the model performance by calculating the following evaluation metrics:

[0116] 1. Accuracy:

[0117]

[0118] 2. Recall:

[0119]

[0120] 3. Precision:

[0121]

[0122] 4. F1 score:

[0123]

[0124] Among them, TP, TN, FP, and FN represent the number of true positives, true negatives, false positives, and false negatives, respectively.

[0125] The fifth step is hyperparameter optimization.

[0126] In order to obtain the best model performance, the present invention implements a set of hyperparameter optimization mechanisms. During the training process, the system will adaptively adjust key hyperparameters according to the model performance, including:

[0127] 1. Learning rate decay: When the performance of the validation set does not improve for n consecutive cycles, the learning rate is adjusted to:

[0128]

[0129] in, is the new learning rate after adjustment, is the current learning rate, is the decay rate.

[0130] 2. Batch size adjustment: According to the system resource usage, Dynamically adjust batch size within range:

[0131]

[0132] in, is the new batch size after adjustment, is the current batch size, is the adjustment factor used to increase or decrease the batch size, is the minimum batch allowed, is the maximum batch allowed.

[0133] 3. Transformer layer parameter optimization: Determine the optimal number of encoder layers num_layers, number of attention heads n_head, and feedforward network dimension dim_feedforward through grid search method.

[0134] 3. Incremental Learning Module

[0135] The present invention provides an incremental learning method based on dynamic experience replay, stability constraint and double-layer meta-update, which is used to solve the catastrophic forgetting problem of deep learning models in the incremental learning process.

[0136] like Figure 4 As shown in FIG. 1 , the data processing flow of the incremental learning module specifically includes the following steps:

[0137] The first step is to build the basic network architecture.

[0138] The incremental learning module includes two core components: the experience buffer and the pre-trained malicious traffic detection model. The pre-trained malicious traffic detection model is represented by a mapping function:

[0139]

[0140] in, represents the parameterized neural network mapping function, represents the input feature space, represents the output label space, Represents the set of learnable parameters of the model.

[0141] To achieve the storage of historical knowledge, the experience buffer is defined as a set of triples:

[0142]

[0143] in, represents the experience buffer, represents the feature vector of the i-th sample, represents the true label of the i-th sample, Represents the historical logic output of the i-th sample, M represents the upper limit of the buffer capacity, and i represents the sample index.

[0144] After the malicious traffic detection model completes pre-training, a preset number of historical training samples are randomly selected from the sample set, and the selected historical training samples and their model output results and sample labels are added to the experience buffer to initialize the experience buffer.

[0145] The second step is data preprocessing.

[0146] Based on the above infrastructure, the input training data needs to be preprocessed. The training data set D is divided into N batches, which can be expressed as:

[0147]

[0148] in, represents the feature matrix of the jth batch, b represents the batch size, and d represents the feature dimension; Represents the corresponding label vector, j represents the batch index, and N represents the total number of batches.

[0149] The third step is parameter initialization and forward propagation.

[0150] After completing data preprocessing, the model training phase begins. For training round e, first obtain the initial parameters ; For batch j in training round e, record the initial model parameters . The model output is calculated by forward propagation:

[0151]

[0152] Wherein, c represents the number of categories, and in the present invention, c=2; represents the logical matrix output of the jth batch, and e represents the current training round.

[0153] The fourth step is to calculate the current task loss.

[0154] Based on the above model output , the cross entropy loss function is used to calculate the classification loss of the current task:

[0155]

[0156] in, represents the cross entropy loss value, b represents the batch size, and c represents the number of categories. represents the true label value of the kth category of the i-th sample in the j-th batch, Represents the corresponding model output value, and softmax represents the normalization function.

[0157] Step 5: Calculate the stability constraint loss.

[0158] After obtaining the classification loss of the current batch After that, this step introduces a stability constraint mechanism to prevent drastic changes in the model output. Specifically, m samples are randomly sampled from the experience buffer. , calculate the mean square error loss:

[0159]

[0160] in, represents the mean square error loss value, which is used as the stability constraint loss; m represents the number of sampling samples, represents the output of the current model for the i-th buffer sample, represents the historical output stored for the i-th sample, Represents the square of the Euclidean norm.

[0161] Step 6: Calculate the dynamic playback coefficient.

[0162] This step introduces a dynamic experience replay mechanism. Based on the stability constraint loss calculated in the previous step, this step dynamically adjusts the number of replay samples through an adaptive mechanism:

[0163]

[0164] in, represents the dynamic playback coefficient, represents the adjustable temperature coefficient used to control the function curvature, represents the stability constraint loss calculated in the fifth step, and exp represents the natural exponential function.

[0165] Step 7: Calculate the experience replay loss.

[0166] Using the dynamic playback coefficient obtained in step 6 , sampled from the experience buffer Samples , calculate the replay loss:

[0167]

[0168] in, Represents the experience replay loss value, represents the number of playback samples after rounding down, c represents the number of categories, represents the true label value of the kth category of the i-th playback sample, Represents the model's predicted output for the kth category of the i-th playback sample.

[0169] Step 8: Multiple loss integration and parameter update.

[0170] The loss functions calculated in the previous steps are weighted and integrated to construct the total loss function:

[0171]

[0172] in, represents the total loss value, represents the trade-off coefficient of knowledge distillation loss, represents the trade-off coefficient of the experience replay loss. Update the parameters by stochastic gradient descent:

[0173]

[0174] in, Indicates the current parameters of the model. represents the learning rate, Denotes the total loss with respect to the parameter gradient.

[0175] Step 9: Perform batch meta-update.

[0176] After completing the basic parameter update, this step introduces the intra-batch meta-update mechanism and uses the Reptile algorithm to perform batch internal parameter meta-optimization:

[0177]

[0178] in, represents the updated parameters of the batch element, represents the initial parameters of the jth batch in the eth round, Indicates the current parameters. represents the intra-batch meta-update rate, e represents the training round index, and j represents the batch index.

[0179] Return to the third step above to start the in-batch model parameter optimization for the next training batch. After traversing all training batches in the current training round, execute the tenth step.

[0180] Step 10: Perform inter-batch meta-updates.

[0181] After completing a full training round, this step introduces an inter-batch meta-update mechanism to further improve the generalization ability of the model by performing inter-batch meta-update optimization:

[0182]

[0183] in, represents the updated parameters between batches, represents the initial parameters of the e-th round, Indicates the current parameters. Represents the inter-batch meta-update rate.

[0184] Return to the third step above to start the next training round of batch model parameter optimization until the preset training round is reached, and the malicious traffic detection model with final parameter optimization is obtained according to the parameters updated by the final batch element.

[0185] Step 11: Model evaluation.

[0186] After completing the parameter update through the two-layer meta-update mechanism including the intra-batch meta-update mechanism and the inter-batch meta-update mechanism, the following evaluation indicators are calculated:

[0187] Calculate the accuracy:

[0188]

[0189] Among them, TP represents the number of true positive examples (True Positive), TN represents the number of true negative examples (True Negative), FP represents the number of false positive examples (False Positive), and FN represents the number of false negative examples (False Negative).

[0190] Calculate the precision:

[0191]

[0192] Calculate the recall rate (Recall):

[0193]

[0194] Calculate the F1 score:

[0195]

[0196] Among the above evaluation indicators, accuracy reflects the overall prediction accuracy of the model, precision reflects the accuracy of positive example prediction, recall reflects the completeness of positive example detection, and F1 score comprehensively considers precision and recall to provide a balanced performance measurement.

[0197] Step 12: Update the experience buffer.

[0198] After completing model training and evaluation, this step updates the experience buffer to prepare for subsequent incremental learning tasks. Update the last batch of data to the experience buffer:

[0199]

[0200] Where B represents the experience buffer zone, represents the feature vector of the i-th sample in the last batch, represents the corresponding true label, represents the corresponding model output value, represents the number of samples in the last batch, and ∪ represents the set union operator.

[0201] When the number of samples in the last batch of data exceeds the preset capacity M, a random sampling strategy is used to maintain a fixed size. Specifically, samples to be replaced are randomly selected from the last batch of data and stored in the experience buffer, ensuring that the buffer size does not exceed the preset capacity M while maintaining the randomness of the sample distribution.

[0202] The present invention realizes a new incremental learning method through the above steps, wherein the stability constraint mechanism effectively maintains the model's memory of historical knowledge by limiting the output changes of the model on historical data; the dynamic experience playback mechanism realizes the targeted review of historical knowledge by adaptively adjusting the number of playback samples; the two-layer meta-update mechanism significantly improves the generalization performance of the model in continuous learning tasks through intra-batch and inter-batch parameter optimization; the synergistic effect of the above three mechanisms enables the present invention to effectively alleviate the catastrophic forgetting problem of deep learning models in the incremental learning process, while maintaining the model's ability to learn new knowledge. In addition, the method of the present invention has good scalability and adaptability, and can be flexibly adjusted and optimized according to specific application scenarios, providing strong support for incremental learning tasks in different fields.

[0203] (IV) Malicious encrypted traffic detection module

[0204] like Figure 5 As shown in FIG. 1 , the data processing flow of the malicious encrypted traffic detection module specifically includes the following steps:

[0205] In the first step, the encrypted network traffic to be inspected is received as a network flow object.

[0206] In the second step, the feature sequence of network flow objects is extracted using the processing method of the original traffic processing module.

[0207] The third step is to test the model trained by the pre-training module and the incremental learning module.

[0208] The fourth step is to output the detection results.

[0209] The effects of the present invention are further described below in conjunction with simulation experiments.

[0210] Simulation experiment conditions: The operating environment of the simulation experiment of this invention is a Linux server with NVIDIA GeForce RTX 3090 GPU. The libraries used include NumPy 1.21.2, Pytorch 1.9.1, Tensorflow 2.10.0 and Cuda 11.7, and the programming environment is Python 3.8.5.

[0211] The datasets used in the simulation are the DoHBrw dataset and the CIC dataset, where:

[0212] The DoHBrw dataset provides a mix of benign and malicious DNS-over-HTTPS (DoH) traffic, all of which is encrypted traffic. Normal traffic is generated by querying benign DNS servers using the DoH protocol. Tunneling tools such as dns2tcp, DNSCat2, and Iodine are used to generate malicious DoH traffic.

[0213] The CIC dataset collects a rich variety of malicious attacks from multiple sources. Malicious traffic samples come from 42 unique malware families and can be divided into four categories: adware, ransomware, scareware, and SMS malware.

[0214] The existing comparison methods used in the simulation are as follows:

[0215] 1. RAPIER makes full use of the different distributions of normal and malicious traffic data in the feature space, where normal data is densely distributed in a certain area, while malicious data is scattered throughout the feature space to increase the training data for model training.

[0216] 2. FS-Net adopts a multi-layer encoder-decoder structure, which can deeply mine the potential sequential features of the stream and introduce a reconstruction mechanism that can enhance the effectiveness of the features.

[0217] 3. ETGuard is a novel malicious encrypted traffic detection framework that can automatically detect malicious encrypted packet sequences and gradually learn from new malicious packet sequences.

[0218] Simulation experiment 1: Comparison of the malicious encrypted traffic identification accuracy of the present invention and three existing methods.

[0219] The simulation process is as follows:

[0220] First, the original traffic processing module of the present invention performs data processing, and the present invention and three existing encrypted traffic detection methods are used to obtain encrypted traffic data detection results.

[0221] Secondly, according to the encrypted traffic data recognition results, the number of correctly identified target encrypted traffic TP, the number of correctly identified other encrypted traffic TN, the number of incorrectly identified target encrypted traffic FP, and the number of missed target encrypted traffic FN are counted, and then the accuracy and recall are calculated respectively. According to the calculation results of the accuracy and recall, the F1 score is calculated as shown in Table 1 below:

[0222] Table 1

[0223]

[0224] Table 1 above shows the F1 scores of malicious encrypted traffic detection of each method on the corresponding data set. It is obvious that the present invention has been significantly improved compared with other malicious traffic detection methods, indicating that the malicious encrypted traffic identification accuracy of the present invention is higher than that of the three existing methods.

[0225] In this embodiment, a malicious encrypted traffic detection system based on incremental learning for implementing the above detection method is also provided, including:

[0226] The original traffic processing module is used to process the original traffic into a standardized feature sequence using a double-layer nested data organization structure, wherein the original traffic includes normal encrypted traffic and malicious encrypted traffic, and construct a sample set. In this embodiment, the module serves as a data processing link, and uses a double-layer data structure (network flow object and data burst object) to parse and extract features from the original encrypted traffic, and finally generates a standardized feature sequence by reading network data packets, multi-layer parsing (link layer, network layer, transport layer), direction identification, and network session maintenance.

[0227] A pre-training module is used to pre-train the malicious traffic detection model using a sample set; in this embodiment, the module completes the malicious traffic detection model pre-training through data standardization, batch training, cross entropy loss function optimization, and hyperparameter adjustment.

[0228] The incremental learning module is used to optimize the model parameters of the pre-trained malicious traffic detection model by incremental learning. During the incremental learning process, an experience buffer is provided for storing historical training samples and their model output results and sample labels. The experience buffer is continuously updated as the model parameter optimization process progresses. This module is committed to solving the problem of catastrophic forgetting of the model during continuous learning. By maintaining the experience buffer to store historical knowledge, combined with dynamic experience playback, stability constraints and a two-layer meta-update mechanism, the module can achieve continuous optimization and updating of the malicious traffic detection model.

[0229] The incremental learning method is used to optimize the model parameters, including: based on the model classification loss, using a stability constraint mechanism to limit the fluctuation of the model output results of the historical training samples and calculating the stability constraint loss, using an experience replay mechanism to adaptively adjust the number of samples from the experience buffer and calculate the experience replay loss; combining the model classification loss, stability constraint loss and experience replay loss, using a double-layer meta-update mechanism to iteratively optimize the intra-batch model parameters and the inter-batch model parameters;

[0230] The malicious encrypted traffic detection module is used to detect the traffic to be detected by using the trained detection model to detect whether the traffic to be detected is malicious encrypted traffic.

[0231] For the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment, and the implementation methods of the remaining modules will not be repeated here. The system embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0232] The embodiments of the system of the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. The system embodiments can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, the corresponding computer program instructions in the non-volatile memory are read into the memory by the processor of any device with data processing capabilities and run.

[0233] The above-mentioned embodiments only express several implementation modes of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. For those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A method for detecting malicious encrypted traffic based on incremental learning, characterized in that: The following steps are involved: (1) A double-layer nested data organization structure is used to process the original traffic into a standardized feature sequence, wherein the original traffic includes normal encrypted traffic and malicious encrypted traffic, and a sample set is constructed; the double-layer nested data organization structure includes an outer network flow object and an inner data burst object, wherein the characteristics of the network flow object include session identification information, time series information, statistical measurement information, and a burst sequence container for storing all data burst objects contained in the original traffic; the characteristics of the data burst object include direction identification information, timestamp information, and the number and cumulative byte length of all data packets contained in the data burst object; (2) Use the sample set to pre-train the malicious traffic detection model; (3) The pre-trained malicious traffic detection model is optimized using an incremental learning method. During the incremental learning process, an experience buffer is provided for storing historical training samples and their model output results and sample labels. The experience buffer is continuously updated as the model parameter optimization process progresses; The incremental learning method is used to optimize the model parameters, including: based on the model classification loss, using a stability constraint mechanism to limit the fluctuation of the model output results of the historical training samples and calculating the stability constraint loss, using an experience replay mechanism to adaptively adjust the number of samples from the experience buffer and calculate the experience replay loss; combining the model classification loss, stability constraint loss and experience replay loss, using a double-layer meta-update mechanism to iteratively optimize the intra-batch model parameters and the inter-batch model parameters; (4) Use the malicious traffic detection model with optimized parameters to detect whether the encrypted traffic is malicious encrypted traffic.

2. The method for detecting malicious encrypted traffic based on incremental learning according to claim 1 is characterized in that: The session identification information includes the source IP address, source port number, target IP address and target port number, the time series information includes the start timestamp and the last activity timestamp of the original traffic session, the statistical metering information includes the number of data packets, the direction identification information represents the data transmission direction, and the timestamp information includes the start and end timestamps of the data burst object.

3. The method for detecting malicious encrypted traffic based on incremental learning according to claim 1 is characterized in that: The double-layer nested data organization structure is used to process the original traffic into a standardized feature sequence, including: (1.1) Read the original traffic file and parse the data packet, verify the legitimacy of the data packet protocol type, and remove illegal data packets; (1.2) Extract the total length, source IP address, destination IP address, source port number and destination port number of each data packet, and construct the session identifier of the data packet based on the source port number and destination port number to represent the data transmission direction; (1.3) Determine whether the current data packet belongs to an existing network flow object based on the source IP address, target IP address, source port number, and target port number of the current data packet: If so, update the characteristics of the network flow object, and maintain the characteristics of the data burst object according to the session identifier of the current data packet: if the session identifier of the current data packet is consistent with the direction identification information of the current data burst object, update the characteristics of the data burst object, otherwise create a new data burst object and initialize the characteristics; If not, create a network stream object and initialize the first data burst object; (1.4) Processing the original traffic into several network flow objects through steps (1.1) to (1.3), each network flow object contains several data burst objects, integrating the characteristics of the network flow objects and the characteristics of the data burst objects to obtain a standardized feature sequence of the original traffic.

4. The method for detecting malicious encrypted traffic based on incremental learning according to claim 1, characterized in that: The malicious traffic detection model uses a Transformer encoder.

5. The method for detecting malicious encrypted traffic based on incremental learning according to claim 1, characterized in that: The process of continuously updating the experience buffer as the model parameter optimization process progresses includes: After the malicious traffic detection model completes pre-training, a preset number of historical training samples are randomly selected from the sample set, and the selected historical training samples and their model output results and sample labels are added to the experience buffer to complete the initialization of the experience buffer; In the incremental learning process, after a round of model parameter optimization is completed, a preset number of historical training samples are randomly selected from the last batch of training samples in that round of optimization, and the selected historical training samples and their model output results and sample labels are added to the experience buffer.

6. The method for detecting malicious encrypted traffic based on incremental learning according to claim 1, characterized in that: The described combination of the model classification loss, stability constraint loss and experience replay loss adopts a double-layer meta-update mechanism to iteratively optimize the intra-batch model parameters and the inter-batch model parameters, including: (3.1) Divide the sample set into training batches and preset the number of training rounds; (3.2) Record the model parameter A at the beginning of the current training round and the model parameter B at the beginning of the current training batch in the current training round; (3.3) For the current training batch in the current training round, calculate the model classification loss using the model parameters B; (3.4) randomly sampling a number of historical training samples from the experience buffer to calculate the stability constraint loss under the current model parameters, wherein the stability constraint loss refers to the loss between the model output results of the sampled historical training samples under the current model parameters and the model output results recorded in the experience buffer; (3.5) Adaptively adjusting the preset number of samples from the experience buffer according to the stability constraint loss, randomly sampling the preset number of historical training samples to calculate the experience replay loss under the current model parameters, wherein the experience replay loss refers to the loss between the model output results and the sample labels of the sampled historical training samples under the current model parameters; (3.6) Update the current model parameters by combining the model classification loss, stability constraint loss and experience replay loss; (3.7) Update the model parameter B according to the current model parameter, return to step (3.3) to start the in-batch model parameter optimization of the next training batch, traverse all training batches in the current training round, and execute step (3.8); (3.8) Update the model parameter A according to the current model parameter, return to step (3.2) to start the next training round of batch model parameter optimization, until the preset training round is reached, and execute step (3.9); (3.9) The malicious traffic detection model after final parameter optimization is obtained according to the model parameter A.

7. The method for detecting malicious encrypted traffic based on incremental learning according to claim 6 is characterized in that: The method of adaptively adjusting the preset number of samples from the experience buffer according to the stability constraint loss includes: Calculate the dynamic playback factor : ； in, represents the adjustable temperature coefficient, represents the stability constraint loss, exp represents the natural exponential function; According to the dynamic playback factor , sample a preset number of samples from the experience buffer of historical training samples; among them, represents the training batch size, Indicates the floor symbol.

8. The method for detecting malicious encrypted traffic based on incremental learning according to claim 6, characterized in that: When updating model parameters A and model parameters B according to the current model parameters, the intra-batch meta-update rate and the inter-batch meta-update rate are set, and the product of the difference between the current model parameters and model parameter B and the intra-batch meta-update rate is used as the update amount of model parameter B, and the sum of model parameter B before the update and the update amount is used as the updated model parameter B; the product of the difference between the current model parameters and model parameter A and the inter-batch meta-update rate is used as the update amount of model parameter A, and the sum of model parameter A before the update and the update amount is used as the updated model parameter A.

9. A malicious encrypted traffic detection system based on incremental learning, characterized in that: include: The original traffic processing module is used to process the original traffic into a standardized feature sequence by adopting a double-layer nested data organization structure, wherein the original traffic includes normal encrypted traffic and malicious encrypted traffic, and construct a sample set; the double-layer nested data organization structure includes an outer network flow object and an inner data burst object, wherein the characteristics of the network flow object include session identification information, time series information, statistical measurement information, and a burst sequence container for storing all data burst objects contained in the original traffic; the characteristics of the data burst object include direction identification information, timestamp information, and the number of all data packets contained in the data burst object and the cumulative byte length; A pre-training module, which is used to pre-train a malicious traffic detection model using a sample set; An incremental learning module is used to optimize the model parameters of the pre-trained malicious traffic detection model by incremental learning. During the incremental learning process, an experience buffer is provided for storing historical training samples and their model output results and sample labels. The experience buffer is continuously updated as the model parameter optimization process progresses; The incremental learning method is used to optimize the model parameters, including: based on the model classification loss, using a stability constraint mechanism to limit the fluctuation of the model output results of the historical training samples and calculating the stability constraint loss, using an experience replay mechanism to adaptively adjust the number of samples from the experience buffer and calculate the experience replay loss; combining the model classification loss, stability constraint loss and experience replay loss, using a double-layer meta-update mechanism to iteratively optimize the intra-batch model parameters and the inter-batch model parameters; The malicious encrypted traffic detection module is used to detect whether the encrypted traffic is malicious encrypted traffic by using the malicious traffic detection model after parameter optimization.

Images