Label propagation-based small sample malicious traffic detection method and system

Abstract

The invention discloses a small sample malicious traffic detection method and system based on label propagation, and the method comprises the steps: carrying out the preprocessing of an obtained original traffic file, obtaining a session representation sequence, extracting a packet header feature sequence and a load feature sequence from the session representation sequence, and obtaining a comprehensive representation vector based on the sequences, and mapping the comprehensive representation vector into a hyperbolic space, taking the comprehensive representation vector labeled with a label as a support set sample, taking the comprehensive representation vector not labeled with the label as a query set sample, and constructing a sample relation graph based on a K-nearest neighbor algorithm, thereby greatly enhancing the representation capability of the model for the complex network flow relation. And the labels of the support set samples are propagated to the similar query set samples by using a label propagation algorithm based on the sample relation graph to obtain the prediction labels of the query set samples, and the malicious traffic in the to-be-detected traffic file is identified according to the prediction labels of the query set samples, so that the malicious traffic detection precision and generalization ability are improved.

Description

Technical Field

[0001] The present invention relates to the field of network security, and in particular to a small sample malicious traffic detection method and system based on label propagation. Background Art

[0002] The continuous expansion of the Internet has enabled it to carry more and more services, and has also caused the scale of network traffic data to become larger and more complex, which has created troubles for Internet supervision and created opportunities for criminals. More and more network attack traffic appears in the network environment, network security incidents occur frequently, and network security issues are becoming increasingly serious. Network attacks are usually hidden. When a network attack is discovered, it has often caused very significant losses and often attracts widespread social attention. Therefore, how to detect malicious traffic in network traffic and provide safer network services has attracted more and more attention.

[0003] According to different implementation methods, the current research on malicious traffic detection can be roughly divided into three types: rule-matching-based detection methods, machine learning-based detection methods, and deep learning-based detection methods. The rule-matching-based malicious traffic detection method detects malicious behavior in network traffic by extracting specific fields from network traffic data and matching them with a pre-defined rule base. Due to the variability of network traffic and the widespread application of encrypted traffic, the missed detection rate of the rule-matching-based method is greatly increased. The machine learning-based malicious traffic detection method combines machine learning methods with features manually extracted from raw traffic to detect malicious network traffic. The performance of this method is limited by the quality of manually extracted features. The deep learning-based malicious traffic detection method combines feature extraction with classification, automatically learns feature representations in data through a carefully designed neural network model, and detects malicious traffic based on the extracted features. Compared with the first two methods, it has higher flexibility and generalization ability, and can achieve high-precision traffic detection.

[0004] Although deep learning-based detection methods have many advantages over other methods, they still have some problems. Deep learning-based methods usually have very deep network layers, and require a large amount of data to train the model to achieve good performance. In real environments, for zero-day attacks, only a few samples can be collected for training. When there are few training samples, deep learning-based methods are prone to overfitting, resulting in a decrease in the detection performance of the model. At the same time, when faced with new categories of samples, deep learning-based methods need to retrain the entire model and cannot quickly build detection capabilities for new categories. How to quickly build good malicious traffic detection capabilities with a few samples has become a thorny problem in the field of malicious traffic detection. Small sample learning aims to solve classification problems with a small number of samples, which is very suitable for solving this problem. There have been many research works on its application in the field of image recognition, but its application in the field of malicious traffic detection is still relatively small.

[0005] However, the existing small sample detection models in the field of malicious traffic detection pay too much attention to the commonalities of categories. Since malicious traffic is usually disguised as normal traffic, and the amount of normal traffic is large and complex, it is easy to learn too much commonality of normal traffic and neglect the detection and generalization of malicious traffic, which affects the performance of the model in detecting malicious traffic. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a small sample malicious traffic detection method and system based on label propagation, which can improve the accuracy and generalization ability of malicious traffic detection.

[0007] In order to solve the above technical problems, a technical solution adopted by the present invention is:

[0008] A small sample malicious traffic detection method based on label propagation, comprising the steps of:

[0009] Preprocessing the acquired original traffic files to obtain a session representation sequence, wherein the original traffic files include traffic files to be detected and traffic files with marked labels;

[0010] Extracting a packet header feature sequence and a load feature sequence from the session representation sequence, and obtaining a comprehensive representation vector based on the packet header feature sequence and the load feature sequence;

[0011] Mapping the comprehensive representation vector into a hyperbolic space, taking the labeled comprehensive representation vector as a support set sample, taking the unlabeled comprehensive representation vector as a query set sample, and constructing a sample relationship graph of the support set sample and the query set sample based on a K-nearest neighbor algorithm;

[0012] Based on the sample relationship graph, a label propagation algorithm is used to propagate the labels of the support set samples to similar query set samples to obtain predicted labels of the query set samples, and malicious traffic in the traffic file to be detected is identified according to the predicted labels of the query set samples.

[0013] In order to solve the above technical problems, another technical solution adopted by the present invention is:

[0014] A small sample malicious traffic detection system based on label propagation includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented:

[0015] Preprocessing the acquired original traffic files to obtain a session representation sequence, wherein the original traffic files include traffic files to be detected and traffic files with marked labels;

[0016] Extracting a packet header feature sequence and a load feature sequence from the session representation sequence, and obtaining a comprehensive representation vector based on the packet header feature sequence and the load feature sequence;

[0017] Mapping the comprehensive representation vector into a hyperbolic space, taking the labeled comprehensive representation vector as a support set sample, taking the unlabeled comprehensive representation vector as a query set sample, and constructing a sample relationship graph of the support set sample and the query set sample based on a K-nearest neighbor algorithm;

[0018] Based on the sample relationship graph, a label propagation algorithm is used to propagate the labels of the support set samples to similar query set samples to obtain predicted labels of the query set samples, and malicious traffic in the traffic file to be detected is identified according to the predicted labels of the query set samples.

[0019] The beneficial effects of the present invention are: preprocessing the acquired original traffic file to obtain a session representation sequence, extracting the packet header feature sequence and the load feature sequence from the session representation sequence, and obtaining a comprehensive representation vector based on the packet header feature sequence and the load feature sequence, thereby avoiding missing key feature information, and then mapping the comprehensive representation vector to the hyperbolic space, using the labeled comprehensive representation vector as a support set sample, and using the unlabeled comprehensive representation vector as a query set sample, and constructing a sample relationship graph of the support set sample and the query set sample based on the K nearest neighbor algorithm, more accurately establishing the relationship between the samples, greatly enhancing the model's representation ability for complex network traffic relationships, and using a label propagation algorithm based on the sample relationship graph to propagate the labels of the support set samples to similar query set samples, obtaining predicted labels of the query set samples, and identifying malicious traffic in the traffic file to be detected based on the predicted labels of the query set samples, effectively solving the label scarcity problem in small sample learning, and enhancing the generalization ability of the model, especially when facing scarce label data, being able to improve classification performance, thereby improving the accuracy and generalization ability of malicious traffic detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flowchart of a method for detecting small sample malicious traffic based on label propagation according to an embodiment of the present invention;

[0021] Figure 2 A schematic diagram of the structure of a small sample malicious traffic detection system based on label propagation according to an embodiment of the present invention;

[0022] Figure 3 It is a schematic diagram of the overall process of a small sample malicious traffic detection method based on label propagation in an embodiment of the present invention;

[0023] Figure 4 A schematic diagram of feature extraction in a small sample malicious traffic detection method based on label propagation according to an embodiment of the present invention;

[0024] Figure 5 Schematic diagram of traffic detection report in a small sample malicious traffic detection method based on label propagation according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to explain the technical content, achieved objectives and effects of the present invention in detail, the following is an explanation in combination with the implementation modes and the accompanying drawings.

[0026] Please refer to Figure 1 , a small sample malicious traffic detection method based on label propagation, comprising the steps of:

[0027] Preprocessing the acquired original traffic files to obtain a session representation sequence, wherein the original traffic files include traffic files to be detected and traffic files with marked labels;

[0028] Extracting a packet header feature sequence and a load feature sequence from the session representation sequence, and obtaining a comprehensive representation vector based on the packet header feature sequence and the load feature sequence;

[0029] Mapping the comprehensive representation vector into a hyperbolic space, taking the labeled comprehensive representation vector as a support set sample, taking the unlabeled comprehensive representation vector as a query set sample, and constructing a sample relationship graph of the support set sample and the query set sample based on a K-nearest neighbor algorithm;

[0030] Based on the sample relationship graph, a label propagation algorithm is used to propagate the label of the support set sample to similar query set samples to obtain the predicted label of the query set sample, and malicious traffic in the traffic file to be detected is identified according to the predicted label of the query set sample.

[0031] From the above description, it can be seen that the beneficial effects of the present invention are: preprocessing the acquired original traffic file to obtain a session representation sequence, extracting the packet header feature sequence and the load feature sequence from the session representation sequence, and obtaining a comprehensive representation vector based on the packet header feature sequence and the load feature sequence, thereby avoiding missing key feature information, and then mapping the comprehensive representation vector to the hyperbolic space, using the labeled comprehensive representation vector as a support set sample, and using the unlabeled comprehensive representation vector as a query set sample, and constructing a sample relationship graph of the support set sample and the query set sample based on the K nearest neighbor algorithm, more accurately establishing the relationship between the samples, greatly enhancing the model's ability to represent complex network traffic relationships, and using a label propagation algorithm based on the sample relationship graph to propagate the labels of the support set samples to similar query set samples, obtaining predicted labels of the query set samples, and identifying malicious traffic in the traffic file to be detected based on the predicted labels of the query set samples, effectively solving the label scarcity problem in small sample learning, and enhancing the generalization ability of the model, especially when facing scarce label data, can improve the classification performance, thereby improving the accuracy and generalization ability of malicious traffic detection.

[0032] Furthermore, the preprocessing of the acquired original traffic file to obtain the session representation sequence includes:

[0033] Splitting the acquired original traffic file into a plurality of session files, each of the session files containing an independent session;

[0034] Filter invalid session files from the multiple session files to obtain multiple filtered session files;

[0035] Selecting a preset number of data packets from each filtered session file as session representations, and performing anonymization processing on the session representations to obtain anonymized session representations;

[0036] Performing padding and alignment on the anonymized session representation to obtain a padded and aligned session representation, and performing normalization processing on the padded and aligned session representation to obtain a normalized session representation;

[0037] A session representation sequence is obtained according to the normalized session representation.

[0038] From the above description, it can be seen that multiple session files are filtered, anonymized, filled, aligned and normalized in sequence to ensure that different session data are at the same scale, thereby improving the subsequent malicious traffic detection effect.

[0039] Further, extracting a packet header feature sequence and a load feature sequence from the session representation sequence, and obtaining a comprehensive representation vector based on the packet header feature sequence and the load feature sequence includes:

[0040] Splitting the session representation sequence to obtain a packet header feature sequence and a load feature sequence;

[0041] Inputting the packet header feature sequence into a first preset feature extraction unit for feature extraction, and outputting a packet header feature representation vector;

[0042] Inputting the load feature sequence into a second preset feature extraction unit for feature extraction, and outputting a load feature representation vector;

[0043] The packet header feature representation vector and the load feature representation vector are concatenated to obtain a comprehensive representation vector of the entire session.

[0044] From the above description, it can be seen that the session representation sequence is split to obtain the packet header feature sequence and the load feature sequence, and independent feature extraction is performed on each of them, which can capture the potential timing relationship and important information in the packet header feature sequence. At the same time, the content of the load feature sequence has a large change due to the difference from the packet header feature application layer protocol, and needs to be processed separately to extract the key information therein. The final spliced ​​comprehensive representation vector of the entire session is more comprehensive and reliable, so as to facilitate the subsequent construction of the sample relationship diagram.

[0045] Furthermore, mapping the comprehensive representation vector into a hyperbolic space, taking the labeled comprehensive representation vector as a support set sample, and taking the unlabeled comprehensive representation vector as a query set sample comprises:

[0046] The comprehensive representation vector is mapped to the hyperbolic space through exponential mapping, and the comprehensive representation vector with labeled labels is used as the support set sample, and the comprehensive representation vector without labeled labels is used as the query set sample, specifically:

[0047]

[0048] In the formula, v i represents the representation vector of the i-th sample in the hyperbolic space, represents the exponential mapping operation, z i Represents the comprehensive representation vector of the i-th sample before mapping.

[0049] From the above description, it can be seen that the comprehensive representation vector is mapped to the hyperbolic space as the support set sample and the query set sample through the exponential mapping. The geometric properties of the hyperbolic space can help better model the relationship between samples.

[0050] Furthermore, constructing a sample relationship graph of the support set samples and the query set samples based on the K-nearest neighbor algorithm includes:

[0051] Calculate the similarity between each two samples in the support set samples and the query set samples by using a hyperbolic Gaussian similarity function;

[0052] Constructing an initial neighbor relationship matrix using a K-nearest neighbor algorithm based on the similarity;

[0053] Obtaining an intermediate matrix according to the initial neighbor relationship matrix, and performing symmetric normalization on the intermediate matrix to obtain a final neighbor relationship matrix;

[0054] A sample relationship graph is obtained according to the final neighbor relationship matrix.

[0055] From the above description, we can know that we first calculate the similarity between samples to determine which samples are more similar, and then use the K nearest neighbor algorithm to construct an initial neighbor relationship matrix based on the similarity. This matrix describes the relationship between samples. We obtain an intermediate matrix based on the initial neighbor relationship matrix, and symmetric normalize the intermediate matrix to obtain the final neighbor relationship matrix. The final neighbor relationship matrix represents the adjacency relationship between all samples. The sample relationship graph obtained based on the final neighbor relationship matrix more accurately reflects the relationship between samples.

[0056] Further, the calculating the similarity between each two samples in the support set samples and the query set samples by using a hyperbolic Gaussian similarity function includes:

[0057]

[0058] In the formula, a ij Represents sample v i and its neighbor sample v j The similarity between them, exp(·) represents the hyperbolic Gaussian similarity function, Represents sample v i and its neighbor sample v j The hyperbolic distance, σ represents the standard deviation of the Gaussian kernel, Represents sample vi The set of neighbor samples.

[0059] From the above description, it can be seen that the hyperbolic Gaussian similarity function can accurately evaluate the similarity between two samples in the hyperbolic space, and the similarity between the samples finally obtained is reasonable and reliable.

[0060] Further, obtaining an intermediate matrix according to the initial neighbor relationship matrix includes:

[0061] W=A+A T ;

[0062] In the formula, W represents the intermediate matrix, A represents the initial neighbor relationship matrix, and A T Represents the transposed matrix of the initial neighbor relationship matrix;

[0063] The step of symmetric normalizing the intermediate matrix to obtain a final neighbor relationship matrix comprises:

[0064]

[0065] Where S represents the final neighbor relationship matrix, and D represents the diagonal matrix containing the sum of each row of the intermediate matrix W.

[0066] From the above description, it can be seen that the initial neighbor relationship matrix is ​​added to its transposed matrix to ensure that the obtained intermediate matrix is ​​symmetric and non-negative, and then the intermediate matrix is ​​symmetrically normalized so that the connection strength between each sample is balanced.

[0067] Furthermore, the using a label propagation algorithm based on the sample relationship graph to propagate the label of the support set sample to similar query set samples to obtain the predicted label of the query set sample includes:

[0068] For an N-Way-K-shot task, determine the number of categories N, the number of support set samples K for each category, and the number of query set samples Q for each category;

[0069] Determine the matrix size as N×(K+Q) according to the number of categories N, the number of support set samples K of each category, and the number of query set samples Q of each category;

[0070] Determine the first N×K rows of the matrix as the one-hot encoding of the labels of the support set samples, and determine the last N×Q rows of the matrix as zero;

[0071] Generate a sample label matrix according to the matrix size, the first N×K rows of the matrix, and the last N×Q rows of the matrix;

[0072] Iterate using a label propagation algorithm based on the sample label matrix and the final neighbor relationship matrix to diffuse the labels of the support set samples, and update the label prediction distribution in each iteration until the predicted labels of the query set samples are obtained.

[0073] From the above description, it can be seen that the matrix size is determined according to the number of categories N, the number of support set samples K of each category, and the number of query set samples Q of each category. The first N×K rows of the matrix are determined as the one-hot encoding of the labels of the support set samples, and the last N×Q rows of the matrix are determined as zero, indicating that each sample in the query set has no initial label. In this way, a sample label matrix is ​​constructed. The label propagation algorithm is used to iterate based on the sample label matrix and the final neighbor relationship matrix to spread the labels of the support set samples. In each iteration, the label prediction distribution is updated until the predicted labels of the query set samples are obtained, thereby realizing label prediction for the query set samples and improving the accuracy and generalization ability of malicious traffic detection.

[0074] Further, the updating of the label prediction distribution includes:

[0075] D t+1 =αSD t +(1-α)Y;

[0076] Where D t+1 represents the label prediction distribution of the t+1th iteration, α represents the parameter that controls the probability of label propagation, S represents the final neighbor relationship matrix, and D t represents the label prediction distribution of the tth iteration, and Y represents the sample label matrix.

[0077] From the above description, it can be seen that by continuously updating the label prediction distribution, the predicted labels are converged, so as to obtain the final predicted labels of the query set samples, which is convenient for identifying malicious traffic, thereby improving the accuracy of malicious traffic detection.

[0078] Please refer to Figure 2 Another embodiment of the present invention provides a small sample malicious traffic detection system based on label propagation, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, each step in the above-mentioned small sample malicious traffic detection method based on label propagation is implemented.

[0079] The above-mentioned small sample malicious traffic detection method and system based on label propagation of the present invention can be applied to malicious traffic detection scenarios, which are described below through specific implementation methods:

[0080] Please refer to Figure 1 , Figure 3-Figure 5 , Embodiment 1 of the present invention is:

[0081] A small sample malicious traffic detection method based on label propagation, comprising the steps of:

[0082] S1. Preprocess the acquired original traffic file to obtain a session representation sequence. The original traffic file includes a traffic file to be detected and a traffic file with a label. The traffic file with a label is prepared in advance, such as Figure 3 As shown, specifically including S11-S15:

[0083] S11. Divide the obtained original traffic file into multiple session files, each of which contains an independent session.

[0084] Since the original traffic file may contain multiple interactive processes, simply segmenting by session identifier may result in multiple interactions in one file. Specifically, identify the three-way handshake and four-wave identifiers from the original traffic file, segment the original traffic file according to the three-way handshake and four-wave identifiers to obtain a TCP session file; determine whether the time interval between the data packets in the original traffic file exceeds the preset time, if so, determine that they belong to the same interactive process, and determine the data packets with a time interval exceeding the preset time as UDP session files; use the sequence number and confirmation number to reorganize the disordered data packets in the TCP session file and the UDP session file, and filter the retransmitted packets to obtain multiple session files. This ensures that each session file of the multiple session files obtained by segmentation contains only one independent session.

[0085] In an optional implementation, the preset time is 60 seconds or 120 seconds.

[0086] In an optional implementation, S11 includes:

[0087] Use the Splitcap tool (a tool specifically used to process network traffic data) to split the obtained raw traffic file into multiple session files.

[0088] S12: Filter invalid session files from the multiple session files to obtain multiple filtered session files.

[0089] In an optional implementation, the invalid session file is a session file having only one or two data packets, because these files usually cannot provide effective detection features.

[0090] S13. Select a preset number of data packets from each filtered session file as session representations, and perform anonymization processing on the session representations to obtain anonymized session representations.

[0091] In an optional implementation, it also includes:

[0092] If there is a filtered session file whose number of data packets is less than the preset number, after the filtered session file is filled, the preset number of data packets before the filled filtered session file are selected as session representations.

[0093] In an optional implementation, the anonymizing the session representation to obtain the anonymized session representation includes:

[0094] The MAC address, source IP address and destination IP address in the session representation are removed, the source IP address is set to 0.0.0.0, and the destination IP address is set to 255.255.255.255 to retain the transmission direction information, thereby obtaining an anonymized session representation.

[0095] S14: performing padding and alignment on the anonymized session representation to obtain a padded and aligned session representation, and performing normalization processing on the padded and aligned session representation to obtain a normalized session representation.

[0096] In an optional implementation, the padding and aligning the anonymized session representation to obtain the padded and aligned session representation includes:

[0097] The IP header of each data packet in the anonymized session representation is filled to 60 bytes, the transport layer part of each data packet is filled to 68 bytes, and the first 128 bytes are intercepted from the Payload part of each data packet so that each data packet forms a 256-byte feature vector to obtain the padded and aligned session representation.

[0098] In an optional implementation, normalizing the padded and aligned session representation to obtain the normalized session representation includes:

[0099] The value of each byte in the padded and aligned session representation is divided by 255 to obtain a normalized session representation, thereby unifying the range of data packet features to between 0 and 1, ensuring that different session data are at the same scale, and improving the effectiveness of malicious traffic detection.

[0100] S15. Obtain a conversation representation sequence according to the normalized conversation representation.

[0101] After preprocessing, each session will be represented as a feature sequence of fixed size, that is, the session representation sequence S (N×M) , where N represents the number of packets in the session, and M represents the length of the feature vector of each packet. The session representation sequence can be represented as a combination of two matrices, specifically:

[0102] S (N×M) =[HP];

[0103] Where H represents the header feature matrix, which is of size N×128 and contains the header information of each data packet in the session. P represents the load feature matrix, which is of size N×(M-128) and contains the load information of each data packet in the session.

[0104] S2, extracting a packet header feature sequence and a load feature sequence from the session representation sequence, and obtaining a comprehensive representation vector based on the packet header feature sequence and the load feature sequence, such as Figure 3 As shown, specifically including S21-S24:

[0105] S21. Split the session representation sequence to obtain a packet header feature sequence and a load feature sequence.

[0106] S22: Input the packet header feature sequence into a first preset feature extraction unit for feature extraction, and output a packet header feature representation vector.

[0107] The packet header contains information such as the protocol, port, and sequence number of the data packet. This information has a fixed format, and the size of the packet header features of each data packet is consistent. Among them, the first preset feature extraction unit is composed of a multi-layer bidirectional gated recurrent unit (bi-GRU), which can capture the potential timing relationship and important information in the packet header feature sequence. The packet header feature sequence is input into the first preset feature extraction unit for feature extraction, and the output packet header feature representation vector is specifically:

[0108]

[0109] In the formula, Represents the high-level abstract information of the packet header feature sequence H, that is, the packet header feature representation vector, and FECell(H) represents the first preset feature extraction unit of the input packet header feature sequence H.

[0110] S23, input the load feature sequence into a second preset feature extraction unit for feature extraction, and output a load feature representation vector, such as Figure 4 As shown, specifically:

[0111]

[0112] In the formula, represents a load feature representation vector, which includes key timing information and features in the load feature sequence P, and FECell(P) represents a second preset feature extraction unit of the input load feature sequence P.

[0113] Different from the packet header features, the content of the payload part varies greatly due to different application layer protocols, and therefore needs to be processed separately.

[0114] S24, concatenate the packet header feature representation vector and the load feature representation vector to obtain a comprehensive representation vector of the entire session, such as Figure 4 As shown, specifically:

[0115]

[0116] In the formula, Cat(·,·) represents the concatenation operation, and z represents the comprehensive representation vector, that is, the comprehensive representation of all data packet features in a session, which provides complete feature input for subsequent classification tasks.

[0117] S3, mapping the comprehensive representation vector to a hyperbolic space, taking the labeled comprehensive representation vector as a support set sample, taking the unlabeled comprehensive representation vector as a query set sample, and constructing a sample relationship graph of the support set sample and the query set sample based on a K-nearest neighbor algorithm, specifically including S31-S35:

[0118] S31, mapping the comprehensive representation vector to the hyperbolic space through exponential mapping, taking the labeled comprehensive representation vector as the support set sample, and taking the unlabeled comprehensive representation vector as the query set sample, specifically:

[0119]

[0120] In the formula, v i represents the representation vector of the i-th sample in the hyperbolic space, represents the exponential mapping operation, z i Represents the comprehensive representation vector of the i-th sample before mapping.

[0121] Exponential mapping is a process of mapping vectors from Euclidean space to hyperbolic space.

[0122] S32, calculating the similarity between each two samples in the support set samples and the query set samples by using a hyperbolic Gaussian similarity function, specifically:

[0123]

[0124] In the formula, a ij Represents sample v i and its neighbor sample v j The similarity between them, exp(·) represents the hyperbolic Gaussian similarity function, Represents sample v i and its neighbor sample v j The hyperbolic distance, σ represents the standard deviation of the Gaussian kernel, Represents sample v i The set of neighbor samples.

[0125] The hyperbolic Gaussian similarity function measures the distance of each sample from other samples to determine which samples are more similar.

[0126] S33: construct an initial neighbor relationship matrix using a K-nearest neighbor algorithm according to the similarity.

[0127] Each element a in the initial neighbor relationship matrix A ij Represents the sample v i With sample v j The similarity between them is the strength of their connection.

[0128] S34. Obtain an intermediate matrix according to the initial neighbor relationship matrix, and perform symmetric normalization on the intermediate matrix to obtain a final neighbor relationship matrix.

[0129] Wherein, obtaining an intermediate matrix according to the initial neighbor relationship matrix comprises:

[0130] W=A+A T ;

[0131] In the formula, W represents the intermediate matrix, A represents the initial neighbor relationship matrix, and A T Represents the transposed matrix of the initial neighbor relationship matrix. To ensure that the matrix W is symmetric and non-negative.

[0132] The step of symmetric normalizing the intermediate matrix to obtain a final neighbor relationship matrix comprises:

[0133]

[0134] In the formula, S represents the final neighbor relationship matrix, and D represents the diagonal matrix, which contains the sum of each row of the intermediate matrix W and is used to implement the normalization operation. The final neighbor relationship matrix S represents the adjacency relationship between all samples and provides support for the subsequent label propagation algorithm.

[0135] S35. Obtain a sample relationship graph according to the final neighbor relationship matrix.

[0136] Specifically, a sample relationship graph is obtained according to the final neighbor relationship matrix transformation.

[0137] S4, based on the sample relationship graph, using a label propagation algorithm to propagate the label of the support set sample to similar query set samples, obtain the predicted label of the query set sample, and identify the malicious traffic in the to-be-detected traffic file according to the predicted label of the query set sample, specifically including S41-S46:

[0138] S41. For an N-Way-K-shot task, determine the number of categories N, the number of support set samples K for each category, and the number of query set samples Q for each category.

[0139] S42: Determine a matrix size of N×(K+Q) according to the number of categories N, the number of support set samples K of each category, and the number of query set samples Q of each category.

[0140] S43, determining the first N×K rows of the matrix as the one-hot encoding of the labels of the support set samples, and determining the last N×Q rows of the matrix as zero, indicating that each sample in the query set has no initial label.

[0141] S44. Generate a sample label matrix according to the matrix size, the first N×K rows of the matrix, and the last N×Q rows of the matrix.

[0142] S45, iterating using a label propagation algorithm based on the sample label matrix and the final neighbor relationship matrix to diffuse the labels of the support set samples, and updating the label prediction distribution in each iteration until the predicted labels of the query set samples are obtained, such as Figure 3 shown.

[0143] Wherein, the updating label prediction distribution includes:

[0144] D t+1 =αSD t +(1-α)Y;

[0145] Where D t+1 represents the label prediction distribution of the t+1th iteration, α represents the parameter that controls the probability of label propagation, S represents the final neighbor relationship matrix, and D t represents the label prediction distribution of the tth iteration, and Y represents the sample label matrix.

[0146] The predicted labels of the query set samples are specifically:

[0147] D * =(I-αS) -1 Y;

[0148] Where D * represents the predicted label of the query set sample, and I represents the identity matrix.

[0149] S46. Identify malicious traffic in the traffic file to be detected according to the predicted labels of the query set samples.

[0150] In an optional implementation, before S1, the process further includes: a model training process, during which a weighted sum of sample classification loss and sample representation loss is used as a total loss function, specifically:

[0151] Loss=Loss c +λ×Loss R ;

[0152] In the formula, Loss represents the total loss function, Loss c represents the sample classification loss, λ represents the weight of the sample representation loss in the total loss, and Loss R Indicates that the sample represents the loss.

[0153] By optimizing this loss function, samples can be made closer to representative samples of the same category in the embedding space, and away from representative samples of other categories, thereby improving classification accuracy.

[0154] The sample classification loss is obtained by calculating the cross entropy loss between the predicted label and the true label. Before calculation, the predicted label needs to be processed by Softmax to obtain the probability p of the sample in the query set belonging to each category. C (y=k|x). The Softmax formula is as follows:

[0155]

[0156] In the formula, represents the predicted label of sample i in the query set, Indicates that the prediction is the label of class k.

[0157] The sample classification loss is specifically:

[0158]

[0159] In the formula, Q represents the number of query set samples, y q represents the true label of the qth sample, Represents the Softmax output of the qth sample.

[0160] Before calculating the sample representation loss, we first need to sample the samples to optimize the distance of the samples in the embedding space. The sampling method is divided into two steps:

[0161] (1) Select similar samples: Select all adjacent similar samples according to the sample relationship graph. If there are multiple similar samples, a representative sample is obtained by taking the hyperbolic average. If there is only one similar sample, no averaging is required. If there are no adjacent similar samples, the sample with the closest distance is selected as the representative sample by calculating the hyperbolic distance.

[0162] (2) Select samples of different categories: For each different category, first select the adjacent samples of the same category; if there are multiple samples, a representative sample is obtained by hyperbolic averaging; if there are no adjacent samples, the sample with the closest distance is selected as the representative.

[0163] Through the above sampling method, we get the representative sample of each category, and calculate the hyperbolic distance between the sample and each representative sample. After negating these distances, we get the distance sequence T, and then convert the distance sequence T into probability through the Softmax function, as follows:

[0164]

[0165] In the formula, Represents sample x i To sample q k The hyperbolic distance, k represents the category, and the negative sign makes the sample with smaller distance have greater probability of belonging to the category.

[0166] Therefore, the sample representation loss is obtained by calculating the cross entropy loss between the predicted probability and the true label, specifically:

[0167]

[0168] In the formula, y q represents the true label of the qth sample, Represents the category probability of the sample obtained by the Softmax function.

[0169] Usually, network traffic data contains many different types of communications, such as HTTP (application layer protocol), FTP (file transfer protocol), etc. However, due to the scarcity of malicious traffic data sets, traditional supervised learning methods face the challenge of insufficient labeled data. Therefore, the small sample malicious traffic detection method based on label propagation mentioned above in the present invention is used to monitor malicious traffic in enterprise networks in real time. The goal is to monitor network traffic and detect potential malicious traffic, and timely warn of potential network threats. The following is an example of applying the small sample malicious traffic detection method based on label propagation in the present invention, as follows:

[0170] (1) Capture network data packets through a network traffic monitoring tool (such as Wireshark or tcpdump) and store them in PCAP file format (a commonly used datagram storage format) as the traffic file to be detected, and at the same time obtain the previously prepared labeled traffic file.

[0171] (2) Each data packet is divided into session files according to the five-tuple (source IP address, destination IP address, source port, destination port, and protocol), and irrelevant protocol packets are filtered out, such as heartbeat packets or Ping packets (a tool used to detect network connectivity) in HTTP traffic, retaining only the data packets of interest. After preprocessing, the features of the session are extracted as packet header features (such as source IP, destination IP, source port, destination port, protocol type, packet size and timestamp) and payload features (such as payload length and specific byte frequency, etc.).

[0172] (3) Use the K nearest neighbor algorithm to map all session samples after feature extraction into the hyperbolic space and calculate the similarity between samples. Each sample selects the K samples that are most similar to it and establishes a neighbor relationship, laying the foundation for the subsequent label propagation process. If two samples are found to be very similar in packet header and payload features, such as the source IP, destination IP and port number are close, and the payload byte length is consistent, then the two samples will be considered similar and a neighbor relationship will be established.

[0173] (4) The label propagation algorithm is used to predict the labels of query set samples. The labeled malicious traffic samples (labeled as 1) and normal traffic samples (labeled as 0) are used to influence the unlabeled query set samples. That is, the labels of the labeled support set samples will be propagated to similar samples in the query set, thereby performing malicious traffic detection on new samples.

[0174] (5) Identify abnormal traffic categories and generate detection reports containing attack information, such as Figure 5 shown.

[0175] In the process of verifying the core algorithm, the IDS-FS dataset was used for multi-classification experiments and ablation experiments to evaluate the performance of the above method of the present invention. This dataset contains malicious traffic and normal traffic data and is suitable for IDS (network intrusion detection system) tasks. In order to evaluate the performance of the model, it was verified under multiple experimental conditions, including combinations of different numbers of categories and numbers of samples. The experimental process is as follows:

[0176] Experimental Design: In the multi-classification experiment, we first conducted 5-way-1-shot and 5-way-5-shot experiments on the IDS-FS dataset to test the performance of the model under conditions with fewer samples (1 sample) and more samples (5 samples), respectively.

[0177] Construction of training and test sets: From the malicious traffic and normal traffic of the IDS-FS dataset, 15 different categories of data are randomly selected to construct the training set, and the number of traffic samples in each category is the same. The remaining 8 categories of malicious traffic and normal traffic constitute the test set, and 1,000 tasks are randomly selected from each category as test tasks.

[0178] Experimental task settings: In the 5-way-1-shot setting, each task contains samples from 5 categories, and each category contains only one sample for training. In the 5-way-5-shot setting, each task also contains 5 categories, but each category contains 5 samples for training.

[0179] Performance evaluation: For each test task, the performance of the model is evaluated by calculating its performance indicators (such as precision, recall, and F1 score). Finally, the average of each performance indicator of all test tasks is taken as the final evaluation result of the model, as shown in Table 1.

[0180] Table 1 Experimental results

[0181]

[0182] The models ProtoNet in Table 1 are from Snell J, Swersky K, Zemel R. Prototypical networks for few-shot learning[C]. Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017: 4080-4090; the model HyProtoNet is from Khrulkov V, Mirvakhabova L, Ustinova E, et al. Hyperbolic image embeddings[C]. Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, 2020: 6418-6428; the model TPN is from Liu Y, Lee J, Park M, et al. Learning to propagate labels: Transductive propagation network for few-shot learning[C]. Proceedings of the 7th International Conference on Learning Representations, 2019; the model TF is from Sirinam P, Mathews N, Rahman M S, et al. Triplet fingerprinting: More practical and portable website fingerprinting with n-shot learning[C]. Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, 2019: 1131-1148; the model FCAD is from Feng T, Qi Q, Wang J, et al. Few-shot class-adaptive anomaly detection with model-agnostic meta-learning[C]. Proceedings of the 2021 IFIP Networking Conference(IFIP Networking).IEEE, 2021:1-9; Model FCNet comes from Xu C, Shen J, Du classification[C].Proceedings of The WebConference 2020,2020:13-22. .

[0183] The present invention adopts a combination of hyperbolic space mapping and K nearest neighbors to construct a more complex and accurate sample relationship graph. By representing samples in hyperbolic space, it can more effectively capture the non-Euclidean structure in traffic data, and use the similarity calculation between samples to perform more accurate relationship modeling, which greatly enhances the model's ability to represent complex network traffic relationships. At the same time, the present invention proposes a label propagation mechanism, which can effectively solve the label scarcity problem in small sample learning by propagating the labels of support set samples in the sample relationship graph. The label propagation process enables the labels of query set samples to be expanded from adjacent support set samples, enhancing the generalization ability of the model, especially when facing scarce label data, it can improve classification performance, thereby effectively improving the accuracy and generalization ability of malicious traffic detection.

[0184] Please refer to Figure 2 , Embodiment 2 of the present invention is:

[0185] A small sample malicious traffic detection system based on label propagation includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, each step of the above-mentioned small sample malicious traffic detection method based on label propagation is implemented.

[0186] In summary, the present invention provides a small sample malicious traffic detection method and system based on label propagation, which preprocesses the acquired original traffic file to obtain a session representation sequence, extracts the packet header feature sequence and the load feature sequence from the session representation sequence, and obtains a comprehensive representation vector based on the packet header feature sequence and the load feature sequence, thereby avoiding missing key feature information, and then maps the comprehensive representation vector to the hyperbolic space, uses the comprehensive representation vector of the labeled labels as the support set sample, and uses the comprehensive representation vector of the unlabeled labels as the query set sample, and constructs a sample relationship graph of the support set samples and the query set samples based on the K nearest neighbor algorithm, which more accurately establishes the relationship between the samples, greatly enhancing the model's representation ability for complex network traffic relationships, and uses the label propagation algorithm based on the sample relationship graph to propagate the labels of the support set samples to similar query sets. The method obtains the predicted labels of the query set samples, and identifies the malicious traffic in the traffic file to be detected according to the predicted labels of the query set samples, which effectively solves the label scarcity problem in small sample learning and enhances the generalization ability of the model. Especially when facing scarce label data, it can improve the classification performance, thereby improving the accuracy and generalization ability of malicious traffic detection. At the same time, the similarity between samples is first calculated to determine which samples are more similar. The K nearest neighbor algorithm is used to construct the initial neighbor relationship matrix according to the similarity. The matrix describes the relationship between the samples. The intermediate matrix is ​​obtained according to the initial neighbor relationship matrix, and the intermediate matrix is ​​symmetrically normalized to obtain the final neighbor relationship matrix. The final neighbor relationship matrix represents the adjacency relationship between all samples. The sample relationship diagram obtained according to the final neighbor relationship matrix more accurately reflects the relationship between the samples.

[0187] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A small sample malicious traffic detection method based on label propagation, characterized in that: Includes steps: Preprocessing the acquired original traffic files to obtain a session representation sequence, wherein the original traffic files include traffic files to be detected and traffic files with marked labels; Extracting a packet header feature sequence and a load feature sequence from the session representation sequence, and obtaining a comprehensive representation vector based on the packet header feature sequence and the load feature sequence; Mapping the comprehensive representation vector into a hyperbolic space, taking the labeled comprehensive representation vector as a support set sample, taking the unlabeled comprehensive representation vector as a query set sample, and constructing a sample relationship graph of the support set sample and the query set sample based on a K-nearest neighbor algorithm; Based on the sample relationship graph, a label propagation algorithm is used to propagate the labels of the support set samples to similar query set samples to obtain predicted labels of the query set samples, and malicious traffic in the traffic file to be detected is identified according to the predicted labels of the query set samples.

2. According to claim 1, a small sample malicious traffic detection method based on label propagation is characterized in that: The preprocessing of the acquired original traffic file to obtain the session representation sequence includes: Splitting the acquired original traffic file into a plurality of session files, each of the session files containing an independent session; Filter invalid session files from the multiple session files to obtain multiple filtered session files; Selecting a preset number of data packets from each filtered session file as session representations, and performing anonymization processing on the session representations to obtain anonymized session representations; Performing padding and alignment on the anonymized session representation to obtain a padded and aligned session representation, and performing normalization processing on the padded and aligned session representation to obtain a normalized session representation; A session representation sequence is obtained according to the normalized session representation.

3. According to claim 1, a small sample malicious traffic detection method based on label propagation is characterized in that: The step of extracting a packet header feature sequence and a load feature sequence from the session representation sequence, and obtaining a comprehensive representation vector based on the packet header feature sequence and the load feature sequence comprises: Splitting the session representation sequence to obtain a packet header feature sequence and a load feature sequence; Inputting the packet header feature sequence into a first preset feature extraction unit for feature extraction, and outputting a packet header feature representation vector; Inputting the load feature sequence into a second preset feature extraction unit for feature extraction, and outputting a load feature representation vector; The packet header feature representation vector and the load feature representation vector are concatenated to obtain a comprehensive representation vector of the entire session.

4. According to claim 1, a small sample malicious traffic detection method based on label propagation is characterized in that: The step of mapping the comprehensive representation vector to a hyperbolic space, taking the comprehensive representation vector with labeled labels as a support set sample, and taking the comprehensive representation vector with unlabeled labels as a query set sample comprises: The comprehensive representation vector is mapped to the hyperbolic space through exponential mapping, and the comprehensive representation vector with labeled labels is used as the support set sample, and the comprehensive representation vector without labeled labels is used as the query set sample, specifically: In the formula, v i represents the representation vector of the i-th sample in the hyperbolic space, represents the exponential mapping operation, z i Represents the comprehensive representation vector of the i-th sample before mapping.

5. According to the method of detecting small sample malicious traffic based on label propagation in claim 1, it is characterized in that: The step of constructing a sample relationship graph of the support set samples and the query set samples based on the K nearest neighbor algorithm includes: Calculate the similarity between each two samples in the support set samples and the query set samples by using a hyperbolic Gaussian similarity function; Constructing an initial neighbor relationship matrix using a K-nearest neighbor algorithm based on the similarity; Obtaining an intermediate matrix according to the initial neighbor relationship matrix, and performing symmetric normalization on the intermediate matrix to obtain a final neighbor relationship matrix; A sample relationship graph is obtained according to the final neighbor relationship matrix.

6. According to claim 5, a small sample malicious traffic detection method based on label propagation is characterized in that: Calculating the similarity between each two samples in the support set samples and the query set samples by using a hyperbolic Gaussian similarity function comprises: In the formula, a ij Represents sample v i and its neighbor sample v j The similarity between them, exp(·) represents the hyperbolic Gaussian similarity function, Represents sample v i and its neighbor sample v j The hyperbolic distance, σ represents the standard deviation of the Gaussian kernel, Represents sample v i The set of neighbor samples.

7. According to claim 5, a small sample malicious traffic detection method based on label propagation is characterized in that: The obtaining of the intermediate matrix according to the initial neighbor relationship matrix comprises: W=A+A T ; In the formula, W represents the intermediate matrix, A represents the initial neighbor relationship matrix, and A T Represents the transposed matrix of the initial neighbor relationship matrix; The step of symmetric normalizing the intermediate matrix to obtain a final neighbor relationship matrix comprises: Where S represents the final neighbor relationship matrix, and D represents the diagonal matrix containing the sum of each row of the intermediate matrix W.

8. According to claim 5, a small sample malicious traffic detection method based on label propagation is characterized in that: The step of using a label propagation algorithm based on the sample relationship graph to propagate the label of the support set sample to similar query set samples to obtain the predicted label of the query set sample comprises: For an N-Way-K-shot task, determine the number of categories N, the number of support set samples K for each category, and the number of query set samples Q for each category; Determine the matrix size as N×(K+Q) according to the number of categories N, the number of support set samples K of each category, and the number of query set samples Q of each category; Determine the first N×K rows of the matrix as the one-hot encoding of the labels of the support set samples, and determine the last N×Q rows of the matrix as zero; Generate a sample label matrix according to the matrix size, the first N×K rows of the matrix, and the last N×Q rows of the matrix; Iterate using a label propagation algorithm based on the sample label matrix and the final neighbor relationship matrix to diffuse the labels of the support set samples, and update the label prediction distribution in each iteration until the predicted labels of the query set samples are obtained.

9. According to claim 8, a small sample malicious traffic detection method based on label propagation is characterized in that: The updating of label prediction distribution includes: D t+1 =αSD t +(1-α)Y; Where D t+1 represents the label prediction distribution of the t+1th iteration, α represents the parameter that controls the probability of label propagation, S represents the final neighbor relationship matrix, and D t represents the label prediction distribution of the tth iteration, and Y represents the sample label matrix.

10. A small sample malicious traffic detection system based on label propagation, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, each step of the small sample malicious traffic detection method based on label propagation described in any one of claims 1 to 9 is implemented.

Images