Small sample malicious traffic detection method and system based on label propagation

Abstract

The application discloses a small sample malicious traffic detection method and system based on label propagation, pre-processes an obtained original traffic file to obtain a session representation sequence, extracts a packet header feature sequence and a load feature sequence from the session representation sequence, obtains a comprehensive representation vector based on the packet header feature sequence and the load feature sequence, maps the comprehensive representation vector to a hyperbolic space, takes the comprehensive representation vector with a labeled label as a support set sample, takes the comprehensive representation vector without a labeled label as a query set sample, constructs a sample relation graph based on a K nearest neighbor algorithm, greatly enhances the representation ability of a model to complex network traffic relations, and uses a label propagation algorithm based on the sample relation graph to propagate the label of the support set sample to similar query set samples, obtains a predicted label of the query set sample, and identifies malicious traffic in a to-be-detected traffic file according to the predicted label of the query set sample, thereby improving the precision and generalization ability of malicious traffic detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of network security, and particularly relates to a small sample malicious traffic detection method and system based on label propagation. BACKGROUND

[0002] The continuous expansion of the Internet scale makes it carry more and more services, and also leads to the fact that the network traffic data scale becomes larger and more complex, which causes trouble for the supervision of the Internet and creates an opportunity for the lawbreakers. More and more network attack traffic appears in the network environment, network security incidents occur frequently, and network security problems become increasingly serious. Network attacks usually have concealment, and when it is found that the network is attacked, it has often caused very serious loss and often causes widespread social concern. Therefore, how to detect malicious traffic in network traffic and provide safer network services is increasingly concerned by people.

[0003] The current research on malicious traffic detection can be roughly divided into three kinds according to the implementation method, that is, the detection method based on rule matching, the detection method based on machine learning and the detection method based on deep learning. The malicious traffic detection method based on rule matching detects malicious behavior in network traffic by extracting specific fields in network traffic data and matching with a pre-defined rule library. Due to the variability of network traffic and the wide application of encrypted traffic, the missing detection rate of the method based on rule matching is greatly increased. The malicious traffic detection method based on machine learning combines machine learning methods with manually extracted features from original traffic to realize detection of malicious network traffic. The performance of this method is limited by the quality of manually extracted features. The malicious traffic detection method based on deep learning combines feature extraction and classification, automatically learns the feature representation in the data through a carefully designed neural network model, and realizes detection of malicious traffic based on the extracted features. Compared with the first two methods, it has higher flexibility and generalization ability, and can realize high-precision traffic detection.

[0004] Although the detection method based on deep learning has many advantages compared with other methods, there are still some problems. The method based on deep learning usually has a very deep network layer, and a large amount of data is needed to train the model to obtain good performance. In the real environment, for zero-day attacks, only a few samples can be collected for training. When the number of training samples is small, the method based on deep learning is prone to overfitting, which leads to the decline of the detection performance of the model. At the same time, when facing new categories of samples, the method based on deep learning needs to retrain the entire model, and cannot quickly build the detection ability for new categories. How to quickly build good malicious traffic detection ability through a few samples has become a thorny problem in the field of malicious traffic detection. Small sample learning aims to solve the classification problem through a small number of samples, which is very suitable for solving this problem. Its application in the field of image recognition has a lot of research work, but its application in the field of malicious traffic detection is still relatively few.

[0005] At present, the small sample detection model in the field of malicious traffic detection pays too much attention to the commonness of categories. Since malicious traffic is usually disguised as normal traffic, the number of normal traffic is large and complex, which leads to the fact that the commonness of normal traffic is learned too much, and the detection and generalization of malicious traffic are ignored, thereby affecting the performance of the model in detecting malicious traffic. SUMMARY

[0006] The technical problem to be solved by the present application is to provide a small sample malicious traffic detection method and system based on label propagation, which can improve the precision and generalization ability of malicious traffic detection.

[0007] To solve the above technical problems, a technical solution adopted by the present application is:

[0008] A small sample malicious traffic detection method based on label propagation, comprising the steps of:

[0009] Pretreating the obtained original traffic file to obtain a session representation sequence, wherein the original traffic file includes a to-be-detected traffic file and a traffic file with a labeled label;

[0010] Extracting a packet header feature sequence and a payload feature sequence from the session representation sequence, and obtaining a comprehensive representation vector based on the packet header feature sequence and the payload feature sequence;

[0011] Mapping the comprehensive representation vector to a hyperbolic space, taking the comprehensive representation vector of the labeled label as a support set sample, taking the comprehensive representation vector of the unlabeled label as a query set sample, and constructing a sample relationship graph of the support set sample and the query set sample based on a K nearest neighbor algorithm;

[0012] Based on the sample relationship graph, the label propagation algorithm is used to propagate the labels of the support set samples to similar query set samples to obtain the predicted labels of the query set samples, and the malicious traffic in the traffic file to be detected is identified according to the predicted labels of the query set samples.

[0013] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is as follows:

[0014] A small-sample malicious traffic detection system based on label propagation includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the following steps:

[0015] The acquired raw traffic files are preprocessed to obtain a session representation sequence. The raw traffic files include traffic files to be detected and traffic files with labels.

[0016] Extract header feature sequences and payload feature sequences from the session representation sequence, and obtain a comprehensive representation vector based on the header feature sequences and payload feature sequences;

[0017] The comprehensive representation vector is mapped to hyperbolic space. The labeled comprehensive representation vectors are used as support set samples, and the unlabeled comprehensive representation vectors are used as query set samples. A sample relationship graph between the support set samples and the query set samples is constructed based on the K-nearest neighbor algorithm.

[0018] Based on the sample relationship graph, the label propagation algorithm is used to propagate the labels of the support set samples to similar query set samples to obtain the predicted labels of the query set samples, and the malicious traffic in the traffic file to be detected is identified according to the predicted labels of the query set samples.

[0019] The beneficial effects of the present application are that the original traffic file obtained is preprocessed to obtain a session representation sequence, the header feature sequence and the payload feature sequence are extracted from the session representation sequence, and the comprehensive representation vector is obtained based on the header feature sequence and the payload feature sequence, avoiding missing key feature information, and then the comprehensive representation vector is mapped into hyperbolic space, the comprehensive representation vector with labeled labels is taken as a support set sample, the comprehensive representation vector without labeled labels is taken as a query set sample, and the sample relationship graph of the support set sample and the query set sample is constructed based on the K nearest neighbor algorithm, the relationship between the samples is more accurately established, the representation ability of the model for complex network traffic relationship is greatly enhanced, and based on the sample relationship graph, the label propagation algorithm is used to propagate the label of the support set sample to the similar query set sample to obtain the predicted label of the query set sample, and the malicious traffic in the to-be-detected traffic file is identified according to the predicted label of the query set sample, effectively solving the label scarcity problem in small sample learning, enhancing the generalization ability of the model, especially when facing scarce label data, the classification performance can be improved, thereby improving the precision and generalization ability of malicious traffic detection. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 A step flow chart of a small sample malicious traffic detection method based on label propagation for an embodiment of the present application;

[0021] Figure 2 A structure schematic diagram of a small sample malicious traffic detection system based on label propagation for an embodiment of the present application;

[0022] Figure 3 A whole flow schematic diagram in a small sample malicious traffic detection method based on label propagation for an embodiment of the present application;

[0023] Figure 4 A feature extraction schematic diagram in a small sample malicious traffic detection method based on label propagation for an embodiment of the present application;

[0024] Figure 5 A traffic detection report schematic diagram in a small sample malicious traffic detection method based on label propagation for an embodiment of the present application. DETAILED DESCRIPTION

[0025] To explain the technical content, the purpose and the effect of the present application in detail, the following will be explained in combination with the embodiments and the drawings.

[0026] Please refer to Figure 1 A small sample malicious traffic detection method based on label propagation, comprising the steps of:

[0027] The obtained original traffic file is preprocessed to obtain a session representation sequence, and the original traffic file includes a to-be-detected traffic file and a traffic file with labeled labels;

[0028] extracting a sequence of packet header features and a sequence of payload features from the sequence of session representations, and obtaining a comprehensive representation vector based on the sequence of packet header features and the sequence of payload features;

[0029] mapping the comprehensive representation vector into a hyperbolic space, taking the comprehensive representation vectors with labeled labels as support set samples, taking the comprehensive representation vectors without labeled labels as query set samples, and constructing a sample relationship graph of the support set samples and the query set samples based on a K-neighbor algorithm;

[0030] propagating the labels of the support set samples to similar query set samples based on the sample relationship graph using a label propagation algorithm, obtaining the predicted labels of the query set samples, and identifying malicious traffic in the to-be-detected traffic file according to the predicted labels of the query set samples.

[0031] From the above description, the beneficial effects of the present application are that the obtained original traffic file is preprocessed to obtain a sequence of session representations, a sequence of packet header features and a sequence of payload features are extracted from the sequence of session representations, and a comprehensive representation vector is obtained based on the sequence of packet header features and the sequence of payload features, avoiding missing key feature information. Then, the comprehensive representation vector is mapped into a hyperbolic space, the comprehensive representation vectors with labeled labels are taken as support set samples, the comprehensive representation vectors without labeled labels are taken as query set samples, and a sample relationship graph of the support set samples and the query set samples is constructed based on a K-neighbor algorithm, which more accurately establishes the relationship between samples, greatly enhances the representation ability of the model to complex network traffic relationship, and based on the sample relationship graph, the labels of the support set samples are propagated to similar query set samples using a label propagation algorithm, the predicted labels of the query set samples are obtained, and the malicious traffic in the to-be-detected traffic file is identified according to the predicted labels of the query set samples. The label scarcity problem in small sample learning is effectively solved, the generalization ability of the model is enhanced, especially when facing scarce label data, the classification performance can be improved, thereby improving the precision and generalization ability of malicious traffic detection.

[0032] Further, the preprocessing of the obtained original traffic file to obtain a sequence of session representations comprises:

[0033] segmenting the obtained original traffic file into a plurality of session files, each of the session files containing an independent session;

[0034] filtering invalid session files from the plurality of session files to obtain a plurality of filtered session files;

[0035] selecting a preset number of data packets from each filtered session file as a session representation, and anonymizing the session representation to obtain an anonymized session representation;

[0036] The padded and aligned conversation representation is normalized to obtain a normalized conversation representation.

[0037] The normalized conversation representation is used to obtain a conversation representation sequence.

[0038] As described above, the plurality of conversation files are sequentially filtered, anonymized, padded and aligned, and normalized, ensuring that different conversation data is in the same scale, improving the subsequent malicious traffic detection effect.

[0039] Further, the packet header feature sequence and the payload feature sequence are extracted from the conversation representation sequence, and a comprehensive representation vector is obtained based on the packet header feature sequence and the payload feature sequence, including:

[0040] The conversation representation sequence is split to obtain a packet header feature sequence and a payload feature sequence.

[0041] The packet header feature sequence is input into a first preset feature extraction unit for feature extraction, and a packet header feature representation vector is output.

[0042] The payload feature sequence is input into a second preset feature extraction unit for feature extraction, and a payload feature representation vector is output.

[0043] The packet header feature representation vector and the payload feature representation vector are spliced to obtain a comprehensive representation vector of the entire conversation.

[0044] As described above, the conversation representation sequence is split to obtain a packet header feature sequence and a payload feature sequence, and independent feature extraction is performed respectively, which can capture the potential timing relationship and important information in the packet header feature sequence. At the same time, the content of the payload feature sequence has great changes due to the difference in application layer protocol with the packet header feature, and needs to be processed separately to extract the key information therein. The comprehensive representation vector of the entire conversation obtained by splicing is more comprehensive and reliable, so as to construct a sample relationship graph subsequently.

[0045] Further, the comprehensive representation vector is mapped into hyperbolic space, the comprehensive representation vector with a labeled label is taken as a support set sample, and the comprehensive representation vector without a labeled label is taken as a query set sample, including:

[0046] The comprehensive representation vector is mapped into hyperbolic space by exponential mapping, the comprehensive representation vector with a labeled label is taken as a support set sample, and the comprehensive representation vector without a labeled label is taken as a query set sample, specifically:

[0047]

[0048] where v i denotes the representation vector of the i-th sample in hyperbolic space, denotes the exponential mapping operation, z i denotes the comprehensive representation vector of the i-th sample before mapping.

[0049] As can be seen from the above description, by mapping the comprehensive representation vector to hyperbolic space as support set samples and query set samples through the exponential mapping, the geometric properties of the hyperbolic space can help better model the relationship between the samples.

[0050] Further, the constructing of the sample relationship graph of the support set samples and the query set samples based on the K-Nearest Neighbor algorithm comprises:

[0051] calculating the similarity between each two samples in the support set samples and the query set samples through the hyperbolic Gaussian similarity function;

[0052] constructing an initial neighbor relationship matrix according to the similarity using the K-Nearest Neighbor algorithm;

[0053] obtaining an intermediate matrix according to the initial neighbor relationship matrix, and performing symmetric normalization on the intermediate matrix to obtain a final neighbor relationship matrix;

[0054] obtaining a sample relationship graph according to the final neighbor relationship matrix.

[0055] As can be seen from the above description, the similarity between the samples is first calculated to determine which samples are more similar, and an initial neighbor relationship matrix is constructed according to the similarity using the K-Nearest Neighbor algorithm, which describes the relationship between the samples. An intermediate matrix is obtained according to the initial neighbor relationship matrix, and symmetric normalization is performed on the intermediate matrix to obtain a final neighbor relationship matrix, which represents the adjacency relationship between all samples. The sample relationship graph obtained according to the final neighbor relationship matrix more accurately reflects the relationship between the samples.

[0056] Further, the calculating of the similarity between each two samples in the support set samples and the query set samples through the hyperbolic Gaussian similarity function comprises:

[0057]

[0058] where a ij denotes the similarity between the sample v i and its neighbor sample v j , exp(·) denotes the hyperbolic Gaussian similarity function, denotes the hyperbolic distance between the sample v i and its neighbor sample v j , and σ denotes the standard deviation of the Gaussian kernel, denotes the similarity between the sample vi The set of neighboring samples.

[0059] As described above, the hyperbolic Gaussian similarity function can accurately assess the similarity between two samples in hyperbolic space, and the resulting similarity between the samples is reasonable and reliable.

[0060] Further, obtaining the intermediate matrix based on the initial neighbor relationship matrix includes:

[0061] W = A + A T ;

[0062] In the formula, W represents the intermediate matrix, A represents the initial neighbor relation matrix, and A T This represents the transpose of the initial neighbor relation matrix;

[0063] The step of performing symmetric normalization on the intermediate matrix to obtain the final neighbor relationship matrix includes:

[0064]

[0065] In the formula, S represents the final neighbor relationship matrix, and D represents a diagonal matrix containing the sum of each row of the intermediate matrix W.

[0066] As described above, the initial neighbor relation matrix is ​​added to its transpose to ensure that the resulting intermediate matrix is ​​symmetric and non-negative. Then, the intermediate matrix is ​​symmetrically normalized to balance the connection strength between each sample.

[0067] Further, the step of using a label propagation algorithm based on the sample relationship graph to propagate the labels of the support set samples to similar query set samples, and obtaining the predicted labels of the query set samples, includes:

[0068] For an N-Way-K-shot task, determine the number of classes N, the number of support set samples K for each class, and the number of query set samples Q for each class;

[0069] The matrix size is determined to be N×(K+Q) based on the number of categories N, the number of support set samples K for each category, and the number of query set samples Q for each category;

[0070] The first N×K rows of the matrix are determined as the one-hot encodings of the labels of the support set samples, and the last N×Q rows of the matrix are determined as zero;

[0071] Generate a sample label matrix according to the matrix size, the first N×K rows of the matrix, and the last N×Q rows of the matrix;

[0072] iteratively using a label propagation algorithm based on the sample label matrix and the final neighbor relation matrix to diffuse the labels of the support set samples, and updating a label prediction distribution at each iteration until obtaining the predicted labels of the query set samples.

[0073] As can be seen from the above description, the matrix size is determined according to the number of categories N, the number of support set samples K of each category, and the number of query set samples Q of each category, the first N×K rows of the matrix are determined as the one-hot encoding of the labels of the support set samples, and the last N×Q rows of the matrix are determined as zero, indicating that each sample in the query set has no initial label, thereby constructing a sample label matrix, iteratively using a label propagation algorithm based on the sample label matrix and the final neighbor relation matrix to diffuse the labels of the support set samples, and updating a label prediction distribution at each iteration until obtaining the predicted labels of the query set samples, thereby realizing label prediction of the query set samples and improving the precision and generalization ability of malicious traffic detection.

[0074] Further, the updating of the label prediction distribution comprises:

[0075] D t+1 = αSD t + (1-α)Y;

[0076] In the formula, D t+1 represents the label prediction distribution of the t+1th iteration, α represents a parameter for controlling the label propagation probability, S represents the final neighbor relation matrix, D t represents the label prediction distribution of the tth iteration, and Y represents the sample label matrix.

[0077] As can be seen from the above description, the label prediction distribution is continuously updated to make the predicted labels converge, thereby obtaining the final predicted labels of the query set samples, facilitating identification of malicious traffic, and thereby improving the precision of malicious traffic detection.

[0078] Please refer to Figure 2 , another embodiment of the present application provides a small sample malicious traffic detection system based on label propagation, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements each step in the above-mentioned small sample malicious traffic detection method based on label propagation when executing the computer program.

[0079] The above-mentioned small sample malicious traffic detection method and system based on label propagation of the present application can be applied to the malicious traffic detection scene, and the following will be described through a specific implementation manner:

[0080] Please refer to Figure 1 , Figures 3-5 , the first embodiment of the present application is:

[0081] A small sample malicious traffic detection method based on label propagation, comprising the steps of:

[0082] S1, preprocessing the obtained original traffic file to obtain a session representation sequence, wherein the original traffic file includes a to-be-detected traffic file and a labeled traffic file, and the labeled traffic file is prepared in advance, such as Figure 3 as shown, specifically comprising S11-S15:

[0083] S11, the obtained original traffic file is divided into a plurality of session files, and each session file contains an independent session.

[0084] Since the original traffic file may contain multiple interaction processes, simply dividing according to the session identifier may cause multiple interactions in one file. Specifically, the three-way handshake and four-way handshake identifiers are identified from the obtained original traffic file, the original traffic file is divided according to the three-way handshake and four-way handshake identifiers, and TCP session files are obtained; it is judged whether the time interval between the data packets in the original traffic file exceeds a preset time, if yes, it is determined that it belongs to the same interaction process, and the data packets with a time interval exceeding the preset time are determined as UDP session files; the out-of-order data packets in the TCP session files and the UDP session files are reorganized using sequence numbers and acknowledgement numbers, and retransmission packets are filtered, to obtain a plurality of session files. In this way, it can be ensured that each session file of the plurality of session files obtained by dividing contains only one independent session.

[0085] In an optional embodiment, the preset time is 60 seconds or 120 seconds.

[0086] In an optional embodiment, S11 comprises:

[0087] The obtained original traffic file is divided into a plurality of session files using the Splitcap tool (a tool specially used for processing network traffic data).

[0088] S12, filtering invalid session files from the plurality of session files to obtain a plurality of filtered session files.

[0089] In an optional embodiment, the invalid session file is a session file with only one or two data packets, because these files usually cannot provide effective detection features.

[0090] S13, selecting a preset number of data packets from each filtered session file as session representations, and anonymizing the session representations to obtain anonymized session representations.

[0091] In an optional embodiment, it further comprises:

[0092] If the number of data packets of the filtered session file is less than the preset number, the filtered session file is padded, and the first preset number of data packets of the padded filtered session file are selected as the session representation.

[0093] In an optional implementation, the anonymizing the session representation to obtain the anonymized session representation comprises:

[0094] The MAC address, the source IP address, and the destination IP address in the session representation are removed, the source IP address is set to 0.0.0.0, the destination IP address is set to 255.255.255.255, and the transmission direction information is reserved to obtain the anonymized session representation.

[0095] S14, padding and aligning the anonymized session representation to obtain a padded and aligned session representation, and normalizing the padded and aligned session representation to obtain a normalized session representation.

[0096] In an optional implementation, the padding and aligning the anonymized session representation to obtain the padded and aligned session representation comprises:

[0097] The IP header of each data packet in the anonymized session representation is padded to 60 bytes, the transmission layer part of each data packet is padded to 68 bytes, and the first 128 bytes of the Payload part of each data packet are intercepted, so that each data packet forms a feature vector of 256 bytes, and the padded and aligned session representation is obtained.

[0098] In an optional implementation, the normalizing the padded and aligned session representation to obtain the normalized session representation comprises:

[0099] The value of each byte in the padded and aligned session representation is divided by 255 to obtain the normalized session representation. In this way, the range of the data packet features is unified to 0-1, and the effectiveness of malicious traffic detection is improved under the same scale.

[0100] S15, obtaining a session representation sequence according to the normalized session representation.

[0101] After the preprocessing is completed, each session is represented as a fixed-size feature sequence, that is, a session representation sequence S (N×M) , where N represents the number of data packets in the session, and M represents the length of the feature vector of each data packet. The session representation sequence can be represented as a combination of two matrices, specifically:

[0102] S (N×M) = [H P]

[0103] In the formula, H represents a packet header feature matrix, with a size of N x 128, containing the packet header information of each data packet in the session, and P represents a payload feature matrix, with a size of N x (M-128), containing the payload information of each data packet in the session.

[0104] S2, extracting a packet header feature sequence and a payload feature sequence from the session representation sequence, and obtaining a comprehensive representation vector based on the packet header feature sequence and the payload feature sequence, as shown in the formula (2), specifically comprising S21-S24: Figure 3

[0105] S21, splitting the session representation sequence to obtain a packet header feature sequence and a payload feature sequence.

[0106] S22, inputting the packet header feature sequence into a first preset feature extraction unit for feature extraction, and outputting a packet header feature representation vector.

[0107] The packet header part contains the protocol, port, sequence number and other information of the data packet, which has a fixed format and the packet header feature size of each data packet is consistent. The first preset feature extraction unit is composed of multiple bi-GRU (bi-directional gated recurrent unit) layers, which can capture the potential timing relationship and important information in the packet header feature sequence. The inputting the packet header feature sequence into the first preset feature extraction unit for feature extraction and outputting the packet header feature representation vector is specifically:

[0108]

[0109] In the formula, represents the high-level abstract information of the packet header feature sequence H, i.e. the packet header feature representation vector, and FECell(H) represents the first preset feature extraction unit inputting the packet header feature sequence H.

[0110] S23, inputting the payload feature sequence into a second preset feature extraction unit for feature extraction, and outputting a payload feature representation vector, as shown in the formula (3), specifically: Figure 4

[0111]

[0112] In the formula, represents the payload feature representation vector, containing the key timing information and features in the payload feature sequence P, and FECell(P) represents the second preset feature extraction unit inputting the payload feature sequence P.

[0113] Unlike the packet header feature, the content of the payload part has great changes due to the difference of the application layer protocol, and therefore needs to be processed separately.

[0114] ​​S24, concatenating the packet feature representation vector and the load feature representation vector to obtain a comprehensive representation vector of the entire session, as shown in Figure 4 Specifically, as shown in

[0115]

[0116] In the formula, Cat(·,·) represents a concatenation operation, and z represents a comprehensive representation vector, i.e., a comprehensive representation of all packet features in a session, providing complete feature input for subsequent classification tasks.

[0117] S3, mapping the comprehensive representation vector into hyperbolic space, taking the comprehensive representation vector of the labeled label as a support set sample, taking the comprehensive representation vector of the unlabeled label as a query set sample, and constructing a sample relationship graph of the support set sample and the query set sample based on a K nearest neighbor algorithm, specifically including S31-S35:

[0118] S31, mapping the comprehensive representation vector into hyperbolic space through exponential mapping, taking the comprehensive representation vector of the labeled label as a support set sample, and taking the comprehensive representation vector of the unlabeled label as a query set sample, specifically:

[0119]

[0120] In the formula, v i represents the representation vector of the i-th sample in the hyperbolic space, represents an exponential mapping operation, and z i represents the comprehensive representation vector of the i-th sample before mapping.

[0121] Exponential mapping is a process of mapping vectors in Euclidean space to hyperbolic space.

[0122] S32, calculating the similarity between each two samples in the support set sample and the query set sample through a hyperbolic Gaussian similarity function, specifically:

[0123]

[0124] In the formula, a ij represents the similarity between the sample v i and its neighbor sample v j , exp(·) represents a hyperbolic Gaussian similarity function, represents the hyperbolic distance between the sample v i and its neighbor sample v j , σ represents the standard deviation of the Gaussian kernel, represents the neighbor sample set of the sample v i .

[0125] The hyperbolic Gaussian similarity function measures the distance between each sample and other samples, thereby determining which samples are more similar.

[0126] S33, constructing an initial neighbor relation matrix according to the similarity using a K-Nearest Neighbor algorithm.

[0127] Each element a ij in the initial neighbor relation matrix A i represents the similarity between the sample v j , that is, their connection strength.

[0128] S34, obtaining an intermediate matrix according to the initial neighbor relation matrix, and performing symmetric normalization on the intermediate matrix to obtain a final neighbor relation matrix.

[0129] The intermediate matrix obtained according to the initial neighbor relation matrix includes:

[0130] W = A + A T ;

[0131] In the formula, W represents the intermediate matrix, A represents the initial neighbor relation matrix, and A T represents the transpose matrix of the initial neighbor relation matrix. To ensure that the matrix W is symmetric and non-negative.

[0132] The symmetric normalization of the intermediate matrix to obtain the final neighbor relation matrix includes:

[0133]

[0134] In the formula, S represents the final neighbor relation matrix, D represents the diagonal matrix, and the sum of each row of the intermediate matrix W is included to realize the normalization operation. The final neighbor relation matrix S represents the adjacency relationship between all samples and provides support for the subsequent label propagation algorithm.

[0135] S35, obtaining a sample relation graph according to the final neighbor relation matrix.

[0136] Specifically, the sample relation graph is obtained by transformation according to the final neighbor relation matrix.

[0137] S4, using a label propagation algorithm based on the sample relation graph to propagate the label of the support set sample to similar query set samples to obtain the predicted label of the query set sample, and identifying the malicious traffic in the to-be-detected traffic file according to the predicted label of the query set sample, specifically including S41-S46:

[0138] S41, for an N-Way-K-shot task, determining the number of classes N, the number of support set samples K of each class, and the number of query set samples Q of each class.

[0139] S42, determining a matrix size of N x (K+Q) according to the number of categories N, the number of support set samples K of each category, and the number of query set samples Q of each category.

[0140] S43, determining the first N x K rows of the matrix as the one-hot encoding of the labels of the support set samples, and determining the last N x Q rows of the matrix as zero, indicating that each sample in the query set has no initial label.

[0141] S44, generating a sample label matrix according to the matrix size, the first N x K rows of the matrix, and the last N x Q rows of the matrix.

[0142] S45, using a label propagation algorithm based on the sample label matrix and the final neighbor relationship matrix to perform iteration to diffuse the labels of the support set samples, and updating a label prediction distribution at each iteration until obtaining the predicted labels of the query set samples, as shown in the following formula. Figure 3

[0143] Wherein, the updating of the label prediction distribution comprises:

[0144] D t+1 t+1 t = αSD t+1 t +(1-α)Y;

[0145] Wherein, D t t+1 represents the label prediction distribution of the t+1th iteration, α represents a parameter controlling the label propagation probability, S represents the final neighbor relationship matrix, D * t represents the label prediction distribution of the tth iteration, and Y represents the sample label matrix.

[0146] The predicted labels of the query set samples are specifically:

[0147] D -1 t+1 = (I-αS) * Y;

[0148] Wherein, D c t+1 represents the predicted labels of the query set samples, and I represents an identity matrix.

[0149] S46, identifying the malicious traffic in the to-be-detected traffic file according to the predicted labels of the query set samples.

[0150] In an optional embodiment, S1 is further preceded by a model training process, in which a weighted sum of a sample classification loss and a sample representation loss is taken as a total loss function, specifically:

[0151] Loss = Loss R + λ x Loss c ;

[0152] where Loss denotes the total loss function, Loss c denotes the sample classification loss, and λ denotes the weight of the sample representation loss in the total loss, Loss R denotes the sample representation loss.

[0153] By optimizing this loss function, the samples can be made closer to the representative samples of the same class and farther away from the representative samples of other classes in the embedding space, thereby improving the classification accuracy.

[0154] where the sample classification loss is obtained by calculating the cross-entropy loss between the predicted label and the true label. Before calculation, the predicted label needs to be processed by Softmax to obtain the probability p C (y = k | x) that the sample in the query set belongs to each class. The Softmax formula is as follows:

[0155]

[0156] where denotes the predicted label of the sample i in the query set, denotes the label of the predicted k class.

[0157] The sample classification loss is specifically:

[0158]

[0159] where Q denotes the number of query set samples, y q denotes the true label of the qth sample, denotes the Softmax output of the qth sample.

[0160] Before calculating the sample representation loss, the samples need to be sampled first to optimize the distance of the samples in the embedding space. The sampling method is divided into two steps:

[0161] (1) Select the same class samples: select all adjacent same class samples according to the sample relationship graph. If there are multiple same class samples, obtain a representative sample by hyperbolic averaging; if there is only one same class sample, no averaging operation is needed; if there is no adjacent same class sample, select the nearest sample as the representative sample by calculating the hyperbolic distance.

[0162] (2) Select different class samples: for each different class, first select the adjacent same class samples; if there are multiple samples, obtain a representative sample by hyperbolic averaging; if there are no adjacent samples, select the nearest sample as the representative.

[0163] Through the above sampling method, a representative sample of each category is obtained, and the hyperbolic distance between the sample and each representative sample is calculated, and the distance sequence T is obtained after taking the negative of these distances, and then the distance sequence T is converted into probability through the Softmax function, specifically:

[0164]

[0165] In the formula, represents the hyperbolic distance from sample x i to sample q k , k represents the category, and the negative sign makes the probability of the sample belonging to the category greater when the distance is smaller.

[0166] Therefore, the sample representation loss is obtained by calculating the cross-entropy loss between the predicted probability and the true label, specifically:

[0167]

[0168] In the formula, y q represents the true label of the qth sample, represents the category probability of the sample obtained by the Softmax function.

[0169] Generally, network traffic data contains various types of communication, such as HTTP (application layer protocol), FTP (file transfer protocol), etc., but due to the scarcity of malicious traffic data sets, traditional supervised learning methods face the challenge of insufficient labeled data, therefore, the above-mentioned small sample malicious traffic detection method based on label propagation of the application is used to monitor malicious traffic in enterprise networks in real time, and the goal is to monitor network traffic and detect potential malicious traffic to timely warn potential network threats. The above-mentioned small sample malicious traffic detection method based on label propagation of the application will be applied to an example as follows:

[0170] (1) Capture network packets through network traffic monitoring tools (such as Wireshark or tcpdump) and store them in PCAP file format (a commonly used data report storage format) as the traffic file to be detected, and at the same time, obtain the traffic file with labeled labels prepared in advance.

[0171] (2) Divide each data packet into session files according to the five-tuple (source IP address, destination IP address, source port, destination port, and protocol), and filter out irrelevant protocol packets such as heartbeat packets or Ping packets (tools for detecting network connectivity) in HTTP traffic, and only keep the data packets of interest. After preprocessing, the features of the session are extracted as packet header features (such as source IP, destination IP, source port, destination port, protocol type, packet size, and timestamp) and payload features (such as payload length and specific byte frequency, etc.).

[0172] (3) Using K-Nearest Neighbor algorithm to map all feature-extracted conversation samples to hyperbolic space and calculate the similarity between samples. Each sample selects the K most similar samples to establish neighbor relationships, laying the foundation for subsequent label propagation process. Assuming that two samples are found to be very similar in packet header and payload features, such as source IP, target IP and port number close, and payload byte length consistent, the two samples are considered similar and neighbor relationships are established.

[0173] (4) Using label propagation algorithm to predict query set sample labels. The labeled malicious traffic samples (label 1) and normal traffic samples (label 0) are affected by the label propagation algorithm to the unlabeled query set samples, i.e. the labels of the support set samples with labeled labels are propagated to similar samples in the query set, thereby detecting malicious traffic for new samples.

[0174] (5) Identifying abnormal traffic categories and generating detection reports containing attack information, as shown in Figure 5

[0175] During the core algorithm verification process, the IDS-FS dataset was used for multi-classification experiments and ablation experiments to evaluate the performance of the above-mentioned method of the present application. The dataset contains malicious traffic and normal traffic data, suitable for IDS (Network Intrusion Detection System) tasks. To evaluate the performance of the model, multiple experimental conditions were verified, including different combinations of class numbers and sample numbers, and the experimental process was as follows:

[0176] Experimental design: In the multi-classification experiment, 5-way-1-shot and 5-way-5-shot experiments were first conducted on the IDS-FS dataset to test the performance of the model under conditions of fewer samples (1 sample) and more samples (5 samples).

[0177] Construction of training set and test set: From the malicious traffic and normal traffic of the IDS-FS dataset, 15 different categories of data were randomly selected to construct the training set, with the same number of traffic samples for each category. The remaining 8 categories of malicious traffic and normal traffic constitute the test set, and 1000 tasks are randomly selected from each category as test tasks.

[0178] Experimental task setting: In the 5-way-1-shot setting, each task contains 5 categories of samples, each containing only one sample for training. In the 5-way-5-shot setting, each task also contains 5 categories, but each category contains 5 samples for training.

[0179] ​Performance Evaluation: For each test task, the performance of the model is evaluated by calculating its performance indicators such as precision, recall, and F1 score. Finally, the average of all performance indicators of all test tasks is taken as the final evaluation result of the model, as shown in Table 1.

[0180] Table 1 Experimental Results

[0181]

[0182] The model ProtoNet t. in Table 1 is from Snell J, Swersky K, Zemel R. Prototypical networks for few-shot learning [C]. Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017:4080-4090; the model HyProtoNet is from Khrulkov V, Mirvakhabova L, Ustinova E, et al. Hyperbolic image embeddings [C]. Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, 2020:6418-6428; the model TPN is from Liu Y, Lee J, Park M, et al. Learning to propagate labels: Transductive propagation network for few-shot learning [C]. Proceedings of the 7th International Conference on Learning Representations, 2019; the model TF is from Sirinam P, Mathews N, Rahman M S, et al. Triple tfingerprinting: More practical and portable website fingerprinting with n-shot learning [C]. Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, 2019:1131-1148; the model FCAD is from Feng T, Qi Q, Wang J, et al. Few-shot class-adaptive anomaly detection with model-agnostic meta-learning [C]. Proceedings of the 2021 IFIP Networking Conference (IFIP Networking).IEEE, 2021: 1-9; model FCNet from Xu C, Shen J, Du X. A method of few-shot network intrusion detection based on meta-learning framework [J]. IEEE Transactions on Information Forensics and Security, 2020, 15: 3540-3552; model RBRN from Zheng W, Gou C, Yan L, et al. Learning to classify: A flow-based relation network for encrypted traffic classification [C]. Proceedings of The Web Conference 2020, 2020: 13-22.

[0183] The application adopts a combination of hyperbolic space mapping and K-nearest neighbors, constructs a more complex and accurate sample relation graph, represents samples in hyperbolic space, can more effectively capture the non-Euclidean structure in traffic data, and performs more accurate relation modeling by similarity calculation between samples, greatly enhances the representation ability of the model for complex network traffic relations. At the same time, the application proposes a label propagation mechanism, which can effectively solve the label scarcity problem in small sample learning by propagating the label of the support set sample in the sample relation graph. The label propagation process enables the label of the query set sample to be extended from the adjacent support set sample, enhancing the generalization ability of the model. Especially in the face of scarce labeled data, it can improve the classification performance, thereby effectively improving the accuracy and generalization ability of malicious traffic detection.

[0184] Please refer to Figure 2 Embodiment two of the application is:

[0185] A small sample malicious traffic detection system based on label propagation, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement each step of the above-mentioned small sample malicious traffic detection method based on label propagation.

[0186] In summary, the present application provides a small sample malicious traffic detection method and system based on label propagation, pre-processes the obtained original traffic file to obtain a session representation sequence, extracts a packet header feature sequence and a payload feature sequence from the session representation sequence, and obtains a comprehensive representation vector based on the packet header feature sequence and the payload feature sequence, avoiding missing key feature information, and then maps the comprehensive representation vector into hyperbolic space, takes the comprehensive representation vector with a labeled label as a support set sample, takes the comprehensive representation vector without a labeled label as a query set sample, and constructs a sample relationship graph of the support set sample and the query set sample based on a K-nearest neighbor algorithm, more accurately establishes the relationship between samples, greatly enhances the representation ability of the model for complex network traffic relationships, and uses a label propagation algorithm to propagate the label of the support set sample to similar query set samples based on the sample relationship graph, obtains the predicted label of the query set sample, and identifies malicious traffic in the to-be-detected traffic file according to the predicted label of the query set sample, effectively solving the label scarcity problem in small sample learning, enhancing the generalization ability of the model, especially when facing scarce label data, improving the classification performance, thereby improving the precision and generalization ability of malicious traffic detection; at the same time, the similarity between samples is calculated first to determine which samples are more similar, an initial neighbor relationship matrix is constructed using the K-nearest neighbor algorithm according to the similarity, the matrix describes the relationship between samples, an intermediate matrix is obtained according to the initial neighbor relationship matrix, the intermediate matrix is symmetrically normalized to obtain a final neighbor relationship matrix, and the final neighbor relationship matrix represents the adjacency relationship between all samples, and the sample relationship graph obtained according to the final neighbor relationship matrix more accurately reflects the relationship between samples.

[0187] The above only describes the embodiments of the present application, and does not limit the patent scope of the present application, and any equivalent transformation or direct or indirect application in related technical fields based on the content of the specification and drawings of the present application is also included in the patent protection scope of the present application.

Claims

1. A method for small sample malicious traffic detection based on label propagation, characterized in that, The method comprises the steps of: preprocessing an obtained original traffic file to obtain a session representation sequence, the original traffic file comprising a to-be-detected traffic file and a traffic file with a labeled label; extracting a packet header feature sequence and a payload feature sequence from the session representation sequence, and obtaining a comprehensive representation vector based on the packet header feature sequence and the payload feature sequence; mapping the comprehensive representation vector into hyperbolic space, taking the comprehensive representation vector with a labeled label as a support set sample, taking the comprehensive representation vector without a labeled label as a query set sample, and constructing a sample relationship graph of the support set sample and the query set sample based on a K-nearest neighbor algorithm; propagating the label of the support set sample to similar query set samples using a label propagation algorithm based on the sample relationship graph to obtain a predicted label of the query set sample, and identifying malicious traffic in the to-be-detected traffic file according to the predicted label of the query set sample; the step of mapping the comprehensive representation vector into hyperbolic space, taking the comprehensive representation vector with a labeled label as a support set sample, and taking the comprehensive representation vector without a labeled label as a query set sample comprises: mapping the comprehensive representation vector into hyperbolic space through exponential mapping, taking the comprehensive representation vector with a labeled label as a support set sample, and taking the comprehensive representation vector without a labeled label as a query set sample, specifically: ； wherein represents the representation vector of the i-th sample in hyperbolic space, represents the exponential mapping operation, represents the combined representation vector of the i-th sample before mapping; the step of constructing a sample relationship graph of the support set sample and the query set sample based on a K-nearest neighbor algorithm comprises: calculating the similarity between each two samples in the support set sample and the query set sample through a hyperbolic Gaussian similarity function; constructing an initial neighbor relationship matrix using a K-nearest neighbor algorithm according to the similarity; obtaining an intermediate matrix according to the initial neighbor relationship matrix, and performing symmetric normalization on the intermediate matrix to obtain a final neighbor relationship matrix; obtaining a sample relationship graph according to the final neighbor relationship matrix.

2. The method of claim 1, wherein, The step of preprocessing the obtained original traffic file to obtain a session representation sequence comprises: segmenting the obtained original traffic file into a plurality of session files, each session file containing an independent session; filtering invalid session files from the plurality of session files to obtain a plurality of filtered session files; selecting a preset number of data packets from each filtered session file as a session representation, and performing anonymization processing on the session representation to obtain an anonymized session representation; performing padding alignment on the anonymized session representation to obtain a padding-aligned session representation, and performing normalization processing on the padding-aligned session representation to obtain a normalized session representation; obtaining a session representation sequence according to the normalized session representation.

3. The method of claim 1, wherein, The step of extracting a packet header feature sequence and a payload feature sequence from the session representation sequence, and obtaining a comprehensive representation vector based on the packet header feature sequence and the payload feature sequence comprises: splitting the session representation sequence to obtain a packet header feature sequence and a payload feature sequence; inputting the packet header feature sequence into a first preset feature extraction unit for feature extraction, and outputting a packet header feature representation vector; inputting the load feature sequence into a second preset feature extraction unit to perform feature extraction and output a load feature representation vector; concatenating the packet header feature representation vector and the load feature representation vector to obtain a comprehensive representation vector of the entire session.

4. The method of claim 1, wherein, The calculation of the similarity between each two samples in the support set and the query set by using the hyperbolic Gaussian similarity function comprises: ； wherein, denotes the similarity between a sample and its neighbor samples , denotes the hyperbolic Gaussian similarity function, denotes the similarity between a sample and its neighbor samples , denotes the standard deviation of the Gaussian kernel, denotes the set of neighbor samples of a sample .

5. The method of claim 1, wherein, The obtaining of the intermediate matrix according to the initial neighbor relation matrix comprises: ； In the formula, W represents an intermediate matrix, A represents an initial neighbor relation matrix, A represents an initial neighbor relation matrix, and AT represents a transposed matrix of the initial neighbor relation matrix. The symmetric normalization of the intermediate matrix to obtain the final neighbor relation matrix comprises: ； In the formula, S represents the final neighbor relation matrix, and D represents a diagonal matrix containing the sum of each row of the intermediate matrix W.

6. The method of claim 1, wherein, The propagation of the label of the support set sample to similar query set samples based on the sample relation graph using a label propagation algorithm to obtain the predicted label of the query set sample comprises: For an N-Way-K-shot task, the number of categories N, the number of support set samples per category K, and the number of query set samples per category Q are determined; According to the number of categories N, the number of support set samples per category K, and the number of query set samples per category Q, a matrix with a size of N×(K+Q) is determined; The first N×K rows of the matrix are determined as the one-hot encoding of the label of the support set sample, and the last N×Q rows of the matrix are determined as zero; A sample label matrix is generated according to the matrix size, the first N×K rows of the matrix, and the last N×Q rows of the matrix; Based on the sample label matrix and the final neighbor relation matrix, a label propagation algorithm is used for iteration to diffuse the label of the support set sample, and at each iteration, the label prediction distribution is updated until the predicted label of the query set sample is obtained.

7. The method of claim 6, wherein, The updating of the label prediction distribution comprises: ； wherein denotes the label prediction distribution of the t+1th iteration, denotes a parameter controlling the label propagation probability, S denotes the final neighborhood relation matrix, denotes the label prediction distribution of the tth iteration, Y denotes the sample label matrix. 8.A small sample malicious traffic detection system based on label propagation, comprising a memory, a processor and a computer program stored on the memory and capable of running on the processor, characterized in that, The processor executes the computer program to realize each step in the label propagation-based small sample malicious traffic detection method according to any one of claims 1 to 7.

Images