Network flow detection method
By constructing the AT-Transformer model, using sliding windows and autoregressive inference mode to process network traffic data, the problems of large time overhead, poor interpretation and weak noise resistance in network traffic detection are solved, and more efficient and interpretable detection effects are achieved.
Patent Information
- Application Number
- CN202411790363.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-06
- Publication Date
- 2025-03-07
AI Technical Summary
The traditional Transformer model has problems such as high time overhead, poor interpretation and weak noise interference in network traffic detection.
The time-sequential data is processed through the sliding window method, and the AT-Transformer model is built. The model includes two sets of encoders and decoders. It optimizes the weight of the Transformer using autoregressive inference mode and adversarial training to enhance the robustness and interpretability of the model.
Improve the accuracy and robustness of network traffic detection, reduce time overhead, and improve the interpretability of the model through visualization technology.
Smart Images
Figure CN119583180A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of network flow detection, and in particular to a network flow detection method. Background Art
[0002] The Transformer model is essentially a self-attention mechanism that can capture complex patterns in network traffic and detect traffic through a classifier. Transformer is often used to process long sequence data, such as DDoS attacks, port scanning, malicious traffic propagation, and other intrusion types. Its workflow is described as follows:
[0003] 1. Data preprocessing: including data collection, data cleaning, and feature extraction. The data cleaning part will mostly remove noise data and only retain features that are useful for detection.
[0004] 2. Input embedding layer: The extracted traffic features are converted into vector representations and input into the Transformer model, capturing the temporal order characteristics of data packets through position encoding.
[0005] 3. Self-attention mechanism: The self-attention mechanism can interact with the information at any two positions in the input sequence, identify potential dependencies in the traffic data, and calculate the attention of different dimensions in parallel through the multi-head attention mechanism, which can better capture the network traffic characteristics at different levels.
[0006] 4. Feedforward neural network: The output of the attention layer is further nonlinearly transformed through a multi-layer feedforward neural network to extract higher-order traffic features.
[0007] 5. Classification output: The feature extraction result is input into a classifier, usually a Softmax layer, which is used to classify network traffic into normal traffic or abnormal traffic (such as DDoS attacks, malware propagation, etc.).
[0008] Traditional Transformer methods mostly remove noise data and retain only features that are easy to detect during data cleaning. They use 6 to 8 sets of decoders and encoders and are black box models like other deep learning methods, such as Figure 1 shown.
[0009] In the traditional Transformer method, although it performs well in processing complex time series data, it still has some limitations:
[0010] 1. Weak ability to resist noise interference: Since the data preprocessing stage relies heavily on feature extraction, the model shows weak robustness when facing noise interference.
[0011] 2. High time cost: The self-attention mechanism and large-scale parameter training cause the model to consume a lot of time when processing large-scale data, making it difficult to meet real-time detection requirements.
[0012] 3. Poor interpretability: The black box nature of the model makes its decision-making process difficult to understand, reducing transparency and credibility in application scenarios.
[0013] These problems affect the application effect of the Transformer model in network traffic intrusion detection. Summary of the invention
[0014] The purpose of the present invention is to provide a network traffic detection method to solve the problems of large time overhead, poor interpretability and weak anti-noise interference ability of traditional network traffic detection methods.
[0015] To achieve the above-mentioned object of the invention, the present invention provides a network traffic detection method, the method comprising:
[0016] S101, obtaining a network traffic data set about a network attack, wherein the network traffic data set is composed of a plurality of flow data, and preprocessing the network traffic data set to obtain time-series data;
[0017] S102, processing the time series data by a sliding window method to obtain a data stream sequence sample;
[0018] S103, calculating the anomaly score of the data stream sequence sample;
[0019] S104, build AT-Transformer model;
[0020] S105. Input the data stream sequence samples into the AT-Transformer model to perform network traffic detection, and use visualization technology to visualize the detection results.
[0021] Furthermore, the network traffic data set is preprocessed, including the following operations:
[0022] Remove features in the stream data that are irrelevant to the stream attribute features;
[0023] Merge all features that characterize the source of a network attack into one feature, and merge all features that characterize the target of a network attack into one feature;
[0024] Handling missing values in network traffic datasets;
[0025] Balance the normal traffic data and abnormal traffic data in the network traffic data set;
[0026] Perform time series processing on the network traffic data set to obtain time series data.
[0027] Furthermore, the time series data is processed by the sliding window method, specifically: the time series data is grouped into fixed length l w The window is used as a data stream sequence sample to be input into the model for processing. The window is represented by The goal is to use the decoder to convert the window from its length l w -l s The shifted version of is reconstructed into a length of l w The original window, l s Indicates the length of the sample data stream.
[0028] Furthermore, the anomaly score is evaluated by measuring the loss function of the data stream sequence sample. Given a stream x k Assigned to the window The anomaly score is then calculated by measuring the reconstruction error, assuming that the streaming data is assigned to the window W x , the corresponding loss function Score(x) is expressed as:
[0029] Score(x)=‖W x -D(E(W x ),W x,shifted )‖
[0030] Where E(W) represents the Euclidean modulus, E(W x ) represents the output calculated by the encoder for window W, D(E(W x ),W x,shifted ) represents the encoder output for the shift window and the subsequent encoder output, where W x,shifted W x A shifted version of arrive k represents the test window size.
[0031] Furthermore, the AT-Transformer model, including two sets of encoders and decoders, predicts the reconstructed window in two parts by acting as an encoder-decoder network using an autoregressive inference mode at each timestamp to achieve prediction of reconstruction of each input sequence window.
[0032] Furthermore, the window reconstructed by predicting the two parts specifically includes:
[0033] In the first part, an approximate reconstruction of the input sequence window is generated. The deviation from this inference helps the attention network inside the AT-Transformer encoder to focus on subsequences with high deviations when extracting temporal trends. The encoder uses context-based attention to convert the input window W∈R K×m Convert it to a compressed representation, which is then converted to output W by the following formula1 :
[0034] W = Sigmoid(FeedForward(P))
[0035] Where m represents the vector dimension, K represents the local context window length, and K>l w , R represents the vector matrix, Sigmoid() represents the activation function, FeedForward() represents the feedforward layer, and P represents the compressed potential representation of attention;
[0036] In the second part, the reconstruction loss of the decoder is used as the anomaly score, and the score matrix F = L in the second part is obtained. 1 , and then perform regression calculation again to get the output W of the second part 3 .
[0037] Furthermore, in the first part, the reconstruction loss L of the decoder is defined as 1 and L 2 As the input of the second part, it is expressed as:
[0038] L 1 =‖W 1 -W 2 ‖ 2
[0039] L 2 =‖W 3 -W 2 ‖ 2
[0040] When defining the adversarial loss in the second part, the second decoder maximizes ‖W 3 -W 2 ‖ 2 The difference between the input window and the candidate reconstruction generated by the first decoder in the first part is used to distinguish the input window and the candidate reconstruction generated by the first decoder in the first part, and the first decoder releases the signal to the second decoder through the reconstruction input, producing a degraded anomaly score so that the second decoder generates the same anomaly score as W at this stage. 3 The same output, training object is represented as:
[0041] min max||W 3 -W 2 || 2
[0042] Decoder1 Decoder2
[0043] Among them, Decoder1 represents the first decoder, and Decoder2 represents the second decoder.
[0044] Furthermore, the goal of the first decoder is to minimize the reconstruction error, and the goal of the second decoder is to maximize the reconstruction error, which is achieved by using the following loss function:
[0045] L 1 =+‖W 3 -W 2 ‖ 2
[0046] L 2 =-‖W 3 -W 2 ‖ 2
[0047] The reconstruction loss and adversarial loss from the first and second parts are combined using a comprehensive loss function to determine the total loss for each decoder, expressed as:
[0048] L 1 = l -n ‖W 1 -W 2 ‖ 2 +(1-l -n )‖W 3 -W 2 ‖ 2
[0049] L 2 = l -n ‖W 3 -W 2 ‖ 2 +(1-l -n )‖W 3 -W 2 ‖ 2
[0050] Among them, n represents the training cycle and l is a training parameter close to 1.
[0051] Compared with the prior art, the present invention has the following beneficial effects:
[0052] The present invention provides a network traffic detection method, which first performs time-series processing on the data to be detected to simulate a real attack situation, and then uses adversarial training to optimize the weight of the Transformer, thereby improving the detection accuracy by amplifying the deviation reconstruction error to avoid missing abnormal data. Secondly, the model is simplified, the number of encoders and decoders is reduced, and a small amount of data is used for training to save time and cost. At the same time, the detection data part is visualized to solve the problem of poor interpretability of the traditional Transformer method. In terms of data cleaning, the actual interference process is simulated by retaining noise data and randomly deleting and modifying normal data, thereby enhancing the stability of the method in the face of noise. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only preferred embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0054] Figure 1 This is a schematic diagram of the traditional Transformer model workflow.
[0055] Figure 2 It is a schematic diagram of the overall flow of a network traffic detection method provided by an embodiment of the present invention.
[0056] Figure 3 It is a schematic diagram of the overall structure of the AT-Transformer model provided in an embodiment of the present invention.
[0057] Figure 4 4 is a schematic diagram of a curve showing a change in F1-score with a noise interference ratio p provided by an embodiment of the present invention.
[0058] Figure 5 4 is a schematic diagram of anomaly score visualization provided by an embodiment of the present invention.
[0059] Figure 6 This is a time cost comparison chart between the AT-Transformer model provided by an embodiment of the present invention and the traditional Transformer model. DETAILED DESCRIPTION
[0060] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The enumerated embodiments are only used to explain the present invention and are not used to limit the scope of the present invention.
[0061] Reference Figure 1 This embodiment provides a network traffic detection method, the method comprising the following steps:
[0062] S101. Obtain a network traffic data set related to a network attack, wherein the network traffic data set is composed of a plurality of flow data, and pre-process the network traffic data set to obtain time-series data.
[0063] In this embodiment, the network traffic data set may be a network traffic data set for various attacks on the industrial Internet of Things network. The network traffic data set is a binary classification label, and normal traffic and abnormal traffic (including DoS, Command Injection, Reconnaissance, Backdoor and other abnormal types) are marked with 0 and 1 respectively. Exemplarily, the network traffic data set may select the industrial Internet of Things data set WUSTL-IIOT-2021, which collects 2.7GB of data for about 53 hours. There are about 1.2 million flow data in the data set, of which normal traffic records account for 92.72% of all traffic records, and each record includes 41 features such as StartTime, LastTime, SrcAddr, and DstAddr.
[0064] As a possible implementation method, the network traffic data set is preprocessed, specifically including the following operations:
[0065] A. Remove features in the stream data that are irrelevant to the stream attribute features. Specifically, remove features such as Mean, SrcLoss, and DstLoss that are irrelevant to the stream attribute features.
[0066] B. Merge all features that characterize the source of network attacks into one feature, and merge all features that characterize the target of network attacks into one feature. For example, merge the attack source address SrcAddr and the source port number Sport into one feature SrcAddr (e.g. 192.168.0.20:55841), and perform the same operation on the target IP address and the target port number.
[0067] C. Processing missing values in the network traffic data set: We process the missing values in the data set by assigning them 0 or random values to meet the data format we need.
[0068] D. Balance the normal traffic data and abnormal traffic data in the network traffic data set. Since the data set itself simulates the real world, the data volume of abnormal data and normal data is very different, so the method of partially using normal traffic data is adopted to alleviate the problem of data set imbalance.
[0069] E. Perform time series processing on the network traffic data set to obtain time series data.
[0070] S102: Process the time-series data by using a sliding window method to obtain data stream sequence samples.
[0071] As an optional implementation, the time series data is processed by a sliding window method, specifically: the time series data is grouped into fixed length l wThe window is used as a data stream sequence sample to be input into the model for processing. The window is represented by The goal is to use the decoder to convert the window from its length l w -l s The shifted version of is reconstructed into a length of l w The original window of the intrusion detection algorithm can effectively capture and learn the temporal features in the data and improve the accuracy and robustness of intrusion detection.
[0072] S103: Calculate the anomaly score of the data stream sequence samples.
[0073] As a possible implementation, the anomaly score is evaluated by measuring the loss function of the data stream sequence sample. Given a stream x k Assigned to the window The anomaly score is then calculated by measuring the reconstruction error, assuming that the streaming data is assigned to the window W x , the corresponding loss function Score(x) is expressed as:
[0074] Score(x)=‖W x -D(E(W x ),W x,shifted )‖
[0075] Where E(W) represents the Euclidean modulus, E(W x ) represents the output calculated by the encoder for window W, D(E(W x ),W x,shifted ) represents the encoder output for the shift window and the subsequent encoder output, where W x,shifted W x A shifted version of arrive k represents the test window size.
[0076] S104. Build an AT-Transformer model and use data stream sequence samples to train the AT-Transformer model.
[0077] In the model building stage, this embodiment simplifies the traditional Transformer model. The simplified model diagram is as follows: Figure 3As shown. The AT-Transformer model constructed in this embodiment uses the same encoder-decoder structure as the traditional model, but uses two sets of encoders and decoders to reduce the complexity of the model. In addition, in order to solve the problem that the traditional encoder-decoder model cannot capture short-term trends and often misses anomalies if the deviation is too small, we use an autoregressive reasoning mode. The prediction of the reconstruction of each input sequence window is achieved by acting as an encoder-decoder network at each timestamp, and the reconstructed window is predicted in two parts.
[0078] In the first part, an approximate reconstruction of the input sequence window is generated. The deviation from this inference helps the attention network inside the AT-Transformer encoder to focus on subsequences with high deviations when extracting temporal trends. The encoder uses context-based attention to convert the input window W∈R K×m Convert it to a compressed representation, which is then converted to output W by the following formula 1 :
[0079] W = Sigmoid(FeedForward(P))
[0080] Where m represents the vector dimension, K represents the local context window length, and K>l w , R represents the vector matrix, Sigmoid() represents the activation function, FeedForward() represents the feedforward layer, and P represents the compressed potential representation of attention.
[0081] In the second part, the reconstruction loss of the decoder is used as the anomaly score, and the score matrix F = L in the second part is obtained. 1 , and then perform regression calculation again to get the output W of the second part 3 . The anomaly score generated in the first part indicates the deviation of the reconstructed output from the given input. This modifies the attention weights in the second part and gives higher neural network activations to specific input subsequences to extract short-term temporal trends. This design makes the model amplify the deviation because the reconstruction error acts as an activation in the attention part of the encoder, and adversarial training can also improve the generalization ability, making the model robust to different input sequences.
[0082] In the first part, we define the reconstruction loss L of the decoder 1 and L 2 As the input of the second part, it is expressed as:
[0083] L 1 =‖W 1 -W 2 ‖ 2
[0084] L2 =‖W 3 -W 2 ‖ 2
[0085] When defining the adversarial loss in the second part, the second decoder maximizes ‖W 3 -W 2 ‖ 2 The difference between the input window and the candidate reconstruction generated by the first decoder in the first part is used to distinguish the input window and the candidate reconstruction generated by the first decoder in the first part. The first decoder releases the signal to the second decoder through the reconstructed input, producing a degraded anomaly score. This part is a zero vector, so that the second decoder generates the same anomaly score as W at this stage. 3 The same output, while its goal in the first part is to match the input, the training object is expressed as:
[0086] min max||W 3 -W 2 || 2
[0087] Decoder1 Decoder2
[0088] Among them, Decoder1 represents the first decoder, and Decoder2 represents the second decoder.
[0089] The goal of the first decoder is to minimize the reconstruction error, and the goal of the second decoder is to maximize the reconstruction error, which is achieved by using the following loss function:
[0090] L 1 =+‖W 3 -W 2 ‖ 2
[0091] L 2 =-‖W 3 -W 2 ‖ 2
[0092] The reconstruction loss and adversarial loss from the first and second parts are combined using a comprehensive loss function to determine the total loss for each decoder, expressed as:
[0093] L 1 = l -n ‖W 1 -W 2 ‖ 2 +(1-l -n )‖W 3 -W 2 ‖ 2
[0094] L 2=e -n ‖W 3 -W 2 ‖ 2 +(1-e -n )‖W 3 -W 2 ‖ 2
[0095] Among them, n represents the training cycle, and l is a training parameter close to 1. Because the loss curve during neural network training usually changes in an exponential function, during the training process, we use the form of l -n The multi-head attention mechanism of the Transformer model can ensure data parallel processing, thereby shortening the training time.
[0096] S105. Input the data stream sequence samples into the AT-Transformer model to perform network traffic detection, and use visualization technology to visualize the detection results.
[0097] Before using the AT-Transformer model for network traffic detection, the model’s performance is evaluated and analyzed, and indicators such as accuracy, precision, recall, and AUC are used to measure the model. Other models are compared to test whether the effect meets the expected goal, and finally anti-noise comparison and visualization are performed.
[0098] In order to demonstrate the effectiveness of the proposed method in network traffic detection, four similar deep learning classification algorithms are selected for comparison. These algorithms are adjusted to ensure their good performance. The number of encoder and decoder layers of the AT-Transformer model is 6, and the number of attention heads we use in multi-head attention is 6. In actual operation, we use l s Fixed to one unit, window length l w Set to 25 to 50. The nonlinear activation function uses the ELU activation function. To prevent overfitting, the attention part uses the dropout method with a rate of 0.1. The model framework uses Pytorch, and the Adam optimizer is used to update the weights of the neural network during training to minimize the loss function. The number of neurons in the feedforward network layer is set to 256. The learning rate of the WUSTL_IIOT_2021 dataset is set to 0.001. The dataset is divided into training, validation, and test sets at a ratio of 80%, 10%, and 10%, and the entire training process lasted 15 training cycles (epochs). Table 1 shows the performance of different detection methods on the DoS data of the dataset WUSTL_IIOT_2021.
[0099] Table 1 Performance comparison of different detection methods on DoS data
[0100]
[0101] In addition to the DoS data, we also compared the performance of the Recon, Comm, and Backdoor data of the WUSTL_IIOT_2021 dataset, as shown in Table 2.
[0102] Table 2 Performance comparison of different detection methods on Recon, Comm, and Backdoor data
[0103]
[0104] In order to more comprehensively compare the anti-interference performance of the above anomaly detection algorithms, noise that satisfies the Gaussian distribution is randomly applied to them. The purpose of interference training is achieved by randomly adding noise to the training set. The scale of adding noise to the training set is controlled by the parameter p. The larger the proportion of P, the larger the scale of adding noise to the training set. Finally, the F1-score value of various methods changes with the noise proportion p. Figure 4 As shown, the shaded part is floating up and down.
[0105] It can be seen that after verification, the F1-score of this method changes little after being disturbed by noise, and the noise resistance is better, which proves the outstanding effect of this method.
[0106] The use of visualization technology to display the anomaly scores in the detection process improves the interpretability of the model. The model is no longer a completely black box model and has a certain degree of transparency. The specific anomaly scores of some serialized data are shown in Figure 5 middle.
[0107] In terms of time cost, we conducted a detailed comparison with the traditional Transformer method, and the results are as follows Figure 6 As shown in the figure. Through experiments with different training data set ratios, it can be found that no matter what the ratio of the training data set is, the time overhead of the AT-Transformer method is significantly reduced. Specifically, compared with the current optimal model in this field, the AT-Transformer model reduces the time overhead by an average of 85%. This significant optimization shows that the AT-transformer model greatly improves the computational efficiency while maintaining high detection accuracy, verifying the effectiveness of the method.
[0108] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A network traffic detection method, characterized in that: The method comprises: S101, obtaining a network traffic data set about a network attack, wherein the network traffic data set is composed of a plurality of flow data, and preprocessing the network traffic data set to obtain time-series data; S102, processing the time series data by a sliding window method to obtain a data stream sequence sample; S103, calculating the anomaly score of the data stream sequence sample; S104, build AT-Transformer model; S105. Input the data stream sequence samples into the AT-Transformer model to perform network traffic detection, and use visualization technology to visualize the detection results.
2. A network traffic detection method according to claim 1, characterized in that: Preprocess the network traffic dataset. The following operations are included: Remove features in the stream data that are irrelevant to the stream attribute features; Merge all features that characterize the source of a network attack into one feature, and merge all features that characterize the target of a network attack into one feature; Handling missing values in network traffic datasets; Balance the normal traffic data and abnormal traffic data in the network traffic data set; Perform time series processing on the network traffic data set to obtain time series data.
3. A network traffic detection method according to claim 1, characterized in that: The time series data is processed by the sliding window method, specifically: the time series data is grouped into fixed length l w The window is used as a data stream sequence sample to be input into the model for processing. The window is represented by The goal is to use the decoder to convert the window from its length l w -l s The shifted version of is reconstructed into a length of l w The original window, l s Indicates the length of the sample data stream.
4. A network traffic detection method according to claim 3, characterized in that: The anomaly score is evaluated by measuring the loss function of the data stream sequence sample. Given a stream x k Assigned to the window The anomaly score is then calculated by measuring the reconstruction error, assuming that the streaming data is assigned to the window W x , the corresponding loss function Score(x) is expressed as: Score(x)=‖W x -D(E(W x ),W x,shifted )‖ Where E(W) represents the Euclidean modulus, E(W x ) represents the output calculated by the encoder for window W, D(E(W x ),W x,shifted ) represents the encoder output for the shift window and the subsequent encoder output, where W x,shifted W x A shifted version of arrive k represents the test window size.
5. A network traffic detection method according to claim 1, characterized in that: The AT-Transformer model, including two sets of encoders and decoders, predicts the reconstructed windows in two parts by acting as an encoder-decoder network using an autoregressive inference mode at each timestamp to achieve prediction for the reconstruction of each input sequence window.
6. A network traffic detection method according to claim 5, characterized in that: The window predicted and reconstructed in two parts specifically includes: In the first part, an approximate reconstruction of the input sequence window is generated. The deviation from this inference helps the attention network inside the AT-Transformer encoder to focus on subsequences with high deviations when extracting temporal trends. The encoder uses context-based attention to convert the input window W∈R K×m Convert it to a compressed representation, which is then converted to output W1 by the following formula: W = Sigmoid(FeedForward(P)) Where m represents the vector dimension, K represents the local context window length, and K>l w , R represents the vector matrix, Sigmoid() represents the activation function, FeedForward() represents the feedforward layer, and P represents the compressed potential representation of attention; In the second part, the reconstruction loss of the decoder is used as the anomaly score to obtain the score matrix F=L1 of the second part, and then regression calculation is performed again to obtain the output W3 of the second part.
7. A network traffic detection method according to claim 6, characterized in that: The reconstruction losses L1 and L2 of the decoder defined in the first part are used as the input of the second part, expressed as: L1=‖W1-W2‖2 L2=‖W3-W2‖2 When defining the adversarial loss of the second part, the second decoder distinguishes the input window from the candidate reconstruction generated by the first decoder of the first part by maximizing the difference of ‖W3-W2‖2, using the first decoder to release a signal to the second decoder through the reconstructed input, producing a degraded anomaly score so that the second decoder generates the same output as W3 at this stage. The training object is expressed as: min max||W3-W2||2 Decoder1 Decoder2 Among them, Decoder1 represents the first decoder, and Decoder2 represents the second decoder.
8. A network traffic detection method according to claim 7, characterized in that: The goal of the first decoder is to minimize the reconstruction error, and the goal of the second decoder is to maximize the reconstruction error, which is achieved by using the following loss function: L1=+‖W3-W2‖2 L2=-‖W3-W2‖2 The reconstruction loss and adversarial loss from the first and second parts are combined using a comprehensive loss function to determine the total loss for each decoder, expressed as: Where n represents the training cycle, is a training parameter close to 1.