An abnormal network traffic detection method based on a bidirectional time convolutional neural network and a multi-head self-attention mechanism

Abstract

The application provides an abnormal network traffic detection method based on a bidirectional time convolution neural network and a multi-head self-attention mechanism. The method comprises the following steps: step 1, capturing an original network traffic file, segmenting the traffic according to different traffic data features, finding and deleting repeated and blank traffic, formulating corresponding rules to intercept or complete the segmented traffic data, generating a byte sequence of the traffic, and marking the type of the traffic; step 2, constructing an abnormal traffic detection model based on a bidirectional time convolution network and a multi-head self-attention mechanism, putting the traffic sequence obtained in step 1 into the model, and obtaining an abnormal traffic classification result. The application provides a new and efficient solution for the detection of abnormal network traffic, and its good performance in the face of the above abnormal traffic makes its application effect in the financial field more prominent.

Description

Technical Field

[0001] This invention belongs to the field of abnormal network traffic detection, and relates to an abnormal network traffic detection method based on bidirectional temporal convolutional neural network and multi-head self-attention mechanism. Background Technology

[0002] With the continuous development of internet technology, while the internet has shortened the distance between people and provided convenience, its ubiquity and complexity have also led to a diversification of cyberattack methods and types. As the cyber domain has become as important as territory, airspace, and territorial waters, the number of cyber hackers has surged dramatically. A constant stream of cyber viruses and abnormal traffic is rampant across the internet. Especially in recent years, with the continuous rise in Bitcoin prices, malware such as Dridex has begun to proliferate globally. These viruses share the common characteristics of high stealth, causing severe economic losses, and are frequently used to attack banks or new investment ventures. Victims often lose significant economic or computing resources without even realizing it. Similar cybersecurity issues have become increasingly prominent worldwide in recent years. As a relatively new field, cybersecurity urgently needs methods and theories to reliably address these cybersecurity problems, and accurately detecting these malicious cyberattacks is one of the keys to solving cybersecurity issues.

[0003] Anomaly network traffic detection, which identifies abnormal attack behaviors to ensure cybersecurity, has become an important research direction in the field of cybersecurity. Currently, the mainstream approach to anomaly network traffic detection involves first collecting network traffic data, then extracting relevant features, and classifying the traffic into normal and abnormal data, as well as their attack types—essentially, data classification. While machine learning has achieved some success in data classification, it still suffers from low accuracy, insufficient ability to fit complex data distributions, and reliance on manual traffic feature extraction. To address these issues, deep learning has been increasingly applied to anomaly network traffic monitoring in recent years. Four common deep learning techniques are: Fully Connected Neural Networks (FCN), Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), and Generative Adversarial Networks (GAN). These methods effectively avoid the inaccuracies and subjectivity associated with manual traffic feature extraction. However, when faced with highly concealed (or encrypted) traffic such as Coinminer and Dridex, as mentioned above, the accuracy of these models decreases, and they suffer from unstable feature extraction. Summary of the Invention

[0004] To address the limitations of traditional unidirectional network structures in capturing bidirectional semantic features of network traffic, and the inherent time-sensitivity and continuity of network traffic, this method proposes a technique for detecting abnormal network traffic that utilizes a bidirectional temporal convolutional neural network combined with a multi-head self-attention mechanism. This technique includes:

[0005] Step 1: Capture publicly available raw network traffic files, segment the traffic based on different traffic data characteristics, identify and delete duplicate and blank traffic; formulate corresponding rules to truncate or complete the segmented traffic data, generate the byte sequence of the traffic, and mark the type of traffic;

[0006] Step 2: Construct an abnormal traffic detection model based on a bidirectional temporal convolutional network and a multi-head self-attention mechanism. Put the traffic sequence obtained in Step 1 into the model to obtain the abnormal traffic classification results.

[0007] Firstly, the specific steps for obtaining code slices in step 1 above are as follows:

[0008] Step 1.1: Capture public network traffic and save it as a PCP binary format traffic file;

[0009] Step 1.2: Based on the size of the saved traffic file, appropriately select traffic data features with different granularities for flow and session to segment the data and obtain multiple traffic data.

[0010] Step 1.3: Delete feature information irrelevant to the content of the segmented traffic, and fill in the deleted content in a random manner to ensure the smooth progress of training; traverse the traffic data according to the traffic-related features, find duplicate and blank irrelevant traffic and delete it.

[0011] Step 1.4: Traverse the traffic data, extract the first 784 bytes of each traffic stream. If the traffic length is insufficient, pad it with '0'. The final traffic byte sequence is represented as A = (a0, a1, ... a... K );

[0012] Step 1.5: Mark the data according to the sequence of data bytes and set corresponding category labels for different markers.

[0013] Secondly, the specific steps of step 2 above are as follows:

[0014] Step 2.1: Randomly divide the preprocessed dataset into 20 parts, use 10 parts for training, and use the remaining 10 parts as the test set.

[0015] Step 2.2: Establish a temporal convolutional neural network composed of causal convolution, dilated convolution, and residual connections. The final fully connected layer of the neural network is replaced with a convolutional layer, ensuring that subsequent layers have the same length as previous layers and guaranteeing that the network's input and output lengths are identical. Causal convolution ensures that the receptive field of the network can be expanded simply by changing the kernel size, without changing the layer depth. Furthermore, causal convolution significantly improves the modeling ability for tasks requiring long-term memory sequences. Dilated convolution allows for interval sampling of the input, adding a dilation parameter d, where d = 2. n (where n represents the nth layer of the network). After adding the dilation parameter, the receptive field of the previously established causal network expands further due to the exponential growth of d, ensuring that more information can be processed. The mathematical expression for the dilated convolution function F with kernel size k and input x is as follows:

[0016]

[0017] Wherein, filter F = (f(1), f(2), ..., f(k)), sequence X = (x1, x2, ..., x), c is the network traffic feature, "*" represents convolution operation, d represents hole parameter, also known as dilation coefficient, and s-di represents the directional information of a certain historical traffic.

[0018] To address the vanishing gradient problem in the positive interval and reduce feature loss, ELU is used to introduce a nonlinear factor, the specific formula of which is as follows:

[0019]

[0020] The value of α is between 0 and 1.

[0021] Adding residual connections ensures that information can be passed across layers in the network, enabling layers to learn and modify efficiently, thus improving convergence speed and overall efficiency. A temporal neural network model is formed by combining causal convolution, dilated convolution, and residual connections. Training the training set on this model yields preliminary positive flow training results.

[0022] Step 2.3: Construct a multi-head self-attention mechanism model. The multi-head self-attention mechanism calculates the weights of the encoded data and uses these weights to determine more important features of abnormal traffic, thereby further improving the model's accuracy. The specific steps are as follows:

[0023] Step 2.3.1: For the input data x1 and x2, obtain the input feature a through the Embedding layer. 1 With a 2 .

[0024] Step 2.3.2: Calculate a1 With a 2 Correlation has three parameters q i (query), k i (key) and v i (value).

[0025] Where: q i =w q a i ;k i =w k a i ;v i =w v a i where w is obtained through network training, and w q w k w v These are weighting factors related to the correlation of three multi-head self-attention mechanisms, obtained through network training.

[0026] Step 2.3.3: Obtain the self-attention matrix using the scaled dot product method. Specifically, this involves scaling the q obtained in the previous step... 1 ,q 2 ,…… q i Value and k 1 ,k 2 ,……k j Multiply the values ​​of q by the square root of d, where q i The value reflects the question value while k j The value reflects the current official value, and the general formula is as follows:

[0027]

[0028] Where d is the expansion coefficient in step 2.

[0029] Step 2.3.4: Softmax normalize the α obtained in Step 3.3 to obtain γ, thereby increasing the nonlinearity of the model. Calculate the correlation between γ and v from Step 3.2 to obtain the final feature b. The specific formula is as follows:

[0030]

[0031] Step 2.3.5: Put the preliminary training results from Step 2 into the self-attention mechanism model for further training to obtain the training results of positive flow.

[0032] Step 2.4: Reverse the training result sequence, and feed the reversed matrix sequence into the temporal neural network model and the self-attention mechanism model for training to obtain the feature b of the reverse flow. 2 The nonlinear representation F is obtained using the ELU function. 2 .

[0033] Step 2.5: Fuse the final feature extraction results of the forward and reverse traffic using an activation function to obtain a global abnormal traffic feature F. 总 The network traffic is classified based on the final traffic characteristics to obtain an abnormal traffic detection model. The test set is then fed into the abnormal traffic detection model to obtain the final result.

[0034] This invention employs a special type of convolutional neural network—the Temporal Convolutional Network (TCN)—and constructs a Bidirectional Temporal Convolutional Network (BiTCN) considering the characteristics of network traffic serialization and bidirectional semantics. Because temporal convolutional networks have a larger receptive field and can process traffic data in parallel, they achieve superior results in traffic anomaly detection. Compared to RNNs, TCN's gradients are more stable along the network depth direction due to internal residual connections; compared to LSTMs, TCN's temporal memory effectively compensates for LSTM's poor memory and degradation over time. Therefore, in this invention, we further improve its detection accuracy by constructing a bidirectional TCN structure model.

[0035] Meanwhile, considering the time-sensitive and continuous nature of network traffic, and to improve the processing capability of global information and thus better optimize the detection model, this invention adds a multi-head self-attention mechanism to the BiTCN model. The self-attention mechanism assigns corresponding weights to different data distributions, focusing on data with higher weights and giving less attention to data with lower weights, thereby improving the efficiency of global data processing. Finally, we obtained a network traffic anomaly detection model combining a bidirectional temporal convolutional neural network and a self-attention mechanism, and experimental results demonstrate its superior performance, especially in dealing with highly concealed, harmful, and widely intrusive anomaly traffic from Coinminer, Dridex, and other sources. This invention provides a novel and efficient solution for anomaly network traffic detection, and its excellent performance in dealing with the aforementioned anomaly traffic makes its application in the financial field even more prominent.

[0036] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0037] 1. Compared to RNNs, TCNs exhibit more stable gradient changes along the network depth direction due to their internal residual connections. Compared to LSTMs, TCNs' temporal memory effectively compensates for LSTMs' poor memory and degradation over time. Furthermore, we pay close attention to the bidirectional semantic features of network traffic; therefore, in this paper, we further improve the detection accuracy by constructing a bidirectional TCN structure model.

[0038] 2. Self-attention mechanism. The self-attention mechanism can assign corresponding weights to different data distributions, giving more attention to data with larger weights and less attention to data with smaller weights, thereby improving the efficiency of global data processing.

[0039] 3. By combining BiTCN with a multi-head self-attention mechanism, this model achieves more accurate detection capabilities for highly concealed abnormal traffic. Testing shows that this model significantly outperforms other existing models in detecting traffic from Coinminer, Dridex, and Trickbot. Especially against common financial viruses like Coinminer and Dridex, the application of this model can more effectively reduce the economic losses they cause. Attached Figure Description

[0040] Figure 1 This is a flowchart of an abnormal network traffic detection method based on a bidirectional temporal convolutional neural network and a multi-head self-attention mechanism.

[0041] Figure 2 This is a diagram of the model structure of a bidirectional temporal convolutional neural network.

[0042] Figure 3 The results show the average detection efficiency (including precision, recall, F1-measure, and accuracy) of the invented BiTCN+self-attention model as a function of epochs on the CTU dataset.

[0043] Figure 4 The results show the accuracy comparison between the invented BiTCN+self-attention model and four other models—TCN, LSTM, 1D-CNN, and BiTCN—on the CTU dataset.

[0044] Table 1 shows the detection results of the invented BiTCN+self-attention model, as well as four other models—TCN, LSTM, 1D-CNN, and BiTCN—on the CTU dataset.

[0045] Table 2 shows the classification results of the invented BiTN+self-attention model, along with four other models—TCN, LSTM, 1D-CNN, and BiTCN—on 10 types of abnormal traffic and 1 type of normal traffic on the CTU dataset. Detailed Implementation

[0046] The present invention will be further described below with reference to the accompanying drawings and embodiments. It should be noted that the described embodiments are only intended to facilitate the understanding of the present invention and do not constitute any limitation thereof.

[0047] This invention aims to address the problems of low accuracy and efficiency in anomaly traffic detection, particularly the unstable extraction of highly concealed anomaly traffic features by tools such as Coinminer and Dridex. It proposes an anomaly traffic detection method based on a bidirectional temporal convolutional neural network (TCN) and a multi-head self-attention mechanism. The method extracts and fuses anomaly traffic features using a bidirectional TCN, and improves accuracy with a multi-head self-attention mechanism, thereby establishing an efficient anomaly traffic detection model.

[0048] like Figure 1 As shown, the present invention proposes an abnormal network traffic detection method based on a bidirectional temporal convolutional neural network and a multi-head self-attention mechanism, comprising:

[0049] Step 201: Capture publicly available raw network traffic files, segment the traffic according to different traffic data characteristics, identify and delete duplicate and blank traffic; formulate corresponding rules to truncate or complete the segmented traffic data, generate the byte sequence of the traffic, and mark the type of traffic.

[0050] The purpose of traffic segmentation in this invention is that the original traffic files contain a large amount of blank and duplicate traffic. Neural networks cannot be directly trained on captured and saved binary PCAP traffic files. Therefore, without traffic segmentation, the learning of neural network features will be insufficient. Segmenting network traffic according to different characteristics reduces the amount of training data, improves accuracy, and removes irrelevant traffic, reducing experimental errors and further improving model accuracy.

[0051] Step 2011: Capture publicly available network traffic and save it as a PCP binary format traffic file;

[0052] Step 2012: Based on the size of the saved traffic file, select appropriate traffic data features with different granularities for flow and session to segment the data and obtain multiple traffic data.

[0053] Step 2013: Delete feature information irrelevant to the content in the segmented traffic, and fill in the deleted content in a random manner to ensure the smooth progress of training; traverse the traffic data according to the traffic-related features, find duplicate and blank irrelevant traffic and delete it.

[0054] Step 2014: Traverse the traffic data, extract the first 784 bytes of each traffic stream. If the traffic length is insufficient, pad it with '0'. The final traffic byte sequence is represented as A = (a0, a1, ... a... K );

[0055] Step 2015: Tag the data according to the data flow byte sequence, and set corresponding category labels for different tags.

[0056] The reason for implementing traffic data truncation and completion in this invention is that the first part of network traffic has a greater impact on the traffic and often determines its data type. If the traffic data is not truncated and completed, training with all traffic would significantly reduce training efficiency while failing to improve the model's detection accuracy. Therefore, selecting the first 784 bytes of traffic data for training not only preserves the network traffic type but also improves training efficiency and ensures that the obtained traffic sequence is identical to the input traffic sequence in the model.

[0057] Step 202: Construct an abnormal traffic detection model based on a bidirectional temporal convolutional network and a multi-head self-attention mechanism. Put the traffic sequence obtained in step 1 into the model to obtain the abnormal traffic classification result.

[0058] Step 2021: The preprocessed dataset is randomly divided into 20 parts, 10 of which are used for training, and the remaining 10 are used as the test set.

[0059] Step 2022: Establish a temporal convolutional neural network composed of causal convolution, dilated convolution, and residual connections. The final fully connected layer of the neural network is replaced with a convolutional layer, ensuring that subsequent layers have the same length as previous layers and guaranteeing that the network's input and output lengths are identical. Causal convolution ensures that the receptive field of the network can be expanded simply by changing the kernel size, without changing the layer depth. Furthermore, causal convolution significantly improves the modeling ability for tasks requiring long-term memory sequences. Dilated convolution allows for interval sampling of the input, adding a dilation parameter d, where d = 2. n (where n represents the nth layer of the network). After adding the dilation parameter, the receptive field of the previously established causal network expands further due to the exponential growth of d, ensuring that more information can be processed. The mathematical expression for the dilated convolution function F with kernel size k and input x is as follows:

[0060]

[0061] Where c represents network traffic characteristics, "*" indicates convolution operation, d represents the hole parameter, also known as the dilation coefficient, and s-di represents the directional information of a certain historical traffic flow.

[0062] To address the vanishing gradient problem in the positive interval and reduce feature loss, ELU is used to introduce a nonlinear factor, the specific formula of which is as follows:

[0063]

[0064] The value of α is between 0 and 1.

[0065] Adding residual connections ensures that information can be passed across layers in the network, enabling layers to learn and modify efficiently, thus improving convergence speed and overall efficiency. A temporal neural network model is formed by combining causal convolution, dilated convolution, and residual connections. Training the training set on this model yields preliminary positive flow training results.

[0066] Step 2023: Construct a multi-head self-attention mechanism model. The multi-head self-attention mechanism calculates the weights of the encoded data and uses these weights to determine more important features of abnormal traffic, thereby further improving the model's accuracy. The specific steps are as follows:

[0067] Step 2.3.1: For the input data x1 and x2, obtain the input feature a through the Embedding layer. 1 With a 2 .

[0068] Step 2.3.2: Calculate a 1 With a 2 Correlation has three parameters q i (query), k i (key) and v i (value).

[0069] Where: q i =w q a i ;k i =w k a i ;v i =w v a i , where w is obtained by training the network.

[0070] Step 2.3.3: Obtain the self-attention matrix using the scaled dot product method. The specific formula is as follows:

[0071]

[0072] Where d is the expansion coefficient in step 2.

[0073] Step 2.3.4: Softmax normalize the α obtained in Step 2.3.3 to obtain γ, thereby increasing the nonlinearity of the model. Calculate the correlation between γ and v from Step 3.2 to obtain the final feature b. The specific formula is as follows:

[0074]

[0075] Step 2.3.5: Put the preliminary training results from Step 2 into the self-attention mechanism model for further training to obtain the training results of positive flow.

[0076] Step 2.4: Reverse the training result sequence, and feed the reversed matrix sequence into the temporal neural network model and the self-attention mechanism model for training to obtain the feature b of the reverse flow. 2 The nonlinear representation F is obtained using the ELU function. 2 .

[0077] Step 2.5: Fuse the final feature extraction results of the forward and reverse traffic using an activation function to obtain a global abnormal traffic feature F. 总 The network traffic is classified based on the final traffic characteristics to obtain an abnormal traffic detection model. The test set is then fed into the abnormal traffic detection model to obtain the final result.

[0078] This invention primarily targets the detection of abnormal network traffic. It proposes a method for abnormal network traffic detection using a bidirectional temporal convolutional neural network and a multi-head self-attention mechanism. Referring to relevant experiments, the CTU dataset, which demonstrates significant experimental results, was selected for testing. The CTU dataset includes 11 types of network traffic data from 2016 to 2019, comprising 10 types of abnormal traffic and 1 type of normal traffic.

[0079] This invention compares the proposed bidirectional temporal neural network plus self-attention mechanism model (BiTCN+self-attention) with TCN, 1D-CNN, LSTM and BiTCN models, calculates the average detection efficiency of the five models (including precision, recall, F1-measure and accuracy), and demonstrates the efficiency improvement of the present invention through data comparison.

[0080] Table 1 shows the detection performance of the five models on the CTU dataset. It can be clearly seen that the BiTCN+self-attention model proposed in this invention significantly improves the accuracy of anomaly traffic detection, reaching 99.278%. With increasing attention paid to anomaly traffic in networks, the need for high-precision and high-efficiency anomaly traffic detectors is also growing. While the invention of BiTCN has significantly improved the accuracy of anomaly traffic detection, the BiTCN+self-attention model proposed in this invention improves the accuracy by nearly 0.1% compared to the aforementioned models, which already exceed 99%, and also exhibits good portability. In addition to accuracy, this invention also shows significant improvements in precision, recall, and F1-measure compared to the other four models. Compared to the LSTM model, the recall value of this invention is improved by nearly 3.2%; compared to 1D-CNN, the F1-measure value of this invention is improved by nearly 5%; overall, on the CTU dataset, compared to TCN, LSTM, 1D-CNN and BiTCN models, the accuracy of this invention is improved by 1.51%, 2.16%, 2.56% and 0.23%, respectively, and the correctness is improved by 0.45%, 0.1922%, 1.51% and 0.08%, respectively.

[0081] Table 1

[0082] Model TCN LSTM 1D-CNN BiTCN BiTCN+ATTENTION Accuracy 96.05 95.39 95.00 97.33 97.56 Recall rate 96.29 94.51 94.08 97.36 97.72 F1-measure 96.15 94.75 92.84 97.28 97.62 accuracy 98.83 98.35 97.77 99.19 99.28

[0083] from Figure 3 and Figure 4 It can be seen that the BiTCN+self-attention model achieves higher detection accuracy than the other four models across different epochs. Compared to other models, this model, during training, assigns weights to different data distributions through a self-attention mechanism, focusing on data with higher weights and giving less attention to data with lower weights. This improves the efficiency of global data processing. Combined with the bidirectional semantic features of BiTCN, this invention can accurately extract the main features of network traffic. Furthermore, the use of a temporal convolutional network algorithm significantly accelerates its convergence efficiency. These two advantages enable this model to be more accurate than the other five models during initial training. When the epoch reaches 6, the accuracy curve of the BiTCN model tends to stabilize; while the other five models generally stabilize at epochs 7 or 8. Experimental results show that the BiTCN+self-attention model achieves higher detection accuracy when the epoch is set to 10; therefore, the epoch value for this model is set to 10.

[0084] Table 2 shows the comparative experimental results of the BiTCN+self-attention model of this invention with conventional detection models such as TCN, LSTM, 1D-CNN, and BiTCN, used to detect 11 types of network traffic, including three evaluation metrics: precision, recall, and F1-measure. Combining Tables 1 and 2, it can be seen that the BiTCN+self-attention model of this invention performs better than all other models. It has a particularly significant advantage in detecting three types of anomalous traffic: coinminer, Dridex, and TrickBot. Coinminer is a highly stealthy virus, Dridex is a currently active and technologically advanced banking Trojan, and TrickBot is a Trojan with extremely strong mutation capabilities and high risk; both belong to encrypted anomalous network traffic. The commonality among these three is the risk of causing significant economic losses, and current detection methods for these three are relatively lacking. Other traditional anomaly detection models showed unsatisfactory performance for these three types of traffic. For coinminer-type traffic, BiTCN+self-attention outperformed other models in recall, precision, and F1-measure, with precision reaching 100%. For Dridex-type traffic, the CNN model achieved a recall of 98.373%, but its precision and F1-measure were only 50.37% and 66.626%, respectively. The BiTCN+self-attention model, however, achieved the highest precision and F1-measure (82.667% and 86.036%, respectively), exceeding the second-highest by 1.869% and 1.932%. For TrickBot anomaly traffic, the CNN model achieved a precision of 96.417%, but its recall and F1-measure were only 41.457% and 57.982%, respectively. This indicates that the model incorrectly predicted many actually abnormal traffic flows as normal traffic, leading to inadequate protection and significant losses. The accuracy and F1-measure of the BiTCN+self-attention model of this invention can reach 86.555% and 88.857% respectively, which are 2.801% and 1.719% higher than the second highest model. It can be seen that even for encrypted traffic that is currently relatively difficult to detect, the BiTCN+self-attention model of this invention can efficiently and accurately detect encrypted abnormal traffic with the same amount of hardware resources.

[0085] Table 2

[0086]

Claims

1. A method for detecting abnormal network traffic based on a bidirectional time convolutional neural network and a multi-head self-attention mechanism, characterized in that, The method comprises the following steps: Step 1, capturing the original network traffic file, segmenting the traffic according to different traffic data features, finding and deleting repeated and blank traffic; formulating corresponding rules to intercept or complete the segmented traffic data, generating byte sequences of the traffic, and marking the type of the traffic; Step 2, constructing an abnormal traffic detection model based on a bidirectional time convolutional neural network and a multi-head self-attention mechanism, putting the traffic sequence obtained in step 1 into the model to obtain the abnormal traffic classification result; The specific implementation of step 2 comprises the following steps: Step 2.1, randomly divide the preprocessed data set into 20 parts, and use 10 parts for data training and the remaining 10 parts as a test set; Step 2.2, establish a time series convolutional neural network composed of causal convolution, dilated convolution, residual connection part, replace the last fully connected layer of the neural network with a convolutional layer, so that the length of the subsequent layer is uniform with the previous layer, and ensure that the length of the network input and output is the same, the causal convolution network ensures that the receptive field of the network is expanded only by changing the size of the convolution kernel under the condition that the depth of the layer is unchanged; dilated convolution allows the input to exist interval sampling, increases a hole parameter d, also known as the expansion coefficient; wherein , n represents the nth layer of the network, after adding the hole parameter, due to the exponential growth of d, the receptive field of the previously established causal network is further expanded, ensuring that more information is processed, for a convolution kernel size of k, the filter , sequence =( , ,……, )The expression of dilated convolution at is: ； Wherein c is the network traffic feature, "*" represents convolution operation, d represents the hole parameter, also known as the expansion coefficient, and s-di represents the direction information of a certain traffic in history; In order to solve the problem of positive interval gradient disappearance and reduce feature loss, ELU is introduced to introduce non-linear factors, and its specific formula is as follows: ​ wherein The value is between 0 and 1; the causal convolution, the dilated convolution, and the residual connection are combined to form a time sequence neural network model, the training set is put into the time sequence neural network model for training, and a preliminary forward flow training result is obtained. Step 2.3, constructing a multi-head self-attention mechanism model, the multi-head self-attention mechanism calculates the weight of the encoding, and the more important features of the abnormal traffic are calculated from the weight value, thereby further improving the accuracy of the model; Step 2.4: Reverse the training result sequence, put the reversed matrix sequence into the time sequence neural network model and the self-attention mechanism model for training to obtain the characteristics of the reverse traffic , and obtain its nonlinear representation with the ELU function ; Step 2.5: The final feature extraction results of the forward and reverse traffic are fused by an activation function to obtain a global abnormal traffic feature The network traffic is classified according to the final traffic feature to obtain an abnormal traffic detection model, and the test set is put into the abnormal traffic detection model to obtain a final result. The specific implementation of step 2.3 comprises the following steps: Step 2.3.1: For the input data With , the input feature is obtained through the Embedding layer With ; Step 2.3.2: Calculation With relevance, there are three parameters (query), (key) and (value); wherein: ; ; wherein , is a weight factor of the three multi-head self-attention mechanism dependencies trained by the network. Step 2.3.3: Get the self-attention matrix by the method of scaled dot-product, which is to multiply the , ,…… values with , ,…… values, then divide by the square root of , where values reflect the question values and values reflect the current formal values, the formula is as follows: ​ Wherein d is the expansion coefficient; Step 2.3.4: Calculate the correlation of the features obtained in step 2.3.3 and step 2.3.2, and obtain the final feature b Step 2.3.5: Perform softmax normalization on the final feature b , thereby increasing the nonlinearity of the model, and obtaining the final output Step 2.3.6: Calculate the correlation of the features obtained in step 2.3.5 and step 2.3.2, and obtain the final feature b i Step 2.3.7: Perform softmax normalization on the final feature b i , thereby increasing the nonlinearity of the model, and obtaining the final output ​ Step 2.3.5: Put the above preliminary training result into the self-attention mechanism model for further training to obtain the training result of the forward traffic.

2. The method of claim 1, wherein, The specific implementation of step 1 comprises the following steps: Step 1.1, capturing the public network traffic and saving it as a pacp binary format traffic file; Step 1.2, according to the size of the saved traffic file, appropriately selecting traffic data features with different granularities of flow and session for segmentation, obtaining multiple traffic data; Step 1.3, deleting the feature information irrelevant to the content of the segmented traffic, and complementing the deleted content in a randomly generated manner to ensure smooth training; according to the relevant features of the traffic, the traffic data is traversed and found, and the repeated and blank irrelevant traffic is deleted; Step 1.4, traverse the traffic data, intercept the first 784 bytes of each traffic, if the traffic length is insufficient, then use '0' to complete the insufficient part, finally the traffic byte sequence obtained is represented as A=(a0, a1,...a K ); Step 1.5, according to the byte sequence of the traffic, marking it, and setting corresponding category labels for different marks.

Images