Edge anomaly traffic detection method based on multi-scale aggregation Transform

Abstract

The invention relates to an edge anomaly traffic detection method based on a multi-scale aggregation Transform, and the method comprises the steps: S1, obtaining edge network traffic data, carrying out the data preprocessing and feature extraction, and obtaining a training sample set; s2, constructing an edge anomaly traffic detection model based on multi-scale aggregation Transform, wherein the edge anomaly traffic detection model comprises an input layer, a coding layer, a classification layer and an output layer; the input layer extracts feature information from the flow data; the coding layer adopts a layered structure to carry out multi-scale time sequence representation, a coding module of each layer is composed of a time sequence representation module and a Transform coding module, the time sequence representation module carries out segmentation mapping on feature data, and the Transform coding module encodes a mapping vector by using a multi-head self-attention mechanism; after the output of the coding module is aggregated, a detection result is obtained through the classification layer and is output through the output layer; training the model through the training sample set; and S3, applying the trained model to an edge system to assist the edge system in detecting and filtering abnormal traffic. The method is beneficial for improving the accuracy of abnormal flow detection.

Description

Technical Field

[0001] The present invention relates to the technical field of abnormal traffic detection, and in particular to an edge abnormal traffic detection method based on a multi-scale aggregation Transformer. Background Art

[0002] With the rapid development of the Internet of Things (IoT), the era of the Internet of Everything is coming, and it plays an important role in all walks of life, including smart homes, industrial control, smart cities, etc. In order to process the massive data generated by IoT terminal devices more efficiently and in real time, Mobile Edge Computing (MEC) was born. By deploying computing, storage and other resources at the edge of the network, it realizes the local processing and analysis of data. This computing mode reduces network latency and data transmission costs, and has become the key to realizing the connectivity, centralization and intelligence of IoT technology. With the gradual popularization and widespread application of IoT technology, users are increasingly subject to abnormal traffic attacks and various types (such as denial of service attacks, data leakage and privacy infringement). In order to prevent these attacks, network traffic is usually encrypted through traffic encryption technology to ensure edge network security to ensure data confidentiality, integrity and identity authentication. Although traffic encryption technology plays an important role in preventing network attacks and ensuring edge network security, it also indirectly makes malicious traffic difficult to detect, which makes it difficult to accurately detect potential abnormal phenomena in encrypted traffic. This seriously affects the system's Quality-of-Service (QoS) and also poses a serious threat to edge network security.

[0003] Abnormal traffic detection is an important technology to ensure the security of edge networks. By monitoring and analyzing network traffic, abnormal traffic detection technology can promptly detect and block malicious traffic to ensure the security of edge networks. Most of the work related to abnormal traffic detection focuses on cloud data centers, and less work involves edge environments and fails to fully consider the characteristics of edge environments. The cloud computing model usually transmits and aggregates data to cloud data centers for processing, which can easily cause high network latency and cannot meet applications with high real-time requirements. In contrast, MEC sinks computing and storage resources to edge nodes close to terminal devices. Therefore, deploying abnormal traffic detection modules on edge nodes can effectively avoid high latency in data transmission and reduce the pressure on cloud data center network bandwidth. At present, edge abnormal traffic security still faces the following challenges:

[0004] (1) During the daily operation of the system, IoT terminal devices and edge servers generate massive amounts of data, which are usually high-dimensional. If all traffic generated by the system is monitored throughout the entire process, it will undoubtedly cause excessive abnormal traffic detection costs and also affect the real-time and efficiency of the detection. Therefore, the edge abnormal traffic detection method should have efficient data processing capabilities and be able to quickly analyze and process data.

[0005] (2) The widespread application of traffic encryption technology has seriously increased the difficulty of detecting abnormal network traffic. This requires the edge abnormal traffic detection method to select appropriate features for abnormal traffic modeling and detection to accurately identify abnormal traffic. However, how to fully consider the representativeness and separability of features in the abnormal traffic detection process is still a challenging problem.

[0006] (3) The edge environment is highly dynamic. The behavior of different devices and the operating environment they are in may change over time, which requires the edge abnormal traffic detection method to be able to adapt to the dynamics of this environment. Most existing works use convolutional neural networks (CNN) and recurrent neural networks (RNN) and their improved variants to detect abnormal traffic using features such as statistical features of data packets and payload information. However, they still have shortcomings in capturing long-distance dependencies and parallel processing capabilities and require manual feature statistics. Summary of the invention

[0007] The purpose of the present invention is to provide an edge abnormal traffic detection method based on multi-scale aggregation Transformer, which is conducive to improving the accuracy of abnormal traffic detection.

[0008] In order to achieve the above object, the technical solution adopted by the present invention is: a method for detecting abnormal edge traffic based on a multi-scale aggregation Transformer, comprising the following steps:

[0009] S1, obtain edge network traffic data from the edge system, and preprocess and extract features of the data to obtain a training sample set;

[0010] S2. Construct an edge abnormal traffic detection model based on multi-scale aggregation Transformer; the edge abnormal traffic detection model includes an input layer, an encoding layer, a classification layer and an output layer; the input layer extracts feature information from the input traffic data to form a multi-dimensional time series, and each dimension describes the change of the corresponding feature; the encoding layer adopts a hierarchical structure for multi-scale time series representation, wherein the encoding module of each layer is composed of a time series representation module and a Transformer encoding module, the time series representation module divides and maps the feature data, and the Transformer encoding module uses a multi-head self-attention mechanism to encode the mapping vector, and the two work together to capture the changing laws and patterns of the traffic; the classification layer includes an average pooling layer and a Softmax classifier, after the outputs of all encoding modules are averaged and aggregated, a comprehensive feature vector is extracted through the average pooling layer, and then it is input into the Softmax classifier; the output layer outputs the detection result; the edge abnormal traffic detection model is trained through a training sample set;

[0011] S3. Apply the trained edge abnormal traffic detection model to the edge system to assist it in detecting and filtering abnormal traffic.

[0012] Furthermore, the implementation method of step S1 is:

[0013] First, the original edge network traffic data is obtained from the edge system, and then the key features for classification are extracted from the original edge network traffic data; when processing the original edge network traffic data, it is split into independent sessions according to the five-tuple, which is: source IP, destination IP, source port, destination port and transport layer protocol; then, for each session, the first t packets are extracted, where the two features of payload length L and arrival interval T are extracted from each packet, and T of the first packet is set to 0; the i-th training traffic sample is defined as:

[0014]

[0015] Among them, t represents the total number of time steps, that is, the number of sample data packets extracted; the sample corresponding to the jth time step is Represents the characteristic data of the data packet at this time step, recorded as in and Respectively represent the payload length of the jth data packet in the i-th sample and the arrival interval time with the j-1th data packet; when extracting data packet features, discard the DNS data packet and fill it with empty packets; arrange the extracted features by time to obtain the training sample

[0016] Therefore, the training sample set is:

[0017]

[0018] Where N is the total number of training samples; Y i For sample X i The label is denoted by Y i ∈{1,2,...,K}, where K represents the total number of abnormal traffic types.

[0019] Furthermore, in step S2, the coding layer is implemented as follows:

[0020] The coding layer adopts a hierarchical structure to support multi-scale time series representation and is composed of multiple coding modules. Feature extraction is performed at each layer, and the characteristics of the input data are fully understood by aggregating the feature vectors of each layer. The coding module uses the time series representation module to extract the characteristic time series of the data packet. Split to shorten its length; assuming the slice size is v, the input sequence contains m features, and the time dimension is t; after sequence segmentation, the length of the new sequence is The size of each slice is v×m; then, these slices are projected onto a new dimension C and normalized; finally, the normalized sequence is added to the position code to obtain a new time series

[0021] The position encoding includes absolute and relative position encoding, which represent static and learnable position embedding respectively. By representing the context information between adjacent time slices, the two position encodings are fused to improve the classification performance. Specifically, a one-dimensional convolution is used to capture the position relationship between adjacent time slices, where the convolution kernel size is set to k and the convolution kernel size is set to k. Zero padding is used to prevent the scale reduction problem and to highlight the absolute position;

[0022] Next, the new temporal representation is input into the multi-head self-attention layer of the Transformer encoding module, with the input being the query Q, key K and value V, and the input matrix is ​​calculated as:

[0023]

[0024] Among them, d k represents the dimension of the input query Q and key K; the calculation process of multi-head self-attention is:

[0025] MultiHead(Q,K,V)=Concat(head1,...,head h )W O (4)

[0026] Where h represents the number of attention heads, Concat represents the serial concatenation operation; the calculation process of each head is:

[0027] head i =Attention(QW i Q ,KW i K ,VW i K ) (5)

[0028] Among them, the matrix is the linear projection parameter, and the dimension of each head is

[0029] After being processed by the multi-head self-attention layer, the result is passed to the fully connected feedforward neural network FNN, which then extracts features from multiple angles. The feedforward neural network FNN contains two linear transformations and embeds the ReLU activation function in it, specifically:

[0030] FNN(x)=max(0,xW1+b1)W2+b2 (6)

[0031] Among them, W1 and W2 both represent weight matrices in FNN, b1 and b2 both represent bias vectors;

[0032] Finally, the output results of the FNN of each encoding module are aggregated and passed to the classification layer to complete abnormal traffic detection.

[0033] Furthermore, in step S2, the classification layer is implemented as follows:

[0034] The classification layer includes an average pooling layer and a Softmax classifier. After the output results of each encoding module are averaged and aggregated, the results are input into the average pooling layer to reduce the dimension of the feature vector, thereby preventing the model from overfitting and retaining key information. Then, the feature vector processed by the average pooling layer is sent to the Softmax classifier to calculate the probability distribution of each category. The specific process is as follows:

[0035]

[0036] Among them, p i (x) represents the probability of the i-th category, x represents the model output, and K represents the number of categories of training samples; the Softmax function converts the model output into a probability distribution and selects the label with the largest probability as the predicted label.

[0037] Furthermore, in step S2, the implementation method of training the edge abnormal flow detection model through the training sample set is:

[0038] Input is edge traffic data X i , training round N and training batch B; first, initialize the model parameters; in each training round and training batch, input the sample into the time series representation module, split the sequence to obtain vectors Then use convolution to transform the vector The dimension is mapped to a new dimension C and a new vector representation is obtained Next, Normalize and extract The position information is added to the normalized vector to obtain the vector X containing the position information. i '; Then, vector X i 'Input it into the multi-head self-attention layer to encode it and get the vector Then, the features are extracted through FNN to obtain h i ; Next, h i Perform average pooling and use Softmax classifier to get the predicted label Y i ';Finally, calculate the cross entropy loss loss based on the true label and the predicted label i The AdamW optimizer is used to update the network parameters of the model to optimize the performance of abnormal traffic detection.

[0039] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides an edge abnormal traffic detection method based on a multi-scale aggregated Transformer, which combines a multi-scale time series representation method with a Transformer self-attention mechanism to accurately detect abnormal traffic based on the effective load length and time interval. First, the feature data is segmented and mapped through a time series representation module; then, the mapping vector is encoded using a multi-head self-attention mechanism; finally, the feature vectors of each layer are aggregated and input into a classifier to obtain the abnormal traffic classification result. Based on a real edge traffic dataset, the effectiveness of this method is verified through experiments. Experimental results show that compared with other benchmark methods, this method can achieve higher abnormal traffic classification accuracy on different indicators, and exhibits superior generalization capabilities for different traffic. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a flow chart of a method implementation of an embodiment of the present invention;

[0041] Figure 2 is an example of a confusion matrix in an embodiment of the present invention;

[0042] Figure 3 is an architecture diagram of an edge abnormal traffic detection model in an embodiment of the present invention;

[0043] Figure 4 is a structural diagram of an encoding module in an embodiment of the present invention;

[0044] Figure 5 is the accuracy of MA-Former when different numbers of data packets are used in the embodiment of the present invention;

[0045] Figure 6 is the accuracy and efficiency of MA-Former when different hidden layer sizes are used in the embodiments of the present invention;

[0046] Figure 7 is the confusion matrix of MA-Former on ACI-IoT-2023 in an embodiment of the present invention;

[0047] Figure 8 is the confusion matrix of 1D CNN on ACI-IoT-2023 in an embodiment of the present invention;

[0048] Fig. 9 is the confusion matrix of TSCRNN on ACI-IoT-2023 in an embodiment of the present invention;

[0049] Fig.10 is the confusion matrix of Deep Packet on ACI-IoT-2023 in an embodiment of the present invention;

[0050] Fig.11 is the confusion matrix of MA-Former in the embodiment of the present invention on USTC-TFC-2016;

[0051] Fig.12 It is the confusion matrix of MA-Former on CIC-IOT-2023 in the embodiment of the present invention. DETAILED DESCRIPTION

[0052] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0053] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0054] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0055] In response to the problem of increased complexity caused by high-dimensional traffic data and the challenges posed by traffic encryption technology to accurate traffic classification, this paper proposes an edge abnormal traffic detection method based on multi-scale aggregated Transformer (Multi-scale Aggregated Transformer, MA-Former), aiming to improve the accuracy and efficiency of abnormal traffic classification, thereby ensuring the QoS and high reliability of the edge system.

[0056] like Figure 1 As shown, the present embodiment provides a method for detecting abnormal edge traffic based on a multi-scale aggregation Transformer, comprising the following steps:

[0057] S1, obtain edge network traffic data from the edge system, and preprocess and extract features of the data to obtain a training sample set;

[0058] S2. Construct an edge abnormal traffic detection model based on multi-scale aggregation Transformer; train the edge abnormal traffic detection model through the training sample set;

[0059] S3. Apply the trained edge abnormal traffic detection model to the edge system to assist it in detecting and filtering abnormal traffic.

[0060] First, the raw edge network traffic data is obtained from the edge system, and then the key features for classification are extracted from the raw edge network traffic data. When processing the raw edge network traffic data, it is split into independent sessions according to the five-tuple (i.e., source IP, destination IP, source port, destination port, and transport layer protocol). Next, the first t packets are extracted for each session, where two features, namely, payload length L and arrival interval T, are extracted from each packet, and T of the first packet is set to 0. Specifically, the i-th training traffic sample is defined as:

[0061]

[0062] Among them, t represents the total number of time steps, that is, the number of sample data packets extracted. The sample corresponding to the jth time step is Represents the characteristic data of the data packet at this time step, recorded as in and Respectively represent the payload length of the jth packet in the i-th sample and the arrival interval with the j-1th packet. In particular, when extracting packet features, the DNS packet is discarded and filled with empty packets, because it mainly contains the mapping information from domain names to IP addresses, which has no effect on abnormal traffic detection. The extracted features are arranged by time to obtain the training sample

[0063] Therefore, the training sample set is defined as:

[0064]

[0065] Where N is the total number of training samples. i For sample X i The label is denoted by Y i ∈{1,2,...,K}, where K represents the total number of abnormal traffic types.

[0066] The goal of abnormal traffic detection is to build an efficient model on a given dataset D to keep the sample X i The predicted label Y i 'With the true label Y i The consistency between them is then sent to the edge system to help it filter abnormal traffic. In order to evaluate the performance of the model, the present invention uses indicators such as accuracy, precision, recall and F1-score, which are defined as:

[0067]

[0068]

[0069]

[0070]

[0071] Among them, TP (True Positive) indicates the number of samples predicted by the model as positive examples and whose actual labels are also positive examples. TN (True Negative) indicates the number of samples predicted by the model as negative examples and whose actual labels are also negative examples. FP (False Positive) indicates the number of samples predicted by the model as positive examples but whose actual labels are negative examples. FN (False Negative) indicates the number of samples predicted by the model as negative examples but whose actual labels are positive examples.

[0072] In addition, the present invention also uses a confusion matrix to evaluate model performance. Figure 2 As shown in the figure, in the confusion matrix, the rows and columns represent the actual category and predicted category of the sample respectively. The depth of the diagonal color intuitively reflects the degree of classification confusion of each category, where the darker the color, the better the classification effect.

[0073] The edge abnormal traffic detection model constructed in this embodiment is as follows: Figure 3As shown. The edge abnormal traffic detection model combines the multi-scale time series representation method with the Transformer self-attention mechanism, and is mainly composed of four parts: input layer, encoding layer, classification layer and output layer. The input layer extracts feature information from the input traffic data to form a multi-dimensional time series, and each dimension describes the change of the corresponding feature. The encoding layer adopts a hierarchical structure for multi-scale time series representation, wherein the encoding module of each layer is composed of a time series representation module and a Transformer encoding module. The time series representation module divides and maps the feature data, and the Transformer encoding module uses a multi-head self-attention mechanism to encode the mapping vector. The two work together to capture the changing laws and patterns of traffic. The classification layer includes an average pooling layer and a Softmax classifier. After averaging the outputs of all encoding modules, a comprehensive feature vector is extracted through the average pooling layer, and then input into the Softmax classifier. The output layer outputs the detection result.

[0074] Encoding layer:

[0075] The encoding layer uses the Transformer self-attention mechanism to efficiently encode input features. Considering that traditional Transformers can only process fixed-length sequences, it is difficult to efficiently process long time sequences of network traffic. To address this problem, the encoding layer combines multi-scale temporal representation with the Transformer self-attention mechanism. In Transformer, position encoding is crucial because it can effectively preserve word order information. For abnormal traffic detection, the timing information of data packets is also critical, reflecting the order of data packets. Therefore, the encoding layer introduces position encoding in the timing representation module and adopts a hierarchical architecture for multi-scale temporal representation. This layer consists of multiple encoding modules, performs feature extraction at each layer, and aggregates the feature vectors of each layer to fully understand the characteristics of the input data, thereby improving the classification accuracy.

[0076] The structure of the encoding module is as follows Figure 4 First, use the time series representation module to represent the packet feature time series. Slice the sequence to shorten its length. Assume that the slice size is v, the input sequence contains m features, and the time dimension is t. After the sequence is sliced, the length of the new sequence is The size of each slice is v×m. Next, these slices are projected onto a new dimension C and normalized. Finally, the normalized sequence is added to the positional encoding to obtain a new time series

[0077] Position encoding is a key step in Transformer, including absolute and relative position encoding, which represent static and learnable position embeddings respectively. To improve classification performance, the encoding layer can fuse the two position encodings by representing the context information between adjacent time slices. Specifically, a one-dimensional convolution is used to capture the position relationship between adjacent time slices, where the convolution kernel size is set to k and the kernel size is set to k. Zero padding is used to prevent scaling issues, which helps to emphasize absolute positions.

[0078] Next, the new time series representation is input into the multi-head self-attention layer, with the input being the query Q, key K and value V, and the input matrix can be calculated as:

[0079]

[0080] Among them, d k Represents the dimension of the input query Q and key K. For single-head self-attention, averaging suppresses the representation subspace information from different positions, while multi-head self-attention solves this problem well. The calculation process is:

[0081] MultiHead(Q,K,V)=Concat(head1,...,head h )W O (8)

[0082] Where h represents the number of attention heads and Concat represents the concatenation operation. The calculation process of each head is:

[0083] head i =Attention(QW i Q ,KW i K ,VW i K ) (9)

[0084] Among them, the matrix is the linear projection parameter, and the dimension of each head is

[0085] After being processed by the multi-head self-attention layer, the result will be passed to the fully connected feedforward neural network (FNN), which will then extract features from multiple angles. The structure contains two linear transformations and the ReLU activation function is embedded in it, specifically:

[0086] FNN(x)=max(0,xW1+b1)W2+b2 (10)

[0087] Among them, W1 and W2 both represent weight matrices in FNN, b1 and b2 both represent bias vectors;

[0088] Finally, the output results of the FNN of each encoding module are aggregated and passed to the classification layer to complete abnormal traffic detection. It is worth noting that as the number of network layers increases, the gradient will gradually decay during the back propagation process, resulting in a decrease in model training performance. In order to effectively avoid the gradient vanishing problem, residual connections are introduced in the multi-head self-attention modules. By applying residual connections in the network layer, the input tensor can be directly connected to the output tensor. This method allows the gradient to be passed to deeper networks to avoid excessive attenuation in the activation function of each layer.

[0089] Classification layer:

[0090] The classification layer includes an average pooling layer and a Softmax classifier. After the results obtained by each encoding module are averaged and aggregated, the results are input into the average pooling layer. This layer can effectively reduce the dimension of the feature vector, while preventing the model from overfitting and retaining key information. Then, the feature vector processed by the pooling layer is sent to the Softmax classifier to calculate the probability distribution of each category. The specific process is as follows:

[0091]

[0092] Among them, p i (x) represents the probability of the i-th category, x represents the model output, and K represents the number of categories of training samples. The Softmax function converts the model output into a probability distribution and selects the label with the highest probability as the predicted label.

[0093] The training process of the edge abnormal traffic detection model is shown in Algorithm 1. The input is the edge traffic data X i , training round N and training batch B. First, initialize the model parameters (line 1). In each training round and training batch, input the sample into the time series representation module, split the sequence into vectors (Line 4), and then use the convolutional network to transform the vector The dimension is mapped to a new dimension C and a new vector representation is obtained (Line 5). Next, Normalize and extract The position information is added to the normalized vector to obtain the vector X containing the position information. i '(Line 6). Then, vector X i 'Input it into the multi-head self-attention encoding module to encode it and get the vector (Line 7), and then extract features through FNN to get h i (Line 8). Next, hi Perform average pooling and use Softmax classifier to get the predicted label Y i '(Lines 9-10). Finally, the cross entropy loss loss is calculated based on the true label and the predicted label. i (Line 11), and update the network parameters of this method through the AdamW optimizer to optimize the performance of abnormal traffic detection (Line 12).

[0094]

[0095]

[0096] Method evaluation:

[0097] The experiments in this example were conducted on a workstation equipped with a GeForce RTX 3090 GPU, CUDA driver version 11.4, and operating system Ubuntu 22.04.2. This method was implemented based on the deep learning framework PyTorch 1.11.0, and a large number of experiments were conducted on the following three real datasets:

[0098] (1) ACI-IoT-2023, the Internet of Things network traffic dataset of the U.S. Army Cyber ​​Institute. This dataset is 35GB and contains various attack records within a week, covering various network attack types such as reconnaissance (Recon), denial of service (DoS) attacks, brute force cracking (BruteForce) and spoofing (Spoofing), and provides two file formats: original network traffic data and statistical features.

[0099] (2) USTC-TFC-2016, a dataset of network traffic from the University of Science and Technology of China. This dataset is 3.71 GB and contains network traffic data from different applications. Normal traffic includes Facetime, FTP, MySQL, etc., and abnormal traffic includes Neris, Nsis, Virut, etc. All are stored in pcap format files.

[0100] (3) Canadian Cybersecurity Research IoT Network Traffic Dataset CIC-IOT-2023. This dataset is obtained by executing 105 attacks on the IoT topology of 33 devices. It contains pcap files of raw network traffic and csv files of 46 statistical features.

[0101] During the experiment, the categories with too few samples in the dataset were eliminated to ensure the stability of the model. The specific classification is shown in Table 1. The ACI-IoT-2023 dataset contains 4 types of abnormal traffic. The original USTC-TFC-2016 dataset contains 10 types of normal traffic and 10 types of abnormal traffic. 6 types of abnormal traffic were selected and 4 categories with too few samples were eliminated. At the same time, the 10 types of normal traffic were classified into 1 category because the goal of the present invention is to detect and classify abnormal traffic. The original CIC-IOT-2023 dataset contains 7 types of abnormal traffic, and distributed DoS and DoS are merged into 1 category and 3 categories with too few samples are eliminated.

[0102] Table 1 Dataset description

[0103]

[0104] For this method, the number of encoding modules is set to 3, the slice size of each module is set to 2, the training rounds are set to 50, the batch size is set to 64, the learning rate is set to 0.001, the number of self-attention mechanism heads is set to 4, and the number of self-attention head layers is set to 6. In addition, data imbalance will cause the model to be biased towards categories with more samples, making it difficult to learn the features of categories with fewer samples, thereby affecting the generalization ability of the model. Therefore, random sampling is adopted to balance the sample data. Specifically, for the ACI-IoT-2023 dataset, 3000 samples are extracted from each category; for the USTC-TFC-2016 and CIC-IOT-2023 datasets, 1000 samples are extracted from each category. The ratio of the training set, validation set, and test set is 6:2:2.

[0105] In order to test the performance of the method proposed in this invention, a large number of comparative experiments were conducted with the following benchmark methods:

[0106] (1) 1D CNN: One-dimensional CNN is used to process time series data, which can extract key information from time series data and use it for classification tasks.

[0107] (2) TSCRNN: It uses a sampling strategy to collect samples from long-term streams, extracts abstract spatial features through CNN, and then introduces a stacked bidirectional LSTM to learn temporal features.

[0108] (3) Deep Packet: By combining SAE and CNN, the payload of raw network traffic packets is extracted and analyzed for classification.

[0109] Figure 5 and Figure 6The effects of the number of packets and the size of the hidden layer of the encoding module on the classification accuracy and efficiency of this method were tested respectively. The experimental results show that when the number of packets is 8, the accuracy reaches a peak of 0.9992. In addition, as the hidden layer size increases, the accuracy also increases. When the hidden layer size is 128, the accuracy reaches a peak of 0.9993, but it also leads to a significant increase in training time. When the hidden layer size is 64, the accuracy drops slightly to 0.9992, but the training time is shortened by 10.52 seconds. Considering the above, the number of packets and the hidden layer size of 64 are selected for this method (MA-Former).

[0110] Table 2 Comparison of different methods on ACI-IoT-2023

[0111]

[0112] Table 2 compares the abnormal traffic classification performance of different methods on the ACI-IoT-2023 dataset. The F1-score index of the method proposed in this invention reaches 0.9993, which is 9.89%, 6.51% and 42.30% higher than other methods. At the same time, this method also shows significant advantages in other performance indicators. This is due to the fact that this method can accurately obtain the timing representation and location information, effectively retain the time information of the original network traffic and significantly reduce the data dimension. At the same time, this method introduces a multi-head self-attention mechanism, which enables it to automatically learn and extract key classification features. In contrast, the 1D CNN method has limited ability to extract and learn time features. Although the TSCRNN method uses bidirectional LSTM for feature learning, the learning effect is not good when it only relies on the packet payload length and arrival interval. The DeepPacket method only extracts features through the payload and cannot effectively handle the problem of encrypted traffic classification.

[0113] Further, Figure 7 , Figure 8 , Fig. 9 and Fig.10The confusion matrix was used to evaluate the classification performance of different methods. The experimental results show that the proposed method can achieve the best classification performance. It is worth noting that for BruteForce and DoS attacks, the classification accuracy of this method reached 100%. For Recon attacks, the classification accuracy of this method is also as high as 99.89%, while the classification accuracy of 1D CNN, TSCRNN and Deep Packet for Recon attacks is only 58.11%, 74.22% and 42.67% respectively. This is because both Recon and Brute Force attacks involve a large number of attempts and detections on the target system, while Recon attacks usually have no obvious attack signs and are more subtle than Brute Force attacks, so they are easily misjudged. However, these methods are difficult to effectively extract Recon attack features, resulting in confusion with Brute Force attacks. In comparison, this method can effectively distinguish these two types of traffic. This is because the designed multi-head self-attention mechanism can capture the difficult-to-distinguish features of these two types of attacks, but other methods have poor learning ability for complex network traffic packet features. The experimental results verify the effectiveness and superiority of the proposed method.

[0114] Table 3 Performance of this method on different datasets

[0115]

[0116]

[0117] Fig.11 and Fig.12 The generalization ability of the proposed method on different data sets was evaluated by confusion matrix. Fig.11 As shown in the figure, on the USTC-TFC-2016 dataset, this method has some confusion between Neris attacks and Virut attacks, with 8% of Neris attacks being misclassified as Virut attacks and 12% of Virut attacks being misclassified as Neris attacks. This is because the two attacks are similar in their propagation methods and the hidden techniques they use. In some cases, the two attacks may lack obvious distinguishing features, thus leading to a certain degree of confusion. On this dataset, the classification accuracy of this method for other categories remains above 95%. Fig.12As shown, the method also shows excellent performance on the CIC-IOT-2023 dataset. Although there are some minor misclassifications, such as 0.54% of Mirai attacks were misclassified as DDoS / DoS attacks and Spoofing attacks, respectively, the overall classification effect is excellent. In addition, as shown in Table 3, the method also achieved superior performance in other indicators. For the USTC-TFC-2016 dataset, the accuracy of this method is 0.9641 and the F1 score is 0.9550. For the CIC-IOT-2023 dataset, its accuracy is as high as 0.9916, and the F1 score also reaches 0.9909. The above results fully demonstrate that the method proposed in the present invention can be adapted to different datasets and exhibits excellent generalization ability.

[0118] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0119] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0120] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0121] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0122] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A method for detecting abnormal edge traffic based on multi-scale aggregation Transformer, characterized in that: The following steps are involved: S1, obtain edge network traffic data from the edge system, and preprocess and extract features of the data to obtain a training sample set; S2. Construct an edge abnormal traffic detection model based on multi-scale aggregation Transformer; the edge abnormal traffic detection model includes an input layer, an encoding layer, a classification layer and an output layer; the input layer extracts feature information from the input traffic data to form a multi-dimensional time series, and each dimension describes the change of the corresponding feature; the encoding layer adopts a hierarchical structure for multi-scale time series representation, wherein the encoding module of each layer is composed of a time series representation module and a Transformer encoding module, the time series representation module divides and maps the feature data, and the Transformer encoding module uses a multi-head self-attention mechanism to encode the mapping vector, and the two work together to capture the changing laws and patterns of the traffic; the classification layer includes an average pooling layer and a Softmax classifier, after the outputs of all encoding modules are averaged and aggregated, a comprehensive feature vector is extracted through the average pooling layer, and then it is input into the Softmax classifier; the output layer outputs the detection result; the edge abnormal traffic detection model is trained through a training sample set; S3. Apply the trained edge abnormal traffic detection model to the edge system to assist it in detecting and filtering abnormal traffic.

2. According to claim 1, the method for detecting abnormal edge traffic based on multi-scale aggregation Transformer is characterized in that: The implementation method of step S1 is: First, the original edge network traffic data is obtained from the edge system, and then the key features for classification are extracted from the original edge network traffic data; when processing the original edge network traffic data, it is split into independent sessions according to the five-tuple, which is: source IP, destination IP, source port, destination port and transport layer protocol; then, for each session, the first t packets are extracted, where the two features of payload length L and arrival interval T are extracted from each packet, and T of the first packet is set to 0; the i-th training traffic sample is defined as: Among them, t represents the total number of time steps, that is, the number of sample data packets extracted; the sample corresponding to the jth time step is Represents the characteristic data of the data packet at this time step, recorded as in and Respectively represent the payload length of the jth data packet in the i-th sample and the arrival interval time with the j-1th data packet; when extracting data packet features, discard the DNS data packet and fill it with empty packets; arrange the extracted features by time to obtain the training sample Therefore, the training sample set is: Where N is the total number of training samples; Y i For sample X i The label is denoted by Y i ∈{1,2,...,K}, where K represents the total number of abnormal traffic types.

3. According to claim 1, the method for detecting abnormal edge traffic based on multi-scale aggregation Transformer is characterized in that: In step S2, the coding layer is implemented as follows: The coding layer adopts a hierarchical structure to support multi-scale time series representation and is composed of multiple coding modules. Feature extraction is performed at each layer, and the characteristics of the input data are fully understood by aggregating the feature vectors of each layer. The coding module uses the time series representation module to extract the characteristic time series of the data packet. Split to shorten its length; assuming the slice size is v, the input sequence contains m features, and the time dimension is t; after sequence segmentation, the length of the new sequence is The size of each slice is v×m; then, these slices are projected onto a new dimension C and normalized; finally, the normalized sequence is added to the position code to obtain a new time series The position encoding includes absolute and relative position encoding, which represent static and learnable position embedding respectively. By representing the context information between adjacent time slices, the two position encodings are fused to improve the classification performance. Specifically, a one-dimensional convolution is used to capture the position relationship between adjacent time slices, where the convolution kernel size is set to k and the convolution kernel size is set to k. Zero padding is used to prevent the scale reduction problem and to highlight the absolute position; Next, the new temporal representation is input into the multi-head self-attention layer of the Transformer encoding module, with the input being the query Q, key K and value V, and the input matrix is ​​calculated as: Among them, d k represents the dimension of the input query Q and key K; the calculation process of multi-head self-attention is: MultiHead(Q,K,V)=Concat(head1,...,head h )W O (4) Among them, h represents the number of attention heads, Concat represents the serial concatenation operation; the calculation process of each head is: head i =Attention(QW i Q ,KW i K ,VW i K ) (5) Among them, the matrix is the linear projection parameter, and the dimension of each head is After being processed by the multi-head self-attention layer, the result is passed to the fully connected feedforward neural network FNN, which then extracts features from multiple angles. The feedforward neural network FNN contains two linear transformations and embeds the ReLU activation function in it, specifically: FNN(x)=max(0,xW1+b1)W2+b2(6) Among them, W1 and W2 both represent weight matrices in FNN, b1 and b2 both represent bias vectors; Finally, the output results of the FNN of each encoding module are aggregated and passed to the classification layer to complete abnormal traffic detection.

4. The edge abnormal traffic detection method based on multi-scale aggregation Transformer according to claim 1 is characterized in that: In step S2, the classification layer is implemented as follows: The classification layer includes an average pooling layer and a Softmax classifier. After the output results of each encoding module are averaged and aggregated, the results are input into the average pooling layer to reduce the dimension of the feature vector, thereby preventing the model from overfitting and retaining key information. Then, the feature vector processed by the average pooling layer is sent to the Softmax classifier to calculate the probability distribution of each category. The specific process is as follows: Among them, p i (x) represents the probability of the i-th category, x represents the model output, and K represents the number of categories of training samples; the Softmax function converts the model output into a probability distribution and selects the label with the largest probability as the predicted label.

5. The edge abnormal traffic detection method based on multi-scale aggregation Transformer according to claim 1 is characterized in that: In step S2, the implementation method of training the edge abnormal flow detection model through the training sample set is: Input is edge traffic data X i , training round N and training batch B; first, initialize the model parameters; in each training round and training batch, input the sample into the time series representation module, split the sequence to obtain vectors Then use convolution to transform the vector The dimension is mapped to a new dimension C and a new vector representation is obtained Next, Normalize and extract The position information is added to the normalized vector to obtain the vector X containing the position information. i '; Then, vector X i 'Input it into the multi-head self-attention layer to encode it and get the vector Then, the features are extracted through FNN to obtain h i ; Next, h i Perform average pooling and use Softmax classifier to get the predicted label Y i ';Finally, calculate the cross entropy loss loss based on the true label and the predicted label i The AdamW optimizer is used to update the network parameters of the model to optimize the performance of abnormal traffic detection.

Images