Edge anomaly traffic detection method based on multi-scale aggregation transformer

Abstract

This invention relates to an edge anomaly traffic detection method based on multi-scale aggregation Transformer, comprising: S1, acquiring edge network traffic data and performing data preprocessing and feature extraction to obtain a training sample set; S2, constructing an edge anomaly traffic detection model based on multi-scale aggregation Transformer, which includes an input layer, an encoding layer, a classification layer, and an output layer; the input layer extracts feature information from the traffic data; the encoding layer adopts a hierarchical structure for multi-scale temporal representation, and each layer's encoding module consists of a temporal representation module and a Transformer encoding module. The temporal representation module segments and maps the feature data, and the Transformer encoding module uses a multi-head self-attention mechanism to encode the mapping vector; after aggregation of the outputs of the encoding modules, the detection result is obtained through the classification layer and output through the output layer; the model is trained using the training sample set; S3, the trained model is applied to the edge system to assist in detecting and filtering anomaly traffic. This method is beneficial for improving the accuracy of anomaly traffic detection.

Description

Technical Field

[0001] This invention relates to the field of abnormal traffic detection technology, and specifically to an edge abnormal traffic detection method based on multi-scale aggregation Transformer. Background Technology

[0002] With the rapid development of the Internet of Things (IoT), the era of interconnected everything is approaching, playing a vital role in various industries, including smart homes, industrial control, and smart cities. To process the massive amounts of data generated by IoT terminal devices more efficiently and in real-time, Mobile Edge Computing (MEC) has emerged. By deploying computing and storage resources at the network edge, it enables local data processing and analysis. This computing model reduces network latency and data transmission costs, becoming key to achieving connectivity, centralization, and intelligence in IoT technology. With the gradual popularization and widespread application of IoT technology, users are increasingly subjected to abnormal traffic attacks of various types (such as denial-of-service attacks, data breaches, and privacy violations). To prevent these attacks, network traffic is typically encrypted using traffic encryption technology to ensure edge network security, guaranteeing data confidentiality, integrity, and authentication. While traffic encryption technology plays a crucial role in preventing network attacks and ensuring edge network security, it also indirectly makes malicious traffic difficult to detect, making it difficult to accurately detect potential anomalies within encrypted traffic. This severely impacts the system's Quality of Service (QoS) and poses a serious threat to edge network security.

[0003] Anomaly detection is a crucial technology for ensuring the security of edge networks. By monitoring and analyzing network traffic, anomaly detection technology can promptly identify and block malicious traffic, thus safeguarding edge network security. Most work on anomaly detection focuses on cloud data centers, with less attention paid to edge environments and a failure to fully consider their characteristics. Cloud computing typically transmits and aggregates data to cloud data centers for processing, which can lead to high network latency and fail to meet the real-time requirements of applications. In contrast, MEC (Multi-access Edge Computing) pushes computing and storage resources down to edge nodes closer to end devices. Therefore, deploying anomaly detection modules on edge nodes can effectively avoid high data transmission latency and alleviate network bandwidth pressure on cloud data centers. Currently, edge anomaly detection still faces several challenges:

[0004] (1) During the daily operation of the system, IoT terminal devices and edge servers generate massive amounts of data, which often have high dimensionality. If all traffic generated by the system is monitored throughout the entire process, it will undoubtedly lead to excessive costs for abnormal traffic detection, and will also affect the real-time performance and efficiency of the detection. Therefore, edge abnormal traffic detection methods should have efficient data processing capabilities and be able to quickly analyze and process data.

[0005] (2) The widespread application of traffic encryption technology has significantly increased the difficulty of detecting abnormal network traffic. This requires edge abnormal traffic detection methods to select appropriate features for abnormal traffic modeling and detection in order to accurately identify abnormal traffic. However, how to fully consider the representativeness and separability of features in the abnormal traffic detection process remains a challenging problem.

[0006] (3) The edge environment is highly dynamic, and the behavior of different devices and their operating environment may change over time. This requires edge anomaly traffic detection methods to adapt to the dynamism of this environment. Most existing works use Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) and their improved variants to detect anomaly traffic by utilizing features such as statistical characteristics of data packets and payload information. However, they still have shortcomings in capturing long-distance dependencies and parallel processing capabilities, and manual feature statistics are still required. Summary of the Invention

[0007] The purpose of this invention is to provide an edge anomaly traffic detection method based on multi-scale aggregation Transformer, which is beneficial to improving the accuracy of anomaly traffic detection.

[0008] To achieve the above objectives, the technical solution adopted by this invention is: an edge anomaly traffic detection method based on multi-scale aggregation Transformer, comprising the following steps:

[0009] S1. Obtain edge network traffic data from the edge system, and preprocess and extract features from the data to obtain a training sample set;

[0010] S2. Construct an edge anomaly traffic detection model based on multi-scale aggregation Transformer; the edge anomaly traffic detection model includes an input layer, an encoding layer, a classification layer, and an output layer; the input layer extracts feature information from the input traffic data to form a multi-dimensional time series, with each dimension depicting the changes in the corresponding feature; the encoding layer adopts a hierarchical structure for multi-scale temporal representation, where each layer's encoding module consists of a temporal representation module and a Transformer encoding module. The temporal representation module segments and maps the feature data, and the Transformer encoding module uses a multi-head self-attention mechanism to encode the mapping vector. The two work together to capture the changing patterns and rules of traffic; the classification layer includes an average pooling layer and a Softmax classifier. After averaging and aggregating the outputs of all encoding modules, the average pooling layer extracts a comprehensive feature vector, which is then input into the Softmax classifier; the output layer outputs the detection results; the edge anomaly traffic detection model is trained using a training sample set;

[0011] S3. Apply the trained edge abnormal traffic detection model to the edge system to assist it in detecting and filtering abnormal traffic.

[0012] Furthermore, the implementation method of step S1 is as follows:

[0013] First, raw edge network traffic data is obtained from the edge system. Then, key features for classification are extracted from the raw edge network traffic data. When processing the raw edge network traffic data, it is split into independent sessions according to a 5-tuple, where the 5-tuple consists of: source IP, destination IP, source port, destination port, and transport layer protocol. Next, for each session, the first t packets are extracted, where each packet extracts two features: payload length L and arrival interval T, and T for the first packet is set to 0. The i-th training traffic sample is defined as:

[0014]

[0015] Where t represents the total number of time steps, i.e., the number of sample data packets extracted; the sample corresponding to the j-th time step is... The characteristic data of this time step data packet is denoted as in and Let represent the payload length of the j-th data packet in the i-th sample and the arrival interval between the j-th and (j-1)-th data packets, respectively. When extracting data packet features, DNS data packets are discarded and padded with empty packets. The extracted features are arranged chronologically to obtain the training samples.

[0016] Therefore, the training sample set is obtained as follows:

[0017]

[0018] Where N is the total number of training samples; Y i For sample X i The label is denoted as Y. i ∈{1,2,...,K}, where K represents the total number of abnormal traffic types.

[0019] Further, in step S2, the implementation method of the coding layer is as follows:

[0020] The encoding layer adopts a hierarchical structure to support multi-scale temporal representation and consists of multiple encoding modules. Feature extraction is performed at each layer, and the characteristics of the input data are comprehensively understood by aggregating the feature vectors from each layer. The encoding module uses the temporal representation module to process the time series features of the data packet. The sequence is segmented to shorten its length; assuming the slice size is v, the input sequence contains m features, and the time dimension is t; after segmentation, the length of the new sequence is... Each slice is v×m in size; then, these slices are projected onto a new dimension C and normalized; finally, the normalized sequence is added to the positional encoding to obtain a new time series.

[0021] The location encoding includes absolute and relative location encoding, representing static and learnable location embeddings, respectively. By representing the contextual information between adjacent time slices, the two location encodings are fused to improve classification performance. Specifically, a one-dimensional convolution is used to capture the positional relationship between adjacent time slices, where the convolution kernel size is set to k, and... Zeros are used to fill the gaps to prevent scaling issues and to highlight the absolute position;

[0022] Next, the new temporal representation is input into the multi-head self-attention layer of the Transformer encoding module, with the inputs being query Q, key K, and value V, and the input matrix is ​​then calculated as follows:

[0023]

[0024] Where, d k The dimensions represent the input query Q and the key K; the calculation process for multi-head self-attention is as follows:

[0025] MultiHead(Q,K,V)=Concat(head1,...,head h W O (4)

[0026] Where h represents the number of attention heads, and Concat represents the concatenation operation; the calculation process for each head is as follows:

[0027] head i =Attention(QW i Q ,KW i K VW i K (5)

[0028] Among them, matrix These are linear projection parameters, and the dimensions of each head are...

[0029] After processing by a multi-head self-attention layer, the result is passed to a fully connected feedforward neural network (FNN) for in-depth feature extraction from multiple perspectives. The FNN contains two linear transformations, in which a ReLU activation function is embedded, specifically:

[0030] FNN(x)=max(0,xW1+b1)W2+b2 (6)

[0031] Where W1 and W2 both represent the weight matrices in the FNN, and b1 and b2 both represent the bias vectors;

[0032] Finally, the outputs of the FNN from each encoding module are aggregated and passed to the classification layer to complete the abnormal traffic detection.

[0033] Furthermore, in step S2, the classification layer is implemented as follows:

[0034] The classification layer includes an average pooling layer and a softmax classifier. The outputs of each encoding module are averaged and aggregated, and the result is then input into the average pooling layer to reduce the dimensionality of the feature vectors, preventing overfitting while preserving key information. Then, the feature vectors processed by the average pooling layer are fed into the softmax classifier to calculate the probability distribution for each category. The specific process is as follows:

[0035]

[0036] Where, p i (x) represents the probability of the i-th class, x represents the model output, and K represents the number of classes in the training samples; the Softmax function converts the model output into a probability distribution and selects the label with the highest probability as the predicted label.

[0037] Furthermore, in step S2, the method for training the edge anomaly traffic detection model using a training sample set is as follows:

[0038] The input is edge traffic data X i The training epochs N and training batches B are defined. First, the model parameters are initialized. In each training epoch and training batch, the samples are input into the time-series representation module, and the sequence is segmented to obtain vectors. Then use convolution to convert the vector The dimension is mapped to a new dimension C and a new vector representation is obtained. Next, regarding Perform normalization and extract The location information is then added to the normalized vector to obtain a vector X containing the location information. i '; Subsequently, vector X i The input is fed into a multi-head self-attention layer for encoding, resulting in a vector. Then, features are extracted using FNN to obtain h. i Next, h i Perform average pooling and use a softmax classifier to obtain the predicted label Y. i Finally, based on the true and predicted labels, the cross-entropy loss is calculated. i The network parameters of the model are updated using the AdamW optimizer to optimize the performance of abnormal traffic detection.

[0039] Compared with existing technologies, this invention has the following advantages: This invention provides an edge anomaly traffic detection method based on multi-scale aggregation Transformer. This method integrates multi-scale temporal representation methods with Transformer self-attention mechanisms to accurately detect anomaly traffic based on payload length and time intervals. First, the feature data is segmented and mapped using a temporal representation module; then, the mapping vector is encoded using a multi-head self-attention mechanism; finally, the feature vectors from each layer are aggregated and input into a classifier to obtain the anomaly traffic classification result. The effectiveness of this method is verified through experiments using a real edge traffic dataset. Experimental results show that, compared with other benchmark methods, this method achieves higher anomaly traffic classification accuracy across different metrics and exhibits superior generalization ability for different traffic types. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating the method implementation of an embodiment of the present invention;

[0041] Figure 2 This is an example of a confusion matrix in an embodiment of the present invention;

[0042] Figure 3 This is an architecture diagram of the edge abnormal traffic detection model in an embodiment of the present invention;

[0043] Figure 4 This is a structural diagram of the encoding module in an embodiment of the present invention;

[0044] Figure 5 This refers to the accuracy of MA-Former when using different numbers of data packets in the embodiments of this invention;

[0045] Figure 6 This refers to the accuracy and efficiency of MA-Former when using different hidden layer sizes in the embodiments of the present invention;

[0046] Figure 7 This is the confusion matrix of MA-Former on ACI-IoT-2023 in this embodiment of the invention;

[0047] Figure 8 This is the confusion matrix of the 1D CNN on ACI-IoT-2023 in this embodiment of the invention;

[0048] Figure 9 This is the confusion matrix of TSCRNN on ACI-IoT-2023 in this embodiment of the invention;

[0049] Figure 10 This is the confusion matrix of Deep Packet on ACI-IoT-2023 in this embodiment of the invention;

[0050] Figure 11 This is the confusion matrix of MA-Former on USTC-TFC-2016 in this embodiment of the invention;

[0051] Figure 12 This is the confusion matrix of MA-Former on CIC-IOT-2023 in this embodiment of the invention. Detailed Implementation

[0052] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0053] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0054] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0055] To address the increased complexity caused by high-dimensional traffic data and the challenges posed by traffic encryption technology to accurate traffic classification, this invention proposes an edge anomaly traffic detection method based on multi-scale aggregated Transformer (MA-Former). This method aims to improve the accuracy and efficiency of anomaly traffic classification, thereby ensuring the QoS and high reliability of edge systems.

[0056] like Figure 1 As shown in the figure, this embodiment provides an edge anomaly traffic detection method based on multi-scale aggregation Transformer, which includes the following steps:

[0057] S1. Obtain edge network traffic data from the edge system, and preprocess and extract features from the data to obtain a training sample set;

[0058] S2. Construct an edge anomaly traffic detection model based on multi-scale aggregation Transformer; train the edge anomaly traffic detection model using a training sample set;

[0059] S3. Apply the trained edge abnormal traffic detection model to the edge system to assist it in detecting and filtering abnormal traffic.

[0060] First, raw edge network traffic data is obtained from the edge system, and then key features for classification are extracted from this data. When processing the raw edge network traffic data, it is split into independent sessions according to a five-tuple (source IP, destination IP, source port, destination port, and transport layer protocol). Next, for each session, the first t packets are extracted, where each packet extracts two features: payload length L and arrival interval T. The T value for the first packet is set to 0. Specifically, the i-th training traffic sample is defined as:

[0061]

[0062] Where t represents the total number of time steps, i.e., the number of sample data packets extracted. The sample corresponding to the j-th time step is... The characteristic data of this time step data packet is denoted as in and Let represent the payload length of the j-th data packet in the i-th sample and the arrival interval with the (j-1)-th data packet, respectively. Specifically, when extracting data packet features, DNS data packets were discarded and padded with empty packets, as they primarily contain domain name-to-IP address mapping information, which has no impact on abnormal traffic detection. Arranging the extracted features by time yields the training samples.

[0063] Therefore, the training sample set is defined as follows:

[0064]

[0065] Where N is the total number of training samples. i For sample X i The label is denoted as Y. i ∈{1,2,...,K}, where K represents the total number of abnormal traffic types.

[0066] The goal of anomaly traffic detection is to build an efficient model on a given dataset D, while maintaining the performance of samples X. i Predicted label Y i 'With real label Y i The consistency between the data points is ensured, and the detection results are then sent to the edge system to assist in filtering abnormal traffic. To evaluate model performance, this invention uses metrics such as accuracy, precision, recall, and F1 score, which are defined as follows:

[0067]

[0068]

[0069]

[0070]

[0071] Wherein, TP (True Positive) represents the number of samples that the model predicts as positive and the actual label is also positive. TN (True Negative) represents the number of samples that the model predicts as negative and the actual label is also negative. FP (False Positive) represents the number of samples that the model predicts as positive but the actual label is negative. FN (False Negative) represents the number of samples that the model predicts as negative but the actual label is positive.

[0072] Furthermore, this invention also uses a confusion matrix to evaluate model performance. For example... Figure 2 As shown, in the confusion matrix, rows and columns represent the actual and predicted classes of the samples, respectively. The intensity of the diagonal color intuitively reflects the degree of confusion for each class, with darker colors indicating better classification performance.

[0073] The edge anomaly traffic detection model constructed in this embodiment is as follows: Figure 3As shown, the edge anomaly traffic detection model integrates multi-scale temporal representation methods and the Transformer self-attention mechanism, and mainly consists of four parts: an input layer, an encoding layer, a classification layer, and an output layer. The input layer extracts feature information from the input traffic data, forming a multi-dimensional time series, with each dimension depicting the changes in the corresponding feature. The encoding layer adopts a hierarchical structure for multi-scale temporal representation, where each layer's encoding module consists of a temporal representation module and a Transformer encoding module. The temporal representation module segments and maps the feature data, while the Transformer encoding module uses a multi-head self-attention mechanism to encode the mapping vector. Both work together to capture the changing patterns and dynamics of the traffic. The classification layer includes an average pooling layer and a Softmax classifier. After averaging and aggregating the outputs of all encoding modules, the average pooling layer extracts a comprehensive feature vector, which is then input into the Softmax classifier. The output layer outputs the detection results.

[0074] Encoding layer:

[0075] The encoding layer employs the Transformer self-attention mechanism to efficiently encode input features. Considering that traditional Transformers can only handle fixed-length sequences and struggle to efficiently process long-term network traffic sequences, the encoding layer addresses this issue by combining multi-scale temporal representation with the Transformer self-attention mechanism. In the Transformer, positional encoding is crucial because it effectively preserves word order information. Similarly, for anomaly traffic detection, the temporal information of data packets is critical, reflecting the packet order. Therefore, the encoding layer introduces positional encoding into the temporal representation module and adopts a hierarchical architecture for multi-scale temporal representation. This layer consists of multiple encoding modules, performing feature extraction at each layer. By aggregating the feature vectors from each layer, the characteristics of the input data are comprehensively understood, thereby improving classification accuracy.

[0076] The structure of the encoding module is as follows Figure 4 As shown. First, the time sequence representation module is used to analyze the time series characteristics of data packets. The sequence is segmented to shorten its length. Assume the slice size is v, the input sequence contains m features, and the time dimension is t. After segmentation, the new sequence length is... Each slice is v×m in size. Next, these slices are projected onto a new dimension C and normalized. Finally, the normalized sequence is added to the positional encoding to obtain a new time series.

[0077] Positional encoding is a key step in the Transformer, including absolute and relative positional encoding, representing static and learnable positional embeddings, respectively. To improve classification performance, the encoding layer can fuse these two positional encodings by representing the contextual information between adjacent time slices. Specifically, a one-dimensional convolution is used to capture the positional relationship between adjacent time slices, where the convolution kernel size is set to k, and... Padding with zeros prevents scaling issues and helps to highlight absolute positions.

[0078] Next, the new temporal representation is input into the multi-head self-attention layer, with the inputs being query Q, key K, and value V, thus the input matrix can be calculated as follows:

[0079]

[0080] Where, d k This represents the dimensions of the input query Q and the key K. For single-head self-attention, averaging suppresses information from the representation subspaces at different locations, while multi-head self-attention effectively addresses this issue. Its computation process is as follows:

[0081] MultiHead(Q,K,V)=Concat(head1,...,head h W O (8)

[0082] Where h represents the number of attention heads, and Concat represents the concatenation operation. The calculation process for each head is as follows:

[0083] head i =Attention(QW i Q ,KW i K VW i K (9)

[0084] Among them, matrix These are linear projection parameters, and the dimensions of each head are...

[0085] After processing by a multi-head self-attention layer, the result is passed to a fully connected feedforward neural network (FNN) for in-depth feature extraction from multiple perspectives. This structure contains two linear transformations with a ReLU activation function embedded within them, specifically:

[0086] FNN(x)=max(0,xW1+b1)W2+b2 (10)

[0087] Where W1 and W2 both represent the weight matrices in the FNN, and b1 and b2 both represent the bias vectors;

[0088] Finally, the outputs of the FNN from each encoding module are aggregated and passed to the classification layer to complete abnormal traffic detection. It is worth noting that as the number of network layers increases, the gradient gradually decays during backpropagation, leading to a decrease in model training performance. To effectively avoid the vanishing gradient problem, residual connections are introduced in the multi-head self-attention modules. By applying residual connections in the network layers, the input tensor can be directly connected to the output tensor. This approach allows the gradient to propagate to deeper layers of the network, avoiding excessive decay in each activation function.

[0089] Category layer:

[0090] The classification layer consists of an average pooling layer and a softmax classifier. The results from each encoding module are averaged and aggregated, then fed into the average pooling layer. This layer effectively reduces the dimensionality of the feature vectors, preventing overfitting while preserving key information. Next, the feature vectors processed by the pooling layer are fed into the softmax classifier to calculate the probability distribution for each class. The specific process is as follows:

[0091]

[0092] Where, p i (x) represents the probability of the i-th class, x represents the model output, and K represents the number of classes in the training samples. The Softmax function converts the model output into a probability distribution and selects the label with the highest probability as the predicted label.

[0093] The training process of the edge anomaly traffic detection model is shown in Algorithm 1. The input is the edge traffic data X. i The training epochs N and training batches B are defined. First, the model parameters are initialized (line 1). In each training epoch and training batch, the samples are input into the time-series representation module, and the sequence is segmented to obtain vectors. (Line 4) Then use a convolutional network to convert the vector The dimension is mapped to a new dimension C and a new vector representation is obtained. (Line 5). Next, regarding... Perform normalization and extract The location information is then added to the normalized vector to obtain a vector X containing the location information. i (Line 6). Then, vector X... i The input is fed into a multi-head self-attention encoding module to encode it, resulting in a vector. (Line 7), and then extract features through FNN to obtain h. i (Line 8). Next, hi Perform average pooling and use a softmax classifier to obtain the predicted label Y. i (Lines 9-10). Finally, calculate the cross-entropy loss based on the true and predicted labels. i (Line 11), and update the network parameters of this method using the AdamW optimizer to optimize the performance of abnormal traffic detection (Line 12).

[0094]

[0095]

[0096] Method evaluation:

[0097] The experiments in this embodiment were conducted on a workstation equipped with a GeForce RTX 3090 GPU, CUDA driver version 11.4, and Ubuntu 22.04.2 operating system. Based on the deep learning framework PyTorch 1.11.0, this method was implemented, and extensive experiments were performed on the following three real-world datasets:

[0098] (1) U.S. Army Cyber ​​Research Institute Internet of Things Network Traffic Dataset ACI-IoT-2023. This dataset is 35GB in size and contains various attack records from one week, covering multiple network attack types such as recon, denial-of-service (DoS) attacks, brute-force attacks, and spoofing. It provides both raw network traffic data and statistical features in file formats.

[0099] (2) USTC-TFC-2016 Application Network Traffic Dataset from the University of Science and Technology of China. This dataset is 3.71GB in size and contains network traffic data from different applications. Normal traffic includes types such as Facetime, FTP, and MySQL, while abnormal traffic includes types such as Neris, Nsis, and Virut. All data are stored in pcap format.

[0100] (3) Canadian Internet of Things Network Traffic Dataset CIC-IOT-2023. This dataset was obtained by performing 105 attacks on an IoT topology of 33 devices and includes pcap files of raw network traffic and csv files of 46 statistical features.

[0101] During the experiment, categories with insufficient sample sizes were removed from the dataset to ensure model stability. Specific classifications are shown in Table 1. The ACI-IoT-2023 dataset contains 4 categories of anomalous traffic. The original USTC-TFC-2016 dataset contains 10 categories of normal traffic and 10 categories of anomalous traffic; 6 categories of anomalous traffic were selected, and 4 categories with insufficient sample sizes were removed. Simultaneously, the 10 categories of normal traffic were grouped into 1 category, as the goal of this invention is to detect and classify anomalous traffic. The original CIC-IOT-2023 dataset contains 7 categories of anomalous traffic; Distributed DoS and DoS were merged into 1 category, and 3 categories with insufficient sample sizes were removed.

[0102] Table 1. Dataset Description

[0103]

[0104] For this method, the number of encoding modules is set to 3, the slice size of each module is 2, the number of training epochs is 50, the batch size is 64, the learning rate is 0.001, the number of self-attention mechanism heads is 4, and the number of self-attention head layers is 6. Furthermore, data imbalance can cause the model to favor the class with more samples, making it difficult to learn the features of the class with fewer samples, thus affecting the model's generalization ability. Therefore, a random sampling method was adopted to balance the sample data. Specifically, for the ACI-IoT-2023 dataset, 3000 samples were drawn from each class; for the USTC-TFC-2016 and CIC-IOT-2023 datasets, 1000 samples were drawn from each class respectively. The ratio of training set, validation set, and test set is 6:2:2.

[0105] To test the performance of the method proposed in this invention, numerous comparative experiments were conducted with the following benchmark methods:

[0106] (1) 1D CNN: One-dimensional CNN is used to process time series data, which can extract key information from time series data and use it for classification tasks.

[0107] (2)TSCRNN: Uses a sampling strategy to collect samples from long-term streams, extracts abstract spatial features through CNN, and then introduces stacked bidirectional LSTM to learn temporal features.

[0108] (3) Deep Packet: By combining SAE and CNN, the payload of the original network traffic data packets is extracted and analyzed for classification.

[0109] Figure 5 and Figure 6The effects of the number of data packets and the size of the hidden layer in the encoding module on the classification accuracy and efficiency of this method were tested. Experimental results show that the accuracy reaches a peak of 0.9992 when using 8 data packets. Furthermore, the accuracy increases with the increase of the hidden layer size. When the hidden layer size is 128, the accuracy reaches a peak of 0.9993, but this also leads to a significant increase in training time. When the hidden layer size is 64, the accuracy slightly decreases to 0.9992, but the training time is shortened by 10.52 seconds. Considering all these factors, the data packet number of this method (MA-Former) is selected as 8, and the hidden layer size as 64.

[0110] Table 2 Comparison of different methods on ACI-IoT-2023

[0111]

[0112] Table 2 compares the performance of different methods in anomaly traffic classification on the ACI-IoT-2023 dataset. The proposed method achieves an F1-score of 0.9993, representing improvements of 9.89%, 6.51%, and 42.30% compared to other methods. Furthermore, this method also demonstrates significant advantages in other performance metrics. This is attributed to its ability to accurately acquire temporal representation and location information, effectively preserving the temporal information of the original network traffic and significantly reducing data dimensionality. Additionally, this method introduces a multi-head self-attention mechanism, enabling it to automatically learn and extract key classification features. In contrast, 1D CNN methods have limited ability to extract and learn temporal features. While the TSCRNN method uses bidirectional LSTM for feature learning, its learning performance is poor when relying solely on packet payload length and arrival interval. The DeepPacket method, which extracts features only from the payload, cannot effectively handle encrypted traffic classification.

[0113] Furthermore, Figure 7 , Figure 8 , Figure 9 and Figure 10A confusion matrix was used to evaluate the classification performance of different methods. Experimental results show that the proposed method achieves the best classification performance. Notably, the method achieves 100% classification accuracy for Brute Force and DoS attacks. For Recon attacks, the method also achieves a high classification accuracy of 99.89%, while 1D CNN, TSCRNN, and Deep Packet methods only achieve 58.11%, 74.22%, and 42.67% accuracy, respectively. This is because both Recon and Brute Force attacks involve numerous attempts and probes of the target system, and Recon attacks typically lack obvious attack indicators, making the differences from Brute Force attacks subtle and prone to misclassification. These methods struggle to effectively extract Recon attack features, leading to confusion with Brute Force attacks. In contrast, the proposed method effectively distinguishes between these two types of traffic. This is because the designed multi-head self-attention mechanism can capture the difficult-to-distinguish features in these two types of attacks, while other methods are less effective at learning the features of complex network traffic packets. Experimental results validate the effectiveness and superiority of the proposed method.

[0114] Table 3 shows the performance of this method on different datasets.

[0115]

[0116]

[0117] Figure 11 and Figure 12 The generalization ability of the proposed method on different datasets was evaluated using a confusion matrix. Figure 11 As shown, on the USTC-TFC-2016 dataset, our method exhibits some confusion between Neris and Virut attacks; 8% of Neris attacks are misclassified as Virut attacks, and 12% of Virut attacks are misclassified as Neris attacks. This is because the two attacks share similarities in their propagation methods and the hiding techniques they employ. In some cases, these two attacks may lack obvious distinguishing features, thus leading to a degree of confusion. On this dataset, our method maintains a classification accuracy of over 95% for other categories. Figure 12As shown, this method also demonstrates superior performance on the CIC-IOT-2023 dataset. Although there are some minor misclassifications, such as 0.54% of Mirai attacks being misclassified as DDoS / DoS attacks and Spoofing attacks respectively, the overall classification effect is excellent. Furthermore, as shown in Table 3, this method also achieves superior performance in other metrics. For the USTC-TFC-2016 dataset, the accuracy is 0.9641 and the F1 score is 0.9550. For the CIC-IOT-2023 dataset, its accuracy reaches 0.9916 and the F1 score is 0.9909. These results fully demonstrate that the proposed method can adapt to different datasets and exhibits superior generalization ability.

[0118] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0119] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0120] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0121] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0122] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for detecting edge anomaly traffic based on multi-scale aggregation Transformer, characterized in that, Includes the following steps: S1. Obtain edge network traffic data from the edge system, and preprocess and extract features from the data to obtain a training sample set; S2. Construct an edge anomaly traffic detection model based on multi-scale aggregation Transformer; the edge anomaly traffic detection model includes an input layer, an encoding layer, a classification layer, and an output layer; the input layer extracts feature information from the input traffic data to form a multi-dimensional time series, with each dimension depicting the changes in the corresponding feature; the encoding layer adopts a hierarchical structure for multi-scale temporal representation, where each layer's encoding module consists of a temporal representation module and a Transformer encoding module. The temporal representation module segments and maps the feature data, and the Transformer encoding module uses a multi-head self-attention mechanism to encode the mapping vector. The two work together to capture the changing patterns and rules of traffic; the classification layer includes an average pooling layer and a Softmax classifier. After averaging and aggregating the outputs of all encoding modules, the average pooling layer extracts a comprehensive feature vector, which is then input into the Softmax classifier; the output layer outputs the detection results; the edge anomaly traffic detection model is trained using a training sample set; S3. Apply the trained edge abnormal traffic detection model to the edge system to assist it in detecting and filtering abnormal traffic.

2. The edge anomaly traffic detection method based on multi-scale aggregation Transformer according to claim 1, characterized in that, The implementation method of step S1 is as follows: First, raw edge network traffic data is obtained from the edge system. Then, key features for classification are extracted from the raw edge network traffic data. When processing the raw edge network traffic data, it is split into independent sessions according to a 5-tuple, where the 5-tuple consists of: source IP, destination IP, source port, destination port, and transport layer protocol. Next, for each session, the first t packets are extracted, where each packet extracts two features: payload length L and arrival interval T, and T for the first packet is set to 0. The i-th training traffic sample is defined as: Where t represents the total number of time steps, i.e., the number of sample data packets extracted; the sample corresponding to the j-th time step is... The characteristic data of this time step data packet is denoted as in and Let represent the payload length of the j-th data packet in the i-th sample and the arrival interval between the j-th and (j-1)-th data packets, respectively. When extracting data packet features, DNS data packets are discarded and padded with empty packets. The extracted features are arranged chronologically to obtain the training samples. Therefore, the training sample set is obtained as follows: Where N is the total number of training samples; Y i For sample X i The label is denoted as Y. i ∈{1,2,...,K}, where K represents the total number of abnormal traffic types.

3. The edge anomaly traffic detection method based on multi-scale aggregation Transformer according to claim 1, characterized in that, In step S2, the coding layer is implemented as follows: The encoding layer adopts a hierarchical structure to support multi-scale temporal representation and consists of multiple encoding modules. Feature extraction is performed at each layer, and the characteristics of the input data are comprehensively understood by aggregating the feature vectors from each layer. The encoding module uses the temporal representation module to process the time series features of the data packet. The sequence is segmented to shorten its length; assuming the slice size is v, the input sequence contains m features, and the time dimension is t; after segmentation, the length of the new sequence is... Each slice is v×m in size; then, these slices are projected onto a new dimension C and normalized; finally, the normalized sequence is added to the positional encoding to obtain a new time series. The location encoding includes absolute and relative location encoding, representing static and learnable location embeddings, respectively. By representing the contextual information between adjacent time slices, the two location encodings are fused to improve classification performance. Specifically, a one-dimensional convolution is used to capture the positional relationship between adjacent time slices, where the convolution kernel size is set to k, and... Zeros are used to fill the gaps to prevent scaling issues and to highlight the absolute position; Next, the new temporal representation is input into the multi-head self-attention layer of the Transformer encoding module, with the inputs being query Q, key K, and value V, and the input matrix is ​​then calculated as follows: Where, d k The dimensions represent the input query Q and the key K; the calculation process for multi-head self-attention is as follows: MultiHead(Q,K,V)=Concat(head1,...,head h )W O (4) Where h represents the number of attention heads, and Concat represents the concatenation operation; the calculation process for each head is as follows: head i =Attention(QW i Q ,KW i K ,VW i K ) (5) Among them, matrix These are linear projection parameters, and the dimensions of each head are... After processing by a multi-head self-attention layer, the result is passed to a fully connected feedforward neural network (FNN) for in-depth feature extraction from multiple perspectives. The FNN contains two linear transformations, in which a ReLU activation function is embedded, specifically: FNN(x)=max(0,xW1+b1)W2+b2(6) Where W1 and W2 both represent the weight matrices in the FNN, and b1 and b2 both represent the bias vectors; Finally, the outputs of the FNN from each encoding module are aggregated and passed to the classification layer to complete the abnormal traffic detection.

4. The edge anomaly traffic detection method based on multi-scale aggregation Transformer according to claim 1, characterized in that, In step S2, the classification layer is implemented as follows: The classification layer includes an average pooling layer and a softmax classifier. The outputs of each encoding module are averaged and aggregated, and the result is then input into the average pooling layer to reduce the dimensionality of the feature vectors, preventing overfitting while preserving key information. Then, the feature vectors processed by the average pooling layer are fed into the softmax classifier to calculate the probability distribution for each category. The specific process is as follows: Where, p i (x) represents the probability of the i-th class, x represents the model output, and K represents the number of classes in the training samples; the Softmax function converts the model output into a probability distribution and selects the label with the highest probability as the predicted label.

5. The edge anomaly traffic detection method based on multi-scale aggregation Transformer according to claim 1, characterized in that, In step S2, the method for training the edge abnormal traffic detection model using a training sample set is as follows: The input is edge traffic data X i The training epochs N and training batches B are defined. First, the model parameters are initialized. In each training epoch and training batch, the samples are input into the time series representation module, and the sequence is segmented to obtain vectors. Then use convolution to convert the vector The dimension is mapped to a new dimension C and a new vector representation is obtained. Next, regarding Perform normalization and extract The location information is then added to the normalized vector to obtain a vector X containing the location information. i '; Subsequently, vector X i The input is fed into a multi-head self-attention layer for encoding, resulting in a vector. Then, features are extracted using FNN to obtain h. i Next, h i Perform average pooling and use a softmax classifier to obtain the predicted label Y. i Finally, based on the true and predicted labels, the cross-entropy loss is calculated. i The network parameters of the model are updated using the AdamW optimizer to optimize the performance of abnormal traffic detection.

Images