Network intrusion detection system based on time convolution network and attention mechanism

Abstract

The application discloses a network intrusion detection system based on a time convolution network and an attention mechanism, and relates to the technical field of network security. The data processing module of the system processes CIC-IDS 2018 data sets. The TCNSE network model module comprises a TCN and an SE module, the TCN extracts time characteristics through causality, inflation convolution and residual connection, and the SE module adjusts channel weights; the loss function and the optimizer module adopt a binary Focal Loss function and an AdamW optimizer to cope with class imbalance; the experimental analysis and result module evaluates the model through various performance indicators, compares the model performance under different conditions, and compares the model with other models; and the model reasoning acceleration module introduces an ONNX format to improve CPU reasoning performance. The system can effectively improve detection accuracy, reduce computational complexity, is suitable for resource-constrained devices, and has a good application prospect in the field of network security.

Description

Technical Field

[0001] This invention relates to the field of network security technology, and more specifically, to a network intrusion detection system based on temporal convolutional networks and attention mechanisms. Background Technology

[0002] With the development of information technology and the popularization of the Internet, the number and complexity of cyberattacks have increased significantly, posing a serious threat to global cybersecurity, such as causing information leaks, identity theft, and data tampering. In 2023, the United States lost more than $12.5 billion to cybercrime, an increase of 22% compared to 2022. At the same time, the widespread use of IoT devices has made them vulnerable targets. For example, in 2016, the Mirai botnet launched a large-scale DDoS attack, using infected IoT devices to paralyze many websites and online services.

[0003] Intrusion Detection Systems (IDS) are an important part of network security defense systems and can be divided into two categories: signature-based detection and anomaly behavior-based detection. Traditional IDS mostly rely on signature-based detection methods, which have high accuracy in detecting known and high-frequency attacks, but are ineffective against unknown or variant attacks. In contrast, anomaly behavior-based detection has better detection performance for both known and unknown attacks and is currently a research focus.

[0004] Deep learning-based systems are better able to cope with dynamic and complex network threats. Against this backdrop, a TCNSE model combining Temporal Convolutional Network (TCN) and Squeeze-and-Excitation (SE) modules is proposed. TCN can effectively model temporal dependencies of data and capture long-distance dependency information, while the SE module can enhance the ability to focus on key features by modeling inter-channel relationships. This model has low computational complexity, is suitable for deployment in IoT devices, and performs excellently in real-time detection scenarios, providing a new approach for building efficient and accurate network intrusion detection systems. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a network intrusion detection system based on temporal convolutional networks and attention mechanisms.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A network intrusion detection system based on temporal convolutional networks and attention mechanisms includes a data processing module, a TCNSE network model module, a loss function and optimizer module, an experimental analysis and results module, and a model inference acceleration module.

[0008] The data processing module is used to process the intrusion detection dataset. The processing includes data preprocessing, feature selection, data standardization, data segmentation, data imbalance handling, and label conversion.

[0009] The TCNSE network model module includes a temporal convolutional network (TCN) and a channel attention (SE) module.

[0010] Preferably, the data preprocessing involves multi-step preprocessing operations on the CIC-IDS2018 dataset, including deleting rows containing 'infinity' or other invalid values, deleting duplicate rows, deleting constant columns, deleting highly correlated columns, deleting specified columns and rows, converting time to timestamps, and merging labels.

[0011] Preferably, the feature selection involves choosing a subset from the original dataset as the input to the model. Specifically, an algorithm based on the comprehensive evaluation of multiple feature selection methods is adopted, combining information gain, chi-square test, random forest, CatBoost and LightGBM feature selection methods, standardizing the importance scores of each method and weighting and summing them to select the top k key features.

[0012] Preferably, the data standardization calculation process is as follows:

[0013] x' = x-x min

[0014] x max -x min

[0015] Here, the standardized value is denoted by x', and the maximum and minimum values ​​of the dataset are denoted by x' and x', respectively. max and x min express;

[0016] The data splitting method uses sklearn's train_test_split method to divide the dataset into 80% training set and 20% test set;

[0017] The data imbalance handling adopts a hybrid sampling method, which first reduces the number of majority samples through random downsampling, and then increases the proportion of minority samples through SMOTE oversampling, ultimately achieving the effect of balancing the proportion of samples in each category; the hybrid sampling method is only applied to the training set, while the test set remains unchanged;

[0018] The label conversion, after sampling, divides the labels into two categories: Benign and Malicious.

[0019] Preferably, the Temporal Convolutional Network (TCN) uses the characteristics of causal convolution, dilated convolution, and residual connections to process temporal data;

[0020] The causal convolution is used to make the output data y at time t... t To prevent future information leakage, information can only originate from time t and earlier. TCN incorporates causal convolution. In standard convolution, padding is typically added to both ends of the input sequence to ensure the output sequence matches the input sequence. However, causal convolution only pads one side of the sequence, usually on the left. The calculation formula is p. l = (k-1)×d; where, p l The padding length is indicated; k represents the kernel size; d is the inflation factor, which determines the spacing between kernel elements and typically grows exponentially.

[0021] Valid input sequence x = [x1, x2, ... x T The convolution kernel size is k, and the weights are ω = [ω1, ω2, ... ω]. k The standard one-dimensional convolution outputs y at position t. t Defined as

[0022] Causal convolution adjusts the index so that: The input depends only on x t ,x t-1 ,…,x t-k+1 , that is, the current and previous inputs;

[0023] The dilated convolution introduces holes to expand the receptive field; given an input sequence x, a kernel size of k, and a dilation factor of d, the output y of the dilated convolution at position t is calculated as follows: Where ω is the weight of the convolution kernel, and d is the dilation factor, which determines the spacing between the elements of the convolution kernel. It usually grows exponentially [1,2,4…]. When d=1, the dilated convolution degenerates into a standard convolution.

[0024] The residual connection alleviates the gradient vanishing problem, optimizes network training, and achieves efficient temporal feature extraction. By introducing a shortcut path that bypasses several layers, the residual connection allows information to be directly transmitted to deeper layers. The core idea of ​​this connection structure is to re-express the original transformation function H(x) as H(x) = F(x) + x, where F(x) represents the residual part to be learned, x is the input, and it is directly added to the output through the residual connection.

[0025] Preferably, the SE module includes two operations: compression and calculation.

[0026] The compression performs global average pooling on each feature channel to obtain a vector with the length of the number of channels, containing global information for each channel;

[0027] The activation is achieved by learning the parameter ω and then re-weighting each feature, enabling the module to focus more on channels with higher weights.

[0028] Preferably, the loss function and optimizer module includes:

[0029] Given the highly imbalanced class distribution in the CIC-IDS2018 dataset, binary Focal Loss was chosen as the loss function. This function effectively improves the model's learning ability for minority or difficult-to-classify samples by dynamically adjusting sample weights, preventing the model from overemphasizing majority or easily classified samples. The calculation formula is FL(p t )=-α t (1-p t ) γ log(p t ); where p t This is the model's prediction of the positive class; when the true label is 1, p t =p; when the actual label is 0, p t =1-p; α t The time balance factor is used to balance the importance of positive and negative samples, and its value is usually between 0 and 1; the adjustment factor is used to reduce the weight of easily distinguishable samples, and its value is usually between 0 and 5.

[0030] The optimizer chosen is AdamW, which combines Adam's adaptive learning rate and fast convergence advantages. By separating weight decay and gradient update, it ensures that the weight decay effect is consistent with optimizers such as SGD, improves the regularization effect, and enhances the model's generalization ability. This allows the model to be gradually optimized during training, achieving better network intrusion detection performance.

[0031] Preferably, the experimental analysis and results module includes performance index definition, model training and performance analysis, and comparison with existing work, including:

[0032] The performance metrics definition details 10 commonly used metrics for measuring the performance of intrusion detection models, including: Recall, Accuracy, F1-score, Precision, False Positive Rate, ROC curve (Receiver Operating Characteristic), True Negative Rate, Confusion matrix, Area under the ROC curve, and Training time. The confusion matrix has four components: True Positive, True Negative, False Positive, and False Negative.

[0033] The model training and performance analysis details the training process of the intrusion detection model, as well as the evaluation and comparison of model metrics, specifically:

[0034] Configure the hardware and software versions of the training platform, and obtain the optimal hyperparameters through experimental comparison;

[0035] The importance of all features in the CIC-IDS2018 dataset was ranked using an integrated feature selection algorithm. The model performance was evaluated under different numbers of features, the number of features that resulted in the best model performance was selected, and the experimental results were analyzed.

[0036] By comparing the changes in various metrics during model training with learning rates of 0.01 and 0.001, the learning rate that improves the model metrics is selected and analyzed.

[0037] With a learning rate of 0.001 and using 20 selected features, the TCNSE model compares the training error, test error, and various metrics during the training process of different temporal convolutional blocks, and selects the structural model with the best performance in each metric.

[0038] The comparison with existing work compares the TCNSE model with traditional machine learning methods and other advanced deep learning intrusion detection models.

[0039] Preferably, the model inference acceleration module introduces the ONNX format to optimize model inference speed; the trained model is exported to the ONNX format, and after model optimization, it runs in the ONNX Runtime inference engine. This engine is optimized for CPU, improving the model's inference performance on the CPU, meeting real-time detection requirements, and is especially suitable for IoT devices lacking GPUs.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] 1. In this invention, a meticulously designed data processing flow effectively removes noisy data and redundant features, selecting the most discriminative feature subset to provide high-quality data for model training. Combining the efficient extraction capability of the Temporal Convolutional Network (TCN) in the TCNSE network model for temporal features, and the enhanced attention to key feature channels by the Channel Attention (SE) module, the model can accurately capture complex patterns and abnormal behaviors in network traffic, thus achieving excellent detection performance. On the CIC-IDS2018 dataset, it achieved an accuracy of 98.4%, a precision of 99.40%, a recall of 98.70%, and an F1 score of 99.10%, demonstrating high sensitivity and accurate identification capabilities against various network attacks.

[0042] 2. In this invention, the parallel computing characteristics of TCN significantly improve its computational speed when processing time-series data. Compared to traditional algorithms such as RNN, it avoids the inefficiency caused by processing time points sequentially, significantly reducing computational complexity. Simultaneously, feature selection effectively reduces the number of features, lowering the computational load during model training and inference. This enables the system to operate efficiently on resource-constrained Internet of Things (IoT) devices, achieving rapid intrusion detection, meeting the real-time requirements of application scenarios, and exhibiting low hardware resource requirements in actual deployment, demonstrating good adaptability.

[0043] 3. In this invention, to address the common challenge of imbalanced data classes, a hybrid sampling method is employed to process the training set, balancing the proportion of samples from each class. Simultaneously, a binary Focal Loss function is selected to dynamically adjust sample weights, enabling the model to pay more attention to hard-to-classify samples (such as minority class attack samples) during training. This effectively avoids overfitting the model to majority class samples, improves the detection capability for minority class attacks, reduces false positives and false negatives, and enhances the model's generalization detection capability for various attacks in complex network environments.

[0044] 4. In this invention, the experimental analysis and results module delves into the model's performance under different feature counts, learning rates, and network structures, providing a basis for flexibly adjusting the model according to specific needs in practical applications. For example, a suitable model structure can be selected based on resource availability and performance requirements, achieving a balance between performance and computational overhead. Furthermore, the system design provides direction for future optimization, such as employing advanced optimization algorithms to optimize feature selection, applying it to more datasets to verify generalization capabilities, exploring other imbalance handling techniques, and combining containerization technology for efficient deployment. These advancements will help continuously improve the system's detection performance and application breadth, adapting to ever-changing cybersecurity threats. Attached Figure Description

[0045] Figure 1 This invention presents a flowchart illustrating a network intrusion detection system based on temporal convolutional networks and attention mechanisms.

[0046] Figure 2 This invention presents a schematic diagram of the architecture of a network intrusion detection system based on temporal convolutional networks and attention mechanisms.

[0047] Figure 3 This invention presents a schematic diagram illustrating the distribution of attack types in each CSV file of the CIC-IDS2018 dataset.

[0048] Figure 4 This is a schematic diagram showing the distribution of the CIC-IDS2018 dataset after preprocessing, as proposed in this invention.

[0049] Figure 5 This invention presents a schematic diagram of the architecture of a network intrusion detection model based on temporal convolutional networks and attention mechanisms.

[0050] Figure 6 This is a schematic diagram of the process of the extended causal convolution proposed in this invention;

[0051] Figure 7 This is a schematic diagram of the confusion matrix proposed in this invention;

[0052] Figure 8 This is a comparison chart of various indicators under different learning rates proposed in this invention;

[0053] Figure 9 This is a comparison chart of various indicators under different model structures proposed in this invention;

[0054] Figure 10 This is the ROC curve diagram for various attack detection methods proposed in this invention;

[0055] Figure 11 This is a schematic diagram of the confusion matrix structure on the test set for the model [128,64,32] proposed in this invention, under the condition of taking the optimal learning rate and number of features. Detailed Implementation

[0056] The embodiments further illustrate the network intrusion detection system based on temporal convolutional networks and attention mechanisms proposed in this invention.

[0057] Reference Figure 1 A network intrusion detection system based on temporal convolutional networks and attention mechanisms includes a data processing module, a TCNSE network model module, a loss function and optimizer module, an experimental analysis and results module, and a model inference acceleration module.

[0058] The data processing module is used to process the intrusion detection dataset. The processing includes data preprocessing, feature selection, data standardization, data segmentation, data imbalance handling, and label conversion.

[0059] The TCNSE network model module includes a temporal convolutional network (TCN) and a channel attention (SE) module.

[0060] Reference Figures 2 to 7 The datasets used to evaluate the performance of network intrusion detection systems include KDDCUP99, NSL-KDD, CIC-IDS2017, CIC-IDS2018, and UNSW-NB15, as shown in the table below:

[0061]

[0062] This paper details the cleaning process for the CIC-IDS2018 dataset, which contains 10 CSV files. The 02-20-2018.csv file is unique, containing four additional columns: FlowID, SrcIP, SrcPort, and DstIP. These redundant columns need to be removed for standardized processing. The following are the preprocessing steps for the dataset:

[0063] Delete rows containing 'infinity' or other invalid values; delete duplicate rows; delete constant columns (i.e., all rows have the same value in this column, which can be quickly determined by calculating whether the column's variance is 0); delete highly correlated columns (set a threshold of 0.95, if the correlation between two columns exceeds the threshold, only one column will be kept); delete specified columns and rows (delete the 'Protocol' column, because there is already a 'Dst Port' column (one port can represent one protocol); discard rows containing negative numbers in the 'Fwd HeaderLen', 'Flow Duration', and 'Flow IAT Min' columns, because the values ​​in these columns should not be negative, and the deleted data is erroneous; similarly delete the 'Init Fwd Win Byts' and 'Init BwdWin Byts' columns, because nearly half of the data in these two columns is negative, as well as the 'Flow... The Duration column converts time to timestamps (the original dataset's time column stores specific times, but for time series processing models, timestamps are needed) and merges labels (i.e., attacks of the same category are grouped into one large category, as shown in the table below).

[0064]

[0065]

[0066] Based on the completed data preprocessing, the entire dataset contains 69 columns (including label columns) and a total of 155,722,254 data entries.

[0067] Besides these common data preprocessing steps, one point is worth noting: the CIC-IDS2018 dataset is very large (approximately 6.41GB). If imported via Pandas, Pandas will not select an appropriate data type for each column (for example, if the maximum value in a column is less than 255 and the minimum value is greater than 0, Pandas might directly select the int type, but uint8 precision is perfectly sufficient). This will consume a lot of memory and slow down subsequent data processing. This can be achieved through data type compression; after processing, approximately 60% of memory can be saved. The algorithm steps are as follows:

[0068] Input:DataFrame props

[0069] Output:Optimized DataFrame with reduced memory usage

[0070] For each col in props.columns do

[0071] ifcol is numeric then

[0072] ifvalues in col canbe represented as integers then

[0073] ifcol.min()≥0and col.max()<255then

[0074] Downcast col to uint8

[0075] else ifcol.min()≥0and col.max()<65535then

[0076] Downcast col to uint16 This logic canbe extended to othertypes likeuint32,int,etc.,by adding similar conditions

[0077] else

[0078] Convent col to a smaller floating-point type(e.g.,float32)

[0079] end if

[0080] end if

[0081] end if

[0082] end for

[0083] Feature selection involves choosing a subset from the original dataset as input to the model, which avoids the curse of dimensionality and enhances generalization ability. Specifically, an algorithm based on the comprehensive evaluation of multiple feature selection methods is employed, combining common feature selection methods such as information gain, chi-square test, random forest, CatBoost, and LightGBM. The importance scores of each method are standardized and weighted, and the top k key features are selected. This feature selection strategy based on multi-method integration and weighted scoring not only effectively addresses the problem of excessively high feature dimensionality but also ensures the generalization performance of the selected features in the model.

[0084] Common feature selection methods are shown in the table below:

[0085]

[0086] Data standardization plays a crucial role in model performance and training efficiency. Standardization can distribute data with different features within a similar range, preventing numerical instability and gradient vanishing caused by excessive differences in feature values, thereby improving the model's convergence speed and prediction accuracy. Furthermore, standardization can enhance the model's generalization ability, reduce the risk of overfitting, and improve the detection of unknown attacks. The data standardization calculation process is as follows: Here, the standardized value is denoted by x', and the maximum and minimum values ​​of the dataset are denoted by x' and x', respectively. max and x min express.

[0087] The data splitting method uses sklearn's train_test_split method to divide the dataset into 80% training set and 20% test set.

[0088] The data imbalance handling adopts a hybrid sampling method, which first reduces the number of majority samples through random downsampling, and then increases the proportion of minority samples through SMOTE oversampling, ultimately achieving the effect of balancing the proportion of samples in each category; the hybrid sampling method is only applied to the training set, while the test set remains unchanged.

[0089] The label conversion, after sampling, divides the labels into two categories: Benign and Malicious.

[0090] The Temporal Convolutional Network (TCN) uses the characteristics of causal convolution, dilated convolution, and residual connections to process temporal data.

[0091] The causal convolution is used to make the output data y at time t... tTo prevent future information leakage, information can only originate from time t and earlier. TCN incorporates causal convolution. In standard convolution, padding is typically added to both ends of the input sequence to ensure the output sequence matches the input sequence. However, causal convolution only pads one side of the sequence, usually on the left. The calculation formula is p. l = (k-1)×d; where, p l The padding length is indicated; k represents the kernel size; d is the inflation factor, which determines the spacing between kernel elements and typically grows exponentially.

[0092] Valid input sequence x = [x1, x2, ... x T The convolution kernel size is k, and the weights are ω = [ω1, ω2, ... ω]. k The standard one-dimensional convolution outputs y at position t. t Defined as

[0093] Causal convolution adjusts the index so that: The input depends only on x t ,x t-1 ,…,x t-k+1 , which refers to the current and previous inputs.

[0094] The dilated convolution introduces holes to expand the receptive field; given an input sequence x, a kernel size of k, and a dilation factor of d, the output y of the dilated convolution at position t is calculated as follows: Where ω is the weight of the convolution kernel, and d is the dilation factor, which determines the spacing between the elements of the convolution kernel. It usually grows exponentially [1,2,4…]. When d=1, the dilated convolution degenerates into a standard convolution.

[0095] The residual connection alleviates the gradient vanishing problem, optimizes network training, and achieves efficient temporal feature extraction. The residual connection introduces a shortcut path that bypasses several layers, allowing information to be directly transmitted to the deep layers. The core idea of ​​this connection structure is to re-express the original transformation function H(x) as H(x) = F(x) + x. Here, F(x) represents the residual part that needs to be learned, and x is the input, which is directly added to the output through the residual connection. The residual connection

[27] makes the neural network no longer learn the direct mapping H(x), but learns the residual function F(x). This design makes the network easier to optimize because even if some layers in the deep network do not learn effective features (i.e., F(x) is close to 0), the network can at least directly transmit information x through the residual connection. That is, as the model depth increases, the effect will at least not decrease.

[0096] The SE module includes two operations: compression and calculation. The compression performs global average pooling on each feature channel to obtain a vector with the length of the number of channels, containing global information for each channel. The activation uses parameters ω learned through learning to reassign weights to each feature, enabling the module to focus more on channels with high weights.

[0097] The loss function and optimizer module includes:

[0098] Given the highly imbalanced class distribution in the CIC-IDS2018 dataset, binary Focal Loss was chosen as the loss function. This function effectively improves the model's learning ability for minority or difficult-to-classify samples by dynamically adjusting sample weights, preventing the model from overemphasizing majority or easily classified samples. The calculation formula is FL(p t )=-α t (1-p t ) γ log(p t ); where p t This is the model's prediction of the positive class; when the true label is 1, p t =p; when the actual label is 0, p t =1-p; α t The time balance factor is used to balance the importance of positive and negative samples, and its value is usually between 0 and 1; the adjustment factor is used to reduce the weight of easily distinguishable samples, and its value is usually between 0 and 5.

[0099] The optimizer chosen is AdamW, which combines Adam's adaptive learning rate and fast convergence advantages. By separating weight decay and gradient update, it ensures that the weight decay effect is consistent with optimizers such as SGD, improves the regularization effect, and enhances the model's generalization ability. This allows the model to be gradually optimized during training, achieving better network intrusion detection performance.

[0100] The experimental analysis and results module includes performance metric definition, model training and performance analysis, and comparison with existing work, including:

[0101] The performance metrics definition details 10 commonly used metrics for measuring the performance of intrusion detection models, including: Recall, Accuracy, F1-score, Precision, False Positive Rate, ROC curve (Receiver Operating Characteristic), True Negative Rate, Confusion matrix, Area under the ROC curve, and Training time. The confusion matrix has four components: True Positive, True Negative, False Positive, and False Negative.

[0102] The True Positive (TP): The Intrusion Detection System (IDS) successfully identifies abnormal network traffic as abnormal; the True Negative (TN): The Intrusion Detection System correctly identifies legitimate network traffic as normal; the False Positive (FP): The Intrusion Detection System incorrectly identifies normal traffic as abnormal traffic; the False Negative (FN): The Intrusion Detection System fails to identify abnormal traffic and misidentifies it as normal traffic.

[0103] The Accuracy quantifies the proportion of correct classifications by the intrusion detection system, and is calculated as follows: This metric is useful for measuring models trained on common datasets, but accuracy may not be applicable to imbalanced datasets like CIC IDS2018, as the model will be biased towards the most common categories.

[0104] The Recall, also known as the True Positive Rate (TPR), is primarily used to calculate the proportion of anomalous samples correctly classified by the intrusion detection system. The calculation method is as follows: This metric is used to detect the model's sensitivity to abnormal samples and is also the most commonly used metric.

[0105] The Precision metric answers the question: what percentage of samples identified as anomalous by the intrusion detection system are truly anomalous? It is calculated as follows:

[0106] The F1-score addresses the issue that high precision often correlates with low recall, and vice versa. To comprehensively consider these two metrics, the F1-score was developed. It is calculated as a weighted harmonic average of precision and recall, and the calculation method is as follows:

[0107] The False Positive Rate (FPR), also known as the false alarm rate, is used to calculate the proportion of all normal samples that are incorrectly marked as abnormal. A higher FPR indicates that the intrusion detection system frequently classifies normal traffic as abnormal traffic. The calculation method is as follows:

[0108] The Receiver Operating Characteristic (ROC) is a tool used to evaluate the performance of binary classification models, showing the model's classification ability at different thresholds. It reflects the model's performance at different decision thresholds by plotting the relationship between TPR and FPR. The closer the ROC curve is to the upper left corner, the better the model's performance, indicating that it maintains a low FPR while achieving a high TPR. The area under the curve (AUC) is usually used to quantify the overall performance of the model; the closer the AUC value is to the top left corner, the better the model.

[0109] The True Negative Rate (TNR), also known as the Normal Predictive Value (NPV), is used to calculate the proportion of samples correctly classified as normal. The calculation method is as follows:

[0110] The False Negative Rate (FNR) measures the proportion of all anomalous samples in network traffic that are not detected by the intrusion detection system. It is calculated as follows:

[0111] The model training and performance analysis details the training process of the intrusion detection model, as well as the evaluation and comparison of model metrics, specifically:

[0112] The hardware configuration and software version of the training platform are shown in the table below:

[0113]

[0114] The optimal hyperparameters obtained through experimental comparison are shown in the table below:

[0115]

[0116]

[0117] An ensemble feature selection algorithm was used to rank the importance of all features in the CIC-IDS2018 dataset. Model performance was evaluated with different numbers of features, and the optimal number of features for model performance was selected. The experimental results are shown in the table below.

[0118]

[0119] Experimental results show that the model performs best when the number of features is 20, with an accuracy of 98.5%, a recall of 98.9%, a precision and F1-score of 99%, a false positive rate of 3.30%, a true negative rate of 96.70%, and a training time of only 74.4 seconds per epoch. This result clearly shows that appropriately reducing the number of features does not reduce the model's classification ability, but rather improves the overall performance of the model in multiple aspects.

[0120] As the number of features increases, although the model's accuracy and F1-score remain at a high level, other metrics such as the false positive rate increase, the true negative rate decreases slightly, and the training time increases significantly. For example, when the number of features increases to 68, the training time increases by almost 2.5 times, reaching 184.9 seconds, while the false positive rate also rises to 4.80% and the true negative rate drops to 95.20%. This phenomenon can be explained by the curse of dimensionality. High-dimensional feature spaces make data sparse, and the model has difficulty effectively capturing the inherent structure of the data in high-dimensional spaces, leading to a decrease in generalization ability. In addition, more features mean more parameters to learn, increasing computational complexity and thus extending training time. Especially in resource-constrained IoT device environments, this computational overhead has a negative impact on system response time and user experience.

[0121] Feature selection uses multiple algorithms to filter out 20 key features, reducing computational overhead and training time, improving real-time response capabilities, reducing costs, and enhancing robustness and generalization performance.

[0122] Reference Figure 8The figures show the changes in various metrics during model training with learning rates of 0.01 and 0.001, respectively. It can be seen that when the learning rate is 0.001, the model's Precision, FPR, and TNR are all superior to the model with a learning rate of 0.01. This may be attributed to the fact that a lower learning rate allows for more stable parameter updates during training, enabling the model to more closely approximate the optimal solution of the loss function and reducing parameter oscillations or overfitting caused by an excessively large learning rate. In contrast, a learning rate of 0.01 may cause the model to skip optimal regions during training, failing to fully learn the deep features of the data, thus affecting the model's classification performance.

[0123] Reference Figure 9 With a learning rate of 0.001 and 20 selected features, the TCNSE model was trained using different temporal convolutional block structures. The training error, test error, and various metrics were compared during the training process. The table below summarizes the main performance metrics and training time for different model structures.

[0124]

[0125] As shown in the table above, with the increase of model complexity, the model's accuracy, precision, recall, F1 score, and TNR all improve. Among them, the model with the structure [256, 128, 64, 32] performs best in all metrics, achieving an accuracy of 98.69%, a precision of 99.46%, a recall of 98.99%, an F1 score of 99.23%, and a TNR of 96.99%. This performance improvement is mainly attributed to the deeper network structure and larger parameter capacity of the model, enabling it to capture more complex features and temporal dependencies in the data. The introduction of residual connections also contributes to this improvement. This effectively alleviates the vanishing gradient problem commonly encountered in deep network training, ensuring the efficient propagation of information within the network. However, the improved model performance comes at the cost of a significant increase in training time. Training a model with a structure of [256, 128, 64, 32] for one epoch takes 89.61 seconds, almost twice that of a model with a structure of [128, 64, 32] (46.16 seconds). The increase in training time is mainly due to the increase in the number of network layers and parameters, resulting in more computational resources required for each training iteration. In practical applications, especially on Internet of Things (IoT) devices with limited computational resources, this may affect the deployment and real-time performance of the model.

[0126] Therefore, a balance needs to be struck between performance improvement and computational overhead in model design. Although more complex models can provide better performance metrics, training time and inference efficiency are equally critical in practical applications. For applications requiring real-time responses, appropriately reducing model complexity and choosing a lighter structure such as [128, 64, 32] may be a more practical choice.

[0127] Figure 11 It is the confusion matrix on the test set for a model structure of [128, 64, 32] with the optimal learning rate and number of features. Figure 10 These are ROC curves for detecting various types of attacks.

[0128] The performance metrics for different attack types are shown in the table below:

[0129] Attacktype Accuracy TPR DDoSattack 98.85％ 100.00％ DoSattack 98.80％ 100.00％ Botnet 98.78％ 100.00％ Infiltration 98.21％ 96.00％ Brute-force 98.77％ 100.00％ Webattack 98.75％ 99.00％

[0130] Compared with existing work, the proposed model based on Temporal Convolutional Networks and Squeeze-and-Excitation Modules (TCNSE) is compared and analyzed with traditional machine learning methods and various advanced deep learning intrusion detection models. The performance of different research models in key performance indicators such as accuracy, precision, recall, AUC (area under the curve), and F1 score is shown in the table below:

[0131]

[0132]

[0133] As can be seen from the table, the TCNSE model performs excellently on all evaluation metrics. Specifically, the TCNSE model achieves 98.40% accuracy, 99.40% recall, 98.70% AUC, and 99.10% F1 score. These results significantly outperform traditional machine learning models, such as the Random Forest model proposed by Leevy et al. (2021), with an AUC of 95.50% and an F1 score of 93.20%, and the NaiveBayes model. The UC was only 55.40% and the F1 score was 31.40%; compared with other deep learning models, the TCNSE model showed significant advantages; for example, Ferrag et al. used RNN and deep autoencoder models to achieve an accuracy of 97.38% and an AUC of 98.18%; Gamage et al.

[34] achieved an accuracy of 98.4%, a recall of 97.99% and an AUC of 98.27% with their deep feedforward neural network; while Lin et al.'s LSTM model achieved an accuracy of 96.2% and a recall of 96%.

[0134] In comparison, the TCNSE model improved across all key metrics, particularly in recall and F1 score, reaching 99.40% and 99.10% respectively, demonstrating a significant advantage in identifying real intrusion events and maintaining a balance between precision and recall.

[0135] The model inference acceleration module introduces the ONNX format to optimize model inference speed. The trained model is exported to the ONNX format, and after model optimization (operator fusion, constant folding, etc.), it is run in the ONNX Runtime inference engine. This engine is optimized for CPU, which improves the model's inference performance on the CPU, meets the requirements of real-time detection, and is especially suitable for IoT devices that lack GPUs.

[0136] ONNX is an open neural network model representation format designed to facilitate interoperability between different deep learning frameworks. It allows developers to easily convert models between different frameworks, such as from PyTorch to TensorFlow and vice versa. The working principle of ONNX involves the following key steps:

[0137] Model export: First, the model trained in frameworks such as PyTorch will be exported as ONNX format; this process will convert the model's computation graph and parameters into a standardized ONNX representation.

[0138] Model optimization: The exported ONNX model can be processed by various optimization tools; for example, ONNX provides an optimizer that can perform optimizations such as operator fusion and constant folding to reduce computational redundancy.

[0139] High-efficiency inference: The optimized ONNX model can run on high-performance inference engines such as ONNX Runtime; ONNXRuntime is optimized for a variety of hardware architectures, especially for the CPU instruction set and parallel computing capabilities.

[0140] By using ONNX, our network intrusion detection model achieves a significant improvement in inference speed on the CPU; this allows the model to run efficiently on IoT devices lacking GPUs, thus meeting the requirements for real-time detection. The table below compares the inference time of the model on different devices for one batch (4096) of data:

[0141] Inference Method AverageInferenceTime(s) CPU 1799 GPU 3.13 ONNX(CPU) 597.54

[0142] It can be seen that the time is improved compared to using CPU inference directly, but it is still significantly slower than GPU inference.

[0143] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A network intrusion detection system based on temporal convolutional networks and attention mechanisms, characterized in that, It includes a data processing module, a TCNSE network model module, a loss function and optimizer module, an experimental analysis and results module, and a model inference acceleration module; The data processing module is used to process the intrusion detection dataset. The processing includes data preprocessing, feature selection, data standardization, data segmentation, data imbalance handling, and label conversion. The TCNSE network model module includes a temporal convolutional network (TCN) and a channel attention (SE) module.

2. The network intrusion detection system based on temporal convolutional networks and attention mechanisms according to claim 1, characterized in that, The data preprocessing involves multiple steps for the CIC-IDS2018 dataset, including deleting rows containing 'infinity' or other invalid values, deleting duplicate rows, deleting constant columns, deleting highly correlated columns, deleting specified columns and rows, converting time to timestamps, and merging labels.

3. The network intrusion detection system based on temporal convolutional networks and attention mechanisms according to claim 2, characterized in that, The feature selection involves choosing a subset from the original dataset as the input to the model. Specifically, an algorithm based on the comprehensive evaluation of multiple feature selection methods is adopted, combining information gain, chi-square test, random forest, CatBoost and LightGBM feature selection methods. The importance scores of each method are standardized and weighted and accumulated to select the top k key features.

4. A network intrusion detection system based on temporal convolutional networks and attention mechanisms according to claim 3, characterized in that, The data standardization calculation process: x'＝ x-x min x max -x min Here, the standardized value is denoted by x', and the maximum and minimum values ​​of the dataset are denoted by x' and x', respectively. max and x min express; The data splitting method uses sklearn's train_test_split method to divide the dataset into 80% training set and 20% test set; The data imbalance handling adopts a hybrid sampling method, which first reduces the number of majority samples through random downsampling, and then increases the proportion of minority samples through SMOTE oversampling, ultimately achieving the effect of balancing the proportion of samples in each category; the hybrid sampling method is only applied to the training set, while the test set remains unchanged; The label conversion, after sampling, divides the labels into two categories: Benign and Malicious.

5. A network intrusion detection system based on temporal convolutional networks and attention mechanisms according to claim 4, characterized in that, The Temporal Convolutional Network (TCN) uses the characteristics of causal convolution, dilated convolution, and residual connections to process temporal data. The causal convolution is used to make the output data y at time t... t To prevent future information leakage, information can only originate from time t and earlier. TCN incorporates causal convolution. In standard convolution, padding is typically added to both ends of the input sequence to ensure the output sequence matches the input sequence. However, causal convolution only pads one side of the sequence, usually on the left. The calculation formula is p. l = (k-1)×d; where, p l The padding length is indicated; k represents the kernel size; d is the inflation factor, which determines the spacing between kernel elements and typically grows exponentially. Valid input sequence x = [x1, x2, ... x T The convolution kernel size is k, and the weights are ω = [ω1, ω2, ... ω]. k The standard one-dimensional convolution outputs y at position t. t Defined as Causal convolution adjusts the index so that: The input depends only on x t ,x t-1 ,…,x t-k+1 , that is, the current and previous inputs; The dilated convolution introduces holes to expand the receptive field; given an input sequence x, a kernel size of k, and a dilation factor of d, the output y of the dilated convolution at position t is calculated as follows: Where ω is the weight of the convolution kernel, and d is the dilation factor, which determines the spacing between the elements of the convolution kernel. It usually grows exponentially [1,2,4…]. When d=1, the dilated convolution degenerates into a standard convolution. The residual connection alleviates the gradient vanishing problem, optimizes network training, and achieves efficient temporal feature extraction. By introducing a shortcut path that bypasses several layers, the residual connection allows information to be directly transmitted to deeper layers. The core idea of ​​this connection structure is to re-express the original transformation function H(x) as H(x) = F(x) + x, where F(x) represents the residual part to be learned, x is the input, and it is directly added to the output through the residual connection.

6. A network intrusion detection system based on temporal convolutional networks and attention mechanisms according to claim 5, characterized in that, The SE module includes two operations: compression and calculation. The compression performs global average pooling on each feature channel to obtain a vector with the length of the number of channels, containing global information for each channel; The activation is achieved by learning the parameter ω and then re-weighting each feature, enabling the module to focus more on channels with higher weights.

7. A network intrusion detection system based on temporal convolutional networks and attention mechanisms according to claim 6, characterized in that, The loss function and optimizer module includes: Given the highly imbalanced class distribution in the CIC-IDS2018 dataset, binary Focal Loss was chosen as the loss function. This function effectively improves the model's learning ability for minority or difficult-to-classify samples by dynamically adjusting sample weights, preventing the model from overemphasizing majority or easily classified samples. The calculation formula is FL(p t )=-α t (1-p t ) γ log(p t ); where p t This is the model's prediction of the positive class; when the true label is 1, p t =p; when the actual label is 0, p t =1-p; α t The time balance factor is used to balance the importance of positive and negative samples, and its value is usually between 0 and 1; the adjustment factor is used to reduce the weight of easily distinguishable samples, and its value is usually between 0 and 5. The optimizer chosen is AdamW, which combines Adam's adaptive learning rate and fast convergence advantages. By separating weight decay and gradient update, it ensures that the weight decay effect is consistent with optimizers such as SGD, improves the regularization effect, and enhances the model's generalization ability. This allows the model to be gradually optimized during training, achieving better network intrusion detection performance.

8. A network intrusion detection system based on temporal convolutional networks and attention mechanisms according to claim 7, characterized in that, The experimental analysis and results module includes performance metric definition, model training and performance analysis, and comparison with existing work, including: The performance metrics definition details 10 commonly used metrics for measuring the performance of intrusion detection models, including: Recall, Accuracy, F1-score, Precision, False Positive Rate, ROC curve (Receiver Operating Characteristic), True Negative Rate, Confusion matrix, Area under the ROC curve, and Training time. The confusion matrix has four components: True Positive, True Negative, False Positive, and False Negative. The model training and performance analysis details the training process of the intrusion detection model, as well as the evaluation and comparison of model metrics, specifically: Configure the hardware and software versions of the training platform, and obtain the optimal hyperparameters through experimental comparison; The importance of all features in the CIC-IDS2018 dataset was ranked using an integrated feature selection algorithm. The model performance was evaluated under different numbers of features, the number of features that resulted in the best model performance was selected, and the experimental results were analyzed. By comparing the changes in various metrics during model training with learning rates of 0.01 and 0.001, the learning rate that improves the model metrics is selected and analyzed. With a learning rate of 0.001 and using 20 selected features, the TCNSE model compares the training error, test error, and various metrics during the training process of different temporal convolutional blocks, and selects the structural model with the best performance in each metric. The comparison with existing work compares the TCNSE model with traditional machine learning methods and other advanced deep learning intrusion detection models.

9. A network intrusion detection system based on temporal convolutional networks and attention mechanisms according to claim 8, characterized in that, The model inference acceleration module introduces the ONNX format to optimize model inference speed; the trained model is exported to the ONNX format, and after model optimization, it runs in the ONNX Runtime inference engine. This engine is optimized for CPU, improving the model's inference performance on the CPU, meeting real-time detection requirements, and is especially suitable for IoT devices lacking GPUs.

Images