Intrusion detection method based on network flow multi-mode spatio-temporal feature fusion

Abstract

The invention discloses an intrusion detection method based on network flow multi-mode spatio-temporal feature fusion, which comprises the following steps: acquiring and preprocessing original network flow data to obtain model input data; converting the pre-processed data into graphical representation by using a Gramb angle field and a Markov transfer field, and generating network traffic image data; time sequence features and space features are extracted in parallel through a two-channel feature extraction model, the time sequence features are extracted by adopting a gating circulation unit, and the space features are extracted by adopting Swin Transform; performing adaptive fusion on the two features through a multi-head attention mechanism to obtain a multi-modal fusion feature; and intrusion detection classification is completed based on this. According to the method, efficient fusion of the network traffic time sequence and the spatial features is realized, and the recognition capability, the detection precision and the generalization performance of an intrusion detection system on complex and novel attacks are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of network security technology, and in particular relates to an intrusion detection method based on the fusion of multimodal spatiotemporal features of network traffic. Background Technology

[0002] Intrusion Detection Systems (IDS), as a crucial component of network security, bear the core responsibility of monitoring network traffic and identifying potential malicious activities. Traditional intrusion detection methods are mainly divided into two categories: feature-based detection methods and anomaly-based detection methods. The former relies on known attack patterns for detection, boasting high detection accuracy, but struggles to identify unknown attacks or zero-day attacks; the latter identifies anomalies by judging the degree of deviation between traffic behavior and normal behavior distribution, enabling the detection of new types of attacks, but suffers from a high false positive rate.

[0003] Recent advances in machine learning and deep learning technologies have driven the development of numerous network intrusion detection methods. These methods aim to improve detection accuracy, real-time performance, and resilience against complex attacks, and they differ in their architectural design, data representation methods, and strategies for addressing class imbalance and feature extraction problems.

[0004] Numerous studies have explored attack classification models based on neural networks and traditional machine learning techniques. Akusota et al. proposed a classifier based on Extreme Learning Machine (ELM) for IoT applications, integrating multiple feature extraction techniques. This model achieved detection accuracies of 94.5% and 94.3% on the NSL-KDD and UNSW-NB15 datasets, respectively. Vinayakumar et al. designed a flexible and efficient attack classifier based on Deep Neural Network (DNN) for detecting evolving malicious attacks. Extensive experimental validation demonstrates that deep neural networks outperform traditional machine learning classifiers.

[0005] Optimization algorithms have become a key research focus in this field. Varzane et al. cleverly combined Opposition-based Learning (OBL) with the Levy flight mechanism to design an improved balanced optimization algorithm, which was applied to intrusion detection tasks, effectively improving search diversity and preventing the model from getting trapped in local optima. Zhou et al. used an improved Harris Eagle optimization algorithm to optimize the feature selection stage, significantly improving detection accuracy and outperforming existing mainstream intrusion detection schemes.

[0006] To mitigate the impact of class imbalance, Hushmand et al. integrated the Synthetic Minority Oversampling Technique (SMOTE) with K-means clustering into an anomaly detection framework: SMOTE oversamples minority class samples while K-means undersampling is achieved based on clustering. Rahman et al. used Generative Adversarial Networks (GANs) to generate synthetic data and explored using 100% GAN-synthesized data for intrusion detection model training. This method significantly reduces reliance on real data while maintaining high model performance. Yang et al. proposed an unsupervised detection framework based on Conditional Variational Autoencoders (CVAEs), which can reconstruct missing data features, aiding in the discovery and learning of potential sparse representational relationships between features and categories. Min et al. proposed a network intrusion detection method based on memory-enhanced deep autoencoders, which effectively overcomes the limitations of traditional feature-matching-based detection systems and alleviates the data imbalance problem in network intrusion detection tasks.

[0007] Hybrid models incorporating spatiotemporal features have achieved good results in related tasks. Jiang et al. proposed a malicious domain name detection model based on a convolutional neural network-gated recurrent unit-attention mechanism (CNN-GRU-attention), and experimental results show that the model has better convergence and higher detection accuracy. To address the shortcomings of existing methods, Bingbusi et al. proposed a novel unsupervised single-stage deep learning intrusion detection method, integrating a one-dimensional convolutional autoencoder and a class of support vector machines into a unified joint optimization framework. Sinha et al. proposed an intrusion detection model based on a convolutional neural network-bidirectional long short-term memory network (CNN-BiLSTM), which enhances the model's robustness against complex attacks by incorporating spatiotemporal feature learning capabilities, while achieving higher detection rates and lower false positive rates.

[0008] Despite the progress made in previous research, many current methods still extract features from a single modality (temporal or spatial) and employ simple fusion strategies such as splicing and mean calculation. Network attack traffic exhibits both temporal dynamic evolution and spatial distribution characteristics. Detection methods relying solely on single-modal features struggle to fully capture the multidimensional attributes of attack behavior, leading to unstable detection performance when dealing with complex or mutated attacks. Furthermore, even when some methods introduce multimodal traffic feature fusion strategies, the fusion methods are often limited to simple splicing or mean fusion, failing to effectively characterize the complementarity and intrinsic correlation between different modal features. This deficiency directly restricts the model's feature representation ability and generalization performance, and results in insufficient sensitivity to mutated or novel attack patterns. Summary of the Invention

[0009] To address the aforementioned technical problems, this invention proposes an intrusion detection method based on the fusion of multimodal spatiotemporal features of network traffic, thereby resolving the issues present in the prior art.

[0010] To achieve the above objectives, this invention provides an intrusion detection method based on the fusion of multimodal spatiotemporal features of network traffic, comprising: The raw network traffic data to be detected is acquired and preprocessed to obtain model input data. The model input data is then visualized to generate network traffic image data. Parallel feature extraction is performed using a dual-channel spatiotemporal feature extraction model; the network traffic image data is processed using a time-series feature extraction channel based on a gated loop unit to obtain traffic temporal features; and the network traffic image data is processed using a spatial feature extraction channel based on a Swing Transformer to obtain traffic spatial features. A multi-head attention mechanism is used to adaptively fuse the traffic temporal features and the traffic spatial features to obtain multimodal fusion features; Based on the multimodal fusion features, intrusion detection classification is completed.

[0011] Optionally, the process of representing the model input data graphically includes: The one-dimensional model input data is converted into Gram angle field image data using Gram angle field; the network traffic data is converted into Markov transition field image data using Markov transition field; the Gram angle field image data and the Markov transition field image data are combined to generate the network traffic image data.

[0012] Optionally, the step of using Gram angle field technology to convert the one-dimensional model input data into Gram angle field image data includes: generating Gram angle and field images and Gram angle difference field images of the network traffic data, respectively.

[0013] Optionally, the Gram angle field image data and the Markov transition field image data are combined to form the network traffic image data, including: using the Gram angle sum field, Gram angle difference field and Markov transition field image as the red, green and blue channels of the RGB image respectively, and fusing them to generate the three-dimensional network traffic image data.

[0014] Optionally, the network traffic image data is processed based on the temporal feature extraction channel of the gated cyclic unit, including: The network traffic image data is input into a gated recurrent unit network; through the update gating mechanism and reset gating mechanism of the gated recurrent unit network, the fusion state of historical information and current input is adjusted to extract the temporal dependency relationship in the network traffic data and output the traffic temporal features.

[0015] Optionally, the network traffic image data is processed based on the spatial feature extraction channel of the Swing Transformer, including: The network traffic image data is divided into multiple non-overlapping image blocks, and each image block is converted into an image block feature vector; The image patch feature vectors are projected into a high-dimensional feature space through a linear embedding layer to obtain an initial feature vector sequence; The initial feature vector sequence is input into a hierarchical feature extraction structure containing multiple levels for multi-level refinement. Each level of processing includes: after feature fusion of the current level's feature vector sequence through a multi-head self-attention module based on a shift window, adjacent feature vectors in the fused feature vector sequence are merged through an image patch merging operation, and the output is the feature vector sequence that serves as the input for the next level. The traffic space features are generated based on the feature vector sequence obtained from the last stage of processing.

[0016] Optionally, a multi-head attention mechanism is employed to adaptively fuse the temporal features of the traffic with the spatial features of the traffic, specifically including: The traffic temporal features are used as query vectors, and the traffic spatial features are used as key vectors and value vectors, respectively. These are then input into the multi-head attention module for cross-modal interactive computation to obtain the first attention output features. The traffic spatial features are used as query vectors, and the traffic temporal features are used as key vectors and value vectors, respectively. These are input into the multi-head attention module for cross-modal interactive calculation to obtain the second attention output features. The first attention output feature and the second attention feature are fused to generate the multimodal fusion feature.

[0017] Optionally, fusing the first attention output feature with the second attention output feature includes: Dynamic weights are generated through a gating fusion unit; the first attention output feature and the second attention output feature are weighted and summed using the dynamic weights to obtain the multimodal fusion feature.

[0018] Optionally, the process of preprocessing the raw network traffic data to obtain the model input data includes: Perform data cleaning and feature encoding on raw network traffic data; The encoded data is oversampled using the Borderline SMOTE algorithm. The process includes: identifying dangerous samples in the minority class and interpolating between neighboring samples of the dangerous samples to generate synthetic samples. Cluster labels generated based on clustering algorithms are used as meta-features and added to the oversampled data; After adding meta-features, principal component analysis is used to perform feature dimensionality reduction to obtain the model input data.

[0019] Compared with the prior art, the present invention has the following advantages and technical effects: 1. This invention innovatively introduces an image coding strategy to expand the representation of network traffic data and efficiently capture the local and global spatial features of traffic data.

[0020] 2. This invention designs a dual-channel architecture to achieve efficient extraction of network traffic temporal and spatial features, and integrates dynamic correlation information in the temporal dimension with topological distribution features in the spatial dimension to generate high-quality samples that are both representative and discriminative.

[0021] 3. This invention designs a multimodal fusion strategy based on an attention mechanism. By leveraging the multi-head attention mechanism, it adaptively matches the intrinsic correlation of heterogeneous spatiotemporal features of traffic, achieving efficient fusion and improving the attack traffic identification capability of intrusion detection methods.

[0022] 4. On two core benchmark datasets in the field of network intrusion detection, NSL-KDD and UNSW-NB15, the intrusion detection method of this invention achieves detection accuracies of 99.60% and 99.10%, precision of 99.25% and 98.24%, F1 scores of 99.27% ​​and 98.58%, and recall rates of 99.32% and 99.33%, respectively. This method demonstrates superior detection performance and generalization ability in complex network intrusion scenarios. Attached Figure Description

[0023] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is an architecture diagram of the intrusion detection method based on multimodal spatiotemporal feature fusion of network traffic according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the imbalanced network attack data balancing algorithm according to an embodiment of the present invention. Figure 3 This is an internal architecture diagram of GRU, the core algorithm for extracting network traffic temporal features according to an embodiment of the present invention. Figure 4 This is a diagram of the network traffic spatial feature extraction architecture based on Swing Transformer according to an embodiment of the present invention; Figure 5 This is a diagram illustrating the network traffic multimodal spatiotemporal feature fusion mechanism according to an embodiment of the present invention. Figure 6 This is a schematic diagram of the NSL-KDD multi-class attack confusion matrix, a network attack dataset according to an embodiment of the present invention. Figure 7 This is a schematic diagram of the UNSW-NB15 multi-class attack confusion matrix, a network attack dataset according to an embodiment of the present invention. Figure 8 This is a schematic diagram showing the distribution of attack identification precision, recall, and F1 score of different comparison methods in this embodiment of the invention on the network attack dataset NSL-KDD. Figure 9 This diagram illustrates the distribution of attack identification precision, recall, and F1 score for different comparison methods in this embodiment of the invention on the network attack dataset UNSW-NB15. Detailed Implementation

[0024] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0025] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0026] Example 1 like Figure 1As shown, this embodiment provides an intrusion detection method based on the fusion of multimodal spatiotemporal features of network traffic, including four core modules: a data preprocessing model, a network traffic image representation model, a dual-channel spatiotemporal feature extraction model, and a multimodal spatiotemporal feature fusion model.

[0027] Step 1: Construct a preprocessing model for network traffic data: 1) Data Cleaning: First, missing values ​​in the dataset are handled and duplicate samples are managed. For duplicate samples with the same label, only one record is retained to reduce data redundancy; for duplicate samples with conflicting labels, ensemble clustering is used to predict the most reasonable label, effectively reducing data noise. Second, the data types are downgraded from 64-bit integers (int64) and 64-bit floating-point numbers (float64) to 32-bit integers (int32) and 32-bit floating-point numbers (float32) respectively, to reduce memory usage and improve computational efficiency. Finally, numerical features are normalized to standardize the range of feature values ​​and avoid the model's bias towards features with large numerical magnitudes.

[0028] 2) Feature encoding: Perform one-hot encoding on non-numerical features to make them conform to the model input format; at the same time, convert the classification labels into numerical form to adapt to the needs of multi-classification tasks.

[0029] 3) Oversampling: Borderline SMOTE is used to oversample the dataset, generating new samples near the class boundaries to enhance the model's ability to learn the classification decision boundaries. Compared to the traditional SMOTE algorithm that generates samples globally in the feature space, Borderline SMOTE has more targeted sampling and can effectively reduce the risk of noise introduced by oversampling.

[0030] 4) Feature Enhancement: Integrate clustering results as meta-features into the dataset to expand the original feature dimensions. Clustering results, as auxiliary meta-features, can provide additional contextual information, thereby improving the model's learning ability and prediction robustness.

[0031] 5) Dimensionality Reduction: Principal Component Analysis (PCA) is used for feature dimensionality reduction. This step removes highly correlated or irrelevant features from the original data, reducing data noise and computational overhead during model training and inference.

[0032] 6) Dataset Partitioning: The preprocessed dataset is divided into a training set and a validation-test set in a ratio of 8.5:1.5 (the internal ratio can be further split according to needs). The training set is used for model parameter learning; the validation set supports hyperparameter tuning and overfitting suppression; and the test set is reserved for final performance evaluation. The class distribution of the training and test sets after preprocessing is shown in Tables 1 and 2.

[0033] Table 1 Table 2 Step 1.1: Construct an oversampling model based on Borderline SMOTE; Borderline SMOTE is an oversampling algorithm that improves upon the traditional SMOTE algorithm. Unlike traditional SMOTE, which uses all minority class samples to generate new data, this algorithm only selects minority class samples near the class boundary to synthesize new samples. Its core objective is to alleviate the class imbalance problem by optimizing the distribution of the imbalanced dataset.

[0034] like Figure 2 As shown, based on the inherent distribution characteristics of network traffic samples, minority class samples are selected according to the magnitude of the sample size. Then, K-nearest neighbor analysis is performed on each minority class sample to calculate the class distribution of its K nearest neighbors across the entire dataset. Finally, based on the proportion of majority class samples in the nearest neighbor set, the minority class samples are precisely divided into three categories: safe samples, boundary samples, and noise samples. Class A (safe samples): Minority class samples in which most of the k-nearest neighbors are minority class samples; Class B (dangerous samples): Minority samples in which more than half of the k-nearest neighbors belong to the majority class; Class C (noise samples): Minority samples whose k nearest neighbors are all majority class samples.

[0035] The SMOTE oversampling algorithm is applied to minority class traffic samples marked as dangerous samples. This targeted strategy avoids generating redundant samples from safe regions or invalid samples from noisy regions, thereby enhancing the model's ability to learn accurate classification boundaries.

[0036] Step 1.2: Construct a feature dimensionality reduction model based on PCA; Network attack traffic data is mostly high-dimensional datasets, often containing redundant features. This not only consumes significant computational resources but also increases the difficulty of data visualization. Therefore, appropriate dimensionality reduction is crucial for improving the data processing efficiency and performance of models. This invention employs Principal Component Analysis (PCA) for dimensionality reduction. Its core objective is to project high-dimensional data into a low-dimensional space through linear transformation, while preserving the main variability and key information of the data.

[0037] Implementation steps of PCA-based feature dimensionality reduction model: 1) Data standardization; Before performing PCA, data standardization is typically required to reduce the influence of different feature scales. This is because PCA calculations rely on variance and covariance matrices; features with larger magnitudes often have higher variances, which can lead to biases in the PCA results. The standardization formula is as follows: ; in, The original data, The mean of the data. This represents the standard deviation of the data.

[0038] 2) Calculation of the covariance matrix; Standardized data can be represented as a matrix, where each row corresponds to a sample and each column corresponds to a feature. The core of PCA is to analyze the correlation between features, which can be characterized by a covariance matrix (as shown in the following formula), which quantifies the linear dependence between different features.

[0039] ; in, is the sample size, and is the standardized data matrix. for The transpose of .

[0040] 3) Calculation of eigenvalues ​​and eigenvectors; Due to the covariance matrix Since it is a symmetric matrix, its eigenvalues ​​and corresponding eigenvectors can be obtained by solving the eigenvalue equation shown in the following formula.

[0041] ; in, For feature vectors, These are the eigenvalues.

[0042] 4) Principal component selection; Principal components are selected based on the magnitude of their eigenvalues, typically choosing the eigenvectors with the largest eigenvalues ​​as the new coordinate axes. This study experimentally verified that the model performance is optimal when the first 20 principal components are selected; therefore, the final dimensionality reduction dimension is set to 20.

[0043] 5) Project the data into the new space; The selected principal components form a projection matrix, which can be used to reduce the dimensionality of the original high-dimensional data while preserving the main variability of the data.

[0044] ; in, The data matrix after dimensionality reduction. It is the projection matrix composed of the first 20 principal components (eigenvectors).

[0045] Step 2: Construct a network traffic visualization model: Step 2.1: Construct an image representation model based on Gram angle field; This invention employs Gramian Angular Field (GAF) technology to convert one-dimensional network traffic data into a two-dimensional image. The converted image retains the temporal dependencies and potential correlations of the data while exhibiting high sparsity, effectively eliminating redundant information between multiple modalities. Furthermore, the structured image data facilitates subsequent extraction of spatial features.

[0046] Gram angle fields have two implementations: Gram sum angle field (GASF) and Gram difference angle field (GADF). GASF captures the cosine of the sum of polar angles between time points, while GADF characterizes the local dynamics of the data by calculating the sine of the difference of polar angles.

[0047] Implementation steps of the image representation model based on Gram angle field: 1) Normalize the traffic data to the [0,1] interval, using the following formula: ; in, .

[0048] 2) Calculate the corresponding polar angle based on the normalized data, using the following formula: ; in, .

[0049] 3) Construct the Gram matrix elements for GASF and GADF respectively using the following formulas: ; ; in, .

[0050] Step 2.2: Construct an image representation model based on Markov transfer fields; This invention utilizes Markov Transition Field (MTF) technology to transform the target flow sequence Convert to the corresponding pixel matrix.

[0051] Implementation steps of the image representation model based on Markov transfer fields: 1) Define the state space: The state space is the set of all non-repeating elements in the target sequence, such as the set {A,B,C}.

[0052] 2) Constructing the transition matrix: Based on the transition frequencies of adjacent elements in the target sequence, the transition probabilities between states are recorded by statistical counting, and a transition matrix is ​​constructed. Specifically, if in the target sequence, state... Subsequent state The number of times is 3, followed by the state. If the number of times is 2, then the transition probability is... and To generate a state sequence The first element or a random element of the original sequence can be selected as the initial state, and then the transition matrix is ​​used to determine the initial state. Randomly select the next state and repeat this process until a state sequence of the target length is generated.

[0053] 3) Determine image dimensions: Set the image width With height , so that the product of the two ( ) and the generated state sequence The lengths are consistent.

[0054] 4) State mapping to pixel values: Generate a sequence Each state in the matrix is ​​mapped to a corresponding pixel value, ultimately forming a pixel matrix. .

[0055] Using the three methods mentioned above (GASF, GADF, MTF), the original one-dimensional flow data can be converted into three different pixel matrices. , and These three matrices serve as the red (R), green (G), and blue (B) channels of an RGB image, respectively, thus converting one-dimensional time-series data into three-dimensional image data.

[0056] Step 3: Construct a dual-channel network traffic spatiotemporal feature extraction model: To automatically learn the temporal and spatial characteristics of network traffic using deep learning technology, this invention designs a dual-channel network traffic spatiotemporal feature extraction model based on GRU and Swin Transformer.

[0057] Step 3.1: Construct a GRU-based temporal feature extraction model; This invention employs a gated recurrent unit (GRU) to construct a temporal feature extraction model, aiming to extract the temporal dependencies of traffic data while preserving the original information to the greatest extent possible. As an improved architecture of recurrent neural networks (RNNs), GRU introduces update and reset gates to control information flow and memory updates, effectively alleviating the gradient vanishing problem inherent in traditional RNNs. Compared to Long Short-Term Memory (LSTM) networks, GRU has a simpler network architecture, fewer trainable parameters, and lower computational and memory overhead. This structural advantage not only accelerates model training but also improves the model's generalization ability, making it particularly suitable for training tasks on small-scale datasets.

[0058] The internal structure of GRU is as follows Figure 3 As shown. Among them, This represents the input information at the current time step. This is the hidden state of the previous time step (containing observation information from all historical nodes). To pass on the hidden state to the next time step, In the candidate hidden state, To reset the door, To update the door. In addition, This represents the Sigmoid activation function, which maps data to the [0,1] interval; The hyperbolic tangent activation function can adjust the data to the [-1, 1] interval. The core calculation formula of GRU is as follows: ; ; ; ; Step 3.2: Construct a network flow spatial feature extraction model based on Swing Transformer; This invention employs the Swin Transformer to construct a spatial feature extraction model. This module, leveraging a local-global attention mechanism, efficiently captures local and global information as well as long-range dependencies in an image, thereby improving the model's classification performance. The overall architecture of the spatial feature extraction method is as follows: Figure 4 As shown.

[0059] Implementation steps of the network flow spatial feature extraction model based on Swing Transformer: 1) Patch Partition; The input data is a three-channel RGB image generated using the Gram-angle field (GAF) and Markov transfer field (MTF) methods. This step first employs a block-based strategy, dividing the image into blocks of size [size missing]. The RGB image is divided into Non-overlapping image blocks. An image patch is treated as an image patch feature vector (patch token), and a total of [number] are generated. The feature vectors constitute the effective input length of the Transformer model.

[0060] Specifically, if you set the image patch size Number of channels Then the feature dimension of each flattened image patch is 48. At this point, the entire image will be divided into... Feature vectors of image patches.

[0061] 2) Stage 1; The first stage consists of a linear embedding layer and two Swin Transformer blocks. The linear embedding layer is essentially a fully connected layer, and its function is to transform the dimension into a fully connected layer. The tensor mapping is of dimension The feature representations of these image patches are then fed into multiple improved Swin Transformer blocks, which optimize and upgrade the traditional attention mechanism. It is important to note that the number of input and output feature vectors in each Swin Transformer block remains constant. .

[0062] 3) Stages 2-4; The second stage consists of an image patch merging layer and two Swin Transformer blocks. The purpose of introducing the image patch merging layer is to gradually decrease the number of feature vectors as the network depth increases. In the image patch merging layer, adjacent... The image patches are merged, reducing the number of feature vectors in each patch to one-quarter of their original size while expanding the dimension of the feature tensor to four times its initial dimension. Furthermore, the model uses a linear layer to reduce the dimensionality of the merged 4C-dimensional image patch features to 2C-dimensionality. After the image patch merging layer, the Swin Transformer block further transforms the features while maintaining the feature resolution. The image patch merging layer in this stage, together with the subsequent SwinTransformer block, constitutes the second stage. Repeating the second stage twice yields the third and fourth stages, with the output feature resolution decreasing sequentially. and .

[0063] 4)Swin Transformer Block (Swin Transformer Block); Compared to the standard Transformer, the core improvement of the Swin Transformer lies in replacing the original standard Multi-Head Self-Attention Module (MSA) with a Shift-Window Multi-Head Self-Attention Module (SW-MSA), while the remaining components remain unchanged. Figure 4 As shown, a Swing Transformer block contains a shift-window multi-head self-attention module, a multilayer perceptron (MLP) module, and auxiliary components such as layer normalization (LN) and residual connections. Specifically, layer normalization preprocessing is performed before the inputs to the multi-head self-attention module and the multilayer perceptron module; and residual connections are added after the output of each module to promote the effective propagation of features.

[0064] Step 4: Construct a multimodal spatiotemporal feature fusion model for network traffic; The model extracts highly discriminative feature representations from one-dimensional temporal sequences and transformed RGB images through parallel GRU temporal feature branches and Swing Transformer spatial feature branches, respectively. However, simple feature concatenation or weighted summation operations cannot fully capture the complex complementary relationships between these two heterogeneous modalities.

[0065] To achieve adaptive calibration and fusion of cross-modal features, this invention designs a fusion model based on a multi-head attention mechanism, the implementation steps of which are as follows: Figure 5 As shown: 1) The model uses temporal traffic features as the "query vector" in the attention mechanism, and spatial traffic features as the "key vector" and "value vector," respectively, to achieve cross-modal attention interaction. The design goal of this model is to mine feature patterns that are crucial for image region classification from long-distance temporal dependencies.

[0066] 2) A gated fusion unit is introduced to adaptively control the contribution weights of the two attention output features to the final fused feature. Notably, this gating mechanism includes a fully connected layer with a sigmoid activation function, generating a gated weight with a value between 0 and 1. This weight dynamically adjusts the degree to which the original temporal information is preserved in the final fused feature representation.

[0067] 3) The multimodal spatiotemporal fusion features of the traffic obtained above will be input into the fully connected layer to complete the classification task of intrusion behavior.

[0068] This design endows the model with powerful capabilities, enabling it to dynamically focus on and integrate highly discriminative information from different modalities, thereby significantly improving its ability to identify complex and covert attacks.

[0069] In this embodiment, the correctness and effectiveness of the intrusion detection model proposed in this invention are verified through experimental simulation. All experiments of this invention were conducted on a workstation running the Ubuntu 22.04 operating system. The workstation's hardware configuration was: NVIDIA GeForce RTX 3090 graphics card (48 GB VRAM) and 220 GB RAM. The experiments were conducted using the Python language.

[0070] To verify the performance and generalization ability of the intrusion detection model proposed in this invention, this experiment selected two core benchmark datasets in the field of network intrusion detection—NSL-KDD and UNSW-NB15. Table 3 shows the distribution details of the two attack categories.

[0071] The NSL-KDD dataset is an optimized version of the classic KDD99 dataset, released by the University of New Brunswick, Canada. The original traffic comes from the DARPA Network Traffic Collection Project in 1998. The dataset contains 41 features, covering three main categories: basic connectivity features, content features, and traffic statistics features. The attack types focus on traditional network threats, specifically divided into four categories: Denial-of-Service (DoS) attacks, Probe attacks, Remote-to-Local (R2L) attacks, and User-to-Root (U2R) attacks.

[0072] The UNSW-NB15 dataset was developed by the Australian Cyber ​​Security Centre (ACCS) Network Range Lab. The original data is a fusion of real modern network traffic and synthetic attack traffic generated by the IXIA PerfectStorm tool. The dataset contains 49 features, covering three major categories: flow features, packet features, and deep statistical features. The attack types are aligned with modern network threat scenarios, specifically covering 9 categories: fuzzing, vulnerability analysis, backdoors, denial-of-service (DoS) attacks, vulnerability exploitation, generalized attacks, reconnaissance, shellcode, and worms.

[0073] Table 3 1. Model performance evaluation metrics; This invention employs multiple metrics, including accuracy, precision, recall, F1 score, false alarm rate (FAR), and confusion matrix, to comprehensively evaluate the performance of the designed intrusion detection method. All of these metrics are calculated based on the confusion matrix, the definition of which is shown in Table 4.

[0074] Table 4 Among them, true positive (TP) refers to a sample that is actually positive and is correctly predicted; true negative (TN) refers to a sample that is actually negative and is correctly predicted; false positive (FP) refers to a sample that is actually negative but is incorrectly predicted as positive; and false negative (FN) refers to a sample that is actually positive but is incorrectly predicted as negative.

[0075] The calculation formulas for each evaluation indicator are as follows: Accuracy: The proportion of correctly classified samples out of the total sample. ; Precision: Represents the percentage of actual positive samples out of the results predicted as positive. ; Recall: The proportion of actual positive samples that are correctly predicted; ; F1 score: The harmonic mean of precision and recall, used to comprehensively evaluate model performance; ; False positive rate: This represents the proportion of actual negative samples that are incorrectly predicted as positive samples. The lower the value, the better the model performance.

[0076] 2. Hyperparameter settings; The key hyperparameter settings for the intrusion detection method designed in this invention are as follows: - GRU branch: The number of hidden layer neurons is 64, and the output layer dimension is 64; - Swin Transformer branch: Output layer dimension is 128; - Multimodal attention mechanism: The number of attention heads is set to 4; - Training configuration: Using the AdamW optimizer, the learning rate is set to... At the same time, an early stopping mechanism is introduced to dynamically determine the model's convergence status and shorten the training time.

[0077] 3. Experimental Results and Analysis; The performance of the proposed intrusion detection method is verified on two core benchmark datasets in the field of network intrusion detection: NSL-KDD and UNSW-NB15.

[0078] NSL-KDD dataset experimental results: The test set contains a total of 18,606 samples, including 11,514 normal traffic samples and 7,092 intrusion samples. Figure 6 As shown, the model correctly identified 11,469 normal samples and 6,738 intrusion samples. Specific performance metrics are shown in Table 5: accuracy 99.60%, precision 99.25%, recall 99.27%, F1 score 99.32%, and false positive rate 0.346%.

[0079] Table 5 Experimental results on the UNSW-NB15 dataset: The test set contains a total of 15,832 samples, including 12,818 normal traffic samples and 3,014 intrusion samples. For example... Figure 7 As shown, the model correctly identified 12,806 normal samples and 2,931 intrusion samples. Specific performance metrics are shown in Table 6: accuracy 99.10%, precision 98.24%, recall 98.58%, F1 score 99.33%, and false positive rate 0.152%.

[0080] Table 6 The model demonstrates excellent detection performance on both datasets, but detecting shellcode and exploit attacks in the UNSW-NB15 dataset remains challenging. These two types of attacks account for a relatively small percentage of the samples in the dataset and exhibit complex behavioral patterns. Even with oversampling strategies, the limited sample size restricts the model's generalization ability. Nevertheless, the intrusion detection method designed in this invention still achieves the best detection performance among current mainstream methods, validating the effectiveness of its multimodal spatiotemporal fusion.

[0081] 4. Comparative experimental analysis; To further verify the effectiveness of the intrusion detection method based on multimodal spatiotemporal feature fusion of network traffic designed in this invention, this invention conducts a comparative experiment with current mainstream intrusion detection methods. The method is quantitatively evaluated using four standard indicators: accuracy, precision, recall, and F1 score. The year of proposal of each comparative method is also noted to reflect the technological development process in this field.

[0082] Comparison results on the NSL-KDD dataset: All compared methods showed good performance on this dataset. As shown in Table 7, the proposed method outperformed other compared methods in all metrics, achieving an accuracy of 99.60%, a precision of 99.25%, a recall of 99.27%, and an F1 score of 99.32%. Furthermore, the XAI and CNN-BiLSTM methods also demonstrated strong competitiveness, achieving accuracies of 99.57% and 99.22%, respectively.

[0083] Table 7 Comparison results on the UNSW-NB15 dataset: As shown in Table 8, the proposed method also exhibits the best performance, with an accuracy of 99.10%, precision of 98.24%, recall of 98.58%, and F1 score of 99.33%. Furthermore, the LOEO, XAI, and ELM methods also demonstrate good performance, with accuracies of 97.60%, 99.08%, and 94.30%, respectively.

[0084] Table 8 Furthermore, this invention selects three core metrics—precision, recall, and F1 score—to visually compare the performance of the proposed method with mainstream methods (see...). Figure 8 , Figure 9 The results show that the overall performance of the proposed method is superior to that of published methods. Although the precision of the XAI method is slightly higher than that of the proposed method on the NSL-KDD dataset, the proposed method has significant advantages in both recall and F1 score. On the UNSW-NB15 dataset, the proposed method outperforms other comparative models in all metrics, further verifying its robustness in intrusion detection tasks and demonstrating its high application value in practical network intrusion detection tasks.

[0085] 5. Ablation test; This invention includes an ablation experiment to investigate the effects of individual and fused multimodal spatiotemporal features on intrusion detection performance. To verify the effectiveness of the dual-channel multimodal architecture designed in this invention, three comparative models were designed for the experiment: 1) Model 1 extracts global spatial features only from the image representation of network traffic; 2) Model 2 captures local temporal dependencies only from one-dimensional raw sequence data; 3) Model 3 integrates spatial and temporal feature branches, but does not introduce an attention fusion mechanism.

[0086] Table 9 Table 10 The ablation experiment results are shown in Tables 9 and 10. The results demonstrate the significant limitations of single-modal features: although Model 1 outperforms Model 2 due to its ability to capture higher-level spatial abstract features, Model 3 significantly outperforms both, validating the advantages of multimodal spatiotemporal feature fusion. This dual-channel architecture effectively combines temporal dynamic features with spatial semantic features, enabling a more comprehensive representation of network traffic data. Furthermore, the introduction of an attention mechanism in Model 3 further improves model performance, proving its crucial role in adaptive feature weighting and fusion. Notably, employing two different input modalities—one-dimensional sequences and three-dimensional Gram-square field images—is key to the model's ability to perceive and learn complex attack patterns from multiple perspectives.

[0087] The experimental results also show that there is a bottleneck in the detection capability of single-modal features: Model 1, which mainly captures spatial features, performs better overall than Model 2, which mainly captures temporal features; while Model 3, with its dual-channel architecture, combines the advantages of both and can extract richer data features, with both detection performance and robustness being better than the two single-modal models.

[0088] Furthermore, experimental results further validate the effectiveness of the attention-based multimodal spatiotemporal feature fusion strategy—this mechanism can significantly improve model performance. Notably, the intrusion detection method based on network traffic multimodal spatiotemporal feature fusion designed in this invention employs two heterogeneous modal inputs: one-dimensional sequence data and three-dimensional image data, providing multi-perspective information dimensions and enabling the model to learn more comprehensive feature representations.

[0089] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An intrusion detection method based on the fusion of multimodal spatiotemporal features of network traffic, characterized in that, Includes the following steps: The raw network traffic data to be detected is acquired and preprocessed to obtain model input data. The model input data is then visualized to generate network traffic image data. Parallel feature extraction is achieved through a dual-channel spatiotemporal feature extraction model; The network traffic image data is processed by a time-series feature extraction channel based on a gated loop unit to obtain traffic time-series features; the network traffic image data is processed by a spatial feature extraction channel based on a Swing Transformer to obtain traffic spatial features. A multi-head attention mechanism is used to adaptively fuse the traffic temporal features and the traffic spatial features to obtain multimodal fusion features; Based on the multimodal fusion features, intrusion detection classification is completed.

2. The intrusion detection method based on multimodal spatiotemporal feature fusion of network traffic according to claim 1, characterized in that, The process of representing the model input data graphically includes: The one-dimensional model input data is converted into Gram angle field image data using Gram angle field; the network traffic data is converted into Markov transition field image data using Markov transition field; the Gram angle field image data and the Markov transition field image data are combined to generate the network traffic image data.

3. The intrusion detection method based on multimodal spatiotemporal feature fusion of network traffic according to claim 2, characterized in that, The step of using Gram angle field technology to convert the one-dimensional model input data into Gram angle field image data includes: generating Gram angle and field images and Gram angle difference field images of the network traffic data respectively.

4. The intrusion detection method based on multimodal spatiotemporal feature fusion of network traffic according to claim 3, characterized in that, Combining the Gram angle field image data with the Markov transition field image data to form the network traffic image data includes: using the Gram angle sum field, Gram angle difference field, and Markov transition field image as the red, green, and blue channels of an RGB image, respectively, and fusing them to generate the three-dimensional network traffic image data.

5. The intrusion detection method based on multimodal spatiotemporal feature fusion of network traffic according to claim 1, characterized in that, The network traffic image data is processed using a time-series feature extraction channel based on a gated loop unit, including: The network traffic image data is input into a gated recurrent unit network; through the update gating mechanism and reset gating mechanism of the gated recurrent unit network, the fusion state of historical information and current input is adjusted to extract the temporal dependency relationship in the network traffic data and output the traffic temporal features.

6. The intrusion detection method based on multimodal spatiotemporal feature fusion of network traffic according to claim 1, characterized in that, The network traffic image data is processed based on the spatial feature extraction channel of the Swing Transformer, including: The network traffic image data is divided into multiple non-overlapping image blocks, and each image block is converted into an image block feature vector; The image patch feature vectors are projected into a high-dimensional feature space through a linear embedding layer to obtain an initial feature vector sequence; The initial feature vector sequence is input into a hierarchical feature extraction structure containing multiple levels for multi-level refinement. Each level of processing includes: after feature fusion of the current level's feature vector sequence through a multi-head self-attention module based on a shift window, adjacent feature vectors in the fused feature vector sequence are merged through an image patch merging operation, and the output is the feature vector sequence that serves as the input for the next level. The traffic space features are generated based on the feature vector sequence obtained from the last stage of processing.

7. The intrusion detection method based on multimodal spatiotemporal feature fusion of network traffic according to claim 1, characterized in that, A multi-head attention mechanism is employed to adaptively fuse the temporal features of traffic with the spatial features of traffic, specifically including: The traffic temporal features are used as query vectors, and the traffic spatial features are used as key vectors and value vectors, respectively. These are then input into the multi-head attention module for cross-modal interactive computation to obtain the first attention output features. The traffic spatial features are used as query vectors, and the traffic temporal features are used as key vectors and value vectors, respectively. These are input into the multi-head attention module for cross-modal interactive calculation to obtain the second attention output features. The first attention output feature and the second attention feature are fused to generate the multimodal fusion feature.

8. The intrusion detection method based on multimodal spatiotemporal feature fusion of network traffic according to claim 7, characterized in that, The first attention output feature and the second attention output feature are fused together, including: Dynamic weights are generated through a gating fusion unit; the first attention output feature and the second attention output feature are weighted and summed using the dynamic weights to obtain the multimodal fusion feature.

9. The intrusion detection method based on multimodal spatiotemporal feature fusion of network traffic according to claim 1, characterized in that, The process of preprocessing raw network traffic data to obtain model input data includes: Perform data cleaning and feature encoding on raw network traffic data; The encoded data is oversampled using the Borderline SMOTE algorithm. The process includes: identifying dangerous samples in the minority class and interpolating between neighboring samples of the dangerous samples to generate synthetic samples. Cluster labels generated based on clustering algorithms are used as meta-features and added to the oversampled data; After adding meta-features, principal component analysis is used to perform feature dimensionality reduction to obtain the model input data.

Images