Intrusion detection method and system based on multi-scale spatio-temporal feature residual fusion

Abstract

The application discloses an intrusion detection method and system based on multi-scale space-time feature residual fusion. The method comprises the following steps: acquiring network flow data to be detected; preprocessing the network flow data to be detected; inputting the preprocessed network flow data to be detected into a trained intrusion detection model to output an intrusion detection result and an intrusion detection type; wherein the trained intrusion detection model is used for multi-scale feature extraction on the network flow data to be detected, feature fusion on the multi-scale features, classification on the fused features, and obtaining of the intrusion detection result and the intrusion detection type.

Description

Technical Field

[0001] This invention relates to the field of network intrusion detection technology, and in particular to an intrusion detection method and system based on multi-scale spatiotemporal feature residual fusion. Background Technology

[0002] The statements in this section merely refer to the background art related to this invention and do not necessarily constitute prior art.

[0003] In today's society, with the advancement of science and technology, the internet has become increasingly integrated into people's lives and work. However, with the development of the network, its environment has also become increasingly complex, and various network attack methods have emerged one after another. The diversification of attack methods and the high frequency of attacks have led to increasingly frequent global network security breaches, seriously endangering the information security of individuals, enterprises, and nations. Intrusion detection technology, as one of the proactive security defense technologies that enables network traffic to propagate securely, plays an important role in today's network security. Traditional machine learning-based intrusion detection models have not achieved ideal accuracy and have a high false positive rate. Furthermore, with the development of the internet, today's network traffic is increasingly massive, and intrusion data has become more voluminous and complex. The operation of intrusion detection models requires processing and feature extraction of large amounts of traffic data, which traditional machine learning cannot handle well.

[0004] With the rise of supercomputing, the computing power of computers has been greatly improved in recent years, and deep learning has also been widely used. Improving the accuracy of malicious traffic detection and efficiently distinguishing different types of malicious traffic have become urgent priorities. In network intrusion detection, the amount of normal traffic data far exceeds that of abnormal data. Improper handling of imbalanced data can lead to high false negative rates and low recall rates. Traditional CNN structures primarily train on the spatial local features of the data, and the selection of convolutional kernels is crucial. However, since the original CNN structure only has a single convolutional kernel of a fixed size and CNNs are better suited for extracting spatial features, obtaining more accurate global features requires training the data with convolutional kernels of various sizes and improving the ability to extract temporal features. Current network threats are increasingly covert, with key intrusion behaviors often hidden within many normal data packets. Malicious traffic and the number of threat detection samples that can be captured are relatively small, easily leading to a short learning process and insufficient training of the model. This results in traditional network intrusion detection methods having low accuracy and high false alarm rates. Furthermore, simply adding more network layers may cause the task to fail, and after reaching a certain threshold, performance may even degrade. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides an intrusion detection method and system based on multi-scale spatiotemporal feature residual fusion;

[0006] On the one hand, an intrusion detection method based on multi-scale spatiotemporal feature residual fusion is provided, including:

[0007] Acquire network traffic data to be detected;

[0008] Preprocess the network traffic data to be detected;

[0009] The preprocessed network traffic data to be detected is input into the trained intrusion detection model, and the intrusion detection results and intrusion detection type are output.

[0010] The trained intrusion detection model is used to extract multi-scale features from the network traffic data to be detected, fuse the multi-scale features, classify the fused features, and obtain the intrusion detection results and intrusion detection type.

[0011] On the other hand, an intrusion detection system based on multi-scale spatiotemporal feature residual fusion is provided, including...

[0012] The acquisition module is configured to acquire network traffic data to be detected.

[0013] The preprocessing module is configured to preprocess the network traffic data to be detected.

[0014] The intrusion detection module is configured to: input preprocessed network traffic data to be detected into the trained intrusion detection model, and output the intrusion detection result and the intrusion detection type;

[0015] The trained intrusion detection model is used to extract multi-scale features from the network traffic data to be detected, fuse the multi-scale features, classify the fused features, and obtain the intrusion detection results and intrusion detection type.

[0016] Furthermore, an electronic device is also provided, including:

[0017] Memory, used for non-transitory storage of computer-readable instructions; and

[0018] Processor, for executing the computer-readable instructions,

[0019] When the computer-readable instructions are executed by the processor, they perform the method described in the first aspect above.

[0020] In another aspect, a storage medium is also provided for non-transitory storage of computer-readable instructions, wherein when the non-transitory computer-readable instructions are executed by a computer, the instructions of the method described in the first aspect are executed.

[0021] In another aspect, a computer program product is also provided, including a computer program that, when run on one or more processors, is used to implement the method described in the first aspect above.

[0022] One of the above technical solutions has the following advantages or beneficial effects:

[0023] (1) Generative Adversarial Networks (GANs) have the ability to generate new samples that conform to the distribution characteristics of the original dataset. However, GANs suffer from the problem of balancing the quality of generated samples with training stability. Therefore, a method for imbalanced data classification oversampling based on an improved auxiliary classification generative adversarial network (ACGAN) is proposed. First, the self-attention (SA) algorithm is added to the generator of the ACGAN. The SA algorithm can weight different positions of the input, enabling the generator to better handle local details and global structure. The generator of the model is constructed using a residual network structure to accelerate the convergence speed of the training model. BiGRU is added to the generator of the model. Through the long-term dependency modeling ability of BiGRU, the generator of the ACGAN can better capture the temporal correlation between input noise and conditional vectors, thereby generating more coherent and reasonable samples. Using the improved ACGAN to fully learn the distribution of minority class data is beneficial to improving the accuracy of minority class classification.

[0024] (2) Because the traditional CNN convolutional model has a relatively simple structure, it cannot accurately capture the features of intrusive data. Therefore, this invention uses the Inception module with multiple convolutional layers to extract and fuse data features at multiple scales, improving the richness and completeness of the features extracted by the convolutional neural network, increasing the network width, and enabling the network to learn both sparse and non-sparse features simultaneously to obtain a strongly correlated feature set. Furthermore, a self-attention mechanism and a BiGRU model are introduced into the improved Inception module, which weights the extracted spatial features at different scales while also improving the ability to extract temporal features, thereby improving classification accuracy.

[0025] (3) Adding a residual network simplifies the Inception module through skip connections. This not only avoids adding extra parameters and computations to the network but also maintains the complexity of the original network. While ensuring performance, the network can learn deeper features. Therefore, the network converges faster and achieves higher classification accuracy.

[0026] (4) Since network intrusion has multi-dimensional and time-dependent characteristics, the BiGRU model is introduced to improve the overall model’s ability to extract temporal features of intrusion data, making feature extraction more comprehensive. That is, at the cost of a small time increase, higher accuracy is achieved, thus obtaining better results. Attached Figure Description

[0027] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0028] Figure 1 This is a flowchart illustrating the overall model of Embodiment 1 of this application;

[0029] Figure 2 This is a flowchart of the Inception-BiGRU-SA model in Embodiment 1 of this application;

[0030] Figure 3 This is a structural diagram of the ACGAN in Embodiment 1 of this application;

[0031] Figure 4 This is a structural diagram of the improved ACGAN generator according to Embodiment 1 of this application;

[0032] Figure 5 This is a structural diagram of the ACGAN discriminator in Embodiment 1 of this application;

[0033] Figure 6 This is a structural diagram of the Transformer-Encoder in Embodiment 1 of this application. Detailed Implementation

[0034] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0035] Example 1

[0036] This embodiment provides an intrusion detection method based on multi-scale spatiotemporal feature residual fusion;

[0037] Intrusion detection methods based on multi-scale spatiotemporal feature residual fusion include:

[0038] S101: Obtain network traffic data to be detected;

[0039] S102: Preprocess the network traffic data to be detected;

[0040] S103: Input the preprocessed network traffic data to be detected into the trained intrusion detection model, and output the intrusion detection result and the intrusion detection type;

[0041] The trained intrusion detection model is used to extract multi-scale features from the network traffic data to be detected, fuse the multi-scale features, classify the fused features, and obtain the intrusion detection results and intrusion detection type.

[0042] Furthermore, the trained intrusion detection model, the training process includes:

[0043] Construct a training set, which consists of network traffic data with known intrusion detection results and intrusion detection types;

[0044] The training set is input into the intrusion detection model to train the model. Training is stopped when the number of training iterations exceeds a set number, or when the loss function value no longer decreases. The trained intrusion detection model is then obtained.

[0045] Furthermore, the construction of the training set includes:

[0046] Obtain network traffic data with known intrusion detection results and intrusion detection types;

[0047] The network traffic data is converted to different data types, and then normalized.

[0048] For the normalized network traffic data, determine whether the number of samples corresponding to each label exceeds a set threshold. If it exceeds the set threshold, the sample corresponding to the current label is identified as a majority class sample. If it is below the set threshold, the sample corresponding to the current label is identified as a minority class sample, and the training set is obtained.

[0049] For example, the training set is implemented using the CIC-IDS-2017 dataset. The CIC-IDS-2017 dataset is used as input data for network intrusion detection, and machine learning algorithms are used to transform the data type of the input data, removing four features irrelevant to feature extraction: Flow ID, Source IP, Destination IP, and Timestamp. After deleting null values, the data is normalized using MinMaxScaler in sklearn, and the last column of the dataset is extracted as the label.

[0050] For example, the input data is divided into 80% training set and 20% test set, and the data is cleaned. The first 80 dimensions of each data set are feature values. To facilitate the conversion of the data into a two-dimensional format, the 81st dimension is padded with 0, and the 82nd dimension is the type label. The data is divided into 10 data types, and the labels are converted into int types from 0 to 9. The total data volume is 1,053,853.

[0051] Table 1. Composition of Training and Test Sets

[0052]

[0053]

[0054] Furthermore, the normalization process, assuming there are N samples in the dataset, can map each feature attribute of all samples to X = (X1, X2, X3, ..., X...). N ) T The min-max method is used to normalize all values ​​to the range [0,1]. The calculation formula is as follows:

[0055]

[0056] Among them, X min and X max These represent the minimum and maximum values ​​of all values ​​in a certain feature attribute, respectively.

[0057] Since the data is captured from a real network environment, the possibility of invalid data is very high. Invalid data is removed to improve the accuracy of the detection results. In addition, more than 80 features are extracted from the raw data, such as flow duration, total length of forward packets, and total length of backward packets, which are important features for intrusion detection in this invention's model. However, features such as IP address, protocol, and port number are also present, which are not needed. Therefore, this invention removes unnecessary feature fields, such as: Flow ID (unique credential of the sample), Source IP (source host IP address), Destination IP (destination IP), and Timestamp.

[0058] Furthermore, such as Figure 1 As shown, the network structure of the trained intrusion detection model includes:

[0059] The sequence consists of a Generative Adversarial Network (ACGAN) with an auxiliary classifier, a Transformer Encoder module, an Inception-BiGRU-SA module, a 1x1 convolutional layer, a global average pooling layer, a Dropout layer, and a Softmax activation function layer.

[0060] Furthermore, such as Figure 3 , Figure 4 and Figure 5 As shown, the distribution of minority class samples is learned through the Generative Adversarial Network (ACGAN) with an auxiliary classifier, and the generator of the Generative Adversarial Network (ACGAN) with an auxiliary classifier is used to upsample the minority class samples.

[0061] Furthermore, such as Figure 3 As shown, the Generative Adversarial Network (ACGAN) with an auxiliary classifier includes:

[0062] Generator and discriminator;

[0063] The generator's input is used to take real sample labels and random noise as input;

[0064] The generator's output is used to output fake data;

[0065] The discriminator's input is used to input real sample labels and fake data;

[0066] The discriminator outputs the probability value of whether the current input data is true or false.

[0067] The discriminator also outputs the label of the sample.

[0068] Furthermore, such as Figure 4 As shown, the generator includes a first fully connected layer, a first batch of normalization layers, a first activation function layer, a first deconvolution layer, a first attention mechanism layer SA, a second batch of normalization layers, a second activation function layer, a second deconvolution layer, a second attention mechanism layer SA, a first bidirectional gated recurrent unit (BiGRU), a third batch of normalization layers, a third activation function layer, and a third deconvolution layer, all connected in sequence.

[0069] Among them, the output of the first fully connected layer is also connected to the input of the second batch of normalization layers; the output of the first attention mechanism layer SA is also connected to the input of the first bidirectional gated recurrent unit BiGRU; the output of the second attention mechanism layer SA is also connected to the input of the third batch of normalization layers.

[0070] The first and second activation function layers use the ReLU activation function; the third activation function layer uses the tanh activation function.

[0071] Furthermore, such as Figure 5As shown, the discriminator includes: a fourth activation function layer, a convolutional layer C1, a first dropout layer, a fifth activation function layer, a convolutional layer C2, a second dropout layer, a sixth activation function layer, a convolutional layer C3, a third dropout layer, a second fully connected layer, and an output layer connected in sequence.

[0072] The fourth, fifth, and sixth activation function layers are implemented using LeakyReLU.

[0073] For example, in the generator of ACGAN, the deconvolutional layer uses batch normalization (BN) to accelerate model training and ensure training stability, while using the ReLU activation function, and the last deconvolutional layer uses the tanh activation function.

[0074] Random noise and labels are input into the generator, and then passed through the first and second deconvolutional layers with an attention mechanism (SA). The attention mechanism feature maps are used to enrich the feature details. The generator then passes through the first BiGRU layer to capture the temporal correlation between the input noise and the conditional vector, thereby generating coherent and reasonable samples. The generator is built using a residual network structure to accelerate the convergence speed of the training model. Finally, the generated samples are output after passing through a deconvolutional layer.

[0075] The discriminator in ACGAN is a convolutional neural network (CNN), which uses the LeakyReLU activation function with a negative slope of α = 0.2. Dropout layers are also added to prevent overfitting.

[0076] Understandably, this invention first performs data preprocessing on the dataset, and then proposes an imbalanced data classification oversampling method based on an improved Generative Adversarial Network for Classification (ACGAN). First, a self-attention (SA) algorithm is added to the ACGAN generator. The SA algorithm can weight different positions of the input, enabling the generator to better handle local details and global structure. A residual network structure is used to construct the model's generator, accelerating the convergence speed of the training model. BiGRU is added to the model's generator. Through BiGRU's long-term dependency modeling ability, the ACGAN generator can better capture the temporal correlation between input noise and conditional vectors, thus generating more coherent and reasonable samples. The improved ACGAN is used to fully learn the distribution of minority class data, and the generator upsamples the minority class data to construct a balanced dataset. This sample set has a more balanced distribution than the original data, which is beneficial for improving the accuracy of minority class classification.

[0077] Furthermore, the trained intrusion detection model's operation includes:

[0078] The Transformer's Encoder module captures global connections and performs preliminary feature extraction from the input data. The Inception-BiGRU-SA module extracts multi-scale features and uses a self-attention mechanism to weight the extracted spatial features at different scales. The combination of convolutional layers and global average pooling layers reduces the number of training parameters in the network. The Dropout layer reduces the coupling between neurons and reduces overfitting. The Softmax activation function classifies and detects network traffic data.

[0079] Furthermore, such as Figure 2 As shown, the Inception-BiGRU-SA module includes:

[0080] The three branches are arranged in parallel: the first branch, the second branch, and the third branch.

[0081] The first branch includes: a max pooling layer, a first convolutional layer, a fourth batch normalization layer, a seventh activation function layer, a second convolutional layer, a fifth batch normalization layer, and an eighth activation function layer connected in sequence;

[0082] The second branch includes: a third convolutional layer, a sixth batch normalization layer, a ninth activation function layer, a fourth convolutional layer, a seventh batch normalization layer, and a tenth activation function layer connected in sequence;

[0083] The third branch includes: a fifth convolutional layer, an eighth batch normalization layer, an eleventh activation function layer, a sixth convolutional layer, a ninth batch normalization layer, and a twelfth activation function layer connected in sequence;

[0084] The input terminals of the max pooling layer, the third convolutional layer, and the fifth convolutional layer are all connected to the input terminal of the Inception-BiGRU-SA module.

[0085] Among them, the output terminals of the eighth activation function layer, the tenth activation function layer, and the twelfth activation function layer are all connected to the input terminal of the third attention mechanism module SA;

[0086] The output of the third attention mechanism module SA is connected to the input of the second bidirectional gated loop unit BiGRU, and the output of the second bidirectional gated loop unit BiGRU is connected to the input of the first adder.

[0087] The input of the first adder is also connected to the input of the Inception-BiGRU-SA module;

[0088] The output of the first adder is the output of the Inception-BiGRU-SA module.

[0089] The convolution kernels of the first, third, and fifth convolutional layers are all 1*1.

[0090] The second convolutional layer has a 1*1 kernel; the fourth convolutional layer has a 3*1 kernel; and the sixth convolutional layer has a 5*1 kernel.

[0091] Furthermore, such as Figure 2 As shown, the Inception-BiGRU-SA module's operation includes:

[0092] The Inception-BiGRU-SA module deploys multiple convolutional kernels of different scales in the same layer of the neural network to capture features at different scales. Then, it assigns different weights to the features and performs forward and reverse feature extraction on the features with different weights to obtain the extracted features.

[0093] Capturing features at different scales means that the network can learn finer-grained and higher-level features simultaneously, thereby enhancing the network's perception capabilities.

[0094] To reduce the computational complexity of the network, 1×1 convolutional kernels and max pooling layers are used to reduce the dimensionality of the input high-dimensional data, thereby reducing the number of network parameters and computational requirements while maintaining high accuracy, so as to obtain richer local feature information.

[0095] The final output is then formed by concatenating 1×1, 3×1, and 5×1 convolutional kernels. This parallel structure of three branches allows the network to learn features at multiple scales simultaneously, thus improving the network model's expressive power. To improve training speed, 1×1 convolutions are used before convolution and after pooling to control the output feature dimension for feature fusion. To control gradient explosion and accelerate network training and convergence, a Batch Normalization (BN) layer is added after the convolutional layers, followed by ReLU activation to process the data and optimize feature distribution. To highlight the role of important features, an Alternating Values ​​(SA) module is added after the convolutional layers, assigning higher weights to important features to improve recognition accuracy.

[0096] The fused output data from multi-scale convolutions is input into a self-attention mechanism and finally into BiGRU. BiGRU processes the sequence vector from both the forward and reverse directions to ensure that the algorithm can obtain accumulated dependency information not only from the forward order but also from the reverse order, maintaining sensitivity to temporal features and further eliminating redundant information to enrich feature information. This is to obtain key features that are discriminative for network intrusion data and improve detection accuracy.

[0097] It should be understood that this invention improves the Inception structure by connecting 1×1, 3×1, and 5×1 convolutional kernels with pooling operations in parallel. To improve training speed, 1×1 convolutions are used before convolution and after pooling to control the output feature dimension for feature fusion. To control gradient explosion and accelerate network training and convergence, a batch normalization (BN) layer is added after the convolutional layer, followed by ReLU activation to process the data and optimize feature distribution. Two 1×1 convolutional kernels are added to maintain a consistent number of channels while simultaneously reducing dimensionality to achieve multi-feature extraction within the same network layer. A self-attention mechanism is introduced after the network backbone to weight the extracted spatial features at different scales to improve classification accuracy, and a BiGRU model is incorporated to enhance the Inception module's ability to extract global temporal features from network intrusion data.

[0098] It should be understood that, in order to obtain rich features at different scales and spatiotemporal features, a multi-scale convolutional neural network Inception network structure is first used to increase the width of the network, and convolutional kernels of different sizes are put into the Inception module. The convolutional kernels are used to improve the adaptability of the network and expand the receptive field of the module, thereby obtaining rich features at different scales and increasing the width of the feature tensor. While increasing the network depth and width, the number of parameters is reduced, thereby reducing the computational burden.

[0099] By incorporating self-attention mechanism and BiGRU into the Inception module, the ability to extract temporal features is improved. At the same time, the extracted spatial features of different sizes are weighted so that the model can focus on spatial features with high discriminative power.

[0100] The improved Inception model and BiGRU model are restructured by combining the residual network structure, and the self-attention mechanism is introduced into the Inception model to construct the Inception-BiGRU-SA residual model;

[0101] Before the final output, a combination of a convolutional layer and a global average pooling layer is used instead of a fully connected layer to reduce the number of training parameters in the network and avoid overfitting.

[0102] This embodiment proposes an intrusion detection method based on multi-scale spatiotemporal feature residual fusion of imbalanced data. First, the imbalanced dataset is effectively upsampled to construct a balanced dataset. Second, the residual structure composed of Transformer's Encoder module, Inception module with self-attention mechanism, and BiGRU is used to extract features from the preprocessed data. Finally, the Softmax activation function is used to detect and classify the data categories.

[0103] Then, before the final output, a combination of a convolutional layer and a global average pooling layer is used to replace the fully connected layer, reducing the number of training parameters in the network. A dropout layer is added to reduce the coupling between neurons and reduce overfitting. A softmax activation function is used to classify and detect network traffic data.

[0104] Auxiliary Classifier Generative Adversarial Network (ACGAN) is an extended model of Generative Adversarial Networks (GANs) used to generate fake data with specific attributes. ACGAN controls the generation process by simultaneously training a generator and a discriminator. The generator is responsible for generating virtual samples, while the discriminator distinguishes between generated and real samples. To further control the generation process, ACGAN introduces an auxiliary classifier. The auxiliary classifier performs additional attribute classification on the generated samples, helping the generator better understand and control the desired attributes. By simultaneously optimizing the generator, discriminator, and auxiliary classifier during training, ACGAN can progressively improve the quality of generated samples and its attribute control performance. In summary, ACGAN is a method that combines a generator, discriminator, and auxiliary classifier using a generative adversarial network to generate virtual samples with specific attributes. This method allows for a degree of control over the generation process, making the generated samples more consistent with the desired results.

[0105] Self-attention (SA) extracts information only from the input itself, without using any other additional information. It has advantages such as fewer parameters and faster computation speed, making it more suitable for practical engineering applications.

[0106] First, the input data passes through three linear layers to calculate the query matrix Q, key matrix K, and value matrix V. Then, the transposes of Q and K are multiplied, divided by a scaling factor, and then the self-attention weight matrix A is calculated using Softmax. Finally, V and weight matrix A are multiplied to obtain the self-attention weighted result.

[0107] The calculation process is expressed by the following formula:

[0108]

[0109] In the formula, d k It is a scaling factor, which controls the result of matrix multiplication to prevent it from becoming too large; attn(h) is the result of input h after self-attention weighting, that is, the output of the self-attention mechanism.

[0110] The basic idea of ​​a bidirectional GRU (BiGRU) neural network is to configure unidirectional GRUs in both the forward and backward directions of each training sequence, with both GRUs connected to the same output layer. This structure provides the output layer with both the past and future information of a sequence of data points. A BiGRU consists of two GRUs stacked on top of each other. At any time t, the input terminal simultaneously provides the same input data x to the two GRUs in opposite directions. i The output states of the forward GRU are obtained respectively. and the output state of the backward GRU Output GRU_O t State of the forward GRU and the output state of the backward GRU It is assembled according to specific criteria. The calculation process of the BiGRU neural network is as follows:

[0111] Assume the input at time step t is x i The activation function of the hidden layer is θ, and the forward hidden state is... Backward hidden state is So:

[0112]

[0113]

[0114] Among them, W fx W fh W bx W bh These represent the weight parameters, b and b respectively. fh b bh These represent the biases corresponding to the forward and backward hidden states, respectively. The output GRU_O of the output layer is then calculated from this. t :

[0115]

[0116] Among them, W q b represents the weight parameters of the output layer. q Indicates the bias of the output layer. This represents the concatenation of the forward and backward hidden states. By weighted concatenation of the forward and backward hidden states to calculate the final output, effective information can be learned while maintaining the same output dimension. Compared to directly concatenating the hidden states in the two directions, this reduces the dimensionality and helps to reduce training time.

[0117] For the ACGAN model, an SA attention mechanism module is added after the first and second deconvolution layers of its generator to extract global features of attack samples. A residual network structure is used to construct the generator, accelerating the convergence speed of the training model. BiGRU is added to the generator; its long-term dependency modeling ability allows the ACGAN generator to better capture the temporal correlation between input noise and conditional vectors, thus generating more coherent and reasonable samples. The generator uses Batch Normalization (BN) layers to accelerate model training and ensure training stability, and ReLU is used as the activation function. A Dropout layer is added to its discriminator to prevent overfitting.

[0118] The original Transformer model structure includes two parts: encoding and decoding. Due to the specific needs of the intrusion detection task and the fixed length of each data item in the dataset, this model only uses the encoding part of the Transformer and fine-tunes some of its parameters.

[0119] like Figure 6 As shown, the Transformer's Encoder module includes: an input layer, a multi-head attention mechanism layer, a second adder, a first normalization layer, a feedforward neural network layer, a third adder, a second normalization layer, and an output layer connected in sequence; wherein, the input layer is also connected to the input of the second adder, and the output of the first normalization layer is also connected to the input of the third adder.

[0120] The Transformer's Encoder module includes a multi-head attention mechanism layer and a feedforward neural network. The multi-head attention mechanism layer uses dot-product attention, meaning it takes three inputs: query, keys, and values. It calculates a weight score for each value using the query and keys, and then calculates a weighted sum with the values ​​to obtain the output. Dot-product attention allows for parallel computation, reducing training time. Its calculation formula is as follows:

[0121]

[0122] Where Q, K, and V represent the Query, Key, and Value matrices respectively, and d k This represents the dimension of the Key. Due to the characteristics of the input data in this invention, the Mask part in the original model is omitted. To enrich the extracted features, a multi-head attention structure is used. The calculation formula for multi-head attention is as follows:

[0123] Q i =QW i Q ,K i =KW iK V i =VW i V i = 1, 2, ..., n

[0124] head i =Attention(Q) i ,K i V i i = 1, 2, ..., n

[0125] MultiHead(Q,K,V)=Concat(head1,...,head n W O

[0126] The feedforward neural network part is a perceptron with only one hidden layer, and its input and output dimensions are the same. This part uses residual connections to prevent the gradient vanishing problem.

[0127] Inception is characterized by using dense structures to approximate sparse networks, while ResNet can solve the degradation problem of deep networks. Therefore, combining Inception and ResNet can construct networks that are as sparse as possible, while improving network stacking effects and avoiding degradation problems.

[0128] Traditional convolutional neural networks (CNNs) often focus on increasing network depth. However, this can lead to excessive training parameters and overfitting. Inception, on the other hand, widens the network by adding horizontal network structures. Inception performs parallel feature extraction on the input feature map and concatenates all outputs into a new feature map. During parallel feature extraction, the kernel size varies in each layer, resulting in the extraction of different features. This invention improves the Inception structure by connecting 1×1, 3×1, and 5×1 convolutional kernels in parallel with pooling operations. To improve training speed, 1×1 convolutions are used before convolution and after pooling to control the output feature dimension for feature fusion. To control gradient explosion and accelerate training and convergence, a Batch Normalization (BN) layer is added after the convolutional layers, followed by ReLU activation to process the data and optimize feature distribution. One-dimensional convolution is more suitable for sequence processing. Using high-dimensional convolution requires converting the sequence into a high-dimensional vector, which can potentially break down continuous data into two unrelated parts, destroying the original information. Therefore, this invention uses one-dimensional convolution to extract the spatial features of traffic flow, which does not destroy the original relative position of traffic flow data and avoids the impact of information loss on the model's recognition accuracy.

[0129] Batch-normalization (BN) adjusts the input value distribution of neurons in each layer of a neural network to a standard normal distribution with a mean of 0 and a variance of 1. This addresses the difficulties in network training, effectively improving training speed, preventing gradient vanishing, and enhancing generalization ability. The output of batch-normalization is expressed as:

[0130] y i =λx′ i +η

[0131] Where: λ and η are coefficients; x′ i The calculation process is as follows:

[0132]

[0133] Where m is the batch size, x i It is a time series.

[0134] The basic idea of ​​a bidirectional GRU (BiGRU) neural network is to configure unidirectional GRUs in both the forward and backward directions of each training sequence, with both GRUs connected to the same output layer. This structure provides the output layer with both the past and future information of a sequence of data points. A BiGRU consists of two GRUs stacked on top of each other. At any time t, the input terminal simultaneously provides the same input data x to the two GRUs in opposite directions. i The output states of the forward GRU are obtained respectively. and the output state of the backward GRU Output GRU_O t State of the forward GRU and the output state of the backward GRU It is assembled according to specific criteria. The calculation process of the BiGRU neural network is as follows:

[0135] Assume the input at time step t is x i The activation function of the hidden layer is θ, and the forward hidden state is... Backward hidden state is So:

[0136]

[0137]

[0138] Among them, W fx W fh W bx W bh These represent the weight parameters, b and b respectively. fh b bhThese represent the biases corresponding to the forward and backward hidden states, respectively. The output GRU_O of the output layer is then calculated from this. t :

[0139]

[0140] Among them, W q b represents the weight parameters of the output layer. q Indicates the bias of the output layer. This represents the concatenation of the forward and backward hidden states. By weighted concatenation of the forward and backward hidden states to calculate the final output, effective information can be learned while maintaining the same output dimension. Compared to directly concatenating the hidden states in the two directions, this reduces the dimensionality and helps to reduce training time.

[0141] Network traffic anomaly detection cannot rely solely on discrete local features. Instead, it should extract network traffic features at different scales using convolutional kernels of varying scales. Based on the multi-scale feature fusion concept of the Inception structure, this invention uses multi-scale one-dimensional convolutional layers to extract local spatial features from multi-scale network traffic data. The convolutional kernel lengths (i.e., feature extraction scales) used are 1, 3, and 5, and the calculation formulas for the three branches are as follows:

[0142] x out1 =δ(xW1+b1)

[0143] x out2 =δ(xW2+b2)

[0144] x out3 =δ(xW3+b3)

[0145] Among them, W 1∈ P k1 W 2∈ P k2 W 3∈ P k3 Given k1=1, k2=3, k3=5, the activation function is the ReLU function. Feature fusion is achieved through term-by-term addition, i.e., the ADD function, as shown in the following formula:

[0146] x add =x out1 +x out2 +x out3

[0147] If the Concatenate function is used for feature fusion by merging, the information content of a single element remains unchanged, but the feature dimension increases. If the ADD function is used for feature fusion by adding each element sequentially, the information content of a single element increases, but the feature dimension remains unchanged. Since the output dimension is consistent, to reduce the number of subsequent computational parameters, this invention uses the ADD function instead of the Concatenate function for feature fusion.

[0148] By using a batch normalization layer, the output is made to satisfy or approximately follow a normal distribution, thereby accelerating the convergence speed of the model and preventing the gradient vanishing phenomenon.

[0149] The output is fed into a self-attention mechanism, and then into a BiGRU. BiGRU processes the sequence vector in both forward and reverse directions to ensure the algorithm obtains accumulated dependency information not only from the forward sequence but also from the reverse sequence, maintaining sensitivity to temporal features and further eliminating redundant information to enrich feature information. This enhances the detection accuracy by obtaining key features that are discriminative against network intrusion data. A residual structure is added to this module, incorporating identity mappings to achieve residual fitting. Before the final output, a combination of a 1×1 convolutional layer and a global average pooling layer replaces the fully connected layer, reducing the number of training parameters in the network. A Dropout layer with a parameter of 0.5 is added to reduce the coupling between neurons and minimize overfitting.

[0150] The softmax activation function is used to classify and detect network traffic data. The model is trained using a training set and validated using a test set. Accuracy (AC), Precision (PR), and F1-score (F1) are used as evaluation metrics to assess the intrusion detection model's performance, calculated as follows:

[0151]

[0152]

[0153]

[0154] Among them, TP, FP, TN and FN are true positive, false positive, true negative and false negative, respectively, and their physical meanings are shown in Table 2.

[0155] Table 2 Physical meanings of several terms

[0156]

[0157]

[0158] The so-called 10-category experiment divides the CIC-IDS-2017 data into normal samples and nine other attack samples. The nine attack samples are labeled as follows: Dos Hulk (network overload attack for denial-of-service attacks), PortScan (port scanning), DDoS (distributed denial-of-service attacks), Dos GoldenEye (GoldenEye attack for denial-of-service attacks), FTP-Patator (brute-force tool for file transfer protocols), SSH-Parator (brute-force tool for network security protocols), Dos slowloris (resource consumption tool for denial-of-service attacks), Dos Slowhttptest (website stress testing tool for denial-of-service attacks), and Bot (automated attack). The detection results of different models are compared in Table 3.

[0159] Table 3 Comparison of detection results of different models

[0160]

[0161] Example 2

[0162] This embodiment provides an intrusion detection system based on multi-scale spatiotemporal feature residual fusion;

[0163] An intrusion detection system based on multi-scale spatiotemporal feature residual fusion, including

[0164] The acquisition module is configured to acquire network traffic data to be detected.

[0165] The preprocessing module is configured to preprocess the network traffic data to be detected.

[0166] The intrusion detection module is configured to: input preprocessed network traffic data to be detected into the trained intrusion detection model, and output the intrusion detection result and the intrusion detection type;

[0167] The trained intrusion detection model is used to extract multi-scale features from the network traffic data to be detected, fuse the multi-scale features, classify the fused features, and obtain the intrusion detection results and intrusion detection type.

[0168] The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1 above. It should be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.

[0169] The descriptions of each embodiment in the above embodiments have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0170] The proposed system can be implemented in other ways. For example, the system embodiments described above are merely illustrative, and the division of modules described above is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed.

[0171] Example 3

[0172] This embodiment also provides an electronic device, including: one or more processors, one or more memories, and one or more computer programs; wherein, the processor is connected to the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory to cause the electronic device to perform the method described in Embodiment 1.

[0173] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0174] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.

[0175] In the implementation process, each step of the above method can be completed by the integrated logic circuits in the processor hardware or by software instructions.

[0176] The method in Embodiment 1 can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.

[0177] Those skilled in the art will recognize that the units and algorithm steps described in connection with the various examples of this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.

[0178] Example 4

[0179] This embodiment also provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the method described in Embodiment 1.

[0180] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An intrusion detection method based on multi-scale spatio-temporal feature residual fusion, characterized in that, include: Acquire network traffic data to be detected; Preprocess the network traffic data to be detected; The preprocessed network traffic data to be detected is input into the trained intrusion detection model, and the intrusion detection results and intrusion detection type are output. The trained intrusion detection model is used to extract multi-scale features from the network traffic data to be detected, fuse the multi-scale features, classify the fused features, and obtain the intrusion detection results and intrusion detection type. The trained intrusion detection model has the following network structure: The following components are connected in sequence: Generative Adversarial Network (ACGAN) with auxiliary classifier, Transformer Encoder module, Inception-BiGRU-SA module, 1*1 convolutional layer, global average pooling layer, Dropout layer, and Softmax activation function layer. The Generative Adversarial Network (ACGAN) with an auxiliary classifier includes: The generator and discriminator are used as inputs; the generator's input is real sample labels and random noise; the generator's output is fake data; the discriminator's input is real sample labels and fake data; the discriminator's output is the probability value of whether the current input data is real or fake; the discriminator's output also outputs the sample labels. The generator includes a first fully connected layer, a first batch of normalized layers, a first activation function layer, a first deconvolution layer, a first attention mechanism layer SA, a second batch of normalized layers, a second activation function layer, a second deconvolution layer, a second attention mechanism layer SA, a first bidirectional gated recurrent unit (BiGRU), a third batch of normalized layers, a third activation function layer, and a third deconvolution layer, all connected in sequence. Among them, the output of the first fully connected layer is also connected to the input of the second batch of normalization layers; the output of the first attention mechanism layer SA is also connected to the input of the first bidirectional gated recurrent unit BiGRU; the output of the second attention mechanism layer SA is also connected to the input of the third batch of normalization layers. The first and second activation function layers use the ReLU activation function; the third activation function layer uses the tanh activation function. The discriminator includes: a fourth activation function layer, a convolutional layer C1, a first dropout layer, a fifth activation function layer, a convolutional layer C2, a second dropout layer, a sixth activation function layer, a convolutional layer C3, a third dropout layer, a second fully connected layer, and an output layer connected in sequence. The fourth, fifth, and sixth activation function layers are implemented using LeakyReLU. The Inception-BiGRU-SA module operates as follows: The Inception-BiGRU-SA module deploys multiple convolutional kernels of different scales in the same layer of the neural network to capture features at different scales from the input data. Then, it assigns different weights to the features and performs forward and reverse feature extraction on the features with different weights to obtain the extracted features. The Transformer's Encoder module includes: an input layer, a multi-head attention mechanism layer, a second adder, a first normalization layer, a feedforward neural network layer, a third adder, a second normalization layer, and an output layer, connected in sequence; wherein, the input layer is also connected to the input of the second adder, and the output of the first normalization layer is also connected to the input of the third adder.

2. The intrusion detection method based on multi-scale spatiotemporal feature residual fusion as described in claim 1, characterized in that, The trained intrusion detection model, the training process includes: Construct a training set, which consists of network traffic data with known intrusion detection results and intrusion detection types; The training set is input into the intrusion detection model to train the model. Training is stopped when the number of training iterations exceeds a set number, or when the loss function value no longer decreases. The trained intrusion detection model is then obtained.

3. The intrusion detection method based on multi-scale spatiotemporal feature residual fusion as described in claim 2, characterized in that, The construction of the training set includes: Obtain network traffic data with known intrusion detection results and intrusion detection types; The network traffic data is converted to different data types, and then normalized. For the normalized network traffic data, determine whether the number of samples corresponding to each label exceeds a set threshold. If it exceeds the set threshold, the sample corresponding to the current label is identified as a majority class sample. If it is below the set threshold, the sample corresponding to the current label is identified as a minority class sample, and the training set is obtained.

4. The intrusion detection method based on multi-scale spatiotemporal feature residual fusion as described in claim 1, characterized in that, The trained intrusion detection model has the following network structure: The distribution of minority class samples is learned by using a generative adversarial network (ACGAN) with an auxiliary classifier, and the minority class samples are upsampled by the generator of the generative adversarial network (ACGAN) with an auxiliary classifier. The Transformer's Encoder module captures global connections and performs preliminary feature extraction from the input data. The Inception-BiGRU-SA module extracts multi-scale features and uses a self-attention mechanism to weight the extracted spatial features at different scales. The combination of convolutional layers and global average pooling layers reduces the number of training parameters in the network. The Dropout layer reduces the coupling between neurons and reduces overfitting. The Softmax activation function classifies and detects network traffic data.

5. The intrusion detection method based on multi-scale spatiotemporal feature residual fusion as described in claim 1, characterized in that, The Inception-BiGRU-SA module includes: The three branches are arranged in parallel: the first branch, the second branch, and the third branch. The first branch includes: a max pooling layer, a first convolutional layer, a fourth batch normalization layer, a seventh activation function layer, a second convolutional layer, a fifth batch normalization layer, and an eighth activation function layer connected in sequence; The second branch includes: a third convolutional layer, a sixth batch normalization layer, a ninth activation function layer, a fourth convolutional layer, a seventh batch normalization layer, and a tenth activation function layer connected in sequence; The third branch includes: a fifth convolutional layer, an eighth batch normalization layer, an eleventh activation function layer, a sixth convolutional layer, a ninth batch normalization layer, and a twelfth activation function layer connected in sequence; The input terminals of the max pooling layer, the third convolutional layer, and the fifth convolutional layer are all connected to the input terminal of the Inception-BiGRU-SA module. Among them, the outputs of the eighth activation function layer, the tenth activation function layer, and the twelfth activation function layer are all connected to the input of the third attention mechanism layer SA. The output of the third attention mechanism layer SA is connected to the input of the second bidirectional gated loop unit BiGRU, and the output of the second bidirectional gated loop unit BiGRU is connected to the input of the first adder. The input of the first adder is also connected to the input of the Inception-BiGRU-SA module; The output of the first adder is the output of the Inception-BiGRU-SA module; The convolution kernels of the first, third, and fifth convolutional layers are all 1*1. The second convolutional layer has a 1*1 kernel; the fourth convolutional layer has a 3*1 kernel; and the sixth convolutional layer has a 5*1 kernel.

6. An intrusion detection system based on multi-scale spatiotemporal feature residual fusion, used to execute the intrusion detection method based on multi-scale spatiotemporal feature residual fusion as described in any one of claims 1-5, comprising: The acquisition module is configured to acquire network traffic data to be detected. The preprocessing module is configured to preprocess the network traffic data to be detected. The intrusion detection module is configured to: input preprocessed network traffic data to be detected into the trained intrusion detection model, and output the intrusion detection result and the intrusion detection type; The trained intrusion detection model is used to extract multi-scale features from the network traffic data to be detected, and then to fuse these multi-scale features. The fused features are classified to obtain intrusion detection results and intrusion detection types.

7. An electronic device, characterized in that it comprises: Memory is used to store computer-readable instructions in a non-transitory manner. as well as Processor, for executing the computer-readable instructions, When the computer-readable instructions are executed by the processor, they perform the method described in any one of claims 1-5.

8. A storage medium, characterized in that, The computer-readable instructions are stored non-transitory, wherein when the non-transitory computer-readable instructions are executed by a computer, the instructions of the method according to any one of claims 1-5 are executed.

Images