Network intrusion detection method based on multi-task deep learning

Abstract

The invention discloses a network intrusion detection method based on multi-task deep learning, and the method comprises the steps: obtaining an intrusion detection data set, carrying out the mapping and one-hot coding of attack class labels of the intrusion detection data set, and obtaining the features of the intrusion detection data set; preprocessing the intrusion detection data set features to obtain a preprocessed intrusion detection data set; constructing a multi-task learning model based on hard sharing, and inputting the preprocessed intrusion detection data set into the multi-task learning model based on hard sharing for training to obtain a trained multi-task learning model based on hard sharing; and obtaining a target intrusion detection data set, inputting the trained hard sharing-based multi-task learning model for detection, and obtaining a detection result of the intrusion network. According to the method, through knowledge sharing among different tasks, the detection accuracy can be obviously optimized, and various network attacks can be effectively identified and responded.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and network information security technology, and in particular to a network intrusion detection method based on multi-task deep learning. Background Art

[0002] Traditional intrusion detection methods rely on predefined rules or signatures to identify attacks. This method is not only costly to maintain, time-consuming, and prone to misjudgment, but also leads to increased system response delays and missed detection rates, resulting in low accuracy and poor real-time performance of traditional intrusion detection technology. In addition, traditional machine learning methods have limitations in learning changing features, so the misjudgment rate is high. Therefore, manual monitoring and updating of intrusion detection systems is impractical in actual operations, especially in environments that require real-time response. With the rapid development of machine learning, especially deep learning technology, it has shown superior performance to humans in areas such as image recognition, speech processing, and natural language understanding. The successful application of these technologies has brought new research and application directions to the field of intrusion detection. Summary of the invention

[0003] In order to solve the above technical problems, the present invention proposes a network intrusion detection method based on multi-task deep learning. Through knowledge sharing between different tasks, it can significantly optimize the detection accuracy and realize effective identification and response to various network attacks.

[0004] To achieve the above object, the present invention provides a network intrusion detection method based on multi-task deep learning, comprising: obtaining an intrusion detection data set, mapping and one-hot encoding the attack category labels of the intrusion detection data set, and obtaining the intrusion detection data set features;

[0005] Preprocessing the intrusion detection data set features to obtain a preprocessed intrusion detection data set;

[0006] Constructing a multi-task learning model based on hard sharing, inputting the preprocessed intrusion detection data set into the multi-task learning model based on hard sharing for training, and obtaining a trained multi-task learning model based on hard sharing;

[0007] Obtain the target intrusion detection dataset, input the trained hard-sharing-based multi-task learning model for detection, and obtain the detection results of the intrusion network.

[0008] Optionally, obtaining a preprocessed intrusion detection dataset includes:

[0009] The missing parts in the features of the intrusion detection data set are filled and the outlier parts are removed, the maximum and minimum normalization is performed on the numerical features of each column of the intrusion detection data set to obtain the maximum and minimum normalization results, and the non-numerical parts of the maximum and minimum normalization results are uniquely encoded to obtain the preprocessed intrusion detection data set.

[0010] Optionally, building a multi-task learning model based on hard sharing includes:

[0011] A hard parameter sharing method is used to construct an underlying parameter sharing network consisting of K blocks of feature extraction networks based on one-dimensional convolution and self-attention mechanism and flattening layers and inactivation layers;

[0012] Design the top-level task module and construct different sub-models according to different tasks;

[0013] The total loss function of the top-level task module and the bottom-level parameter sharing network is calculated to complete the construction of a multi-task learning model based on hard sharing.

[0014] Optionally, a hard parameter sharing method is used to construct an underlying parameter sharing network consisting of K blocks of feature extraction networks based on one-dimensional convolution and self-attention mechanisms and flattening layers and inactivation layers, including:

[0015] Stack k blocks and learn multiple levels of spatiotemporal features of input data through one-dimensional convolution;

[0016] Perform maximum pooling on the learned spatiotemporal features to obtain local significant features;

[0017] Performing layer normalization processing on the local significant features to obtain the mean and variance of the adjusted activation values ​​of the intermediate layer;

[0018] Based on the expansion self-attention mechanism, an expansion factor is introduced to collect sequences at intervals;

[0019] Inputting the interval-collected sequence into a flattening layer to change the shape of the sequence;

[0020] The reshaped sequence is fed into a dropout layer to freeze some neurons.

[0021] Optionally, an expansion factor is introduced based on the expansion self-attention mechanism, and the interval acquisition sequence includes:

[0022] The input data sequence is multiplied by three different weight matrices to generate the values ​​of the query vector, key vector, and value vector;

[0023] Apply the dilation factor to the index of the input sequence, and for each element in the sequence, calculate the dot product between it and the element within the dilation interval.

[0024] Calculate softmax for each query vector and determine the weight of the key vector corresponding to the query vector;

[0025] Use the output of softmax as the weight to calculate the value vector after weighted average.

[0026] Optionally, the top-level task module includes an auxiliary task subnetwork and a main task subnetwork;

[0027] The auxiliary task sub-network consists of a multi-head attention mechanism and a fully connected layer, which is used to output whether a sample has an attack;

[0028] The main task sub-network consists of a Transformer encoder and a fully connected layer, which is used to output sample attack categories.

[0029] Optionally, whether the auxiliary task sub-network output sample has an attack includes:

[0030] Split a single input data into heads, and each head calculates self-attention independently to obtain features of different parts of the input;

[0031] The features of different parts of the input are input into the fully connected layer, and the results are output through the softmax activation function;

[0032] The binary cross entropy loss is calculated for the output result to obtain an output sample of whether an attack exists.

[0033] Optionally, the main task sub-network output sample attack categories include:

[0034] Inputting input data into a multi-head self-attention mechanism to obtain an output result; performing residual connection on the input data and the output result and then performing layer normalization to obtain a layer normalization result; inputting the layer normalization result into a position feedforward network to obtain a feedforward network output result; performing residual connection on the layer normalization result and the feedforward network output result and then performing layer normalization to obtain a coded output result;

[0035] The encoded output result is input into the fully connected layer, and the main task output result is obtained through the softmax activation function;

[0036] The multi-classification cross entropy function is calculated for the output result of the main task to obtain the normal or attack type of the network.

[0037] Technical effect of the present invention: The present invention discloses a network intrusion detection method based on multi-task deep learning, which obtains an intrusion detection data set, maps and one-hot encodes the attack category labels of the intrusion detection data set, and obtains the characteristics of the intrusion detection data set; preprocesses the characteristics of the intrusion detection data set to obtain the preprocessed intrusion detection data set; constructs a multi-task learning model based on hard sharing, inputs the preprocessed intrusion detection data set into the multi-task learning model based on hard sharing for training, and obtains a trained multi-task learning model based on hard sharing; obtains a target intrusion detection data set, inputs the trained multi-task learning model based on hard sharing for detection, and obtains the detection result of the intrusion network; the present invention can significantly optimize the detection accuracy through knowledge sharing between different tasks, and realizes effective identification and response to various network attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0039] Figure 1 This is a schematic diagram of the model training phase of an embodiment of the present invention;

[0040] Figure 2 Schematic diagram of the internal structure of the Transformer encoder according to an embodiment of the present invention;

[0041] Figure 3 This is a schematic diagram of the model reasoning stage of an embodiment of the present invention;

[0042] Figure 4 Schematic diagram of the process of a network intrusion detection method based on multi-task deep learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0044] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0045] By using deep learning models, features can be automatically learned from a large amount of network traffic data without manually setting rules, thereby effectively identifying and responding to various network attacks. Multi-task learning is one of the methods in deep learning. It can train models to perform multiple related tasks at the same time, rather than optimizing only a single task. Its purpose is to use multiple related tasks in learning for shared learning, so that the model can capture common features between different tasks to improve the performance of each task, thereby improving the generalization ability of the model.

[0046] Multi-task learning has been widely used in natural language processing and computer vision, so the present invention introduces it into network intrusion detection. Through knowledge sharing between different tasks, the accuracy of detection can be significantly optimized. The expansion self-attention mechanism and the multi-head self-attention mechanism are two different branches of the attention mechanism, which can optimize the model to process sequence data containing long-distance dependencies, rather than focusing only on local or short-distance information. They use the global and long-range dependencies in the sequence data to enable the model to capture deeper temporal or spatial relationships by effectively interacting information between different positions. The expansion self-attention mechanism further expands the function of the standard self-attention. By introducing the expansion factor, it can increase the receptive field of the model while maintaining computational efficiency, thereby capturing a wider range of dependencies. Therefore, the present invention uses the expansion self-attention mechanism to construct an underlying parameter sharing network for feature extraction, and uses the multi-head attention mechanism to construct a sub-model for classification. The experimental results show that the combination of the two attention mechanisms can significantly optimize the accuracy of detection. Feature fusion is a common method in machine learning and deep learning, especially when dealing with complex problems with multiple data sources or different types of features. It combines information or features from multiple sources into a unified representation, thereby improving the performance of the model, such as accuracy, robustness, etc. Feature fusion can be applied to data processing at different stages, including early fusion, mid-term fusion, and late fusion. The present invention adopts mid-term fusion, which is a fusion method that fuses features after conversion or extraction, thereby taking into account the complexity and diversity of features.

[0047] like Figure 4 As shown, this embodiment provides a network intrusion detection method based on multi-task deep learning, including: obtaining an intrusion detection data set, mapping and one-hot encoding the attack category labels of the intrusion detection data set, and obtaining the intrusion detection data set features; the specific label is y b and m , where y b Mapped to 0 or 1 to indicate whether the data sample has an attack; y m The value of the one-hot encoding is determined by the attack category of the data set, indicating the attack category of the data sample;

[0048] Preprocessing the features of the intrusion detection data set to obtain a preprocessed intrusion detection data set;

[0049] Construct a multi-task learning model based on hard sharing, input the preprocessed intrusion detection data set into the multi-task learning model based on hard sharing for training, and obtain a trained multi-task learning model based on hard sharing;

[0050] Obtain the target intrusion detection dataset, input the trained hard-sharing-based multi-task learning model for detection, and obtain the detection results of the intrusion network.

[0051] Furthermore, the preprocessed intrusion detection data set includes:

[0052] The missing parts in the features of the intrusion detection data set are filled and the outliers are removed. The maximum and minimum normalization is performed on the numerical features of each column of the intrusion detection data set to obtain the maximum and minimum normalization results. The non-numerical part of the maximum and minimum normalization results is uniquely encoded to obtain the preprocessed intrusion detection data set.

[0053] Specifically, the non-numerical feature part is one-hot encoded. One-hot encoding is a way of encoding a category into a numerical vector. Its basic steps are divided into two steps. The first step is to determine the number of non-numerical features. The second step is to create a unique binary vector for each non-numerical feature. The maximum and minimum normalization formula is shown in formula (1).

[0054]

[0055] Among them, M sca Indicates the result of maximum and minimum normalization, max means taking the maximum value, and min means taking the minimum value.

[0056] Further, such as Figure 1 The multi-task learning model based on hard sharing is shown as follows:

[0057] A hard parameter sharing method is used to construct an underlying parameter sharing network consisting of K blocks of feature extraction networks based on one-dimensional convolution and self-attention mechanism and flattening layers and inactivation layers;

[0058] Design the top-level task module and construct different sub-models according to different tasks;

[0059] Calculate the total loss function of the top-level task module and the underlying parameter sharing network to complete the construction of the multi-task learning model based on hard sharing.

[0060] Furthermore, the hard parameter sharing method is used to construct the underlying parameter sharing network consisting of K blocks of feature extraction networks based on one-dimensional convolution and self-attention mechanism and flattening layers and inactivation layers, including:

[0061] Stack k blocks and learn multiple levels of spatiotemporal features of input data through one-dimensional convolution;

[0062] Perform maximum pooling on the learned spatiotemporal features to obtain local significant features;

[0063] Perform layer normalization on local salient features to obtain the mean and variance of the adjusted activation values ​​of the intermediate layers;

[0064] Based on the expansion self-attention mechanism, an expansion factor is introduced to collect sequences at intervals;

[0065] Input the interval-collected sequence into the flattening layer to change the shape of the sequence;

[0066] The reshaped sequence is fed into a dropout layer to freeze some neurons.

[0067] Specifically, the underlying parameter sharing network is used to learn the spatiotemporal features of multiple levels of input data. One-dimensional convolution allows the network to process multiple features at the same time. Each feature can be used as a channel input, and the convolution layer can learn information between these features. In addition, one-dimensional convolution does not affect the self-attention mechanism to capture temporal information, avoiding the loss of temporal information caused by high-level spatial feature extraction. The one-dimensional convolution calculation is shown in formula (2).

[0068]

[0069] Among them, k represents the length of the convolution kernel, x(t+i) represents the eigenvalue at time t+i, and h(i) represents the weight of the convolution kernel at the i-th position.

[0070] The underlying parameter sharing network also needs to learn the features in the data that have a greater impact on the classification results. Although the self-attention mechanism has a unique learning characteristic that can learn the importance of each feature and make feature learning more targeted, it will bring high computational costs when processing very long sequences. The dilated self-attention mechanism borrows the concept of dilated convolution and introduces a dilation factor to sample the sequence at intervals. This allows the underlying parameter sharing network to focus on the features that have a greater impact on the classification results while keeping the computational cost unchanged. The formulas for the dilated self-attention mechanism are shown in formulas (3), (4), (5), and (6).

[0071] Q=XW Q ,K=XW K ,V=XW V (3)

[0072]

[0073]

[0074]

[0075] The maximum pooling layer and layer normalization layer are also included between the one-dimensional convolution layer and the self-attention layer. The maximum pooling layer is used to reduce the data dimension and capture local significant features, which helps the model to focus on more distinguishing features in subsequent processing. Batch normalization normalizes each batch of data and adjusts the mean and variance of the activation values ​​of the intermediate layer. The calculation formula of the maximum pooling layer is shown in formula (7):

[0076] y[i]=max(x[j:j+k]) (7)

[0077] Among them, x is the input data, k is the pooling window size;

[0078] The calculation formula of batch normalization is shown in formulas (8), (9), (10), and (11).

[0079]

[0080]

[0081]

[0082]

[0083] Wherein, formula (5) is used to calculate the mean of each feature in the batch, m represents the number of data in the batch, x i Represents the time step of a single feature; Formula (6) is used to calculate the variance of each feature; Formula (7) represents the normalization process, where ε is a very small smoothing term to prevent the variance from being divided by 0 in very small cases; Formula (8) represents scaling and translation, γ and β are learning parameters used to allow the model to learn the data distribution that best suits the current task.

[0084] The last two layers of the underlying parameter sharing network are the flattening layer and the random dropout layer. The flattening layer does not perform any calculations but only changes the shape of the data. This is done so that the output of the self-attention layer can be connected to the random dropout layer, so that the extracted features can be output to the auxiliary task and the main task for training the top-level model. The random dropout layer is a regularization technique. During the training process, the random dropout layer will randomly freeze some neurons to reduce the dependence of a single neuron on specific training data, which is used to prevent the model from overfitting.

[0085] Furthermore, based on the expansion self-attention mechanism, an expansion factor is introduced, and the interval acquisition sequence includes:

[0086] The input data sequence x is multiplied by three different weight matrices to generate the values ​​of the query vector Q, key vector K, and value vector V;

[0087] Apply the dilation factor to the index of the input sequence, and for each element in the sequence, calculate the dot product between it and the element within the dilation interval.

[0088] Calculate softmax for each query vector and determine the weight of the key vector corresponding to the query vector;

[0089] Use the output of softmax as the weight to calculate the value vector after weighted average.

[0090] Furthermore, the top-level task module includes an auxiliary task sub-network and a main task sub-network;

[0091] The auxiliary task sub-network consists of a multi-head attention mechanism and a fully connected layer, which is used to output whether the sample has an attack;

[0092] The main task sub-network consists of a Transformer encoder and a fully connected layer, which is used to output sample attack categories.

[0093] Specifically, the top-level task design is to construct different sub-models according to different tasks to achieve the output of different tasks, including the following two sub-tasks: The auxiliary task is a binary classification task, which is used to output whether the sample has an attack. The input accepted by the auxiliary task is the output of the underlying parameter sharing network. The auxiliary task consists of a multi-head attention mechanism and a fully connected layer. The loss function used by the auxiliary personnel is the binary cross entropy loss. The main task is a multi-classification task, which consists of a Transformer encoder and a fully connected layer to output the sample attack category. The input accepted by the main task is the fusion feature obtained by splicing and weighting the output of the underlying parameter sharing network and the multi-head self-attention output in the auxiliary task. The main task uses the multi-classification cross entropy function as the loss function.

[0094] Furthermore, whether the auxiliary task sub-network output sample has an attack includes:

[0095] Split a single input data into heads, and each head calculates self-attention independently to obtain features of different parts of the input;

[0096] The features of different parts of the input are input into the fully connected layer, and the results are output through the softmax activation function;

[0097] The binary cross entropy loss is calculated for the output results to obtain output samples of whether there is an attack.

[0098] Specifically, the multi-head self-attention mechanism is a special multi-head attention mechanism that allows the model to focus on multiple unknowns in the data and process information in parallel in subspaces of different representations. The multi-head attention mechanism divides a single input data into heads, and each head calculates self-attention independently, so that each part can independently focus on the features of different parts of the input. The calculation formulas of the multi-head self-attention mechanism are shown in formulas (12), (13), and (14).

[0099]

[0100] Among them, Q i ,K i ,V i represents the Q, K, V of the i-th head, and is the corresponding weight matrix.

[0101]

[0102] MultiHead(X)=Concat(head1,…,head h )W O (14)

[0103] Among them, head i represents the self-attention output of the i-th head, W O Represents the weight matrix of the output layer. The self-attention outputs of i heads are concatenated with the weight matrix W of the output layer. O Multiply together to get a matrix that combines the self-attention of all heads.

[0104] The output of the self-attention mechanism is input into the fully connected layer and the result is output through the softmax activation function.

[0105] The calculation formula for the binary cross entropy loss is shown in formula (15).

[0106] L b =-(ylog(p)+(1-y)log(1-p)) (15)

[0107] Among them, y represents the actual label, and p represents the probability that the model predicts that the sample is a positive class.

[0108] Furthermore, the main task sub-network output sample attack categories include:

[0109] Input the input data to the multi-head self-attention mechanism to obtain the output result; perform residual connection on the input data and the output result and then perform layer normalization to obtain the layer normalization result; input the layer normalization result to the position feedforward network to obtain the feedforward network output result; perform residual connection on the layer normalization result and the feedforward network output result and then perform layer normalization to obtain the encoded output result;

[0110] The encoded output result is input into the fully connected layer, and the main task output result is obtained through the softmax activation function;

[0111] The multi-classification cross entropy function is calculated for the output of the main task to obtain the normal or attack type of the network.

[0112] Specifically, the main task uses the multi-classification cross entropy function as the loss function.

[0113] The input of the main task is shown in formula (16),

[0114] Input=μMultiHead(F(X))+F(X) (16)

[0115] Among them, F represents the underlying parameter sharing network, X represents the original input, μ represents the auxiliary task feature weight parameter, which is used to adjust the contribution of the auxiliary task feature to the main task, and μ is a hyperparameter.

[0116] The Transformer encoder captures the complex relationships in the sequence through a multi-head self-attention mechanism and a feed-forward neural network, providing information-rich context for the classification layer. The Transformer encoder consists of a multi-head self-attention and position feed-forward network. Its structure is as follows Figure 2 As shown, the calculation process is shown in formulas (17), (18) and (19).

[0117] M=LayerNorm(X+Sublayer(X)) (17)

[0118] Where X, represents the original input, and Sublayer(X) represents the output of the multi-head self-attention in the Transformer encoder. The output of the multi-head self-attention and the original input are residually connected and then normalized.

[0119] FFN(M)=max(0,MW1+b1)W2+b2 (18)

[0120] FFN stands for position feedforward network, where W1, b1, W2, b2 are the parameters of the feedforward network.

[0121] Output=LayerNorm(M+FFN(M)) (19)

[0122] The output of the multi-head self-attention and the output of the position feed-forward network are residually connected and then normalized.

[0123] The output is input into the fully connected layer and the result is output through the softmax activation function.

[0124] The calculation formula of the multi-classification cross entropy function is shown in formula (20).

[0125]

[0126] Among them, y c is the one-hot encoding of the actual category label, p c is the probability that the model predicts class c.

[0127] Furthermore, the total loss function is shown in formula (21):

[0128] L=μ b L b +μ m L m (twenty one)

[0129] Among them, L b represents the loss function of the auxiliary task, L m represents the loss function of the main task, μ b and μ m As a weight parameter, it is used to balance different tasks.

[0130] A specific application example of the present invention is as follows:

[0131] S1: CICIDS2017 and CICIOT2023 datasets are used as datasets for training and testing the generalization performance of the model. Figure 1 As shown in Figure 2, the inference phase is as follows: Figure 3 shown.

[0132] The CICIDS2017 dataset contains six types of network attacks, including denial of service attacks, port scanning, brute force cracking, network attacks, botnets, and penetration attacks. The specific message tree of CICIDS2017 is shown in Table 1.

[0133] The CICIOT2023 dataset is a real-time dataset for the IoT environment released in 2023. It includes six types of network attacks, including denial of access attacks, brute force attacks, deception attacks, reconnaissance attacks, network attacks, and botnets. The specific number of CICIOT2023 messages is shown in Table 2.

[0134] Table 1

[0135] normal 22731 Denial of Access Attack 19035 Port Scanning 7946 Brute force 2767 Cyber ​​Attacks 2180 Botnet 1966 Penetration Attack 36

[0136] Table 2

[0137]

[0138]

[0139] S1.1: Map the label fields in the dataset into Arabic numerals 0 and 1 to form y b , and then perform one-hot encoding on the label field to form y m .

[0140] S2: Fill the gaps in the data set with 0, remove abnormal features such as NAN and INFINITE, perform one-hot encoding on non-numerical features, perform maximum and minimum normalization on each column of the data set, and finally divide the data set.

[0141] S3: Model construction stage. Construct a network intrusion detection model based on multi-task fusion learning as described above, set model parameters, and perform model training. The model hyperparameter settings are shown in Table 3.

[0142] S3.1: Number of feature extraction networks (K)

[0143] The size of the feature extraction network affects the data extraction capability of the underlying parameter sharing layer, and also affects the model training and convergence time. The larger the K, the stronger the data feature extraction capability of the underlying parameter sharing layer. However, it will also cause the model to generate too much and cause overfitting problems. The present invention increases the size of K in sequence from [3, 6, 9] to select the optimal K.

[0144] S3.2: Batch Size

[0145] The size of the batch size affects the training speed, memory and performance of the model. The larger the batch size, the more effectively the model can utilize parallel processing capabilities and provide more accurate gradient estimates, which helps the model converge stably. However, it will increase the memory requirements for each iteration, making it impossible to train on resource-constrained devices. The present invention increases the batch size from [64, 128, 256] in sequence to select the optimal batch size.

[0146] S3.2: Training cycle (Epoch)

[0147] The number of training cycles indicates the process in which the entire training data set is used to train the model once. Training the model with multiple epochs can help the model better approach or reach the global optimal solution, but there is also a risk of overfitting. The epoch selected by the present invention according to the characteristics of the selected data set is 100. In addition, the present invention adopts the Early Stopping method to determine whether the performance of the validation set has been significantly improved by monitoring the total loss of the model, thereby stopping the training in advance.

[0148] S3.3: Learning rate

[0149] The learning rate defines the amplitude of parameter updates during the optimization process. If the learning rate is set too high, the model may oscillate around the optimal solution or even diverge, resulting in training failure. If the learning rate is set too low, the training process may be very slow, and the model requires more time and more training cycles to converge, and may even fall into a local minimum. The present invention adopts a learning rate decreasing strategy, and the learning rate is fixed at 0.001 in the first 20 training cycles. Starting from the 21st cycle, the learning rate of each cycle will be recursively multiplied by e to the power of -0.1.

[0150] S3.4: Optimizer Algorithm

[0151] The optimizer algorithm is used to change the weights in the neural network. The goal of the optimizer is to find the minimum value of the loss function by gradually adjusting the network parameters, which is usually achieved by gradient descent or its variants. Commonly used optimizer algorithms include Adam, SGD, and MBGD. The optimizer algorithm selected in the present invention is Adam.

[0152] S3.5: Auxiliary task feature weight parameters

[0153] In the multi-task learning framework, the auxiliary task feature weight parameter plays a vital role. This parameter is generally used to adjust the contribution of the auxiliary task feature to the main task, helping the main task to improve performance through the information learned from the auxiliary task, thereby enhancing the generalization ability of the model and reducing the risk of overfitting. As shown in Table 4, the auxiliary task feature weight parameter selected in the present invention is determined to be 0.7 after multiple experiments.

[0154] Table 3

[0155]

[0156]

[0157] Table 4

[0158] CICIDS2017 CICIOT2023 Auxiliary Tasks ACC 0.9827 0.9941 Main Task ACC 0.9809 0.9902 Training time 17.49 minutes 36.47 minutes Reasoning time 0.02 minutes 0.11 minutes

[0159] S4: In summary, the intrusion detection method proposed in the present invention realizes efficient detection of data sets of two different types of networks and multiple intrusion attacks by utilizing multi-task learning and feature fusion methods. In addition, the training time and inference time of this method are short, and it is suitable for intrusion detection scenarios that require high real-time performance.

[0160] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A network intrusion detection method based on multi-task deep learning, characterized in that: include: Obtain an intrusion detection data set, map and one-hot encode the attack category labels of the intrusion detection data set, and obtain the intrusion detection data set features; Preprocessing the intrusion detection data set features to obtain a preprocessed intrusion detection data set; Constructing a multi-task learning model based on hard sharing, inputting the preprocessed intrusion detection data set into the multi-task learning model based on hard sharing for training, and obtaining a trained multi-task learning model based on hard sharing; Obtain the target intrusion detection dataset, input the trained hard-sharing-based multi-task learning model for detection, and obtain the detection results of the intrusion network.

2. The network intrusion detection method based on multi-task deep learning according to claim 1, characterized in that: The preprocessed intrusion detection dataset includes: The missing parts in the features of the intrusion detection data set are filled and the outlier parts are removed, the maximum and minimum normalization is performed on the numerical features of each column of the intrusion detection data set to obtain the maximum and minimum normalization results, and the non-numerical parts of the maximum and minimum normalization results are uniquely encoded to obtain the preprocessed intrusion detection data set.

3. The network intrusion detection method based on multi-task deep learning according to claim 1, characterized in that: Building a hard-sharing based multi-task learning model involves: A hard parameter sharing method is used to construct an underlying parameter sharing network consisting of K blocks of feature extraction networks based on one-dimensional convolution and self-attention mechanism and flattening layers and inactivation layers; Design the top-level task module and construct different sub-models according to different tasks; The total loss function of the top-level task module and the bottom-level parameter sharing network is calculated to complete the construction of a multi-task learning model based on hard sharing.

4. The network intrusion detection method based on multi-task deep learning as claimed in claim 3, characterized in that: The hard parameter sharing method is used to construct the underlying parameter sharing network consisting of K blocks of feature extraction networks based on one-dimensional convolution and self-attention mechanism and flattening layers and inactivation layers, including: Stack k blocks and learn multiple levels of spatiotemporal features of input data through one-dimensional convolution; Perform maximum pooling on the learned spatiotemporal features to obtain local significant features; Performing layer normalization processing on the local significant features to obtain the mean and variance of the adjusted activation values ​​of the intermediate layer; Based on the expansion self-attention mechanism, an expansion factor is introduced to collect sequences at intervals; Inputting the interval-collected sequence into a flattening layer to change the shape of the sequence; The reshaped sequence is fed into a dropout layer to freeze some neurons.

5. The network intrusion detection method based on multi-task deep learning as claimed in claim 4, characterized in that: Based on the expansion factor introduced by the expansion self-attention mechanism, the interval acquisition sequence includes: The input data sequence is multiplied by three different weight matrices to generate the values ​​of the query vector, key vector, and value vector; Apply the dilation factor to the index of the input sequence, and for each element in the sequence, calculate the dot product of the element with the other elements within the dilation interval; Calculate softmax for each query vector and determine the weight of the key vector corresponding to the query vector; Use the output of softmax as the weight to calculate the value vector after weighted average.

6. The network intrusion detection method based on multi-task deep learning as claimed in claim 3, characterized in that: The top-level task module includes an auxiliary task sub-network and a main task sub-network; The auxiliary task sub-network consists of a multi-head attention mechanism and a fully connected layer, which is used to output whether a sample has an attack; The main task sub-network consists of a Transformer encoder and a fully connected layer, which is used to output sample attack categories.

7. The network intrusion detection method based on multi-task deep learning as claimed in claim 6, characterized in that: Whether the auxiliary task sub-network output sample has an attack includes: Split a single input data into heads, and each head calculates self-attention independently to obtain features of different parts of the input; The features of different parts of the input are input into the fully connected layer, and the results are output through the softmax activation function; The binary cross entropy loss is calculated for the output result to obtain an output sample of whether an attack exists.

8. The network intrusion detection method based on multi-task deep learning as claimed in claim 6, characterized in that: The main task sub-network output sample attack categories include: Inputting input data into a multi-head self-attention mechanism to obtain an output result; performing residual connection on the input data and the output result and then performing layer normalization to obtain a layer normalization result; inputting the layer normalization result into a position feedforward network to obtain a feedforward network output result; performing residual connection on the layer normalization result and the feedforward network output result and then performing layer normalization to obtain a coded output result; The encoded output result is input into the fully connected layer, and the main task output result is obtained through the softmax activation function; The multi-classification cross entropy function is calculated for the output result of the main task to obtain the normal or attack type of the network.

Images