Biaprn-based industrial internet of things attack prediction method

Abstract

The application relates to the technical field of Internet of Things security, in particular to an industrial Internet of Things attack prediction method based on a BiAPRN. First, an improved single-gate structure gated recurrent unit model is introduced; the model significantly improves the performance of the model through a self-attention mechanism and a loss compensation mechanism. Then, a new bidirectional attention simple recurrent network (BiAPRN) neural network model is constructed. The BiAPRN effectively extracts data features by combining the linear combination of forward and backward aprns, more deeply mines the correlation of time series, and significantly reduces the prediction error. Finally, an industrial Internet of Things attack prediction framework is obtained through the BiAPRN model, attack prediction is carried out using the trained BiAPRN model, and accurate and efficient industrial Internet of Things attack prediction is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet of Things security, and particularly relates to an industrial Internet of Things attack prediction method based on BiAPRN. BACKGROUND

[0002] Industrial Internet of Things faces severe security challenges such as data leakage, malicious attacks, and virus infection. In traditional defense systems, techniques such as intrusion detection and key establishment have been implemented, and advanced security devices have been equipped. However, these techniques can only passively defend and cannot actively discover or predict attacks through network behavior and threat analysis. This limitation hinders the effective use of network threat strategies, resulting in high false alarm rates and major challenges.

[0003] In order to carry out active defense, people have adopted an attack prediction technical solution. Attack prediction is an effective security mechanism that predicts future attacks by analyzing historical network traffic data and establishing accurate prediction models. Deploying such a system allows industrial Internet of Things security administrators to monitor and predict attacks, implement defensive measures in advance, and provide real-time protection for industrial Internet of Things devices and systems. In combination with machine learning and deep learning, models such as Deep Neural Network (DNN), Convolutional Neural Network (CNN), and Recurrent Neural Network (RNN) are used to identify network threats. However, existing RNN and its variants, such as Gated Recurrent Unit (GRU), have limitations in processing network traffic data, particularly in long-term and short-term information processing and data representation learning.

[0004] Given the large-scale, complexity, and time series nature of industrial Internet of Things network traffic, these models need to be improved in terms of real-time processing, efficiency, and prediction accuracy. SUMMARY

[0005] The purpose of the present application is to provide an industrial Internet of Things attack prediction method based on BiAPRN, which designs a bidirectional attention simple recurrent network model to eliminate redundant information in past hidden states and achieve accurate and efficient industrial Internet of Things attack prediction.

[0006] To achieve the above purpose, the present application provides an industrial Internet of Things attack prediction method based on BiAPRN, comprising the following steps:

[0007] Step 1: improve the GRU model to obtain an attention simple recurrent network model;

[0008] Step 2: construct a bidirectional attention simple recurrent network model;

[0009] Step 3: Collecting data generated by industrial internet of things devices;

[0010] Step 4: Preprocessing the collected data and completing data set partitioning;

[0011] Step 5: Inputting data of the data set into the bidirectional attention simple recurrent network model for training;

[0012] Step 6: Using the trained bidirectional attention simple recurrent network model to predict the results of the test data set;

[0013] Step 7: Applying the trained bidirectional attention simple recurrent network model to a real industrial internet of things environment and visualizing the results.

[0014] Specifically, during the execution of step 1, a gating mechanism module and a self-attention module are used to replace the update gate in the GRU model;

[0015] In the gating mechanism module, when x t is fed into the gating mechanism module, the gating mechanism enables the network to identify when to update or ignore information in the hidden state, which is expressed by the formula:

[0016] x t = tanh(w h ·x t-1 +b h ) x δ(w s ·x t-1 +b s )+x t-1

[0017] where tanh is the hyperbolic tangent function, δ is the sigmoid function, w h , w s , b h , b s are all learnable parameters of the gating mechanism;

[0018] The reset gate module determines to what extent past information should be forgotten when updating the current state, and how much should be considered. The reset gate r can be calculated as follows:

[0019] r = δ(w ir ·x t +b ir +w hr ·h t-1 +b hr )

[0020] where δ is the sigmoid function, w ir , w hr , bir and b hr are all learnable parameters of the reset gate.

[0021] In the self-attention module, by splicing and h t-1 Construct a candidate set to prepare for the subsequent self-attention mechanism learning data. The formula is as follows:

[0022]

[0023] Among them, h t-1 Indicates past tense The hidden state of the moment, h t Indicates the hidden state at the current moment, represents the candidate set, w in , w hn , b in and b hn are all learnable parameters of the candidate set, stack is a matrix splicing operation, h s It is the result of splicing.

[0024] In the attention-reduced recurrent network model, self-attention is used to learn the relationship between the past hidden state and the current input. The corresponding formula is as follows:

[0025] h′ t-1 =Self-Attention(h s )

[0026]

[0027] Specifically, the output expression of the bidirectional attention simplified recurrent network model in step 2 is as follows:

[0028]

[0029] in, Indicates the future state information being output to the APRN. Indicates the past state information output to the APRN. For the forward APRN model, is the backward APRN model, w1, w2, and b are the learnable parameters of the linear combination.

[0030] The application provides an industrial Internet of Things attack prediction method based on BiAPRN. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0032] Figure 1 is a step flowchart of an industrial Internet of Things attack prediction method based on BiAPRN of the present application.

[0033] Figure 2 is a structure diagram of an APRN model of the present application.

[0034] Figure 3 is a structure diagram of a BiAPRN model of the present application.

[0035] Figure 4 is a working flowchart of attack prediction based on the BiAPRN model of the present application.

[0036] Figure 5 is a confusion matrix diagram of the BiAPRN model based on each data set in the specific embodiment example of the present application.

[0037] Figure 6 is an effectiveness evaluation diagram of each scheme on the data set in the specific embodiment example of the present application.

[0038] Figure 7 is a training time consumption comparison diagram of each scheme in the specific embodiment example of the present application.

[0039] Figure 8 is a test time consumption comparison diagram of each scheme in the specific embodiment example of the present application. DETAILED DESCRIPTION

[0040] Embodiments of the present application are described in detail below with reference to the attached drawing figures, wherein like or similar constituent elements or features may be denoted by like reference characters throughout the drawing figures and the description. The embodiments described below are exemplary in nature and are intended to be illustrative of the present application rather than to limit the present application.

[0041] The following is a semantic explanation of some English term abbreviations appearing in the present application:

[0042] Gate Recurrent Unit, GRU: Gated Recurrent Unit;

[0043] Attention Parsimonious Recurrent Network, APRN: Attention Parsimonious Recurrent Network;

[0044] Bidirectional Attention Parsimonious Recurrent Network, BiAPRN: Bidirectional Attention Parsimonious Recurrent Network.

[0045] Bidirectional Long Short-Term Memory, BiDLSTM: Bidirectional Long Short-Term Memory.

[0046] Parsimonious memory unit for recurrent neural networks: Parsimonious memory unit for recurrent neural networks.

[0047] Please refer to Figure 1 The present application provides an industrial Internet of Things attack prediction method based on BiAPRN, comprising the following steps:

[0048] S1: improving the GRU model to obtain an attention parsimonious recurrent network model;

[0049] S2: constructing a bidirectional attention parsimonious recurrent network model;

[0050] S3: collecting data generated by industrial Internet of Things devices;

[0051] S4: preprocessing the collected data and completing data set partitioning;

[0052] S5: inputting the data of the data set into the bidirectional attention parsimonious recurrent network model for training;

[0053] S6: using the trained bidirectional attention parsimonious recurrent network model to predict the results of the test data set;

[0054] S7: Apply the trained bidirectional attentional simple recurrent network model to a real industrial internet of things environment and visualize the results.

[0055] The following will be further described in combination with specific implementation steps and related technical solutions:

[0056] In step S1, the APRN model introduced by the present application alleviates the problem that the gating structure of the standard GRU may cause omission of key content in a long sequence.

[0057] As shown in Figure 2 , the present application uses r and x t to represent the reset gate and the input at the current time respectively. At the same time, h t-1 represents the hidden state at the past time, h t represents the hidden state at the current time, and h represents the candidate set, h′ t-1 represents the hidden information at the past time after self-attention learning.

[0058] The gating mechanism module, when x t is fed into the gating mechanism module, the gating mechanism enables the network to distinguish when to update or ignore the information in the hidden state. The formula is expressed as:

[0059] x t =tanh(w h ·x t-1 +b h )×δ(w s ·x t-1 +b s )+x t-1

[0060] where tanh is the hyperbolic tangent function, w h , w s , b h , b s are all learnable parameters of the gating mechanism.

[0061] The reset gate module, the role of the reset gate is to determine how much past information should be forgotten when updating the current state, and how much should be considered. The reset gate r can be calculated as follows:

[0062] r=δ(w ir ·x t +b ir +w hr ·h t-1 +b hr )

[0063] where δ is the sigmoid function, w ir , b ir , wh , b hr are learnable parameters of the reset gate r.

[0064] Self-attention module, the application constructs a candidate set through splicing and h t-1 to prepare for subsequent self-attention mechanism learning data. The related formula is as follows:

[0065]

[0066] where w in , w hn , b in and b hn are learnable parameters of the candidate set, stack is a matrix splicing operation, and h s is the splicing result.

[0067] Finally, self-attention is used to learn the relationship between the past hidden state and the current input. The related formula is as follows:

[0068] h′ t-1 = Self-Attention(h s )

[0069]

[0070] In step S2, the BiAPRN expands the traditional APRN model and improves the attack prediction performance. It trains two APRNs: one on the original input data and the other on the reverse input data. This bidirectional modeling allows the network to consider both past and future information simultaneously, capturing patterns and rules in time series data. Unlike traditional bidirectional networks, BiAPRN uses linear combination technology for more effective feature learning. Therefore, in tasks such as network traffic attack prediction, BiAPRN improves the prediction accuracy of the model.

[0071] The specific structure of the BiAPRN model is shown in Figure 3 , the current APRN output, for example y t , is composed of two parts, representing the future state information of the forward APRN output, representing the past state information of the backward APRN output. The formula is as follows:

[0072]

[0073] where, is the forward APRN model, is the backward APRN model, and w1, w2, b are learnable parameters of the linear combination.

[0074] After obtaining the BiAPRN model, as shown in Figure 4 Fig. 1 shows a typical workflow of the attack prediction scheme based on BiAPRN, which will be described in detail in steps S3 to S7 of the method of the present application.

[0075] Step S3: In the data collection layer, a large amount of network traffic data will be generated when the industrial Internet of Things device participates in the industrial production environment, and the generated data needs to be collected.

[0076] Step S4: In the data processing layer, data collection includes collecting raw data from industrial Internet of Things devices. After collection, data preprocessing ensures high-quality and effective deep learning model training. Preprocessing includes normalization, character encoding, and data set partitioning.

[0077] Step S5: In the model training layer, use the data set from the data processing layer. Then input the data into the Bi-APRN model. The BiAPRN model includes forward and backward neural networks to capture temporal dynamics and attention mechanisms for key features. Then the trained BiAPRN model is used for attack prediction.

[0078] Step S6: In the test and prediction layer, the trained Bi-APRN model predicts the results of the test data set. Use Accuracy, Precision, Recall, etc. to measure the performance and generalization of the model, and use F1 score. Through these indicators, insights into the prediction accuracy and reliability are provided.

[0079] Step S7: In the result representation layer, the trained BiAPRN model is applied to a real industrial Internet of Things environment, and the results are visualized, enabling industrial Internet of Things security administrators to monitor potential network attacks.

[0080] Further, in order to further illustrate the benefits of the industrial Internet of Things attack prediction method, the present application proposes specific embodiments, which are illustrated by comparison with existing schemes:

[0081] 1. Theoretical calculation complexity

[0082] Compared with some existing schemes, the comparison results are shown in Table 1. Generally, assuming that the dimension of the input data is m, and the number of hidden units is n. GRU uses reset and update gates to control information flow, so through the GRU double-gated mechanism, the total number of operations is T(3×n×m+6×n 2 +4×n), and the time complexity is O(n 2 ). On the other hand, BiDLSTM is composed of two LSTMs. Therefore, the total number of operations of BiDLSTM is T(2×(4×n×m+4×n2+4×n)), and the time complexity is O(n 2). The gate structure of PMU is much cleaner, so its total number of operations is T(2 x n x m + 2 x n2+ 2 x n) and the time complexity is O(n 2 ) for APRN, it consists of Self-Attention and a gating unit. The total number of operations required by self-attention is T(12 x n2+ 2 x n) and the time complexity is O(n 2 ) for BiAPRN. It can be seen that the total number of operations required by BiAPRN is T(2 x (2 x n x m + 16 x n2+ 6 x n)) and the time complexity is O(n 2 ).

[0083] Table 1 Time complexity comparison

[0084] Method Operation time Complexity GRU [T(3xn xm + 6xn 2 + 4xn)] O(n 2 )]]> PMU [T(2xnxm+2xn 2 +2xn)]]> O(n 2 )]]> BiDLSTM [T(2 x (4 x n x m + 4 x n 2 + 4 x n))] + 4 x n O(n 2 )]]> APRN [T(2xnxm+ 16xn 2 + 6xn)] O(n 2 )]]> BiAPRN [T(2 x (2 x n x m + 16 x n 2 + 6 x n))] - T(2 x (2 x n x m + 16 x n O(n 2 )]]>

[0085] 2. Comparison of multiple experimental data sets:

[0086] 1) Experimental environment: All simulations were performed on a desktop computer running Windows 11 and 16 gb RAM. The desktop was installed with CUDA 10.0, cuDNN 7.4.2, and Python 3.7. The CPU was an Intel Core i5-8300H at 2.30 GHz, and the GPU was an NVIDIA GeForce GTX 1050Ti. The deep learning framework used was PyTorch 1.8.

[0087] 2) Experimental data sets: NSL-KDD is a widely recognized public data set. It includes KDDTrain+ as the training set for model learning, KDDTesn+ and KDDTest-21 as the test set for evaluating the performance of the trained model. In order to enhance the realism of network traffic simulation, the test data set contains many attack types that do not exist in the training set. For optimal model training, KDDTrain+ is mainly used as the training set, while KDDTest+ is used as the test set in the experiment.

[0088] The UNSW-NB15 data set is one of the widely used benchmark data sets. UNSW-NB15, as a modern data set, is very different from the previous NSL-KDD data set, reflecting more complex and diverse network traffic scenarios. The UNSW-NB15 data set covers nine modern and common network attack types.

[0089] The Bot-IoT dataset was created by designing realistic network environments at the UNSW Canberra Networked Battle Lab. The dataset contains a large amount of network traffic from IIoT devices, normal traffic, and botnet attacks. The dataset captures pcap files with a size of 69.3 GB, containing over 72,000,000 records. This embodiment uses the training and test sets in the "5%" folder as experimental datasets.

[0090] 3) Evaluation Criteria: A variety of evaluation parameters are used to evaluate the proposed BiAPRN model's capabilities. The main categories of true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN) are given by the confusion matrix. The present invention will comprehensively use ConfusionMatrix, Accuracy, Precision, Recall, and F1_score as evaluation indicators to ensure the thoroughness of the evaluation.

[0091] 4) Implementation: The BiAPRN model is implemented in Python using Pytorch. Hyperparameters are adjusted according to dataset differences. For the NSL-KDD dataset, the adaptive moment estimation (Adam) optimizer is used with a learning rate of 0.001. The BiAPRN hidden layer has 180 neurons, 256 batches, and 700 training epochs. For the USNW-NB15 dataset, the Adam optimizer is used with a learning rate of 0.001. The hidden layer has 100 neurons, 512 batches, and 300 training epochs. For the BoT-IoT dataset with over 23,000,000 samples, the Adam optimizer is used with a learning rate of 0.001. The hidden layer has 150 neurons, 1024 batches, and 200 training epochs, which reduces time overhead while ensuring performance.

[0092] Specifically, this embodiment compares the attack prediction scheme based on APRN, the attack prediction scheme based on PMU, the attack prediction scheme based on BiDLSTM, and the attack prediction scheme based on GRU, and explains from the following aspects:

[0093] 1) Robustness of attack prediction model: In order to better evaluate the effectiveness and robustness of the BiAPRN model, this embodiment conducts a series of complementary experiments considering ablation. The BiAPRN model is based on the principles of gating mechanism (GM) and self-attention (SA) modules; these components are further aggregated to produce the final model. In this case, this embodiment creates two BiAPRN variants: w / o GM and w / o SA. In the w / o GM variant, the gating mechanism responsible for regulating the flow of information is deleted. In the w / o SA variant, the self-attention mechanism and the remaining connections are deleted.

[0094] The corresponding parameter configurations are as shown above, and are applied to all ablation experiments, and the results are shown in Table 2. It is observed that, on various data sets, all performance indicators of the w / o GM and w / o SA models are significantly lower than those of the BiAPRN model. The overall implementation of BiAPRN achieves excellent results, which indicates that removing any module will have a significant impact on the results. Therefore, the flexibility and effectiveness of the BiAPRN model are verified.

[0095] Table 2 Comparison of ablation experiment results

[0096]

[0097]

[0098] 2) Accuracy and versatility: The training process of deep learning is essentially a self-learning process, and the model learns from a large amount of data through iterative training. In this case, the diagonal line entries of the confusion matrix are particularly important, as they represent the number of correct recognitions for each class. Essentially, a larger number on the diagonal line indicates that the deep learning model has successfully learned and gained more knowledge. Figure 5 Tables 5b and 5c show that the BiAPRN model of the present application effectively learns from the training data set.

[0099] In addition, in order to ensure the comprehensiveness of the evaluation, the evaluation indicators of Accuracy, Precision, Recall and F1 score are comprehensively used in this embodiment to verify the effectiveness of the method of the present application.

[0100] Based on the effectiveness evaluation of the NSL-KDD data set. In the analysis of the NSL-KDD data set, as shown in Table 5a, the BiAPRN scheme proposed in the present application shows excellent performance in multiple evaluation indicators. Among them, the accuracy of the BiAPRN model of the present application reaches 81.44%, which is significantly better than the APRN, PMU, BiDSLTM and GRU models, and the accuracy of the latter is 78.56%, 79.02%, 79.51% and 79.79%, respectively. In terms of Precision, Recall and F1 score, the BiAPRN model of the present application reaches 82.59%, 81.19% and 78.91%, respectively, which is better than the APRN, PMU, BiDSLTM and GRU models. Figure 6 Based on the effectiveness evaluation of the UNSW-NB15 data set. From

[0101] Figure 6 ​As can be seen in b, the scheme proposed by the present application has achieved significant advantages in the four indicators based on the UNSW-NB15 dataset - accuracy, precision, recall and F1 score. For example, the accuracy of the scheme proposed by the present application is 96.5%, while the accuracy of the other four schemes is 94.54%, 94.62%, 94.42% and 93.20% respectively. For another example, the F1 score of the scheme proposed by the present application is 96.60%, which is significantly higher than the F1 scores of the other schemes by 2%, because the F1 scores of the other schemes are 94.59%, 94.42%, 94.50% and 93.31% respectively. Therefore, the comprehensive effectiveness of the scheme proposed by the present application is superior.

[0102] Based on the Bot-IoT dataset, the effectiveness evaluation. Contrary to the UNSW-NB15 dataset, the BoT-IoT dataset is mainly composed of attack traffic, and the normal traffic data accounts for less than 1% of the dataset. From Figure 6 In c, it can be observed that all attack prediction schemes can achieve high Accuracy, Precision, Recall and F1 score. The accuracy, precision, recall and F1 score of BiAPRN proposed by the present application are 99.99%, 96.67%, 99.99% and 98.18% respectively. Overall, BiAPRN is more advantageous than the other four models.

[0103] Overall, compared with some existing schemes, the scheme proposed by the present application has significant improvement in all indicators: accuracy, precision, recall and F1 score. This is because in BiAPRN, the present application achieves better feature extraction through a bidirectional network, and designs a loss compensation mechanism, which significantly reduces the loss of data features. Therefore, BiAPRN has stronger effectiveness, universality and generalization ability.

[0104] 3) Low detection delay: the running time of the experimental test is divided into model training and test time overhead.

[0105] Comparison of training time overhead. In this experiment, the present application trains a deep learning model and tests the time overhead, as Figure 7 shown. The training time overhead of the BiAPRN model on the NSL-KDD, UNSW-NB15 and Bot-IoT datasets is 106.66 min, 104.33 min and 84.63 min respectively. The present application finds that the training time overhead using the UNSW-NB15 dataset is slightly lower than that of the NSL-KDD dataset. Compared with the UNSW-NB15 dataset, the NSL-KDD dataset has a smaller batch size and more training epochs. Due to the forward and reverse sequential input, learning process and loss compensation of BiAPRN, the scheme proposed by the present application has the highest training time overhead.

[0106] Test time overhead comparison. The average test time per 100 samples was calculated to normalize the test time. As shown in Figure 8 a、 Figure 8 b and Figure 8 c, the test overhead of the BiAPRN model on the three datasets was 3.23 ms, 3.9 ms and 3.63 ms, respectively. Compared with other methods, the method proposed in the present application prolongs the processing time. This expansion is due to the inclusion of forward models, inverse models and self-attention mechanisms in BiAPRN, resulting in additional time overhead. Although it takes more time overhead, the scheme proposed in the present application achieves more attractive effectiveness. In addition, the test time overhead and difference are negligible and acceptable without significantly affecting performance. The performance requirement for low latency in the IIoT environment is met, ensuring efficient and timely data transmission.

[0107] The above only discloses a preferred embodiment of the present application, of course, cannot limit the scope of the rights of the present application, those skilled in the art can understand that the implementation of all or part of the above-mentioned processes, and the equivalent changes made according to the claims of the present application, still belong to the scope covered by the present application.

Claims

1.A method for predicting attacks on an industrial internet of things based on BiAPRN, characterized in that, The method comprises the following steps: Step 1: improving the GRU model to obtain an attention-based simple recurrent network model; In the execution process of step 1, a gating mechanism module and a self-attention module are used to replace the update gate in the GRU model; In the gating mechanism module, when x t The gating mechanism makes the network distinguish when to update or ignore the information in the hidden state when it is loaded into the gating mechanism module, which is expressed as: x t = tanh(w h · x t-1 + b h ) x δ(w s · x t-1 + b s ) + x t-1 where tanh is the hyperbolic tangent function, δ is the sigmoid function, w h , w s , b h , b s are learnable parameters of the gating mechanism; The reset gate r of the reset gate module is calculated as follows: r = δ(w ir · x t + b ir + w hr · h t-1 + b hr ) wherein, δ is a sigmoid function, w ir , w hr , b ir and b hr are all learnable parameters of the reset gate; In the self-attention module, by splicing and h t-1 Construct a candidate set to prepare for subsequent self-attention mechanism learning data, as follows: In the attention-based simple recurrent network model, self-attention is used to learn the relationship between the past hidden state and the current input, and the corresponding formula is as follows: h′ t-1 = Self-Attention(h s ) where h t-1 denotes the hidden state at past time, h t denotes the hidden state at current time, denotes the candidate set, w in , w hn , b in and b hn are learnable parameters of the candidate set, stack is a matrix concatenation operation, h s is the concatenation result; Step 2: constructing a bidirectional attention-based simple recurrent network model; The output expression of the bidirectional attention-based simple recurrent network model in step 2 is as follows: wherein, represents future state information output by a forward APRN, represents past state information output by a backward APRN, is a forward APRN model, is a backward APRN model, w1, w2, b are learnable parameters for linear combination; Step 3: collecting data generated by industrial Internet of Things devices; Step 4: preprocessing the collected data and completing data set partitioning; Step 5: inputting the data of the data set into the bidirectional attention-based simple recurrent network model for training; Step 6: using the trained bidirectional attention-based simple recurrent network model to predict the results of the test data set; Step 7: applying the trained bidirectional attention-based simple recurrent network model to a real industrial Internet of Things environment and visualizing the results.

Images