Industrial Internet of Things intrusion detection method based on time sequence clustering

Abstract

The invention belongs to the technical field of intrusion detection, and discloses an industrial Internet of Things intrusion detection method based on time sequence clustering. An intrusion detection model based on TCN-GRU-Temporal Attention is provided, and the model combines the advantages of long and short term memory processing, dynamic feature capture and key information focusing, and is especially suitable for processing complex time sequence tasks. A multi-dimensional time sequence clustering algorithm based on an evaluation index is provided, k-medoids clustering edge nodes based on DTW are used, and the intrusion detection accuracy and the system response speed in a federated learning environment are improved. The industrial network intrusion detection method provided by the invention also shows good performance under non-independent identically distributed data, and effectively improves the recognition capability for complex attack modes.

Description

Technical Field

[0001] The present invention relates to the field of intrusion detection technology, and in particular to an industrial Internet of Things intrusion detection method based on time series clustering. Background Art

[0002] With the rapid development of technologies such as 5G, mobile internet, artificial intelligence, and edge computing, industrial control systems are gradually being integrated into the Industrial Internet of Things (IIoT), significantly enhancing the intelligence and connectivity of devices. However, this trend also presents significant challenges to data privacy and security. The devices and data in the IIoT are highly sensitive, and critical infrastructure across various sectors, such as nuclear power plants, petrochemical plants, the national power grid, and water conservancy and transportation systems, relies on industrial control systems. Many industrial control systems are designed without security considerations, exposing structural vulnerabilities and making them potential targets for attackers. As cyberattack techniques mature, more and more attackers are exploiting network vulnerabilities to severely impact industrial production equipment.

[0003] Using intrusion detection systems to detect attacks in the Industrial Internet of Things (IIoT) can effectively improve network security and enable timely responses to potential threats. Traditional intrusion detection methods are increasingly being replaced by deep learning models due to their inability to efficiently handle large-scale network traffic and emerging threats. While deep learning-based intrusion detection models can improve detection rates, obtaining high-quality data is challenging because the training data often involves enterprise privacy concerns. Furthermore, the heterogeneous distribution of data within an enterprise further complicates improving model performance.

[0004] Federated learning, a distributed machine learning technology, effectively reduces the privacy risks associated with data sharing by enabling each participating node to complete model training locally, and enhances the generalization capabilities of intrusion detection systems in industrial IoT environments. However, due to the non-independent and identically distributed nature of the data at each node, traditional federated averaging algorithms face significant limitations in practical applications.

[0005] The paper "Chen J, Yin S, Cai S, et al. An efficient network intrusion detection model based on temporal convolutional networks [C] / / 2021 IEEE 21st International Conference on Software Quality, Reliability and Security (QRS). IEEE, 2021: 768-775." proposed an efficient network intrusion detection method based on temporal convolutional networks (TCN). This method extracts key features from the data set by mining the characteristics of byte sequences in network traffic packets, achieving more accurate abnormal traffic detection. At the same time, TCN has parallel computing capabilities, which reduces computational complexity and improves the detection speed and performance of the model. However, the TCN method is sensitive to data volume and data quality. In high-dimensional features and complex network environments, the model will have problems of overfitting or insufficient generalization ability. In addition, the TCN method has certain limitations in dealing with new or mutated attack modes and cannot accurately capture unknown threats.

[0006] The paper "Popoola SI, Ande R, Adebisi B, et al. Federated deep learning for zero-day botnet attack detection in IoT-edge devices[J]. IEEE Internet of Things Journal, 2021, 9(5): 3930-3944." proposed a federated learning scheme using a federated averaging algorithm. It uses a deep neural network as the basic classifier and distributes the training of deep learning models on multiple edge devices, avoiding the privacy risks of centralized data collection and improving the security and generalization ability of the model. However, when faced with the problem of non-independent and identically distributed (non-IID) data, this method becomes very difficult to build a good global model due to the significant differences in data distribution among devices, resulting in poor intrusion detection accuracy.

[0007] In summary, existing intrusion detection methods face significant challenges in diverse environments and complex data conditions. They struggle to accurately detect new or mutated attacks, and they lack the ability to handle heterogeneous data distribution and generalize effectively. These methods struggle to be effectively applied in industrial IoT environments, have low overall practicality, and offer limited practical value in real-world scenarios. Summary of the Invention

[0008] The purpose of this invention is to propose an industrial network intrusion detection method based on time series clustering. (1) An intrusion detection model based on TCN-GRU-Temporal Attention is proposed. This model combines the advantages of long-term and short-term memory processing, dynamic feature capture and key information focusing, and is particularly suitable for processing complex time series tasks. (2) A multidimensional time series clustering algorithm based on evaluation indicators is proposed, which uses DTW-based k-medoids to cluster edge nodes, thereby improving the accuracy of intrusion detection and system response speed in a federated learning environment. (3) The proposed industrial network intrusion detection method also shows good performance under non-independent and identically distributed data, effectively improving the ability to recognize complex attack patterns.

[0009] The technical solution of the present invention is as follows: an industrial Internet of Things intrusion detection method based on time series clustering, comprising the following steps:

[0010] Step 1: Preprocess the network traffic data of each edge node. Use stacked sparse autoencoders (SSAE) to perform feature dimensionality reduction on the preprocessed data, extract the most valuable feature representations, and obtain a dataset after feature dimensionality reduction.

[0011] Step 2: The central server sends the initial intrusion detection model parameters w to each edge node 0 , where the federated training parameters include the maximum number of communication rounds R, the number of client local training rounds E, and the learning rate η. For a federated learning system consisting of K edge nodes, the global objective function is:

[0012]

[0013] W n represents the model aggregation parameter of cluster n, w K represents the intrusion detection model parameters uploaded by K edge nodes in the subsequent federated learning process, g n (·) represents the objective function of cluster n;

[0014] Step 3: Each edge node uploads the intrusion detection model parameters after local model training The intrusion detection model is trained using the dataset after feature dimension reduction, and the trained intrusion detection model and evaluation indicators are uploaded to the server;

[0015] Step 4: The server performs time series clustering based on the evaluation indicators uploaded by each edge node in multiple communication rounds to divide the edge node clusters;

[0016] Step 5: Determine the optimal number of clusters using the elbow method in the k-medoids algorithm, that is, determine the optimal k value and divide the edge nodes into n = k edge node clusters;

[0017] Step 6: Aggregate the intrusion detection models in each edge node cluster and integrate the intrusion detection model parameters from each edge node in the edge node cluster to generate a personalized model for the edge node cluster that adapts to its specific needs. The calculation method is as follows:

[0018]

[0019] in, is the intrusion detection model parameter of edge node cluster n in the r+1th round, is the personalized model of the rth round of the sth edge node in edge node cluster n; is the data volume of the sth edge node, and the total data volume of edge node cluster n is D n ;

[0020] Step 7: Personalize the edge node cluster n Send it to each edge node in the edge node cluster;

[0021] Step 8: Repeat steps 3 to 7 until the maximum number of communication rounds R is reached, and obtain the final aggregated personalized intrusion detection model for intrusion detection.

[0022] The intrusion detection model includes a temporal convolutional network, a gated recurrent unit, and a temporal attention mechanism;

[0023] The dataset after feature dimensionality reduction is input into the temporal convolutional network, which convolves the input before time t to obtain the output h at time t. t ;

[0024] The output of the temporal convolutional network is passed to the gated recurrent unit for short-term dependency learning to obtain h t ′; The calculation process formula of the gated recurrent unit neural network is as follows:

[0025] z t =Sigmoid(w z [h t-1 ,x t ]+b z )

[0026] r t =Sigmoid(w r [h t-1 ,x t ]+b r )

[0027]

[0028] Among them, h t-1 is the state of the previous moment relative to the current moment t, r t and z t is the reset gate and update gate in the gated recurrent unit; where h t-1 and h t ′ represents the hidden state information of the previous moment and the hidden state information of the current moment respectively; w z 、w r and w h represents weight; b z 、b r and b h represents the coefficient of variation;

[0029] The temporal attention mechanism is used to calculate the influence of each historical time information on the current moment information on the output of the gated recurrent unit, and the result is In order to strengthen the influence of key time points, the calculation formula of the time attention mechanism is as follows:

[0030]

[0031] where h t ′、h s ′ represents the hidden layer output at two different moments in the above gated recurrent unit, α t′ represents the attention weight of the historical hidden state of the current input, c t is the intermediate transformation vector; the output of the temporal attention mechanism is

[0032] The output of the temporal attention mechanism is classified and outputted through a deep neural network layer to generate a predicted attack classification result, which is converted into a predicted classification value of the sample through a SoftMax activation function.

[0033] The training process of the intrusion detection model is as follows: In federated learning, each edge node uses the intrusion detection model to train on the local dataset, where E is the number of local training rounds specified when the intrusion detection model parameters are issued. During each round of training, each edge node uses the cross-entropy loss function to calculate the back-propagation error. The cross-entropy loss function is as follows:

[0034]

[0035] Afterwards, each edge node uses the intrusion detection model parameter w from the previous round r-1 i The hyperparameters distributed by the server are used to calculate the model update through the gradient descent algorithm. The formula is as follows:

[0036]

[0037] Each edge node will update the intrusion detection model parameters containing learning information and the six evaluation indicators obtained by evaluating the model on the test set Upload to the central server; represents the evaluation index uploaded by the i-th edge node in the r-th round of communication, where Indicates the accuracy, represents the accuracy, represents the recall rate, represents the F1 score, represents the false alarm rate, Represents the value of the cross entropy loss function.

[0038] The stacked sparse autoencoder SSAE includes an encoder and a decoder, and the equation is as follows:

[0039] H=f(W x X+b x )

[0040]

[0041] In the encoder of the stacked sparse autoencoder, a sparsity constraint parameter ρ is introduced into many hidden layer neurons. The sparsity constraint parameter calculation formula is as follows:

[0042]

[0043] in, represents the sparsity constraint parameter, m is the number of training samples in the preprocessed dataset, It means that when given input x (i) When , the activation value of hidden neuron j in the stacked sparse autoencoder

[0044] In order to make the average activation value of each hidden neuron close to the predefined sparsity constraint parameter ρ, a penalty term based on Kullback-Leibler divergence is introduced to penalize the sparsity constraint parameter far away from ρ. Apply a penalty; the calculation formula for KL divergence is:

[0045]

[0046] Among them, the first term represents the penalty when the actual activation rate is lower than the expected one, and the second term represents the penalty when the actual activation rate is higher than the expected one; when ρ and When the distribution of is the same, the value of KL divergence is 0; otherwise, the divergence value gradually increases with the degree of deviation;

[0047] The optimization objective function of the stacked sparse autoencoder is:

[0048]

[0049] Where s is the number of hidden layer neurons and λ is the weight of the sparsity penalty term.

[0050] The fourth step is as follows: in the k-medoids algorithm, k time series are randomly selected from all time series as the initial center points of the cluster, and the DTW distance from each time series to each center point is calculated; for any two edge nodes q of the time series containing the evaluation index, i =[P i r-L+1 ,P i r-L+2 ,…,P i r ]and The DTW distance is calculated using the following formula:

[0051]

[0052] Where d=6 is the dimension of the performance indicator time series, The time series of edge nodes and The twisted path of the squared distance metric in a single dimension, are the elements that make up the warped path, and T is the total number of elements; the similarity between two time series increases as the DTW distance decreases;

[0053] When calculating distance, a cost matrix is ​​created whose size is the product of the lengths of the two time series. Each element of the cost matrix represents the DTW distance between any two time points. The shortest distance between two time series is obtained by finding a twisted path. The cost is accumulated from the lower left corner of the cost matrix using dynamic programming, and the twisted path is obtained by tracing back from the upper right corner by moving upward, rightward, or diagonally.

[0054] Move each edge node containing a time series to the nearest cluster, update the central sequence node of each cluster, and calculate the cost function of k-medoids by the following formula:

[0055]

[0056] Among them, m i Representative cluster C i The central time series of r represents C i Time series nodes within;

[0057] Clustering is repeated until the cost function no longer changes.

[0058] The step 5 is specifically as follows: the position where the inflection point of the curve appears in the k-medoids algorithm represents the optimal number of clusters, thereby determining the optimal partitioning scheme;

[0059] Obtain the relationship between k and SSE through step 4, and calculate the two endpoints (1, SSE1) and (N, SSE N ), is calculated as follows:

[0060]

[0061] For all values ​​of k, calculate the point (k, SSE k ) to the point (k,f(k)), is calculated as follows:

[0062] s k =|SSE k -f(k)|,k=1,2,K,N

[0063] Returns the s with the largest distance k The subscript k of is the optimal k value.

[0064] The preprocessing is normalization processing, specifically:

[0065] Convert the character features in network traffic data into numerical features using one-hot encoding;

[0066] All numerical features are standardized. The standardization formula is as follows:

[0067]

[0068] Among them, r is the original eigenvalue, μ is the mean value of the feature, s is the standard deviation of the feature, and z is the standardized eigenvalue;

[0069] Use the minimum-maximum normalization method to map the standardized values ​​to the [0,1] interval. The normalization formula is as follows:

[0070]

[0071] Among them, Z min is the minimum value of the feature, Z max is the maximum value of the feature, and x is the normalized feature value.

[0072] The beneficial effects of the present invention are as follows: (1) By introducing the intrusion detection model in the form of TCN-GRU-Temporal Attention, the ability to process time series data is enhanced, so that the model can capture long-term and short-term dependencies and dynamic features and focus on key information, thereby better adapting to complex intrusion detection needs.

[0073] (2) A multidimensional time series clustering method based on evaluation indicators is proposed, and the k-medoids algorithm of DTW distance is used for automatic clustering to ensure that similar data can be aggregated together when the data distribution is uneven, thereby improving the accuracy and effectiveness of federated learning under non-IID data.

[0074] (3) An intrusion detection method CFL-TGTA-IDS is proposed, which generates personalized models through clustering and tailors models for edge nodes of different clusters, thereby improving the response speed and accuracy of intrusion detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 This is a flow chart of the intrusion detection method based on temporal clustering;

[0076] Figure 2 This is the architecture diagram of the intrusion detection model in the present invention;

[0077] Figure 3 is the cost matrix for calculating DTW distance;

[0078] Figure 4 The detection accuracy trend chart of different methods on the natural gas pipeline dataset;

[0079] Figure 5 The following is a trend chart of the detection accuracy of different methods on the UNSW-NB15 dataset. DETAILED DESCRIPTION

[0080] This paper proposes an industrial IoT intrusion detection method based on time series clustering. The method designs an intrusion detection model based on a TCN-GRU-Temporal Attention hybrid and a multidimensional time series clustering algorithm based on evaluation indicators. This intrusion detection model combines the advantages of long-term and short-term memory processing, dynamic feature capture, and key information focus, making it particularly suitable for processing complex time series tasks. Compared with traditional deep learning models, the intrusion detection model proposed in this paper has lower time and space overhead during training. Figure 1 As shown, the processing flow of the cluster federated learning intrusion detection method of the present invention includes the following steps:

[0081] Step 1: Preprocess the network traffic data of each edge node to obtain data in a format that meets the input requirements of the neural network. The specific steps are as follows:

[0082] The character features in the data are converted into numerical features using one-hot encoding.

[0083] All numerical features are standardized. The standardization formula is as follows:

[0084]

[0085] Among them, r is the original eigenvalue, μ is the mean value of the feature, s is the standard deviation of the feature, and z is the standardized eigenvalue.

[0086] Use the minimum-maximum normalization method to map the standardized values ​​to the [0,1] interval. The normalization formula is as follows:

[0087]

[0088] Among them, Z min is the minimum value of the feature, Z max is the maximum value of the feature, and x is the normalized feature value.

[0089] The normalized data is subjected to feature dimensionality reduction using stacked sparse autoencoders (SSAE), extracting the most valuable feature representations from high-dimensional network data and reducing the interference of useless features. The specific steps are:

[0090] A stacked sparse autoencoder is used for feature extraction to extract effective features from high-dimensional and complex data. The encoder and decoder equations are as follows

[0091] H=f(W x X+b x )

[0092]

[0093] In the construction of stacked sparse autoencoders, a sparsity constraint parameter ρ is introduced into many hidden layer neurons. The sparsity parameter ρ is calculated as follows:

[0094]

[0095] Where m is the number of training samples in the preprocessed dataset, Indicates that when the network is given input x (i) The activation value of hidden neuron j in the stacked sparse autoencoder when .

[0096] In order to make the average activation value of each hidden neuron close to the predefined sparsity parameter ρ, a penalty term based on Kullback-Leibler divergence is introduced to impose a penalty on the actual average activation that is far away from ρ. The calculation formula of KL divergence is:

[0097]

[0098] The first term represents the penalty when the actual activation rate is lower than the expected one, and the second term represents the penalty when the actual activation rate is higher than the expected one. When the distribution of is the same, the value of KL divergence is 0; otherwise, the divergence value will gradually increase with the degree of deviation.

[0099] The optimization objective function of the stacked sparse autoencoder is:

[0100]

[0101] Where s is the number of hidden layer neurons and λ is the weight of the sparsity penalty term.

[0102] Step 2: The central server sends the initial intrusion detection model parameters w to each edge node 0 , where the federated training related parameters include the maximum number of communication rounds R, the number of client local training rounds E, the learning rate η, etc. For a federated learning system consisting of K edge nodes, the global objective function is:

[0103]

[0104] Step 3: Each edge node performs model training locally and uploads model parameters after training The intrusion detection model is trained using the dataset after feature dimensionality reduction in step 2. The trained intrusion detection model and performance indicators are uploaded to the server. The intrusion detection model architecture is shown in the figure below. Figure 2 The specific steps are as follows:

[0105] The dataset after feature dimensionality reduction is input into the Temporal Convolutional Network (TCN), which performs convolution operations on the time series data to capture long-term dependencies.

[0106] The output of TCN is passed to the gated recurrent unit (GRU) for short-term dependency learning. The calculation process formula of the GRU neural network is as follows:

[0107] z t =Sigmoid(w z [h t-1 ,x t ]+b z )

[0108] r t =Sigmoid(w r [h t-1 ,x t ]+b r )

[0109]

[0110]

[0111] The temporal attention mechanism is used to calculate the influence of each historical time information on the current moment information at the output of the gated recurrent unit GRU, thereby strengthening the influence of key time points. The calculation formula of the temporal attention mechanism is as follows:

[0112]

[0113] where h t ′、h s ′ represents the hidden layer output at two different moments in the above gated recurrent unit, α t′ represents the attention weight of the historical hidden state of the current input, c t is the intermediate transformation vector; the output of the temporal attention mechanism is

[0114] The classification output is performed through the deep neural network (DNN) layer to generate the predicted attack classification result, which is converted into the predicted classification value of the sample through the SoftMax activation function. The SoftMax activation function is as follows:

[0115]

[0116] The training process of the intrusion detection model is as follows: each edge node trains on the local dataset for E rounds, where E is the number of local training rounds specified when the model parameters are issued. During each round of training, each edge node calculates the backpropagation error using the cross-entropy loss function, which is as follows:

[0117]

[0118] Afterwards, each edge node uses the model parameter w from the previous round r-1 i The hyperparameters distributed by the server are used to calculate the model update through the gradient descent algorithm. The formula is as follows:

[0119]

[0120] Each edge node will update the model parameters that implicitly contain learning information and the six evaluation indicators obtained by evaluating the model on the test set Upload to the central server. represents the performance indicator uploaded by the i-th edge node in the r-th round of communication, where Indicates the accuracy, represents the accuracy, represents the recall rate, represents the F1 score, represents the false alarm rate, Represents the value of the cross entropy loss function.

[0121] Step 4: The server performs time series clustering based on the evaluation indicators uploaded by each edge node in multiple communication rounds to divide the edge node clusters. The present invention uses the classic k-medoids algorithm based on dynamic time warping (DTW) distance to divide the edge node clusters.

[0122] Randomly select k time series from all time series as the initial center points of the cluster, and calculate the DTW distance from each time series to each center point. For any two edge nodes q containing the time series of the evaluation index i =[P i r-L+1 ,P i r-L+2 ,…,P i r ]and Their DTW distance can be calculated by the following formula:

[0123]

[0124] Where d=6 is the dimension of the performance indicator time series, The time series of edge nodes and The twisted path of the squared distance metric in a single dimension, are the elements that make up the warp path, and T is the total number of elements. The similarity between two sequences increases as the DTW distance decreases.

[0125] When calculating the distance, you need to create a cost matrix whose size is the product of the lengths of the two time series. Figure 3 As shown in Figure 1. Each element of the matrix represents the DTW distance between any two time points. The shortest distance between two sequences is obtained by finding a twisted path. Using dynamic programming, the cost is accumulated from the lower left corner of the cost matrix. The only way to reach the upper right corner is to move upward, rightward, or diagonally. From there, the twisted path is obtained by backtracking.

[0126] Move each edge node containing a time series to the nearest cluster, update the central sequence node of each cluster, and calculate the cost function of k-medoids by the following formula:

[0127]

[0128] where m i Representative cluster C i The central time series of r represents C i Time series nodes within.

[0129] Clustering is repeated until the cost function no longer changes.

[0130] Step 5: Determine the optimal number of clusters using the elbow method, that is, determine the optimal k value. In the k-medoids algorithm, the inflection point of the curve represents the optimal number of clusters, which is used to determine the best partitioning scheme.

[0131] Obtain the relationship between k and SSE through step 4, and calculate the two endpoints (1, SSE1) and (N, SSE N ), is calculated as follows:

[0132]

[0133] For all values ​​of k, calculate the point (k, SSE k ) to the point (k,f(k)), is calculated as follows:

[0134] s k =|SSE k -f(k)|,k=1,2,K,N

[0135] Returns the s with the largest distance k The subscript k of is the optimal k value.

[0136] Step 6: Aggregate the models in each edge node cluster and integrate the model parameters from each node in the cluster to generate a personalized model for the cluster that adapts to its specific needs. The calculation method is as follows:

[0137]

[0138] in, is the model parameter of edge node cluster n in the r+1th round, It is the personalized model of the rth round of the sth edge node in the edge node cluster n. is the data volume of the sth edge node, and the total data volume of edge node cluster n is D n .

[0139] Step 7: Personalize the model of edge node cluster n Sent to each edge node in the cluster.

[0140] Step 8: Repeat steps 3 to 7 until the maximum number of communication rounds R is reached, and obtain the final aggregated personalized intrusion detection model for intrusion detection.

[0141] The specific implementation of the present invention is further described in detail below with reference to the accompanying drawings and examples.

[0142] This example evaluates the proposed method on the natural gas pipeline (GP) and UNSW-NB15 intrusion detection datasets. The GP dataset contains seven types of attack and benign samples, each consisting of 26 features and one label. The UNSW-NB15 dataset contains nine different attack and benign samples, each consisting of 42 features and one label. In both datasets, each attack type employs a unique strategy targeting a specific target. The distribution of samples in each category for the two datasets is shown in Tables 1 and 2, respectively.

[0143] Table 1 Sample distribution of natural gas pipeline dataset

[0144] Label name quantity 0 Benign samples (Bening) 61156 1 Simple Malicious Response Injection (NMRI) 2763 2 Complex Malicious Response Injection (CMRI) 15466 3 Malicious State Command Injection (MSCI) 782 4 Malicious Parameter Command Injection Attack (MPCI) 7637 5 Malicious Function Command Injection (MFCI) 573 6 Denial of Service (DoS) 1837 7 Reconnaissance Attack (Rec) 6805

[0145] Table 2 Sample distribution of UNSW-NB15 dataset

[0146]

[0147]

[0148] The above dataset was tested using an IIoT federated intrusion detection method based on time series clustering. The federated training parameters included the maximum number of communication rounds R = 200, the number of client local training rounds E = 1, the learning rate η = 0.01, and the cluster length L = 20.

[0149] In order to verify the effectiveness of the method proposed in this invention, a set of experiments were conducted to compare the detection effects of the system model with two more advanced federated learning algorithms, including the traditional FedAVG federated learning model and the personalized federated learning model IFCA, on non-independent and identically distributed data sets.

[0150] We set up a complex non-IID scenario for each client participating in federated learning. Specifically, in our experiment, we set up 100 clients, each with 1000-1500 data points. Each client's data consisted of benign data, random two-type attacks, and a small amount of other attack data. Benign data and random two-type attacks accounted for 80%, while other attack data accounted for 20%. 80% of each client's data was used for training, and 20% for testing.

[0151] The experimental results are shown in Table 3, Table 4, Figure 4 、 Figure 5 As shown in Table 3 and Table 4, it can be seen that the method proposed in the present invention is superior to all other methods and achieves a higher accuracy. Figure 4 and Figure 5It can be seen that under the same number of communication rounds, the method proposed in this invention can achieve higher accuracy on the natural gas pipeline dataset and the UNSW-NB15 intrusion detection dataset, and can improve the accuracy and effectiveness of federated learning under non-IID data.

[0152] Table 3 Detection results of different methods on the natural gas pipeline dataset

[0153]

[0154]

[0155] Table 4 Detection effects of different methods on the UNSW-NB15 dataset

[0156] method Accuracy Accuracy Recall F1-score False alarm rate FedAVG 0.7069 0.5219 0.6198 0.6834 0.0270 IFCA 0.8493 0.6237 0.6196 0.7110 0.0683 CFL-TGTA-IDS 0.8539 0.7132 0.7523 0.7639 0.0247

[0157] The above preferred embodiments are only for illustrating the technical concept and features of the present invention, and the purpose is to enable technicians in this field to understand the content of the present invention and implement it. It cannot be used to limit the scope of protection of the present invention. Any equivalent changes or modifications made according to the essence of the present invention fall within the scope of protection of the present invention.

Claims

1. An industrial Internet of Things intrusion detection method based on time series clustering, characterized in that: The following steps are involved: Step 1: Preprocess the network traffic data of each edge node. Use stacked sparse autoencoders (SSAE) to perform feature dimensionality reduction on the preprocessed data, extract the most valuable feature representations, and obtain a dataset after feature dimensionality reduction. Step 2: The central server sends the initial intrusion detection model parameters w to each edge node 0 , where the federated training parameters include the maximum number of communication rounds R, the number of client local training rounds E, and the learning rate η. For a federated learning system consisting of K edge nodes, the global objective function is: W n represents the model aggregation parameter of cluster n, w K represents the intrusion detection model parameters uploaded by K edge nodes in the subsequent federated learning process, g n (·) represents the objective function of cluster n; Step 3: Each edge node uploads the intrusion detection model parameters after local model training The intrusion detection model is trained using the dataset after feature dimension reduction, and the trained intrusion detection model and evaluation indicators are uploaded to the server; Step 4: The server performs time series clustering based on the evaluation indicators uploaded by each edge node in multiple communication rounds to divide the edge node clusters; Step 5: Determine the optimal number of clusters using the elbow method in the k-medoids algorithm, that is, determine the optimal k value and divide the edge nodes into n = k edge node clusters; Step 6: Aggregate the intrusion detection models in each edge node cluster and integrate the intrusion detection model parameters from each edge node in the edge node cluster to generate a personalized model for the edge node cluster that adapts to its specific needs. The calculation method is as follows: in, is the intrusion detection model parameter of edge node cluster n in the r+1th round, is the personalized model of the rth round of the sth edge node in edge node cluster n; is the data volume of the sth edge node, and the total data volume of edge node cluster n is D n ; Step 7: Personalize the edge node cluster n Send it to each edge node in the edge node cluster; Step 8: Repeat steps 3 to 7 until the maximum number of communication rounds R is reached, and obtain the final aggregated personalized intrusion detection model for intrusion detection.

2. The industrial Internet of Things intrusion detection method based on time series clustering according to claim 1 is characterized in that: The intrusion detection model includes a temporal convolutional network, a gated recurrent unit, and a temporal attention mechanism; The dataset after feature dimensionality reduction is input into the temporal convolutional network, which convolves the input before time t to obtain the output h at time t. t ; The output of the temporal convolutional network is passed to the gated recurrent unit for short-term dependency learning to obtain h t ′; The calculation process formula of the gated recurrent unit neural network is as follows: z t =Sigmoid(w z [h t-1 ,x t ]+b z ) r t =Sigmoid(w r [h t-1 ,x t ]+b r ) Among them, h t-1 is the state of the previous moment relative to the current moment t, r t and z t is the reset gate and update gate in the gated recurrent unit; where h t-1 and h t ′ represents the hidden state information of the previous moment and the hidden state information of the current moment respectively; w z 、w r and w h represents weight; b z 、b r and b h represents the coefficient of variation; The temporal attention mechanism is used to calculate the influence of each historical time information on the current moment information on the output of the gated recurrent unit, and the result is In order to strengthen the influence of key time points, the calculation formula of the time attention mechanism is as follows: where h t ′、h s ′ represents the hidden layer output at two different moments in the above gated recurrent unit, α t′ represents the attention weight of the historical hidden state of the current input, c t is the intermediate transformation vector; the output of the temporal attention mechanism is The output of the temporal attention mechanism is classified and outputted through a deep neural network layer to generate a predicted attack classification result, which is converted into a predicted classification value of the sample through a SoftMax activation function.

3. The industrial Internet of Things intrusion detection method based on time series clustering according to claim 1 is characterized in that: The training process of the intrusion detection model is as follows: In federated learning, each edge node uses the intrusion detection model to train on the local dataset, where E is the number of local training rounds specified when the intrusion detection model parameters are issued. During each round of training, each edge node uses the cross-entropy loss function to calculate the back-propagation error. The cross-entropy loss function is as follows: Afterwards, each edge node uses the intrusion detection model parameter w from the previous round r-1 i The hyperparameters distributed by the server are used to calculate the model update through the gradient descent algorithm. The formula is as follows: Each edge node will update the intrusion detection model parameters containing learning information and the six evaluation indicators obtained by evaluating the model on the test set Upload to the central server; represents the evaluation index uploaded by the i-th edge node in the r-th round of communication, where Indicates the accuracy, represents the accuracy, represents the recall rate, represents the F1 score, represents the false alarm rate, Represents the value of the cross entropy loss function.

4. The industrial Internet of Things intrusion detection method based on time series clustering according to claim 1 is characterized in that: The stacked sparse autoencoder SSAE includes an encoder and a decoder, and the equation is as follows: H=f(W x X+b x ) In the encoder of the stacked sparse autoencoder, a sparsity constraint parameter ρ is introduced into many hidden layer neurons. The sparsity constraint parameter calculation formula is as follows: in, represents the sparsity constraint parameter, m is the number of training samples in the preprocessed dataset, It means that when given input x (i) When , the activation value of hidden neuron j in the stacked sparse autoencoder In order to make the average activation value of each hidden neuron close to the predefined sparsity constraint parameter ρ, a penalty term based on Kullback-Leibler divergence is introduced to penalize the sparsity constraint parameter far away from ρ. Apply a penalty; the calculation formula for KL divergence is: Among them, the first term represents the penalty when the actual activation rate is lower than the expected one, and the second term represents the penalty when the actual activation rate is higher than the expected one; when ρ and When the distribution of is the same, the value of KL divergence is 0; otherwise, the divergence value gradually increases with the degree of deviation; The optimization objective function of the stacked sparse autoencoder is: Where s is the number of hidden layer neurons and λ is the weight of the sparsity penalty term.

5. The industrial Internet of Things intrusion detection method based on time series clustering according to claim 4 is characterized in that: The fourth step is as follows: in the k-medoids algorithm, k time series are randomly selected from all time series as the initial center points of the cluster, and the DTW distance from each time series to each center point is calculated; for any two edge nodes q of the time series containing the evaluation index, i =[P i r-L+1 ,P i r-L+2 ,...,P i r ]and The DTW distance is calculated using the following formula: Where d=6 is the dimension of the performance indicator time series, The time series of edge nodes and The twisted path of the squared distance metric in a single dimension, are the elements that make up the warped path, and T is the total number of elements; the similarity between two time series increases as the DTW distance decreases; When calculating distance, a cost matrix is ​​created whose size is the product of the lengths of the two time series. Each element of the cost matrix represents the DTW distance between any two time points. The shortest distance between two time series is obtained by finding a twisted path. The cost is accumulated from the lower left corner of the cost matrix using dynamic programming, and the twisted path is obtained by tracing back from the upper right corner by moving upward, rightward, or diagonally. Move each edge node containing a time series to the nearest cluster, update the central sequence node of each cluster, and calculate the cost function of k-medoids by the following formula: Among them, m i Representative cluster C i The central time series of r represents C i Time series nodes within; Clustering is repeated until the cost function no longer changes.

6. The industrial Internet of Things intrusion detection method based on time series clustering according to claim 5 is characterized in that: The step 5 is specifically as follows: the position where the inflection point of the curve appears in the k-medoids algorithm represents the optimal number of clusters, thereby determining the optimal partitioning scheme; Obtain the relationship between k and SSE through step 4, and calculate the two endpoints (1, SSE1) and (N, SSE N ), is calculated as follows: For all values ​​of k, calculate the point (k, SSE k ) to the point (k,f(k)), is calculated as follows: s k =|SSE k -f(k)|,k=1,2,K,N Returns the s with the largest distance k The subscript k of is the optimal k value.

7. The industrial Internet of Things intrusion detection method based on time series clustering according to any one of claims 1 to 6, characterized in that: The preprocessing is normalization processing, specifically: Convert the character features in network traffic data into numerical features using one-hot encoding; All numerical features are standardized. The standardization formula is as follows: Among them, r is the original eigenvalue, μ is the mean value of the feature, s is the standard deviation of the feature, and z is the standardized eigenvalue; Use the minimum-maximum normalization method to map the standardized values ​​to the [0,1] interval. The normalization formula is as follows: Among them, Z min is the minimum value of the feature, Z max is the maximum value of the feature, and x is the normalized feature value.

Images