A cyber-physical system intrusion detection method

Abstract

The application discloses a network physical system intrusion detection method, which comprises the following steps: data preprocessing is performed on an intrusion detection data set, the data preprocessing comprises character type data value processing, data normalization processing and data imbalance processing; an optimal feature subset is selected from the preprocessed intrusion detection data set through a binary grey wolf optimization algorithm; a teacher network model is pre-trained according to the selected optimal feature subset; an intrusion detection model training process is as follows: initializing intrusion model parameters, determining the structure of a student network model; two groups of network flows of different categories are input into the intrusion detection model for training based on the optimal feature subset; the error of a K-fold cross-training process is adjusted according to a knowledge distillation loss until the student network model reaches convergence; and the intrusion detection model is tested to obtain the classification result of each data. The application realizes the characteristics of lightweight, real-time and unsupervised of the Internet of Things intrusion detection, reduces the excessive dependence on labels and improves the generalization ability.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of industrial network, and particularly relates to a network physical system intrusion detection method. BACKGROUND

[0002] Cyber-physical systems (CPS) are mechanisms that are based on computer algorithmic control or monitoring, and the whole system is integrated with the network. The cyber-physical system is usually referred to as a large-scale, geographically dispersed, complex and heterogeneous Internet of Things. In recent years, the development and deployment of various types of cyber-physical systems have increased exponentially, bringing great influence to all aspects of daily life, such as power grids, transportation systems, medical devices and home appliances. Many such systems are deployed in critical infrastructure, life support devices, or places that are extremely important to our daily life.

[0003] However, the diversity of CPS applications deployed across networks in the Internet of Things makes it vulnerable to network attacks and physical attacks between different levels of systems, especially in the message transmission of intelligent manufacturing processes. This introduces security risks into the CPS application, causing the program to become out of control and harm people who rely on the program. Industrial CPS attaches great importance to communication and network capabilities, which collect physical world object state data in real time through networks and interfaces and send them to servers. The servers make corresponding processing after receiving the data and return them to the physical end devices to make corresponding changes. Usually, attackers will hack into the CAN bus network and hijack the data sent to the server, thereby endangering the equipment of the industrial CPS. The supervisory control and data acquisition (SCADA) system involves monitoring and collecting signals (such as vibration, temperature and TX&RX packet data) generated across the network, in which a deep learning (DL) based anomaly detection module is deployed to identify anomalies.

[0004] Intrusion detection system (IDS) can detect intrusion behaviors that other security mechanisms cannot prevent, and it plays an important role as a second line of defense in protecting CPS. According to the source of data, intrusion detection systems can be divided into: host-based intrusion detection and network-based intrusion detection. Host-based intrusion detection only monitors the host, needs to be installed on each host, and cannot observe network traffic and analyze network-related behavior information. Network-based intrusion detection observes and analyzes real-time network traffic and monitors multiple hosts, aiming to collect packet information and view the contents therein to detect intrusion behaviors in the network. Modern artificial intelligence technologies, including intelligent sensing and intelligent control, are widely used in behavior monitoring in smart manufacturing. However, there are still some challenges in detecting abnormal traffic in industrial CPS. First, the hybrid cyber-physical environment constructed with cloud facilities is a large and complex distributed system, so there are a large number of industrial data streams (such as instructions, accelerometers, videos, images, etc.) generated through various physical systems and sensors. Another key issue is that such abnormal events occur with a very low probability in the real world, resulting in a lack of good labeled data for model training. Not only that, the missing of monitoring data, which can be caused by different factors such as sensor failure, data transmission error, etc., will bring more difficulties to data collection and model training, making it difficult to achieve anomaly detection. In addition, most nodes in the Internet of Things network are deployed in resource-limited devices, such as limited power, computing, communication and storage capabilities, etc. In order to reduce the damage caused by malicious attacks in industrial CPS, real-time anomaly detection with high precision and timeliness is usually required to facilitate overall performance monitoring based on data streams obtained and transmitted by distributed nodes at different levels across systems.

[0005] In summary, how to compress the size of the intrusion detection model without reducing the efficiency of the model and improve the generalization ability of the model has practical significance. SUMMARY

[0006] To this end, the present application provides an Internet of Things intrusion detection method based on self-supervised learning and self-knowledge distillation, which realizes lightweight, real-time and unsupervised Internet of Things intrusion detection, reduces the over-reliance on labels, and improves the generalization ability.

[0007] In order to achieve the above purpose, the present application provides the following technical scheme: a network physical system intrusion detection method, comprising:

[0008] (1) data preprocessing is performed on the intrusion detection data set, and the data preprocessing includes character type data numerical value processing, data normalization processing and data imbalance processing;

[0009] (2) the optimal feature subset of the preprocessed intrusion detection data set is selected by using a binary grey wolf optimization algorithm.

[0010] (3) Pre-training the teacher network model according to the selected optimal feature subset;

[0011] (4) KD-TCNN intrusion detection model training:

[0012] (41) Initialize the KD-TCNN intrusion detection parameters and determine the structure of the student network model;

[0013] (42) Based on the optimal feature subset, input two groups of network traffic of different categories into the KD-TCNN intrusion detection model for training;

[0014] (43) Adjust the error of the K-fold cross-training process according to the knowledge distillation loss until the student network model converges;

[0015] (5) Test the KD-TCNN intrusion detection model, and input the preprocessed test data set into the student network to obtain the classification result of each data.

[0016] As a preferred scheme of the network physical system intrusion detection method, in step (1), the intrusion detection data set includes the NSL-KDD data set, and the numerical value processing process of the character type data converts the element type of the character type in the NSL-KDD data set into numerical value data.

[0017] As a preferred scheme of the network physical system intrusion detection method, in step (1), the data normalization processing process, according to the actual distribution of the data, the normalization preprocessing formula is:

[0018]

[0019] Wherein, x i is the i-th feature value in the original data, is the minimum value in the i-th feature value, is the maximum value in the i-th feature value, is the result after normalization.

[0020] As a preferred scheme of the network physical system intrusion detection method, in step (2), the most suitable solution is named as alpha, the second and third best solutions are named as beta and delta respectively, and the remaining candidate solutions are assumed as omega, and the steps of the grey wolf optimization algorithm include:

[0021] Surrounding prey stage: a mathematical model of surrounding behavior is established;

[0022] Hunting stage: guided by alpha, beta and delta may participate in hunting; the remaining omega updates the position according to the position of the best search agent;

[0023] Attacking prey stage: simulate approaching prey, parameter a linear update in each iteration;

[0024] Feature subset evaluation stage, using convolutional neural network as learning algorithm, using fitness function to evaluate gray wolf position, selecting the feature subset with the lowest fitness function value for feature selection and dimension reduction to obtain the optimal feature subset.

[0025] As a preferred solution of the network physical system intrusion detection method, in step (42), the KD-TCNN intrusion detection model training adopts a knowledge distillation framework based on a triple convolutional neural network.

[0026] As a preferred solution of the network physical system intrusion detection method, in step (42), three losses are considered in the design of the loss function, including a triple loss L triplet based on the distance between anchor samples and positive and negative samples, hard a cross-entropy loss L soft between the student network output and the label, and a KL divergence loss L hard between the teacher and student networks.

[0027] As a preferred solution of the network physical system intrusion detection method, to constrain the difference between the output of the student network model and the probability distribution of the real label, the cross-entropy loss L hard between the student network model output and the real label is used as part of the model loss function.

[0028] As a preferred solution of the network physical system intrusion detection method, a coefficient is added to the loss term to adjust the contribution of each loss to the overall loss function, and the loss function L of the model is defined as follows:

[0029] L=L KD +θL triplet

[0030] Where θ is a balance coefficient that controls the knowledge distillation loss and the triple loss during model training, L KD is the knowledge distillation part loss, triplet L is the triple loss based on the distance between anchor samples and positive and negative samples.

[0031] As a preferred solution of the network physical system intrusion detection method, the knowledge distillation framework based on a triple convolutional neural network uses a depth separable convolution.

[0032] As a preferred solution of the network physical system intrusion detection method, in step (43), the K-fold cross-training process includes:

[0033] (431) define the model and learning rate, and divide the data set into training and test data sets;

[0034] (432)Divide the training data set into K parts, one part as a validation set, and the remaining K-1 parts as a training set;

[0035] (433)Define a gradient optimizer with a learning rate using a decay strategy, K-1 parts of data are used for model training, and the remaining one part of data is used for testing the model;

[0036] (434)Repeat step (433) K times to obtain the optimal model and the optimal model performance indicator on the test data set.

[0037] The present application has the following advantages: data preprocessing is performed on the intrusion detection data set, the data preprocessing includes character data value processing, data normalization processing and data imbalance processing; the binary grey wolf optimization algorithm is used to select the optimal feature subset of the preprocessed intrusion detection data set; the teacher network model is pre-trained according to the selected optimal feature subset; the KD-TCNN intrusion detection model training process: initializing the KD-TCNN intrusion detection parameters, determining the structure of the student network model; based on the optimal feature subset, two groups of network traffic of different categories are input into the KD-TCNN intrusion detection model for training; the error of the K-fold cross-training process is adjusted according to the knowledge distillation loss until the student network model converges; the KD-TCNN intrusion detection model is tested, and the preprocessed test data set is input into the student network to obtain the classification result of each data. The present application uses knowledge distillation to make the output of the student model as close as possible to the teacher model, which can make the student network learn the information between classes in the teacher network, can process and analyze large-scale data in real time, and reduce the model parameter quantity; the difference between the output of the teacher network model and the student network model can be reduced, thereby improving the performance of the student model; the present application uses deep separable convolution to further reduce the parameter quantity and calculation amount of the model, so that the intrusion detection model can be deployed to the nodes with limited computing power in the Internet of Things network, reduce the intrusion detection time and realize real-time detection; the verification results show that the present application is superior to the traditional deep learning model in terms of parameter quantity and other performance indicators. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed in the following embodiment or prior art description will be briefly introduced. Obviously, the drawings in the following description are only exemplary, and other embodiments can be obtained from the provided drawings without creative labor for those skilled in the art.

[0039] Figure 1 The network physical system intrusion detection method flowchart provided by the embodiment of the present application;

[0040] Figure 2The flowchart of the binary grey wolf optimization algorithm feature selection in the network physical system intrusion detection method provided by the embodiment of the application is as follows:

[0041] Figure 3 The knowledge distillation framework based on the ternary convolutional neural network in the network physical system intrusion detection method provided by the embodiment of the application. DETAILED DESCRIPTION

[0042] The embodiments of the application are described below by specific specific embodiments, and those skilled in the art can easily understand other advantages and effects of the application from the content disclosed in the specification. Obviously, the described embodiments are part of the embodiments of the application, not all. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor belong to the scope of protection of the application.

[0043] Auxiliary Figure 1 The network physical system intrusion detection method provided by the application comprises:

[0044] (1) data preprocessing is performed on the intrusion detection data set, and the data preprocessing comprises character type data value processing, data normalization processing and data imbalance processing;

[0045] (2) the binary grey wolf optimization algorithm is used to select the optimal feature subset of the preprocessed intrusion detection data set;

[0046] (3) the teacher network model is pre-trained according to the selected optimal feature subset;

[0047] (4) KD-TCNN intrusion detection model training:

[0048] (41) the KD-TCNN intrusion detection parameters are initialized, and the structure of the student network model is determined;

[0049] (42) based on the optimal feature subset, two groups of network traffic of different categories are input into the KD-TCNN intrusion detection model for training;

[0050] (43) the error of the K-fold cross-training process is adjusted according to the knowledge distillation loss until the student network model converges;

[0051] (5) the KD-TCNN intrusion detection model is tested, and the preprocessed test data set is input into the student network to obtain the classification result of each data.

[0052] In this embodiment, binary grey wolf optimization algorithm is used to select the optimal feature subset. Grey wolf optimization algorithm is a swarm intelligence algorithm simulating the hunting behavior of grey wolves. According to the social hierarchy of grey wolves, hunting tasks such as surrounding, chasing and attacking are allocated to grey wolves of different levels to complete the hunting behavior, so as to realize the process of global optimization.

[0053] In order to model the social hierarchy of grey wolves when designing the grey wolf optimization algorithm, the best solution is named alpha, the second and third best solutions are named beta and delta respectively, and the remaining candidate solutions are assumed to be omega. In the grey wolf optimization algorithm, the hunting process is guided by alpha, beta and delta, and omega wolves follow the three wolves. The specific steps of the grey wolf optimization algorithm are as follows:

[0054] Step 1. Surrounding prey stage:

[0055] The wolf pack needs to surround the prey first to hunt the prey. In order to establish a mathematical model of surrounding behavior, the surrounding behavior is explained by the following equation:

[0056]

[0057]

[0058]

[0059] Where t is the iteration number, and is a coefficient vector, is the position of the prey, is the position of the grey wolf,

[0060] Alpha decreases linearly from 2 to 0 during iteration, and is a random vector in the range of [0, 1].

[0061] Step 2. Hunting stage:

[0062] Hunting is usually guided by alpha wolf, and beta and delta wolves may participate in hunting. It is assumed that alpha, beta and delta have a better understanding of the potential position of the prey, and other wolves (including omega wolves) are required to update their positions according to the position of the best search agent. The position update formula is as follows:

[0063]

[0064]

[0065] Where, and are the three optimal solutions in the population at a given iteration t, and defined by equation (3). and defined by equations (9)-(11), respectively:

[0066]

[0067] where and defined by equation (4).

[0068] Step 3. Attacking the prey stage:

[0069] When the prey stops moving, the gray wolf completes the hunting process by attacking. To simulate the approach to the prey, the parameter a is linearly updated in each iteration according to equation (12), ranging from 2 to 0.

[0070] a = 2 - t(2 / MaxIter) (12)

[0071] where t is the current iteration number and MaxIter is the maximum number of iterations allowed for optimization.

[0072] In the binary gray wolf optimization algorithm, the update formula of the wolf position is a function of three position vectors, i.e., x α , x β , and x δ , which attracts each wolf to the first three best solutions. In the binary gray wolf optimization algorithm, the solution pool is in binary form at any given time, and all solutions are in the corners of a hypercube. The present invention uses the second model of the binary gray wolf optimization algorithm, bGWO2, in which only the updated gray wolf position vector is binary. The gray wolf position update formula is shown in equation (13):

[0073]

[0074] where rand is a random number uniformly distributed from [0, 1], is the updated binary position in dimension d in iteration t, and the sigmoid function is defined as follows:

[0075]

[0076] The binary gray wolf optimization algorithm adaptively searches the feature space to find the optimal feature subset, which is the feature subset with the highest classification performance and the smallest number of selected features. The fitness function used to evaluate the gray wolf position in the binary gray wolf optimization is shown in equation (15):

[0077]

[0078] Wherein, P is the classification accuracy, L is the number of selected optimal feature subset elements, N is the total number of features, alpha and beta are the weights of classification accuracy and the number of selected feature subsets, alpha is in [0, 1] and beta = 1-alpha.

[0079] The selection of the optimal feature subset of the network traffic data is completed on the training subset. In the feature subset evaluation stage, the convolutional neural network is used as the learning algorithm, and formula (15) is used as the fitness function. The feature subset with the lowest fitness function value is selected, the feature selection and dimension reduction are realized, and thus the feature subset with the best classification effect is obtained. Figure 2 The flowchart of the entire binary grey wolf optimization algorithm feature selection is shown.

[0080] Specifically, knowledge distillation is a common method for model compression. Model compression refers to "distilling" the feature representation learned by a complex and powerful teacher network in a teacher-student framework to a student network with fewer parameters and weaker learning ability, so as to obtain a student network with fast speed and strong ability but small model. In another aspect, knowledge distillation makes the output of the student model as close to the teacher model as possible, so that the student network can learn more softened knowledge from the teacher network, which contains information between classes, which is not available in traditional one-hot encoding. Because the purpose of knowledge distillation is to increase the similarity between the teacher model and the student model, and deep metric learning aims to reduce the distance between similar input samples and increase the distance between different input samples. Therefore, the function of metric learning to reduce the difference between similar inputs can be used for knowledge distillation to reduce the difference between the outputs of the teacher model and the student model, thereby improving the performance of the student model. Usually, siamese neural networks and triplet neural networks are two commonly used neural network architectures for metric learning. Since siamese neural networks can only consider the distance between two samples, siamese neural networks must uniquely define the similarity between two samples, for example: if there are two different male images, they should be judged as similar in the case of gender concept. However, in the case of individual concept, they should be judged as dissimilar. It is difficult to express these multiple concepts in siamese neural networks, while triplet neural networks learn to make the distance between anchor samples and positive samples closer relative to the distance between anchor samples and negative samples. Therefore, multiple similar concepts can be considered without relying on a similar concept, so the present application reduces the difference between the outputs of the teacher model and the student model through the triplet neural network in deep metric learning.

[0081] Referring to Figure 3 , a knowledge distillation framework based on a triplet convolutional neural network is shown.

[0082] To train the KD-TCNN (Knowledge Distillation-Triple Convolutional Neural Network) intrusion detection model, network traffic sample data x a The input is fed into the pre-trained teacher network, and outputs through the softmax output layer. Calculate the probability vectors of each category

[0083]

[0084] Here, T is usually set to a temperature parameter of 1, which corresponds to the softmax activation function. Using a higher value for T will produce a smoother probability distribution on the class, which is also known as soft pseudo-labels.

[0085] With x a Different categories of network traffic sample data x n and network traffic sample data x a The data is fed into the student network and output through the softmax output layer. and Calculate the probability vectors of each category and

[0086]

[0087] To ensure the prediction accuracy and false alarm rate of anomaly detection in industrial CPS data, the loss function design of this invention considers three types of losses, including a triplet loss L based on the distance between the anchor sample and the positive and negative samples. triplet Student network output and label cross-entropy loss L hard KL divergence loss L in teacher-student network soft .

[0088] For the same sample, the outputs of the teacher model and the student model are considered anchored and positive, respectively; similarly, this invention considers samples whose outputs from the student model differ from the positive sample class, referred to as negative samples. The triplet loss is effective in reducing the distance between the anchor sample and the positive sample output, and in increasing the distance between the anchor sample and the negative sample output. This invention introduces this technique into knowledge distillation, defining the triplet loss for knowledge distillation as follows:

[0089]

[0090] Where m is a manually set hyperparameter, and Ω is a set of industrial CPS intrusion detection datasets.

[0091] In order to make the softmax output of the student model close to the softmax output of the teacher model, the present application uses the KL divergence of the softmax outputs of the two models as part of the model training loss, defines the KL divergence loss L of the teacher-student network soft As follows:

[0092]

[0093] Wherein, KL(p,q) is to calculate the KL divergence between the softmax output of the student model and the softmax output of the teacher model, and the formula for calculating the KL divergence is defined as follows:

[0094]

[0095] In order to constrain the difference between the output of the student network and the probability distribution of the real label, the present application takes the cross-entropy loss between the student network output and the real label as part of the model loss function, and defines the cross-entropy loss L between the student network output and the real label hard As follows:

[0096]

[0097] Wherein, y i,k represents the i-th sample is label k; p i,k represents the probability that the i-th sample is predicted to be label k; N is the total number of samples in the data set, and K is the total number of categories.

[0098] The present application defines the knowledge distillation part loss L KD As follows:

[0099] L KD = αT 2 *L hard +(1-α)*L soft (23)

[0100] Wherein, T is the temperature parameter used to soften the label distribution in the above, and a is the weight coefficient for constraining L hard and L soft , which is a hyperparameter set artificially.

[0101] Since the model loss function is composed of multiple parts, the present application needs to add a coefficient to the loss term to adjust the contribution of each loss to the overall loss function, so the loss function L of the model is defined as follows:

[0102] L = L KD + θL triplet (24)

[0103] Wherein θ is a balance coefficient for controlling the balance between the knowledge distillation loss and the triplet loss in the model training process. The application adjusts the error of the training process according to the loss Loss until the student model reaches a convergence state, and saves the optimal student model for use in subsequent test experiments.

[0104] In the embodiment, the core idea of the depth separable convolution is to divide a complete convolution operation into two steps, respectively passing through a depthwise convolution and a pointwise convolution.

[0105] A convolution kernel of the depthwise convolution is responsible for a channel, and a channel is only convolved by one convolution kernel. The number of feature map channels generated by this process is exactly the same as the number of input channels, so the parameter amount of the depthwise convolution is:

[0106] Parameter amount = convolution kernel W * convolution kernel H * input channel number (25)

[0107] The calculation amount of the depthwise convolution is:

[0108] Calculation amount = convolution kernel W * convolution kernel H * (picture W - convolution kernel W + 1) * (picture H - convolution kernel H + 1) * input channel number (26)

[0109] The number of feature maps after the depthwise convolution is the same as the number of channels of the input layer, and the feature map size cannot be expanded. Moreover, this operation independently performs convolution operation on each channel of the input layer, without effectively utilizing the feature information of different channels at the same spatial position, so the pointwise convolution is needed to combine these feature maps to generate new feature maps.

[0110] The operation of the pointwise convolution is similar to the conventional convolution operation, and the size of the convolution kernel thereof is 1*1*M, and M is the number of channels of the previous layer. The convolution operation here will weight and combine the feature maps of the previous step in the depth direction to generate new feature maps, so the parameter amount of the pointwise convolution is:

[0111] Parameter amount = 1 * 1 * input channel number * output channel number (27)

[0112] The calculation amount of the pointwise convolution is:

[0113] Calculation amount = 1 * 1 * feature map W * feature map H * input channel number * output channel number (28)

[0114] By decomposing the conventional convolution operation into two steps, the computational load and the number of parameters of the convolution layer are greatly reduced. For example, assuming that the input feature map size is 224x224x16, the output feature map size is 224x224x32, the convolution kernel size is 3x3, and the number of parameters of the conventional convolution is 3x3x16x32=4608, the computational load is 3x3x(224-2)x(224-2)x16x32≈230 million, and the number of parameters of the depth separable convolution is 3x3x16+1x1x16x32=656, and the computational load is 3x3x(224-2)x(224-2)x16+3x3x16x32≈7.1 million. Obviously, the computational load and the number of parameters of the depth separable convolution are much smaller than those of the conventional convolution. Therefore, the depth separable convolution can be applied to the intrusion detection model, and thus deployed to the nodes with limited computing power in the Internet of Things network. In this way, the intrusion detection time can be greatly reduced.

[0115] In this embodiment, the K-fold cross-training neural network training method is similar to K-fold cross-validation. The K-fold cross-training divides the training data set into K parts, and then performs validation set on each subset data, and the remaining K-1 subset data is used as the training set. Different from K-fold cross-validation, K-fold cross-validation obtains K models, and the average of the classification accuracy of the final validation set of the K models is used as the performance index of the classifier. K-fold cross-training only obtains one model. The model is continuously optimized based on the last training each time, which is similar to the pre-training idea. The model has strong prior knowledge before each training, which can make the model converge faster and avoid falling into local optimum.

[0116] The specific steps of K-fold cross-training are as follows:(431) define the model and the learning rate, and divide the data set into training and test data sets;(432) divide the training data set into K parts, one part as the validation set, and the remaining K-1 parts as the training set;(433) define the gradient optimizer, the learning rate adopts the decay strategy, and K-1 parts of data are used for model training, and the remaining part of data is used for testing the model;(434) repeat step(433) K times to obtain the optimal model and the optimal model performance index in the test data set. The algorithm pseudo code of K-fold cross-training is shown in Algorithm 1:

[0117]

[0118] In order to verify the detection ability of the KD-TCNN intrusion detection model to the network-based industrial CPS intrusion detection system, the application not only carries out intrusion detection on the older intrusion detection data set NSL-KDD, but also carries out intrusion detection on the newer intrusion detection data set CICIDS2017.

[0119] Since the input data set must conform to the input format of the convolutional neural network, the experimental data set needs to be preprocessed, and the preprocessing steps are as follows:

[0120] First, the character type data is numerically processed.

[0121] Taking the NSL-KDD data set as an example, the elements of the three features of protocol, flag and service are character type, which need to be converted into numerical type data, for example, the protocol contains 3 categories of UDP, TCP and ICMP, and the protocol type is processed into the form of 0, 1 and 2, and the processing process of other features is similar. After processing, the dimension of each network flow is 41. In order to conform to the input format of the convolutional neural network, the network flow needs to be resampled, and the network flow of the NSL-KDD data set is converted into an 8x8 gray scale format, and the network flow of the CICIDS2017 data set is converted into a 10x10 gray scale format.

[0122] Second, data normalization processing;

[0123] In order to cancel the dimension, make the gradient always move in the direction of the minimum value and accelerate the convergence, the normalized data after feature mapping needs to be processed. As a linear scale method, Min-Max normalization is used for data preprocessing in machine learning. However, due to its dependence on the minimum and maximum values of the sample, Min-Max normalization has obvious limitations. Therefore, a new scaling method is used to process the case where the value range of each feature changes greatly. Since the value difference of each feature in the NSL-KDD and CICIDS2017 data sets is very large, we use a hybrid data preprocessing method. According to the actual distribution of the data, our normalization preprocessing method is shown in equation (29).

[0124]

[0125] where x i is the i-th feature value in the original data, is the minimum value of the i-th feature value, is the maximum value of the i-th feature value, is the result after normalization.

[0126] Third, data imbalance processing:

[0127] In the industrial CPS (industrial cyber-physical system) intrusion detection scenario, some malicious attack methods only account for a small part of all network traffic. For example, there is a serious data imbalance problem in the NSL-KDD dataset, and the R2L and U2R attacks in the NSL-KDD training set only account for 0.79% and 0.041% of the training set, respectively, so the classification model will tend to be biased to the majority class, resulting in a large false positive rate. In order to alleviate this problem, the present application uses the SVMSMOTE (Support Vector Machine Synthetic Minority Over-sampling) algorithm to oversample the attack types with fewer quantities, but only for the training data set without changing the data distribution in the test data set (avoiding the model over-reliance on generated data).

[0128] Because the network intrusion detection data is relatively complex, the accuracy rate cannot be used as the only evaluation standard for evaluating the model, and there is an obvious data imbalance phenomenon in the data set, so the present application uses the accuracy rate (ACC), weighted precision (Weighted Precision, WPrecision), weighted detection rate (Weighted DR, WDR) and weighted F1 score Weighted FMeasure (WFMeasure) as the evaluation index of the intrusion detection model, and the accuracy and stability of the model are verified by comprehensive comparison of the above indexes.

[0129] In order to further verify the effectiveness of the knowledge distillation intrusion detection model based on the triple convolutional neural network proposed by the present application, an ablation experiment will be performed on the KD-TCNN model. The KD-TCNN model uses four parts of feature selection, deep metric learning, knowledge distillation and K-fold cross training, so the present application performs an ablation experiment on the four parts on the NSL-KDD data set, and the experimental results are shown in Table 1. As can be seen from the table, the accuracy rate of the benchmark student model is the lowest, 96.86%, after adding the feature selection operation, the redundant features are eliminated, and the performance of the model is improved, at this time the model accuracy is 96.88%. Then introduce the knowledge distillation into the intrusion detection model, the model accuracy is improved by 0.39% compared with the benchmark model, introduce the deep metric learning into the knowledge distillation framework, reduce the difference between the output of the teacher model and the student model, and improve the accuracy to 97.98%, and then introduce the K-fold cross training method to train the model, further improve the model accuracy to 98.44%, compared with the teacher model, the accuracy rate only differs by 0.4%, which fully proves the effectiveness of the knowledge distillation intrusion detection model based on the triple convolutional neural network and the K-fold cross training method proposed by the present application.

[0130] Table 1 Ablation experiment on NSL-KDD data set

[0131]

[0132] To sum up, the application carries out data preprocessing on the intrusion detection data set, the data preprocessing includes character type data numerical value processing, data normalization processing and data imbalance processing; the binary grey wolf optimization algorithm is used to select the optimal feature subset of the preprocessed intrusion detection data set; the teacher network model is pre-trained according to the selected optimal feature subset; the KD-TCNN intrusion detection model training process: initializing the KD-TCNN intrusion detection parameters, determining the structure of the student network model; based on the optimal feature subset, two groups of network traffic of different categories are input into the KD-TCNN intrusion detection model for training; according to the knowledge distillation loss, the error of the K-fold cross-training process is adjusted until the student network model reaches convergence; the KD-TCNN intrusion detection model is tested, and the preprocessed test data set is input into the student network to obtain the classification result of each data. The application adopts knowledge distillation to make the output of the student model close to the teacher model as much as possible, can make the student network learn the information between classes in the teacher network, can process and analyze large-scale data in real time and reduce the model parameter quantity; the difference between the outputs of the teacher network model and the student network model can be reduced, so as to improve the performance of the student model; the application adopts the depth separable convolution to further reduce the parameter quantity and the calculation amount of the model, so that the intrusion detection model can be deployed to the nodes with limited calculation capacity in the Internet of Things network, the intrusion detection time is reduced to realize real-time detection; the verification result shows that the application is superior to the traditional deep learning model in the parameter quantity and other performance indicators.

[0133] Although the application has been fully described in the foregoing by general statement and specific embodiments, some modifications or improvements can be made on the basis of the application, which is obvious to those skilled in the art. Therefore, these modifications or improvements made on the basis of not deviating from the spirit of the application, all belong to the scope of protection claimed by the application.

Claims

1. A cyber-physical system intrusion detection method, characterized by, Comprise: (1) data preprocessing of intrusion detection dataset, the data preprocessing includes character data numerical processing, data normalization processing and data imbalance processing; (2) the optimal feature subset selection of the preprocessed intrusion detection dataset is selected by binary grey wolf optimization algorithm; (3) the teacher network model is pre-trained according to the selected optimal feature subset; (4) KD-TCNN intrusion detection model training: (41) initialize the KD-TCNN intrusion detection parameters, and determine the structure of the student network model; (42) based on the optimal feature subset, input two groups of network traffic of different categories into the KD-TCNN intrusion detection model for training; (43) according to the knowledge distillation loss, adjust the error of K-fold cross training process until the student network model reaches convergence; (5) test the KD-TCNN intrusion detection model, input the preprocessed test dataset into the student network to get the classification result of each data.

2. The cyber-physical system intrusion detection method of claim 1, wherein, In step (1), the intrusion detection dataset includes NSL-KDD dataset, and the character data numerical processing process converts the element type of character type in NSL-KDD dataset into numerical data.

3. The cyber-physical system intrusion detection method of claim 1, wherein, In step (1), the data normalization processing process, according to the actual distribution of data, the normalization preprocessing formula is: where x i is the i-th feature value in the original data, is the minimum value in the i-th feature value, is the maximum value in the i-th feature value, is the result after normalization.

4. The cyber-physical system intrusion detection method of claim 1, wherein, In step (2), the optimal solution is named as alpha, the second and third optimal solutions are named as beta and delta respectively, and the remaining candidate solutions are assumed as omega, the grey wolf optimization algorithm steps include: Surrounding prey stage: a mathematical model of surrounding behavior is established; Hunting stage: guided by alpha, beta and delta may participate in hunting; the remaining omega updates the position according to the position of the best search agent; Attack prey stage: simulate approaching prey, and linearly update the parameter alpha in each iteration; In the feature subset evaluation stage, convolutional neural network is used as the learning algorithm, the fitness function for evaluating the position of grey wolf is used, the feature subset with the lowest fitness function value is selected for feature selection and dimension reduction, and the optimal feature subset is obtained.

5. The cyber-physical system intrusion detection method of claim 1, wherein, In step (42), the KD-TCNN intrusion detection model training adopts a knowledge distillation framework based on triple convolutional neural network.

6. The cyber-physical system intrusion detection method of claim 5, wherein, In step (42), the loss function design considers three types of losses, including the triplet loss L based on the distance between the anchor sample and the positive and negative samples. triplet Student network output and label cross-entropy loss L hard KL divergence loss L in teacher-student network soft .

7. The cyber-physical system intrusion detection method of claim 6, wherein, To constrain the difference degree between the output of the student network model and the probability distribution of the real label, the cross-entropy loss of the student network model output and the real label is taken as part of the model loss function, and the cross-entropy loss L of the student network model output and the real label is defined as hard .

8. The cyber-physical system intrusion detection method of claim 7, wherein, A coefficient is added to the loss term to adjust the contribution of each loss to the overall loss function, and the loss function L of the model is defined as follows: L = L KD + θL triplet wherein θ is a balance coefficient for controlling the balance between the knowledge distillation loss and the triplet loss in the model training process, L KD The knowledge distillation partial loss, L triplet The triplet loss based on the distance between the anchor sample and the positive and negative samples.

9. The cyber-physical system intrusion detection method of claim 8, wherein, The knowledge distillation framework based on triple convolutional neural network adopts depth separable convolution.

10. The cyber-physical system intrusion detection method of claim 1, wherein, In step (43), the K-fold cross training process includes: (431) define the model and learning rate, and divide the dataset into training and test dataset; (432) divide the training dataset into K parts, one part as validation set, and the remaining K-1 parts as training set; (433) define the gradient optimizer, the learning rate adopts the decay strategy, K-1 parts of data are used for model training, and the remaining one part of data is used for testing the model; (434) repeat step (433) K times to get the optimal model and get the optimal model performance index in the test dataset.

Images