Iot abnormal traffic detection method based on attention mechanism

Abstract

The application discloses an abnormal traffic detection method for Internet of Things based on an attention mechanism, mainly comprising the following steps: acquiring a training data set with labels, obtaining a gray image with a uniform size after pre-processing the training set data; constructing an abnormal traffic detection model for Internet of Things based on an attention mechanism; training the abnormal traffic detection model for Internet of Things by using the gray image with a uniform size obtained after pre-processing, obtaining the best network parameters of the model; acquiring network traffic of Internet of Things devices in real time, obtaining a gray image with a uniform size after pre-processing the traffic data; inputting the obtained gray image into the trained abnormal traffic detection model for Internet of Things, and the output of the model is the detection result. The application combines a deep separable convolution and a multi-channel attention mechanism, so that the abnormal traffic detection model for Internet of Things is more lightweight and faster in reasoning speed while having high accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of Internet of Things (IoT) security, and relates to a network abnormal traffic detection method, in particular to an abnormal traffic detection method for Internet of Things based on attention mechanism. BACKGROUND

[0002] Internet of Things (IoT) extends the connection between people under the traditional Internet to the connection between people and things and the connection between things, has a wide application prospect and strong industrial driving force, and everything connected will become an inevitable trend of mobile communication. Generally, the architecture of Internet of Things can be disassembled into a perception layer, a transmission layer, a platform layer and an application layer. Among them, the perception layer collects data of the physical world at any time and any place through RFID, bar code, positioning module, sensor, intelligent controller and other components, and then transmits the data to the platform layer through the transmission layer. The platform layer processes and calculates the data and then delivers the results to the application layer. The application layer can further calculate, process and knowledge mine the data transmitted by the platform layer, so as to provide intelligent services for different industries and realize real-time control, accurate management and scientific decision-making of the real physical world.

[0003] With the popularity and development of Internet of Things, more and more devices are connected to the Internet, and the network security problem is increasingly prominent. Among them, the problem of abnormal traffic detection of Internet of Things becomes particularly important. In the traditional abnormal traffic detection method, traffic features need to be extracted manually, and the detection accuracy is not high. In recent years, the abnormal traffic detection method based on representation learning of Internet of Things has attracted more and more attention. Its advantage is that it does not need to extract traffic features manually, but can automatically learn the features of traffic from raw data, and has high detection accuracy. Convolutional neural network (CNN) [1,2] is the most popular representation learning method at present, because its deep architecture can learn rich image features. The abnormal traffic detection model based on CNN is usually too complex and needs to occupy more computing resources and memory resources.

[0004] With the concept of edge computing, it has gradually become a trend to deploy the abnormal traffic detection model of Internet of Things on edge devices. However, the computing resources of Internet of Things edge devices are very limited, which requires the abnormal traffic detection model deployed on edge devices to have detection accuracy while occupying as few computing resources and memory resources of the device as possible. At present, most of the abnormal traffic detection models of Internet of Things based on representation learning are too complex and slow in reasoning, which are not suitable for deployment in resource-limited Internet of Things environment.

[0005] [REFERENCE]

[0006] [1] Kalchbrenner N, Grefenstette E, Blunsom P. A Convolutional Neural Network for Modelling Sentences[J]. Eprint Arxiv, 2014, 1.

[0007] [2] He K, Jian S. Convolutional neural networks at constrained time cost[C]. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015. SUMMARY

[0008] In view of the above prior art, the application provides an Internet of Things abnormal traffic detection method based on an attention mechanism, which is used to solve the problems of low model detection accuracy and overly complex model that cannot be deployed on resource-limited edge gateway devices and the like. The method described in the application has detection accuracy while making the model as lightweight as possible to adapt to its deployment on resource-limited Internet of Things edge devices.

[0009] In order to solve the above technical problems, the application provides an Internet of Things abnormal traffic detection method based on an attention mechanism, which comprises the following steps:

[0010] Step one, obtain a training data set with labels, and obtain a uniform size grayscale image after preprocessing the training set data;

[0011] Step two, construct an Internet of Things abnormal traffic detection model based on an attention mechanism;

[0012] Step three, train the Internet of Things abnormal traffic detection model using the uniform size grayscale image obtained after preprocessing, and obtain the best network parameters of the model;

[0013] Step four, obtain the network traffic of the Internet of Things device in real time, and obtain a uniform size grayscale image after preprocessing the traffic data; input the obtained grayscale image into the trained Internet of Things abnormal traffic detection model, and the output of the model is the detection result.

[0014] Further, the Internet of Things abnormal traffic detection method described in the application, wherein,

[0015] In step one, there are two ways to obtain the training data set, one is a public Internet of Things data set, and the other is to collect Internet of Things traffic using an Internet of Things traffic data capture module, and then manually label and add labels to the data set.

[0016] In step one, the process of obtaining the gray-scale image of uniform size after preprocessing the training set data is as follows:

[0017] 1-1) Flow segmentation: the Internet of Things traffic data packets with the same five-tuple in all collected data are segmented into flows, and stored as original traffic files in pcap format, wherein the five-tuple refers to source IP address, source port, destination IP address, destination port and transmission layer protocol;

[0018] 1-2) Anonymization processing: anonymization processing is performed on the original traffic file to obtain an anonymous traffic file, and the data information affecting traffic feature extraction in the data packet is removed;

[0019] 1-3) Uniform length: the length of the anonymous traffic file is unified to m x m bytes using the reduction or zero method, the process is: when the length of the anonymous traffic file is greater than m x m bytes, the first m x m bytes of the anonymous traffic file are truncated, and when the length of the anonymous traffic file is less than m x m bytes, a number of 0x00 are added at the end of the anonymous traffic file;

[0020] 1-4) Generation of gray-scale image: the traffic file after uniform length is normalized, all bytes are divided by 255 to map to [0, 1], and the normalized traffic file is represented as an m x m size gray-scale image.

[0021] In step 1-2), the data information affecting traffic feature extraction includes source IP address, destination IP address, source MAC address and destination MAC address.

[0022] The structure of the Internet of Things abnormal traffic detection model constructed in step two includes a feature extraction part and a feature classification part; the feature extraction part has three stages, the first stage includes a convolutional neural network and two deep separable convolutional encoders, the second stage and the third stage are the same, including a down-sampling neural network, a deep separable convolutional encoder and a deep separable multi-channel attention encoder; the feature classification part includes an average pooling layer and a fully connected layer, and the fully connected layer is connected with Softmax;

[0023] The structure of the deep separable convolutional encoder comprises a deep convolutional neural network, two point convolutional neural networks and a skip connection: the kernel size of the deep convolutional neural network is 3*3, and local features of the input are obtained through the deep convolutional neural network; the two point convolutional neural networks are used to enrich the local features, and a standard normalization layer and a Gaussian error linear unit are connected after the point convolutional neural networks for nonlinear feature mapping; the skip connection enables information to flow in the network hierarchy; the deep separable convolutional encoder is expressed as follows:

[0024] x i+1 =x i +Linear G (Linear(LN(D w (x i )))) (1)

[0025] In formula (1), x i represents an input feature map of H*W*1, D w is a k*k deep convolution, LN represents a standard normalization operation, Linear represents a linear transformation operation, Linear G represents a function representation of a point convolutional neural network and a Gaussian error linear unit, x i+1 represents an output feature map of the convolutional encoder.

[0026] The structure and expression of the deep separable multi-channel attention encoder are as follows: firstly, adaptive multi-scale feature representations are learned through various spatial hierarchies in the input feature map, comprising: dividing the input tensor HXWXC into s groups, each group being a feature map subset, denoted as x i , and having the same spatial scale as C / s channels, wherein i∈{1, 2, …, s}, and C represents the number of channels; in all feature map subsets, except for the first feature subset x1, a 3*3 deep convolutional neural network d i is used, and the output of the deep convolutional neural network d i is denoted as y i ; and the output y i-1 of d i-1 is also substituted into the feature subset x i , then x i +y i-1 is input into d i ; the number s of feature map subsets is based on different stages t, wherein t∈{2, 3}; y i is expressed as follows:

[0027]

[0028] Then, the global image representation is implicitly encoded, including point multiplication operation using multi-head self-attention in the channel dimension, calculating the inter-covariance on the channel; using the y obtained from formula (2) i After normalization, the tensor Y is obtained, Y e HxWxC, and three linear layers are used to calculate Q, K, V e HWxC projections, as follows:

[0029] Q = W Q Y, K = W K Y, V = W V Y (3)

[0030] Q T and K are calculated in the channel dimension, as follows:

[0031] (CxHW) · (HWxC) (4)

[0032] The result obtained from formula (4) is input to Softmax to obtain a CxC attention score matrix, V is multiplied by the attention score matrix and accumulated, and the transpose attention operation is represented as follows:

[0033]

[0034] In formula (5), X is the input tensor, is the output feature tensor;

[0035] Finally, after two 1x1 point convolution operations, a regularization processing function and a Gaussian error linear unit, a nonlinear feature is generated.

[0036] The specific steps of step three are as follows:

[0037] 3-1) input the gray-scale image of uniform size obtained in step one to the feature extraction part of the Internet of Things abnormal traffic detection model constructed in step two, and extract spatial feature data in the Internet of Things traffic after three stages of processing;

[0038] 3-2) input the spatial feature data extracted above to the feature classification part of the Internet of Things abnormal traffic detection model, output the classification result, and compare the classification result with the label of the training data to calculate the loss through the cross-entropy function;

[0039] 3-3) perform AdamW gradient descent algorithm according to the loss;

[0040] 3-4) stop training when the loss function converges, and thus the network parameters are the best network parameters of the model.

[0041] In step four, the process of obtaining a gray-scale image of uniform size after preprocessing the traffic data is the same as the process of obtaining a gray-scale image of uniform size after preprocessing the training set data in step one.

[0042] Compared with the prior art, the application has the beneficial effects that:

[0043] In order to design efficient convolution suitable for low-power edge devices, the application adopts a depth separable convolution; in order to solve the problem of high-cost calculation caused by the introduction of a self-attention mechanism, the self-attention operation is applied to the cross-channel dimension instead of the spatial dimension. Based on this, the application combines the depth separable convolution and the multi-channel attention mechanism, so that the Internet of Things abnormal traffic detection model is more lightweight and faster in inference speed while maintaining high accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 is a system framework schematic diagram of an embodiment of the application.

[0045] Figure 2 is an Internet of Things abnormal traffic detection model (DSCCA) based on an attention mechanism proposed by the application. DETAILED DESCRIPTION

[0046] The application will be further described below in combination with the drawings and specific embodiments, but the following embodiments are by no means any limitation on the application.

[0047] As shown in Figure 1 , an embodiment of an Internet of Things abnormal traffic detection method based on an attention mechanism proposed by the application is shown, which includes an offline training link and a real-time detection link, and includes the following steps:

[0048] Step one, obtaining a training data set with labels, in this embodiment, the Internet of Things traffic data is collected by using an Internet of Things traffic data capture module, and then manually labeled to add labels to the data set (in other instances, a public Internet of Things data set is not excluded as training set data); after preprocessing the training set data, a gray-scale image of uniform size is obtained. The specific steps of preprocessing are as follows:

[0049] 1-1) Flow segmentation: the Internet of Things traffic data packets with the same five-tuple (source IP address, source port, destination IP address, destination port and transmission layer protocol) in all collected data are segmented into flows and stored as original traffic files in the format of pcap.

[0050] 1-2) Anonymization: anonymize the original traffic file to obtain an anonymous traffic file, remove useless data information in the data packet that may affect traffic feature extraction, such as some data of the data link layer related to Ethernet, mainly including source IP address, destination IP address, source MAC address and destination MAC address. Considering that different types of traffic in the collected data set may correspond to different IP addresses and MAC addresses, this will make the model pay too much attention to the relationship between IP address and MAC address and traffic type in the learning process, so it cannot be well generalized, therefore, the IP address and MAC address of the original traffic data are zeroed.

[0051] 1-3) Uniform length: use the reduction or zeroing method to unify the length of the anonymous traffic file to m x m bytes, the process is: when the length of the anonymous traffic file is greater than m x m bytes, only the first m x m bytes of the anonymous traffic file are intercepted, and when the length of the anonymous traffic file is less than m x m bytes, a number of 0x00 are added at the end of the anonymous traffic file to expand it to m x m bytes in order to unify the input data length of the model.

[0052] 1-4) Generate grayscale image: normalize the traffic file after uniform length. Since the size of each byte is between 0 and 255 in decimal number, it corresponds to the pixel value of the grayscale image. For example, 0x00 corresponds to black pixel value, and 0xFF corresponds to white pixel value. All bytes are uniformly divided by 255 to map them to [0, 1], and represented as an m x m grayscale image.

[0053] Step two, build an Internet of Things abnormal traffic detection model DSCCA based on attention mechanism.

[0054] As shown in Figure 2 The structure of the Internet of Things abnormal traffic detection model DSCCA mainly includes two parts, namely the feature extraction part and the feature classification part, wherein the feature extraction part has three stages, the first stage includes a convolutional neural network and two deep separable convolutional encoders (Conv. Encoder), the second stage and the third stage are the same, including a down-sampling neural network, a deep separable convolutional encoder (Conv. Encoder) and a deep separable multi-channel attention encoder (DSCA Encoder).

[0055] The feature classification part includes an average pooling layer and a fully connected layer, and the fully connected layer is connected with a Softmax classifier.

[0056] The deep separable convolutional encoder comprises a deep convolutional neural network (DW-CNN), two point-wise convolutional neural networks (Point-wise CNN) and a skip-connection: the kernel size of the DW-CNN is 3*3, and local features of the input are obtained through the DW-CNN; the two Point-wise CNNs are used to enrich the local features, and a standard normalization layer (Norm layer) and a Gaussian error linear unit (GELU) are connected after the Point-wise CNNs, for nonlinear feature mapping; the skip-connection enables information to flow in the network hierarchy. The deep separable convolutional encoder can be summarized as the following formula:

[0057] x i+1 =x i +Linear G (Linear(LN(D w (x i )))) (1)

[0058] In formula (1), x i represents an input feature map of H*W*1, D w is a k*k deep convolution, LN represents a standard normalization operation, Linear represents a linear transformation operation, Linear G represents a function representation of the Point-wise CNN and GELU, x i+1 represents an output feature map of the convolutional encoder.

[0059] The deep separable multi-channel attention encoder comprises two parts: the first part learns adaptive multi-scale feature representation by encoding various spatial hierarchies in the input image; and the second part implicitly encodes a global image representation.

[0060] The first part learns adaptive multi-scale feature representation by encoding various spatial hierarchies in the input feature map, comprising: dividing an input tensor H*W*C into s groups, each group being a feature map subset, denoted as x i , and having the same spatial scale as C / s channels, wherein i∈{1, 2, …, s}, and C represents the number of channels; in all feature map subsets, except for the first feature subset x1, a 3*3 deep convolutional neural network d i is used, and the output of the deep convolutional neural network d i is denoted as y i ; and d i-1the output y i-1 is also substituted into the feature subset x i , and then x i +y i-1 is input into d i ; the number s of feature map subsets is based on different stages t, where t e {2, 3}; y i can be expressed as the following formula:

[0061]

[0062] where the second part introduces a multi-head attention mechanism to intuitively encode global image representations. At the same time, in order to eliminate the high computational complexity of the attention mechanism, the model uses the dot product operation of MSA in the channel dimension instead of the spatial dimension to achieve linear complexity, and calculates the mutual covariance in the channel to generate the attention feature map, which has implicit knowledge of global representation.

[0063] After normalizing y i obtained from equation (2), a tensor Y is obtained, Y e H x W x C, and three linear layers are used to calculate Q, K, and V e HW x C projections, as follows:

[0064] Q = W Q Y,K = W K Y,V = W V Y (3)

[0065] The dot product of Q T and K in the channel dimension is calculated as follows:

[0066] (C x HW) cdot (HW x C) (4)

[0067] The result obtained from equation (4) is input into Softmax to obtain a C x C attention score matrix. In order to obtain the final attention map, V is multiplied by the attention score matrix and accumulated, and the transpose attention operation can be represented as follows:

[0068]

[0069] In equation (5), X is the input tensor, is the output feature tensor, and then two 1 x 1 Point-wise operations, a regularization processing function LN, and a GELU activation layer are used to generate nonlinear features.

[0070] Step three, train the Internet of Things abnormal traffic detection model using the preprocessed gray-scale images of uniform size to obtain the best network parameters of the model. The specific steps are as follows:

[0071] 3-1) Input the uniformly sized grayscale image obtained after preprocessing the training set data in step one into the feature extraction part of the DSCCA neural network constructed in step two. After three stages of processing by a depth-separable convolutional encoder and a depth-separable multi-channel attention encoder, the spatial feature data in the IoT traffic is extracted.

[0072] 3-2) Input the spatial feature data extracted above into the Softmax classifier of the DSCCA feature classification part, output the classification result, compare the classification result with the label of the training data, and calculate the loss through the cross-entropy function;

[0073] 3-3) Execute the AdamW gradient descent algorithm based on the loss;

[0074] 3-4) Stop training when the loss function converges. The network parameters at this point are the optimal network parameters for the model.

[0075] Step 4: Acquire network traffic from IoT devices in real time, and preprocess the traffic data to obtain a uniform-sized grayscale image; input the obtained grayscale image into the trained IoT abnormal traffic detection model, and the output of the model is the detection result. The specific steps are as follows:

[0076] 4-1) Real-time acquisition of network traffic of IoT devices flowing through the edge gateway, and preprocessing of traffic data; the process of obtaining a uniform-sized grayscale image after preprocessing traffic data is the same as the process of obtaining a uniform-sized grayscale image after preprocessing training set data in step one.

[0077] 4-2) Input the data processed in step 4-1) into the trained DSCCA neural network, and extract the spatial features in the original IoT traffic through a depth-separable convolutional encoder and a depth-separable multi-channel attention encoder.

[0078] 4-3) Input the data processed in step 4-2) into the Softmax classifier and output the final detection result.

[0079] This invention can be applied to IoT edge devices with limited resources for detecting abnormal traffic in the Internet of Things (IoT). On the IoT-IMC19 and Bot-Iot datasets, the accuracy rate for identifying abnormal traffic reaches over 99%.

[0080] Although the present application has been described above with reference to specific embodiments, the above embodiments are merely illustrative and not restrictive, and many modifications and other embodiments of the present application can occur to those skilled in the art upon reading the foregoing description, which modifications and other embodiments fall within the scope of the present application.

Claims

1. A method for detecting abnormal traffic in the Internet of Things based on an attention mechanism, characterized in that, Includes the following steps: Step 1: Obtain a labeled training dataset and preprocess the training dataset to obtain grayscale images of uniform size. Step 2: Construct an IoT abnormal traffic detection model based on an attention mechanism; The structure of the constructed IoT abnormal traffic detection model includes a feature extraction part and a feature classification part; wherein, the feature extraction part has three stages. The first stage includes a convolutional neural network and two depthwise separable convolutional encoders. The second and third stages are the same, including a downsampling neural network, a depthwise separable convolutional encoder and a depthwise separable multichannel attention encoder. The feature classification part includes an average pooling layer and a fully connected layer, with a Softmax layer connected after the fully connected layer. 2-1) The structure and representation of the depth-separable convolutional encoder described above: The depthwise separable convolutional encoder comprises a deep convolutional neural network, two point convolutional neural networks, and a skip connection: the deep convolutional neural network has a kernel size of 3×3, and the input local features are obtained through this deep convolutional neural network; the two point convolutional neural networks are used to enrich the above local features, and a standard normalization layer and a Gaussian error linear unit are connected after the point convolutional neural networks for non-linear feature mapping; the skip connection enables information to flow in the network hierarchy; The depth-separable convolutional encoder is expressed as follows: x i+1 =x i +Linear G (Linear(LN(D w (x i )))) (1) In equation (1), x i D represents the input feature map of size H×W×1. w It is a k×k depth convolution, LN represents the standard normalization operation, and Linear represents the linear transformation operation. G The function representation of the interaction between a point convolutional neural network and a Gaussian error linear unit, x i+1 This represents the output feature map of the convolutional encoder. The structure and representation of the depth-separable multichannel attention encoder described in 2-2) are as follows: First, adaptive multi-scale feature representations are learned by exploring various spatial levels within the input feature maps. This includes dividing the input tensor H×W×C into s groups, each group representing a subset of feature maps, and using x... i It is represented as i ∈ {1,2,…,s}, and has the same spatial scale as the C / s channel; where i ∈ {1,2,…,s}, and C represents the number of channels; All feature map subsets, except for the first feature subset x1, are processed by a 3×3 deep convolutional neural network d. i This deep convolutional neural network d i The output is represented as y i Furthermore, d i-1 The output y i-1 It is also substituted into the feature subset x i In the middle, then, x i +y i-1 Input to d i In the middle; the number s of feature map subsets is based on different stages t, where t∈{2,3}; y i The expression is as follows: Then, the global image representation is implicitly encoded, including a dot product operation with multi-head self-attention on the channel dimension, and cross-covariance is computed on the channels; The y obtained from equation (2) i After normalization, we obtain the tensor Y, Y∈H×W×C. We then use three linear layers to calculate the projections of Q,K,V∈HW×C, as shown in the following formula: Q=W Q Y,K=W K Y,V=W V Y (3) Calculate Q along the channel dimension T The dot product of K and K is calculated using the following formula: (C×HW)·(HW×C)(4) The result obtained from equation (4) is input into Softmax to obtain a C×C attention score matrix. V is multiplied by the attention score matrix and summed. The transpose attention operation is expressed as the following formula: In equation (5), X is the input tensor. It is the output feature tensor; Finally, nonlinear features are generated through two 1×1 point convolution operations, a regularization function, and a Gaussian error linear unit. Step 3: Train the IoT abnormal traffic detection model using the preprocessed grayscale images of uniform size to obtain the optimal network parameters for the model; Step 4: Acquire network traffic of IoT devices in real time, and preprocess the traffic data to obtain a uniform grayscale image; input the obtained grayscale image into the trained IoT abnormal traffic detection model, and the output of the model is the detection result.

2. The IoT abnormal traffic detection method according to claim 1, characterized in that, In step one, there are two ways to obtain the training dataset: one is to use a publicly available IoT dataset, and the other is to use an IoT traffic data capture module to collect IoT traffic and then manually label it to add labels to the dataset.

3. The IoT abnormal traffic detection method according to claim 2, characterized in that, In step one, the process of preprocessing the training set data to obtain grayscale images of uniform size is as follows; 1-1) Stream segmentation: All collected IoT traffic data packets with the same five-tuple are segmented into streams and stored as raw traffic files in pcap format. The five-tuple refers to the source IP address, source port, destination IP address, destination port, and transport layer protocol. 1-2) Anonymization: The original traffic file is anonymized to obtain an anonymous traffic file, removing data information in the data packets that affects the extraction of traffic features; 1-3) Unified length: Use reduction or zeroing methods to unify the length of the anonymous traffic file to m×m bytes. The process is as follows: when the length of the anonymous traffic file is greater than m×m bytes, the first m×m bytes of the anonymous traffic file are truncated. When the length of the anonymous traffic file is less than m×m bytes, several 0x00s are added to the end of the anonymous traffic file. 1-4) Generate grayscale image: Normalize the traffic file after it has been standardized in length, divide all bytes by 255 to map them to the range [0,1], and represent the normalized traffic file as a grayscale image of size m×m.

4. The IoT abnormal traffic detection method according to claim 3, characterized in that, In steps 1-2), the data information that affects traffic feature extraction includes source IP address, destination IP address, source MAC address, and destination MAC address.

5. The IoT abnormal traffic detection method according to claim 1, characterized in that, The specific steps for step three are as follows: 3-1) Input the uniformly sized grayscale image obtained in step one into the feature extraction part of the IoT abnormal traffic detection model constructed in step two. After three stages of processing, extract the spatial feature data in the IoT traffic. 3-2) Input the spatial feature data extracted above into the feature classification part of the IoT abnormal traffic detection model, output the classification result, compare the classification result with the label of the training data, and calculate the loss through the cross-entropy function; 3-3) Execute the AdamW gradient descent algorithm based on the loss; 3-4) Stop training when the loss function converges. At this point, the network parameters are the optimal network parameters for the model.

6. The IoT abnormal traffic detection method according to claim 4, characterized in that, In step four, the process of preprocessing the traffic data to obtain a uniform-sized grayscale image is the same as the process of preprocessing the training set data to obtain a uniform-sized grayscale image in step one.

Images