Abnormal traffic monitoring method and system based on convolutional network

Abstract

The invention relates to the field of network monitoring, in particular to an abnormal traffic monitoring method and system based on a convolutional network. The invention discloses an abnormal traffic monitoring system based on a convolutional network. The abnormal traffic monitoring system comprises a data packet acquisition module, a to-be-detected traffic data generation module, a traffic anomaly detection model management module and a traffic anomaly monitoring module. According to the method, the data packets are integrated into the image data form, the data features and the time sequence features in the data packets are extracted, the data features can focus on data information of the data packets, and the time sequence features focus on matching information between the data packets; the abnormal traffic can be monitored more comprehensively through the data features and the time sequence features in the data packet, and the influence of the undetected abnormal traffic on network use is avoided.

Description

Technical Field

[0001] This invention relates to the field of network monitoring, and more specifically, to a method and system for abnormal traffic monitoring based on convolutional networks. Background Technology

[0002] With the increasing prevalence of network applications, the number and types of network attacks are also constantly increasing. To reduce the various impacts caused by network attacks and improve network security, network anomaly intrusion detection is of paramount importance. Monitoring abnormal traffic in network transmissions is a highly practical method for network anomaly intrusion detection.

[0003] Currently, methods for monitoring abnormal traffic using neural networks exist. For example, a neural network-based abnormal traffic detection method (publication number CN113194094B) resamples imbalanced traffic samples using sampling techniques to reduce the impact of data imbalance on detection. The sample data is then input into the network for detection, and the network is optimized in terms of spatial structure and feature weights. Furthermore, the network structure is improved to address the characteristics of abnormal traffic, thereby enhancing detection accuracy. However, network traffic data is time-series data with inherent temporal characteristics, such as coordinated network attacks between data packets. Ignoring these temporal characteristics can easily lead to missed detections. Summary of the Invention

[0004] This invention provides an abnormal traffic monitoring method and system based on convolutional networks. By integrating data packets into image data, and extracting data features and temporal features from the data packets, the data features can focus on the data information of the data packets themselves, while the temporal features focus on the coordination information between data packets. By using the data features and temporal features in the data packets, abnormal traffic can be monitored more comprehensively, avoiding the impact of missed abnormal traffic on network usage.

[0005] An abnormal traffic monitoring method based on convolutional networks includes: At the gateway, data packets are acquired sequentially, and each data packet is processed as follows: the bytes of the data packet are filled into a two-dimensional matrix in order to generate the traffic data to be tested. Each byte includes 8 bits, and each bit stores a binary number. That is, each byte is regarded as a vector. Therefore, the traffic data to be tested exists in the form of an n×n×8 multidimensional matrix. If the number of bytes in the data packet exceeds n×n, the extra bytes are deleted. If the number of bytes in the data packet is less than n×n, the missing bytes are padded with 0. The traffic data to be tested is fed into the trained traffic anomaly detection model to detect the traffic features and output the detection results. If the detection result is abnormal traffic, the five-tuple information in the data corresponding to the traffic feature to be tested is packaged and sent to the user terminal to trigger an alarm; otherwise, the next abnormal traffic detection is performed. The traffic anomaly detection model is built on a CNN convolutional neural network and includes a data feature extraction block, a temporal feature extraction block, a fully connected layer, and a classification layer.

[0006] The abnormal traffic monitoring method based on convolutional networks according to claim 1 is characterized in that the abnormal traffic detection model detects the traffic data to be tested, specifically including the following steps: the traffic data to be tested is processed by a data feature extraction block to obtain data features; the data features are sent to a temporal feature extraction block for processing to obtain temporal features; the data features and temporal features are concatenated and fused to obtain the traffic features to be tested; the traffic features to be tested are processed sequentially through a fully connected layer and a classification layer to obtain the detection result; if the probability of abnormal traffic is higher than a threshold, the detection result is output as abnormal traffic; otherwise, the detection result is output as non-abnormal traffic.

[0007] As a preferred aspect of the present invention, the data feature extraction block comprises a first data feature extraction sub-block, a second data feature extraction sub-block, and a third data feature extraction sub-block. The first data feature extraction sub-block comprises a convolutional layer, a ReLU layer, and a max pooling layer. The second data feature extraction sub-block comprises three convolutional layers, three ReLU layers, and a max pooling layer, with each convolutional layer followed by a ReLU layer. The third data feature extraction sub-block comprises three convolutional layers, three ReLU layers, and a max pooling layer, with each convolutional layer followed by a ReLU layer.

[0008] As a preferred aspect of the present invention, the time series feature extraction block is built based on the GRU model and includes an input layer, a hidden layer, an output layer and a GRU layer. The input layer is used to record the current data features, the hidden layer has built-in hidden layer data features and is used to calculate the hidden layer output based on the current data features and the hidden layer data features; the output layer is used to calculate the time series features based on the hidden layer output; and the GRU layer is used to update the hidden layer data features.

[0009] As a preferred aspect of the present invention, obtaining time-series features by processing data features through a time-series feature extraction block specifically includes the following steps: Let x be the data feature to be processed. t Data feature x t After the data is input from the input layer to the hidden layer, the hidden layer determines the data based on the data feature x. t and the built-in hidden layer data features h t-1 Calculate the hidden layer output S t=f(U s ·x t +W s ·h t-1 +b1), where U s W is the output layer weight matrix. s Let b1 be the weight matrix of the first hidden layer, b1 be the first bias value, and f be the first activation function; let the hidden layer output S be... t The data is fed into the output layer to calculate the temporal features O. t =g(V) o ·S t +b2), where V o Let b1 be the output layer weight matrix, b2 be the second bias value, and g be the second activation function.

[0010] As a preferred aspect of the present invention, the GRU layer updating the hidden layer data features based on the current data features specifically includes the following steps: calculating the update gate Z. t =σ(W z ·x t +U z ·h t-1 ), where W z U is the first updated gate weight matrix. z The second update gate weight matrix is ​​σ, where σ is the sigmoid function. The update gate records how much information in the current data features and hidden layer data features needs to be retained in the hidden layer. The reset gate R is calculated. t =σ(W r ·x t +U r ·h t-1 ), where W r U is the first reset gate weight matrix. r The second reset gate weight matrix is ​​used to record how much information in the current data features and hidden layer data features needs to be deleted in the hidden layer; the updated hidden layer data features h are then calculated. t =Z t ·h t-1 +(1-Z t )·tanh(U s ·x t +R t ·W s ·h t-1 ), where tanh refers to the tanh function.

[0011] As a preferred aspect of the present invention, the training of the traffic anomaly detection model specifically includes the following steps: obtaining data packets labeled with abnormal traffic types and non-abnormal traffic types to form a training set sample; selecting an initialized traffic anomaly detection model, disabling the temporal feature extraction block, and pre-training the traffic anomaly detection model that only includes a data feature extraction block, a fully connected layer, and a classification layer; feeding the training set sample into the pre-trained traffic anomaly detection model for training, without using the step of concatenating and fusing data features and temporal features to obtain the traffic features to be tested, but instead directly feeding the temporal features into the fully connected layer; fixing the internal parameters of the temporal feature extraction block, and then feeding the training set sample into the traffic anomaly detection model with the fixed internal parameters of the temporal feature extraction block for training, and outputting the trained traffic anomaly detection model.

[0012] An abnormal traffic monitoring system based on convolutional networks includes: The data packet acquisition module is used to sequentially acquire data packets at the gateway; The traffic data generation module is used to generate corresponding traffic data to be tested based on data packets; The traffic anomaly detection model management module is used to train and store traffic anomaly detection models; The traffic anomaly monitoring module is used to monitor abnormal traffic based on the traffic data to be measured and the traffic anomaly detection model.

[0013] The present invention has the following advantages: This invention integrates data packets into image data and extracts data features and timing features from the data packets. Data features focus on the data information of the data packets themselves, while timing features focus on the coordination information between data packets. By combining data features and timing features in the data packets, abnormal traffic can be monitored more comprehensively, avoiding the impact of missed abnormal traffic on network usage. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the abnormal traffic monitoring system based on convolutional networks used in an embodiment of the present invention.

[0015] Figure 2 This is a schematic diagram of the flow anomaly detection model used in an embodiment of the present invention.

[0016] Figure 3 This is a schematic diagram of the structure of the data feature extraction block used in an embodiment of the present invention.

[0017] Figure 4 This is a schematic diagram of the temporal feature extraction block used in an embodiment of the present invention. Detailed Implementation

[0018] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of this invention.

[0019] Example 1: An abnormal traffic monitoring method based on convolutional networks, comprising: Data packets are sequentially acquired at the gateway, and each data packet is processed as follows: the bytes of the data packet are sequentially filled into a two-dimensional matrix to generate the traffic data to be tested. Each byte includes 8 bits, and each bit stores a binary number. That is, each byte is regarded as a vector. Therefore, the traffic data to be tested exists in the form of an n×n×8 multidimensional matrix. If the number of bytes in the data packet exceeds n×n, the extra bytes are deleted. If the number of bytes in the data packet is less than n×n, the missing bytes are padded with 0. At this time, the traffic data to be tested is analogous to n×n×8 image data, with a length and width of n and 8 channels. Processing the data packets into a form similar to image data facilitates subsequent convolutional network processing. In this embodiment, since most of the transmission protocols at the gateway are TCP protocols, the size of the data packets is limited. Therefore, n is set to 40. The traffic data to be tested is fed into the trained traffic anomaly detection model to detect the traffic features and output the detection results. If the detection result is abnormal traffic, the five-tuple information in the data corresponding to the traffic feature to be tested is packaged and sent to the user terminal to trigger an alarm; otherwise, the next abnormal traffic detection is performed. The traffic anomaly detection model is built based on a CNN convolutional neural network, such as... Figure 2 As shown, the traffic anomaly detection model includes a data feature extraction block, a time-series feature extraction block, a fully connected layer, and a classification layer.

[0020] The detection of traffic anomaly data using the traffic anomaly detection model includes the following steps: First, the traffic data is processed through a data feature extraction block to obtain data features. Then, these data features are processed through a time-series feature extraction block to obtain time-series features. Next, the data features and time-series features are concatenated and fused to obtain the traffic features to be detected. Since these features are obtained by concatenating and fusing data features and time-series features, they include data-level features and the influence of the previous data packet on the current data packet. Therefore, the traffic features to be detected possess more comprehensive feature information and can achieve better detection results. Finally, the traffic features to be detected are processed sequentially through a fully connected layer and a classification layer to obtain the detection result. The classification layer has a built-in softmax function. The one-dimensional feature vector obtained after processing by the fully connected layer is used to calculate the probability using the softmax function. If the probability of abnormal traffic is higher than a threshold, the detection result is output as abnormal traffic; otherwise, the detection result is output as non-abnormal traffic.

[0021] like Figure 3 As shown, the data feature extraction block comprises a first data feature extraction sub-block, a second data feature extraction sub-block, and a third data feature extraction sub-block. The first data feature extraction sub-block includes one convolutional layer, one ReLU layer, and one max pooling layer. The second data feature extraction sub-block includes three convolutional layers, three ReLU layers, and one max pooling layer, with each convolutional layer followed by a ReLU layer. The third data feature extraction sub-block also includes three convolutional layers, three ReLU layers, and one max pooling layer, with each convolutional layer followed by a ReLU layer. The convolutional layers use 3×3 kernels to perform convolution operations on the traffic data to be tested, achieving feature extraction. The ReLU layers incorporate the ReLU activation function to map the features extracted through convolution to a high-dimensional space and reduce linear correlation. The max pooling layers use 2×2 kernels to reduce the size of the features corresponding to the traffic data to be tested. The specific steps for obtaining data features by processing the traffic data to be tested through the data feature extraction block are as follows: The traffic data to be tested is processed sequentially through the first data feature extraction sub-block, the second data feature extraction sub-block, and the third data feature extraction sub-block to obtain the first feature data, the second feature data, and the third feature data, respectively; the third feature data is upsampled and then concatenated with the second feature data to obtain the fourth feature data; the fourth feature data is upsampled and then concatenated with the first feature data to obtain the fifth feature data; the fifth feature data is subjected to a convolution operation to obtain the data features; wherein the upsampling operation adopts transposed convolution. In this application, a convolutional network is used to extract data features from traffic data packets, and by successively fusing the first feature data, the second feature data, and the third feature data, the data features can include both shallow and deep features, thereby improving the accuracy of abnormal traffic detection.

[0022] Temporal feature extraction blocks are built based on the GRU model, such as Figure 4 As shown, the system includes an input layer, a hidden layer, an output layer, and a GRU layer. The GRU (Gated Recurrent Unit) model is a type of recurrent neural network that can extract feature information from temporal data. In network transmission, network traffic has obvious temporal characteristics; the previous data packet may affect the next data packet or be combined with the next data packet as attack traffic. Therefore, this application takes into account the temporal characteristics of data packets to achieve better detection results. The input layer is used to record the current data features, and the hidden layer has built-in hidden layer data features, which are used to calculate the hidden layer output based on the current data features and the hidden layer data features. The output layer is used to calculate temporal features based on the hidden layer output. The GRU layer is used to update the hidden layer data features. The process of extracting time-series features from data features through time-series feature extraction blocks includes the following steps: Let x be the data feature to be processed. t Data feature x t After the data is input from the input layer to the hidden layer, the hidden layer determines the data based on the data feature x. t and the built-in hidden layer data features h t-1 Calculate the hidden layer output S t =f(U s ·x t +W s ·h t-1 +b1), where U s W is the output layer weight matrix. s Here, b1 is the weight matrix of the first hidden layer, b1 is the first bias value, and f is the first activation function, typically the tanh function; the hidden layer output S is... t The data is fed into the output layer to calculate the temporal features O. t =g(V) o ·S t +b2), where V o Here, b1 is the output layer weight matrix, b2 is the second bias value, and g is the second activation function, typically the sigmoid function. The GRU layer updates the hidden layer data features based on the current data features, specifically including the following steps: calculating the update gate Z. t =σ(W z ·x t +U z ·h t-1 ), where W z U is the first updated gate weight matrix. z The second update gate weight matrix is ​​σ, where σ is the sigmoid function. The update gate records how much information in the current data features and hidden layer data features needs to be retained in the hidden layer. The reset gate R is calculated. t =σ(W r ·x t +U r ·h t-1 ), where W r U is the first reset gate weight matrix. r The second reset gate weight matrix is ​​used to record how much information in the current data features and hidden layer data features needs to be deleted in the hidden layer; the updated hidden layer data features h are then calculated. t =Z t ·h t-1 +(1-Z t )·tanh(U s ·x t +R t ·W s ·ht-1 ), where tanh refers to the tanh function; This invention integrates data packets into image data and extracts data features and timing features from the data packets. Data features focus on the data information of the data packets themselves, while timing features focus on the coordination information between data packets. By combining data features and timing features in the data packets, abnormal traffic can be monitored more comprehensively, avoiding the impact of missed abnormal traffic on network usage.

[0023] The training of the traffic anomaly detection model includes the following steps: First, obtain data packets labeled with anomaly and non-anomaly traffic types to form a training set sample. Second, select an initialized traffic anomaly detection model, disable the temporal feature extraction block, and pre-train the model using the ImageNet dataset, which only includes the data feature extraction block, fully connected layer, and classification layer. Third, feed the training set sample into the pre-trained traffic anomaly detection model for training. During this process, instead of concatenating and fusing data features and temporal features to obtain the traffic features to be tested, the temporal features are directly fed into the fully connected layer. Fourth, keep the internal parameters of the temporal feature extraction block unchanged, and then feed the training set sample into the traffic anomaly detection model with the fixed internal parameters of the temporal feature extraction block for training, outputting the trained traffic anomaly detection model.

[0024] Example 2: An abnormal traffic monitoring system based on convolutional networks, such as... Figure 1 As shown, it includes: The data packet acquisition module is used to sequentially acquire data packets at the gateway; The traffic data generation module is used to generate corresponding traffic data to be tested based on data packets; The traffic anomaly detection model management module is used to train and store traffic anomaly detection models; The traffic anomaly monitoring module is used to monitor abnormal traffic based on the traffic data to be measured and the traffic anomaly detection model.

[0025] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims. Parts not described in detail in this specification are prior art known to those skilled in the art.

Claims

1. A method for abnormal traffic monitoring based on convolutional networks, characterized in that, include: At the gateway, data packets are acquired sequentially, and each data packet is processed as follows: the bytes of the data packet are filled into a two-dimensional matrix in order to generate the traffic data to be tested. Each byte includes 8 bits, and each bit stores a binary number. That is, each byte is regarded as a vector. Therefore, the traffic data to be tested exists in the form of an n×n×8 multidimensional matrix. If the number of bytes in the data packet exceeds n×n, the extra bytes are deleted. If the number of bytes in the data packet is less than n×n, the missing bytes are padded with 0. The traffic data to be tested is fed into the trained traffic anomaly detection model to detect the characteristics of the traffic and output the detection results. If the detection result is abnormal traffic, the five-tuple information in the data corresponding to the characteristics of this traffic to be tested will be packaged and sent to the user terminal to trigger an alarm. Otherwise, proceed with the next abnormal traffic detection; The traffic anomaly detection model is built on a CNN convolutional neural network and includes a data feature extraction block, a temporal feature extraction block, a fully connected layer, and a classification layer.

2. The abnormal traffic monitoring method based on convolutional networks according to claim 1, characterized in that, The traffic anomaly detection model detects the traffic data to be tested, specifically including the following steps: The traffic data to be tested is processed through a data feature extraction block to obtain data features; the data features are then processed through a time series feature extraction block to obtain time series features; the data features and time series features are concatenated and fused to obtain the traffic features to be tested; the traffic features to be tested are then processed through a fully connected layer and a classification layer to obtain the detection result. If the probability of abnormal traffic is higher than a threshold, the detection result is output as abnormal traffic; otherwise, the detection result is output as non-abnormal traffic.

3. The abnormal traffic monitoring method based on convolutional networks according to claim 2, characterized in that, The data feature extraction block consists of a first data feature extraction sub-block, a second data feature extraction sub-block, and a third data feature extraction sub-block. The first data feature extraction sub-block includes one convolutional layer, one ReLU layer, and one max pooling layer. The second data feature extraction sub-block includes three convolutional layers, three ReLU layers, and one max pooling layer, with each convolutional layer followed by a ReLU layer. The third data feature extraction sub-block includes three convolutional layers, three ReLU layers, and one max pooling layer, with each convolutional layer followed by a ReLU layer.

4. The abnormal traffic monitoring method based on convolutional networks according to claim 3, characterized in that, The temporal feature extraction block is built based on the GRU model and includes an input layer, a hidden layer, an output layer, and a GRU layer. The input layer is used to record the current data features, the hidden layer contains hidden data features, and the hidden layer output is calculated based on the current data features and the hidden layer data features. The output layer is used to calculate the temporal features based on the hidden layer output. The GRU layer is used to update the features of the hidden layer data.

5. The abnormal traffic monitoring method based on convolutional networks according to claim 4, characterized in that, The process of extracting time-series features from data features through time-series feature extraction blocks includes the following steps: Let x be the data feature to be processed. t Data feature x t After the data is input from the input layer to the hidden layer, the hidden layer determines the data based on the data feature x. t and the built-in hidden layer data features h t-1 Calculate the hidden layer output S t =f(U s ·x t +W s ·h t-1 +b1), where U s W is the output layer weight matrix. s Let b1 be the weight matrix of the first hidden layer, b1 be the first bias value, and f be the first activation function; let the hidden layer output S be... t The data is fed into the output layer to calculate the temporal features O. t =g(V) o ·S t +b2), where V o Let b1 be the output layer weight matrix, b2 be the second bias value, and g be the second activation function.

6. The abnormal traffic monitoring method based on convolutional networks according to claim 5, characterized in that, The GRU layer updates the hidden layer data features based on the current data features, specifically including the following steps: calculating the update gate Z. t =σ(W z ·x t +U z ·h t-1 ), where W z U is the first updated gate weight matrix. z Here is the weight matrix for the second update gate, and σ is the sigmoid function; calculate the reset gate R. t =σ(W r ·x t +U r ·h t-1 ), where W r U is the first reset gate weight matrix. r The second reset gate weight matrix is ​​used; the updated hidden layer data features h are calculated. t =Z t ·h t-1 +(1-Z t )·tanh(U s ·x t +R t ·W s ·h t-1 ), where tanh refers to the tanh function.

7. The abnormal traffic monitoring method based on convolutional networks according to claim 6, characterized in that, The training of the traffic anomaly detection model includes the following steps: First, obtain data packets labeled with anomaly and non-anomaly traffic types to form a training set sample. Second, select an initialized traffic anomaly detection model, disable the temporal feature extraction block, and pre-train the model containing only the data feature extraction block, fully connected layer, and classification layer. Third, feed the training set sample into the pre-trained traffic anomaly detection model for training. During this process, instead of concatenating and fusing data features and temporal features to obtain the traffic features to be tested, the temporal features are directly fed into the fully connected layer. Fourth, keep the internal parameters of the temporal feature extraction block unchanged, and then feed the training set sample into the traffic anomaly detection model with the fixed internal parameters of the temporal feature extraction block for training, outputting the trained traffic anomaly detection model.

8. An abnormal traffic monitoring system based on convolutional networks, characterized in that, The system employs an abnormal traffic monitoring method based on convolutional networks as described in any one of claims 1-7, comprising: The data packet acquisition module is used to sequentially acquire data packets at the gateway; The traffic data generation module is used to generate corresponding traffic data to be tested based on data packets; The traffic anomaly detection model management module is used to train and store traffic anomaly detection models; The traffic anomaly monitoring module is used to monitor abnormal traffic based on the traffic data to be measured and the traffic anomaly detection model.

Images