Fine-grained traffic classification method based on improved residual convolutional network in SDN environment

Abstract

The application relates to a fine-grained traffic classification method based on an improved residual convolutional network in an SDN environment and belongs to the software technical field. In order to provide finer-grained application-aware traffic classification, let network operators better analyze network composition and manage and schedule network resources, fine-grained classification of the specific application programs is very important. Traditional methods tend to classify traffic based on protocols, which is coarse-grained classification. Inspired by the research in computer vision, the method of the residual convolutional network is applied to the identification and classification of network traffic. The method solves the network degradation problem that occurs in the process of fine-grained network traffic identification by traditional deep learning methods with the increase of network depth, can effectively learn deeper network features, and further realizes fine-grained network traffic classification.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of software and relates to a fine-grained traffic classification method based on an improved residual convolutional network in an SDN environment. BACKGROUND

[0002] With the development of the Internet of Things, cloud computing and 5G, the number of nodes accessing the network is increasing, the network transmission rate is growing, and various new applications are emerging, resulting in exponential growth of network traffic, which makes the network overburdened and prone to problems such as resource waste and network congestion. Traffic classification is an important network function, and its purpose is to divide traffic into multiple priorities or multiple services, which provides a method for performing fine-grained network management by identifying different traffic types. With the help of traffic classification, network operators can more effectively handle different businesses and allocate network resources, improving network quality and user experience.

[0003] SDN is an emerging and promising network model that can greatly simplify network management, improve network resource utilization, optimize network performance and reduce operating costs. Software-defined networking (SDN) provides us with a new opportunity to achieve intelligence within the network. Specifically:

[0004] First, the centralized architecture of SDN enables the controller to have a central view of the network, collect various network data, effectively solve the data acquisition problem in deep learning, and facilitate the application of deep learning algorithms. Second, based on real-time and historical network data, deep learning technology can provide intelligence for SDN controllers through data analysis, network optimization and automation of network services. Third, the programmability of SDN allows applications to program the network, enabling the optimal network solutions made by deep learning algorithms to be executed in real time on the network. Fourth, the latest advances in computing technologies such as graphics processing units (GPUs) and tensor processing units (TPUs) provide technical support for applying deep learning technology in the network field. Therefore, it is appropriate and effective to apply deep learning technology in SDN.

[0005] The results of traditional machine learning methods for traffic classification are highly dependent on network traffic feature engineering, which requires a lot of time and manpower, which is not suitable for some core or edge devices. In addition, traditional machine learning methods cannot be applied to the classification and identification of encrypted traffic, greatly reducing the universality of machine learning methods. Deep learning is an end-to-end learning method, and its output is not affected by the features of the input data, eliminating the need for manual feature extraction and relieving the heavy workload of feature learning and selection. Therefore, methods based on deep learning can perform more accurate, faster and more fine-grained traffic classification.

[0006] The results of deep learning algorithms can be divided into two categories according to the classification level: coarse-grained and fine-grained. Coarse-grained classification identifies traffic into several categories according to rough traffic types (such as web browsers, bulk transfers, VOIP, etc.). Fine-grained classification aims to distinguish individual applications or functional models (such as the Facebook application can generate video, voice, audio sharing streams, etc.), rather than classifying them into rough traffic categories, which is beneficial for operators to further analyze traffic composition and user portraits and improve network service quality.

[0007] SDN networks, as a promising network architecture, have been widely discussed and researched, but network security and QoS quality still face great challenges, therefore, the traffic classification problem in SDN network has also become an important research. Currently, there are documents that apply deep learning methods to various challenges and problems in software-defined networks, from the perspectives of traffic classification, routing optimization, QoS / QoE prediction, resource management, and security, and discuss the application of ML algorithms in the field of SDN in detail. The research on traffic classification task has a good overview effect.

[0008] Network traffic classification technology has experienced four stages:

[0009] First, based on port traffic classification, but with the emergence of dynamic ports, hidden ports, the recognition accuracy of this method has decreased significantly.

[0010] Second, based on payload traffic classification, such as deep packet inspection, by searching for application signatures in the payload of IP packets, it can avoid dynamic port problems to some extent. But this method needs to constantly update and maintain the protocol feature library, and will produce great resource consumption in the face of rapidly developing networks, in addition, when encrypted traffic appears, this method is difficult to implement and has high computational overhead.

[0011] Third, based on traditional deep learning traffic classification, this method mainly relies on some external statistical features of traffic, such as packet time, total packet number, traffic duration, etc. to realize the classification of traffic, common ones are support vector machine SVM, random forest RF, decision tree DT, K-means algorithm, this kind of method has high adaptability to dynamic ports and encrypted traffic, but needs complex feature engineering, which increases the complexity of the method.

[0012] Fourth, traffic classification based on deep learning. The feature extraction and selection of deep learning-based methods are automatically completed through model training. Compared with traditional deep learning methods, deep learning methods have higher learning ability. In addition, deep learning methods can directly learn the features of raw traffic without additional artificial design of unique or private traffic information. These characteristics make the DL-based method an ideal traffic classification method. There is an end-to-end encrypted traffic classification method based on one-dimensional convolutional neural network. This method integrates feature extraction, feature selection, and classifier into a unified end-to-end framework, and uses the public ISCX VPN-non VPN traffic dataset for verification. However, the article does not discuss the optimization of model parameters, so the generalization ability of the model needs to be further verified. There is a deep packet framework called Deep Packet. This framework implements SAE and one-dimensional CNN respectively, automatically extracts network traffic to classify traffic, and can handle application recognition and traffic characterization tasks. This article still uses the public traffic dataset ISCX VPN-non VPN for verification, and both networks can accurately classify packets. The author shows that the DeepPacket framework outperforms all similar work on the ISCX VPN-non VPN traffic dataset in terms of application recognition and traffic characterization. However, the author does not solve the multi-channel classification problem (i.e. fine-grained classification, such as different functional flows of the same application). There is a lightweight traffic classification and intrusion detection framework based on deep learning called Deep Full Range (DFR). This framework implements one-dimensional CNN, LSTM, and SAE, and uses two public datasets ISCX VPN non VPN and ISCX 2012 IDS for evaluation and can achieve high accuracy. There is a semi-supervised learning encrypted traffic classification method based on generative adversarial network (GAN) called ByteSGAN, which is embedded in the SDN edge gateway to achieve the goal of traffic classification and further improve network resource utilization. Based on the public dataset "ISCX2012 VPN-non VPN and Crossmarket, the results show that ByteSGAN can effectively improve the performance of the traffic classifier. However, the mode collapse and instability of GAN training are problems that need to be considered and solved in this article. There is a new encrypted traffic classification scheme based on stream space-time features called TSCRNN, which is used for efficient management of industrial Internet of Things. TSCRNN uses CNN to extract abstract features, and then learns time features based on these low-dimensional feature maps through stacked Bi-LTSM. There is a new self-attention deep learning method called DarknetSec for dark web traffic classification and application recognition.The one-dimensional cascading model convolutional neural network (1D CNN) and the bidirectional long short-term memory (Bi-LSTM) network are used to capture local space-time features from the payload content of the data packet, and meanwhile, the self-attention mechanism is integrated into the above feature extraction network to mine the internal relationship and hidden connection between the previously extracted content features. There is a current online multimedia traffic classification framework based on a convolutional neural network (CNN), which can perform fast early classification and class-incremental learning. The authors apply sliding window technology and probability distribution functions to feature design. In order to better support the addition of new services, a class-incremental learning model is developed using knowledge distillation and bias correction techniques. However, the time consumption of feature extraction limits the method to a certain extent.

[0013] Table 1 Summary of related papers

[0014]

[0015]

[0016] In summary, there are several main problems in the current related traffic classification literature: first, the data sets selected by the authors all select traffic data sets with high consistency in each category and obvious distinction between different categories, so high accuracy can be achieved, but in the actual network environment, it is more important to accurately distinguish the fine-grained network streams generated by the same application. Second, there is no unified traffic preprocessing scheme, and various feature processing or traffic extraction methods greatly increase the time consumption in the early stage. Third, in the selection of models, most studies choose CNN and LSTM, and only a few articles explore the performance of other deep learning methods in network traffic classification tasks.

[0017] To solve these problems, this paper first selects a data set with both distinction and similarity between categories from a public data set website, then uses a unified traffic preprocessing method to process the original network traffic data set into image data and further converts it into a form recognizable by the model, and finally proposes an improved residual convolutional network-based model framework to classify the preprocessed data and perform multiple comparison experiments to verify the reliability of the model. SUMMARY

[0018] Therefore, the purpose of the present application is to provide a fine-grained traffic classification method based on an improved residual convolutional network in an SDN environment.

[0019] To achieve the above purpose, the present application provides the following technical solutions:

[0020] The fine-grained traffic classification method based on the improved residual convolutional network in the SDN environment comprises the following steps:

[0021] S1: classify network traffic categories, define network traffic splitting granularity;

[0022] S2: perform traffic classification tasks in SDN;

[0023] S3: data preparation, data preprocessing, data visualization;

[0024] S4: design a fine-grained traffic classification system based on residual convolutional network.

[0025] Optionally, the S1 is specifically:

[0026] Classify traffic categories according to protocols, applications and traffic type granularity;

[0027] Network traffic splitting granularity includes TCP connection, flow, session, service and host; Different splitting granularity will result in different traffic units; Flow is defined as all data packets with the same 5-tuple, i.e. timestamp, source IP, destination IP, protocol and packet length; Session is a bidirectional flow, including traffic in both directions;

[0028] Let network traffic be composed of a series of continuous data packets P=p1, p2,..., p i ,...p n , where p i represents the ith data packet; By using the traffic splitting tool SplitCap, the five-tuple is extracted from the data packet header, and each data packet is defined as p i =(t i ,src i ,dst i ,pro i ,len i ), i=1,2,...,|p|, where t i represents the timestamp, src i represents the source IP address, dst i represents the destination IP address, pro i represents the protocol type, and len i represents the packet length.

[0029] Optionally, the S2 is specifically:

[0030] S21: the switch collects network traffic and sends it to the controller through the OpenFlow protocol;

[0031] S22: the controller extracts traffic data and then sends the features to the preprocessing module to process the original data set;

[0032] S23: send the processing result to the classification module to classify the flow;

[0033] S24: send the classification result to the SDN controller;

[0034] S25: the controller analyzes the traffic composition according to the classification result and makes a decision on management and scheduling.

[0035] Optionally, in the S3, the data preparation is as follows:

[0036] Raw data sets in 11 real environments are obtained from a public data set website, including facebook_audio, facebook_chat, facebook_video, hangout_chat, hangouts_audio, hangouts_video, skype_audio, netflix, vimeo, youtube and email, and are named CIC_TF.

[0037] Optionally, in the S3, the data preprocessing is as follows:

[0038] Raw PCAP format traffic data is converted into IDX format data recognizable by a residual convolutional network, including traffic segmentation, normalization, imaging and PNG to IDX, specifically including:

[0039] 1) Traffic segmentation: the original continuous PCAP traffic data is split into smaller discrete PCAP traffic data in the form of session + all layers as the granularity; empty files and duplicate files are deleted;

[0040] 2) Normalization: all files are trimmed to a uniform length, and the files are unified to 28*28, 784 bytes. When the file size is greater than 784 bytes, it is trimmed to 784 bytes. When the file size is less than 784 bytes, 0x00 is added at the end to supplement to 784 bytes. After trimming, all files are divided into test set and training set in the ratio of 1:9;

[0041] 3) Imaging: this step converts the files with the same size processed in step 3) into grayscale images; each byte of the original file represents a pixel, 0x00 represents black, and 0xff represents white. All files are batched and converted into 28*28 grayscale images;

[0042] 4) PNG to IDX: PNG format image data is converted into a data format recognizable by a network model, and an IDX filler generator is used to convert image data of the same size into a two-dimensional IDX file.

[0043] Optionally, in the S3, the data visualization is as follows:

[0044] Visualize the partial images generated by step 3) visualization, each gray-scale image is 28*28 bytes in size.

[0045] Optionally, the S4 is specifically:

[0046] If each layer is regarded as a nonlinear function, the entire network is regarded as a composite nonlinear multivariate function:

[0047] F(x) = f n (...f3(f2(f1(x)*w1+b1)*w2+b2)...)

[0048] Where: F(x) represents the output of the neural network, f represents the result function after the activation function processing, x represents the input of the neural network, w represents the weight, and b represents the bias; Obviously, the output of this chain structure will change exponentially with the number of network layers and the size of the activation function, which is the typical gradient explosion and gradient disappearance problem in deep learning;

[0049] The deep residual network is stacked by a residual module, and the structure of the residual block is as follows:

[0050] The residual block is divided into identity mapping and residual mapping two parts; let the input of the neural network be X, and the expected output be H(x), through the direct mapping method, the input X is directly transmitted to the output, if F(x) and X have the same channel, then the output result is H(x) = F(x) + X, if F(x) and x have different channels, then use a special one-dimensional convolution kernel in the convolutional neural network to adjust the channel number to make it the same as F(x) channel, and the output result is: H(x) = F(x) + WX, when F(x) = 0, the output results of the two cases are: H(x) = X, that is, identity mapping;

[0051] (1) Convolutional layer

[0052] Let the input shape be n h *n w , the shape of the convolution kernel is k h *k w , the padding size is p, and the step is s, then the output shape is:

[0053]

[0054] The complete feature map is obtained by multiple convolution kernels, and the output of the lth layer and the kth feature map at position (i,j) is represented as follows:

[0055]

[0056] Where, and​ respectively are the weight vector and bias term of the kth convolution kernel of the lth layer, is the receptive field centered at position (i,j) of the lth layer;

[0057] For each Mini-Batch, a training process contains m training instances, the activation value of each neuron in the hidden layer is transformed as follows:

[0058]

[0059] The last convolution layer often needs to use a nonlinear activation function to generate a nonlinear mapping between input and output, further enhance the feature learning ability of the network, and the output after the activation function processing is expressed as:

[0060]

[0061] (2) Adaptive average pooling layer

[0062] Let padding be 0, and the size of the pooling kernel be expressed as:

[0063]

[0064] In the residual convolution network, batch_size=n gray traffic pictures with a size of I=28*28*1 are read from the IDX file; then, two residual layers are passed through, each residual layer includes a convolution layer, a batch normalization layer and an activation function layer, the batchnorm and relu functions are used to optimize the network result, and then a convolution layer and a normalization layer are used to convert it into n_classes feature maps, and finally, an adaptive average pooling layer AdaptiveAvgPool2d is used to output the feature values of each class.

[0065] The beneficial effects of the present application are:

[0066] 1) A traffic classification method based on residual convolution network in SDN environment is proposed, which realizes high-precision classification effect on public traffic data set.

[0067] 2) A traffic data set preprocessing method based on representation learning is designed, which converts the original traffic data into a data form recognizable by the network model.

[0068] 3) High-precision fine-grained traffic classification is realized, which effectively solves the gradient vanishing problem that may occur when traditional deep learning methods identify fine-grained traffic.

[0069] 4) Compared with the current popular deep learning method, the results prove that the method used in this paper can significantly improve the accuracy of fine-grained traffic classification.

[0070] Additional advantages, objects, and features of the application will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following or can be learned by practice of the application. The objects and other advantages of the application can be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings. BRIEF DESCRIPTION OF DRAWINGS

[0071] In order to make the objects, technical solutions and advantages of the present application clearer, the preferred detailed description of the present application will be combined with the drawings to make the objects, technical solutions and advantages of the present application clearer, wherein:

[0072] Figure 1 It is a basic architecture diagram for traffic classification in SDN;

[0073] Figure 2 It is a data preprocessing process;

[0074] Figure 3 It is data visualization;

[0075] Figure 4 It is the training loss value of the CNN model on the CIC_TF data set;

[0076] Figure 5 It is the basic structure of the residual block;

[0077] Figure 6 It is a confusion matrix result graph;

[0078] Figure 7 It is a confusion matrix graph of the model prediction on the USTC-TK data set;

[0079] Figure 8 It is a comparison experiment result graph;

[0080] Figure 9 It is the influence of various hyperparameters on the model effect; Figure 9 (a) is the training period, Figure 9 (b) is the number of residual blocks, Figure 9 (c) is the learning rate, Figure 9 (d) is the batch size. DETAILED DESCRIPTION

[0081] The present application is herein described, by way of example only, with the assistance of the accompanying drawings detailed description. As will be realized by those skilled in the art, the application is capable of other and different embodiments, and its details are capable of modifications in various obvious respects, all without departing from the spirit and scope of the application. Accordingly, the drawings and description are illustrative in nature and not restrictive.

[0082] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the principles of the application. In the drawings:

[0083] The same or similar components in the drawings of the embodiments of the present application correspond to the same or similar components; in the description of the present application, it should be understood that if the terms "upper", "lower", "left", "right", "front", "back" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, therefore the terms describing the positional relationship in the drawings are only for illustrative purposes, and cannot be understood as a limitation of the present application, for those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0084] I. Problem Definition

[0085] (1) Classification granularity

[0086] The first step to build a traffic classifier is to define the classification goal. Typical goals include QoS guarantee, resource allocation, and intrusion detection, etc. To achieve these goals, traffic categories can be classified according to the following granularities:

[0087] 1. Protocol (e.g. FTP, HTTP)

[0088] 2. Application (e.g. Youtube, QQ)

[0089] 3. Traffic type (e.g. video, audio, chat)

[0090] Therefore, the goal is to label each flow with the corresponding traffic category. In this paper, we mainly classify different types of network traffic from the perspective of QoS and resource allocation to achieve more refined network management.

[0091] (2) Flow granularity

[0092] Network traffic splitting granularity includes: TCP connection, flow, session, service and host. Different splitting granularity leads to different traffic units. Flow is defined as all packets with the same 5-tuple, i.e. timestamp, source IP, destination IP, protocol and packet length. Session is a bidirectional flow, including traffic in both directions. In addition, from the OSI seven-layer model of network traffic, on the one hand, the inherent characteristics of traffic are intuitively reflected in the application layer (the seventh layer), on the other hand, the data of other layers should also contain some traffic feature information such as the port information of the transport layer. Therefore, this paper uses the splitting granularity of all layers + session, which maximizes the preservation of the data characteristics of the original traffic.

[0093] In order to better describe the problem, it is expressed as follows:

[0094] Network traffic is composed of a series of continuous packets P = p1, p2,..., p i ,...p n , where p i represents the i-th packet. By using the traffic splitting tool SplitCap to extract the five-tuple from the packet header, each packet can be defined as p i = (t i , src i , dst i , pro i , len i ), i = 1, 2,..., |p|, where t i represents the timestamp, src i represents the source IP address, dst i represents the destination IP address, pro i represents the protocol type, and len i represents the packet length.

[0095] II. Basic architecture

[0096] As shown in Figure 1 , SDN is a new network model that realizes network infrastructure virtualization by separating control and data planes, thus creating a dynamic, flexible, automated and manageable three-layer architecture. SDN controller is the key component of the entire SDN architecture, which mainly controls SDN switches to manage the entire data flow. Most importantly, the scalability and programmability of SDN make it possible to deploy deep learning technology.

[0097] The traffic classification task in SDN can be summarized as follows:

[0098] ①Switch collects network traffic and sends it to the controller through the OpenFlow protocol.

[0099] ②The controller extracts the traffic data and then sends the features to the preprocessing module to process the original dataset.

[0100] ③The processing result is sent to the classification module to classify the flow.

[0101] ④Finally, the classification result is sent to the SDN controller.

[0102] ⑤The controller analyzes the traffic composition according to the classification result and makes efficient management and scheduling decisions.

[0103] Three, data preprocessing

[0104] (1) Data preparation

[0105] In order to achieve the purpose of classifying different traffic types, i.e. fine-grained classification, 11 original datasets in real environment (i.e. facebook_audio, facebook_chat, facebook_video, hangout_chat, hangouts_audio, hangouts_video, skype_audio, netflix, vimeo, youtube, email) are obtained from the public dataset website and named CIC_TF. It involves several common audio, video and conversation traffic datasets, including both coarse-grained different application datasets and fine-grained different conversation datasets of the same application. The number of data packets contained in each category is shown in Table 2, where the proportion of various traffic data is different and the proportion of video traffic data is large, which also conforms to the actual traffic distribution.

[0106] Table 2 Dataset structure

[0107] TrafficType Quantity Percentage (%) facebook_audio 46061 10.68 facebook_chat 5527 1.28 facebook_video 1569 0.36 hangout_chat 7587 1.76 hangouts_audio 87334 20.26 hangouts_video 8083 1.87 skype_audio 60207 13.96 netflix 51932 12.04 vimeo 92705 21.50 youtube 39742 9.22 email 30410 7.05

[0108] (2) Preprocessing process

[0109] The purpose of this process is to convert the original PCAP format traffic data into IDX format data recognizable by the residual convolutional network, while solving the problems of inconsistent data, redundant data, etc. It includes four steps: traffic segmentation, normalization, imaging, PNG to IDX. The basic flow is as follows: Figure 2

[0110] ​1) Flow segmentation: In this process, we use the SplitCap tool to process it, which can split the PCAP file into smaller files according to the 5-tuple standard, etc. First, we split the original continuous PCAP traffic data into smaller discrete PCAP traffic data in the form of session + all layers. But when some split packets have the same content, the same file will be generated, and repeated data will produce bias when training the network model, so empty files and repeated files also need to be deleted to eliminate interference.

[0111] 2) Normalization: This step trims all files to a uniform length. The size of the residual network input data must be fixed, inspired by the classic picture data set MINIST we unify the file to 784 bytes (i.e. 28*28), when the file size is greater than 784 bytes, it will be trimmed to 784 bytes, when the file size is less than 784 bytes, 0x00 will be added at the end to supplement to 784 bytes. After trimming, all files are divided into test set and training set in the ratio of 1:9.

[0112] 3) Imaging: This step converts the files with the same size processed in step 3) into grayscale images. Each byte of the original file represents a pixel, 0x00 represents black, and 0xff represents white. Batch all files into 28*28 grayscale images.

[0113] 4) PNG to IDX: The residual network model cannot directly train image data, this step needs to further convert the PNG format image data into a data format that the network model can recognize. IDX-Ubyte format is a common file format in deep learning, we use IDX filler generator to convert the size consistent image data into two-dimensional IDX file, which is convenient for subsequent model training.

[0114] Algorithm 1 describes the detailed steps of the entire preprocessing process:

[0115]

[0116] The symbols involved in Algorithm 1 are explained in Table 3:

[0117] Table 3 Symbol Description

[0118]

[0119] (3) Data visualization analysis

[0120] This part visualizes part of the images generated in the third step of the data preprocessing process as shown in Figure 3 , where each grayscale image is 28*28 bytes in size.

[0121] FromFigure 3 The visualization analysis can see that there is no obvious degree of distinction between different service type traffic of the same application, such as audio, chat and video traffic in Facebook. There is also no high consistency between the unified service type traffic of different applications, such as facebook_video, hangots_video and youtube. Therefore, in order to distinguish these smaller granularity service traffic to the greatest extent, as many features as possible must be learned.

[0122] Four, design of fine-grained traffic classification system based on residual convolutional network

[0123] The most widely used and best classification effect of traffic classification technology so far is the neural network in deep learning, especially the convolutional neural network. However, we found in the experiment that when there is a certain similarity between the categories of the data set, the classification effect of the general convolutional neural network will be greatly reduced, and with the increase of training epoch, the model gradually begins to degenerate, and the problem of gradient disappearance occurs.

[0124] Figure 4 is the training loss value of the CNN model on the CIC_TF data set, the horizontal coordinate represents epochs, and the vertical coordinate is the loss value amplified by 100 times. From the figure, it can be seen that when epochs is between 0 and 50, the model can learn the data features well, and the training loss value can quickly and smoothly decrease; when epochs is between 50 and 150, the loss value starts to fluctuate greatly up and down, and the learning ability of the model begins to gradually decrease; when epochs is greater than 150, the loss value is even equal to 0, at this time, the model has completely degenerated. As can be seen, the simple CNN model cannot solve the classification problem of fine-grained traffic data set. The residual convolutional network can make the current layer network identity map the input of the last layer through the residual module, so that the effect of the network will not be worse than the last layer, which to a large extent solves the problem of network degeneration, and the introduction of convolutional layer also inherits the advantages of general convolutional neural network.

[0125] In deep learning research, the deeper the network structure, the more it can fit the most real function. However, the deep network is stacked by many nonlinear layers, if each layer is regarded as a nonlinear function, then the whole network can be regarded as a composite nonlinear multivariate function:

[0126] F(x)=f n (...f3(f2(f1(x)*w1+b1)*w2+b2)...)

[0127] Where: F(x) represents the output of the neural network, f represents the result function after the activation function processing, x represents the input of the neural network, w represents the weight, and b represents the bias. Obviously, the output of such a chain structure will change exponentially with the number of network layers and the size of the activation function, which is the typical gradient explosion and gradient disappearance problem in deep learning.

[0128] The deep residual network is stacked by residual modules, and the main idea is identity mapping, which ensures that the deep effect is not weaker than the shallow effect. The basic structure of the residual block is as follows:

[0129] As shown in Figure 5 : The residual block is divided into identity mapping and residual mapping. Assuming that the input of the neural network is X, and the expected output is H(x), the input X is directly mapped to the output by direct mapping (i.e. the curved part on the right in the figure), if F(x) and X have the same channel, then the output result is H(x) = F(x) + X, if F(x) and x have different channels, then use a special one-dimensional convolution kernel in the convolutional neural network to adjust the channel number to make it the same as F(x) channel, and the output result is: H(x) = F(x) + WX, when F(x) = 0, then the output result in the two cases is: H(x) = X, that is, identity mapping.

[0130] (1) Convolutional layer

[0131] The biggest advantage of the convolutional network compared with the traditional neural network is the parameter sharing and sparse connection, which greatly reduces the complexity of the neural network model training, and has been widely used in image recognition, target detection and other fields. The main goal of the convolutional layer is to use the movement of the convolution kernel on the input image to calculate the feature value, and then form a feature map corresponding to the current convolution kernel. The value of the convolution kernel is the so-called weight, and padding is needed when there is no input data at the corresponding position of the convolution kernel. The unit of convolution kernel movement is stride. Assuming that the input shape is n h *n w , the shape of the convolution kernel is k h *k w , the padding size is p, and the stride is s, then the output shape is:

[0132]

[0133] The complete feature map is obtained by multiple convolution kernels, and the output of the lth layer and the kth feature map at position (i, j) is represented as follows:

[0134]

[0135] where, ​and are the weight vector and bias term of the kth convolution kernel of the 1st layer respectively, is the receptive field centered at position (i,j) of the 1st layer.

[0136] As the depth of the network increases, the training becomes more and more difficult, and the convergence speed is also slower and slower. As the neural network is trained, the parameters of each layer of the neural network are constantly changing, so even if the input of the previous layer network obeys the same distribution, after the change of the network parameters, the output data of the previous layer network no longer obeys the same distribution as the output data of the last iteration. Thus, the input data of the current layer neural network of the current iteration and the input data of the last iteration obey different distributions. This imagination is also called internal covariate shift.

[0137] To solve this problem, this paper uses the Batchnorm (BN) method, which basically forces the distribution of the input value of any neuron in each layer of the neural network to the standard normal distribution of the domain through certain normalization means. This makes the activation input value fall in the region where the non-linear function is sensitive to the input, and small changes in the input will cause large changes in the loss function, thereby avoiding the problem of gradient disappearance at the same time, and the large gradient can greatly speed up the training speed.

[0138] For each Mini-Batch, a training process contains m training instances, and the specific BN operation is to transform the activation value of each neuron in the hidden layer as follows:

[0139]

[0140] The last convolution layer often needs to use a nonlinear activation function to generate a nonlinear mapping between input and output, further enhancing the feature learning ability of the network. The output after the activation function processing can be expressed as:

[0141]

[0142] (2) Adaptive average pooling layer

[0143] The addition of the pooling layer can downsample the sample features collected by the convolutional layer, further reducing the parameter amount. The adaptive average pooling layer can dynamically calculate the size of the kernel and the step size of each movement by inputting the original size and the target size, and each operation uses the average value method. Compared with the ordinary pooling layer, it does not need to manually specify specific kernel size, step size, padding value and other parameters. The size of the pooling kernel is rounded up using the maximum principle, and then the equidistant position points are calculated on the feature map. The starting coordinates are rounded down using the minimum principle. Assuming that padding is 0, the size of the pooling kernel can be simplified as:

[0144]

[0145] In the residual convolutional network proposed in this paper, first read batch_size = n gray flow pictures with size I = 28 * 28 * 1 from IDX file. Then pass through two residual layers, each residual layer includes convolutional layer, batch normalization layer and activation function layer. In order to make the network training easier and the network generalization stronger, this paper uses batchnorm and relu function to optimize the network result, and then converts it to n_classes feature maps through a convolutional layer and a normalization layer. Finally, the AdaptiveAvgPool2d output feature value of each class. The specific parameters are shown in Table 4:

[0146] Table 4 Model parameter setting, output size and parameter number

[0147]

[0148] Five, experimental setting and result analysis

[0149] (1) Loss function

[0150] This paper uses cross entropy loss function (Cross Entropy Loss). Cross entropy mainly describes the distance between actual output (probability) and expected output (probability), that is, the smaller the value of cross entropy, the closer the two probability distributions. Assuming that x is the input sample feature value, the probability distribution p is the expected output, the probability distribution q is the actual output, and H(p,q) is the cross entropy, then:

[0151]

[0152] (2) Optimizer

[0153] Gradient descent is the most commonly used method for optimizing neural networks. The ultimate goal of gradient descent is to find a set of suitable θ (w1, w2,..., wn) that can minimize the loss function. The gradient descent algorithm is as follows: n) is a method that minimizes the objective function J(θ) value. The specific method is to update the parameters in the opposite direction of the gradient of the objective function with respect to the parameters until the global minimum or local minimum is reached. The basic calculation formula is as follows:

[0154]

[0155] where α represents the learning rate, denotes the partial derivative of θ.

[0156] This paper chooses to use the Mini-batch gradient descent method, which combines the advantages of batch gradient descent and stochastic gradient descent. This method reduces the variance of parameter updates and makes convergence more stable. At the same time, it can use highly optimized matrix optimization methods in mature deep learning libraries to efficiently solve the gradient of parameters according to each small batch of data. The result of using n small batches of training samples at the i-th update is:

[0157]

[0158] To help SGD speed up in the relevant direction and suppress jitter, a component β can be added to the history of the update vector in the current update vector:

[0159]

[0160] θ = θ - v t

[0161] where v t denotes the estimate of the t-th number, and β is a tunable parameter. The training pseudo code is shown in Algorithm 2:

[0162]

[0163]

[0164] (3) Evaluation index

[0165] Accuracy: For single-label tasks, each sample has only one correct class, so the most intuitive indicator of classification is accuracy (Accuracy), which is used to predict whether the sample is classified correctly, and the calculation method is as follows:

[0166]

[0167] Let the class of sample x i be y i , the class is (0, 1,..., N), and the predicted class function is f, then the calculation method of TOP-1 Accuracy is as follows:

[0168]

[0169] Precision: Precision is for the predicted results, which indicates how many of the predicted positive samples are truly positive samples.

[0170]

[0171] Recall: Recall is for the original samples, which indicates how many positive examples in the sample are predicted correctly.

[0172]

[0173] F1-score: This indicator is an indicator for evaluating recall and precision. Only when both precision and recall are high and the gap is small, F1-score will be high, so F1-score is a balanced performance indicator. The calculation method is as follows:

[0174]

[0175] Where: TP: the number of positive classes predicted as positive classes, FN: the number of positive classes predicted as negative classes, FP: the number of negative classes predicted as positive classes, TN: the number of negative classes predicted as negative classes.

[0176] Since it is a multi-classification problem, it is not convenient to use PR, RC, and F1 for a single type. Therefore, the MacroAverage is used to calculate the average value of precision, recall, and F1 of each class as follows:

[0177]

[0178]

[0179]

[0180] Confusion matrix: Confusion matrix is an error matrix commonly used to visually evaluate the performance of the model. The confusion matrix is a square matrix of size (n_classes, n_classes), where n_classes represents the number of classes. Each row of this matrix represents an instance in the true class, and each column represents an instance in the predicted class.

[0181] The experimental environment of this paper is:

[0182] NVIDIA-SMI 460.32.03GPU as accelerator, Python 3.7.13+Pytorch 1.12.0 as experimental framework, experimental data randomly selects one-tenth of the data as test data, and the rest as training data. The minimum batch size is 128 when reading during training, the loss function selects the cross-entropy loss function, the built-in SGD of Pytorch as the optimizer, the learning rate is 0.01, the momentum parameter is set to 0.9, and the training time is 150 cycles.

[0183] (4) Experimental results

[0184] Table 5 shows the precision, recall and F1 score of each class in the CIC_TF dataset on the residual convolutional network model proposed in this paper, and finally the macro average and weighted average results are calculated, where the macro average is the average of the classification results of all classes, and the weighted average is an improvement of the macro average, considering the proportion of the number of samples of each class in the total samples. The experimental results show that the overall classification accuracy of the model can reach 99.93%, the macro average precision, recall and F1 score can reach 99.98%, 100%, 100%, and the weighted average precision, recall and F1 score can reach 100%, 100%, 100%. In general, whether it is for the ftps_down, skype_audio traffic with less sample size or the FTP traffic with more sample size, high accuracy can be achieved. In addition, different service flows of the same application program, such as the conversation and video traffic models in facebook, can also be accurately classified, and similar service flows of different application programs, such as the video flow of facebook, the video flow of hangouts, and the video flow of skype, can also be well extracted by the model and effectively distinguished. In general, the residual convolutional network model proposed in this paper has good performance in classifying similar fine-grained traffic data.

[0185] Table 5 Precision, recall, F1 score, sample size and accuracy of each class

[0186]

[0187]

[0188] In order to better observe the specific type of each class of traffic classified by the model, we also calculated the confusion matrix diagram of the model classification results as follows: Figure 6The heat value of the classification accuracy is shown on the right side, the higher the accuracy, the higher the classification accuracy, and the darker the color. The horizontal coordinate represents the model prediction result, and the vertical coordinate represents the real label. It can be seen from the figure that the value on the diagonal line is the darkest, which also shows that the model can well distinguish between different traffic. At the same time, it can also be seen that which categories the traffic will be classified into by the model, such as 0.001% of the hangouts_chat traffic will be classified as facebook_chat traffic.

[0189] (5) Comparison experiment

[0190] Dataset

[0191] In order to verify the effectiveness and versatility of the model in the traffic classification task, the same hyperparameters as the above experiment are used to conduct experiments on the USTC-TK traffic dataset, which contains ten kinds of traffic, eight of which are different application traffic, part of which is malware traffic collected from public websites in real network environment, and the other part is normal traffic collected using IXIA BPS traffic simulation equipment. The same preprocessing operation is performed on the dataset before the experiment, and the original PCAP form traffic is converted into IDX form data. The experimental results are as follows: Figure 7 The confusion matrix can also be seen that almost all samples can be correctly classified, and only 0.3% of the FTP traffic is identified as MySQL traffic. Therefore, the model proposed in this paper is also applicable to the classification task of other granularity traffic datasets.

[0192] Table 6 Accuracy, recall, F1 score, sample size and accuracy of each class of USTC-TK dataset

[0193]

[0194]

[0195] (6) Comparison with other advanced methods

[0196] In order to verify the advancement of the model proposed in this paper in the fine-grained traffic classification task, we compare the performance of the classic residual network model ResNet-18 on the CIC_TF dataset, and compare other deep learning methods such as 1D-CNN, LSTM vertically. The specific parameters of the model are shown in Table 7 (the best effect in the experimental process is used as the standard):

[0197] Table 7 Comparison of specific model parameter settings

[0198]

[0199] The accuracy, precision, recall and F1 score results of the experiments are shown in Table 8 and Figure 8 From Table 8, it can be seen that the current advanced deep learning methods can achieve more than 80% classification accuracy. Specifically, the overall classification accuracy of ordinary one-dimensional convolutional neural network, recurrent neural network and classic residual network can reach 86%, 92% and 95% respectively, and the overall accuracy of the improved residual convolutional model proposed in this paper can reach 99%, which is 13%, 7% and 2% higher than the other three methods respectively. From Figure 8 It can be more intuitively seen that the model proposed in this paper is significantly better than other types of deep learning methods.

[0200] Table 8 Comparison of experimental results (Accuracy, Precision, Recall, F1-score)

[0201] Methods Accuracy Precision Recall F1-score Proposed 0.99 0.99 1.00 1.00 ResNet-18 0.95 0.89 0.87 0.90 1D-CNN 0.86 0.85 0.83 0.84 RNN 0.92 0.91 0.88 0.90

[0202] (7) Comparison of hyperparameters

[0203] We conducted a series of experiments on the CIC_TF dataset to explore the influence of hyperparameter settings on model accuracy. The main hyperparameters include: training period epoch, number of residual blocks block_num, learning rate lr, batch_size. For each experiment, only the value of one parameter is changed, and the values of other parameters use the default values. Figure 9 The influence of various hyperparameters on the model effect. As shown in Figure 9 (a) shows that as the epoch increases, the model accuracy increases, and when the epoch is 150, the optimal accuracy is reached, but when the epoch continues to increase, the learning ability of the model begins to decline. Figure 9 (b) shows the influence of the number of residual blocks on the model effect. When the number of residual blocks is only one, the model cannot learn all the features well, and when the number of residual blocks increases to 2, the model effect is significantly improved, but the number of residual blocks is not the more the better to learn, on the contrary, with the increase of the number of residual blocks, the model may begin to degrade. Figure 9 (c) shows the influence of learning rate on the model effect. The size of the learning rate is updated by 0.5 times during training. It can be seen that for the model in this paper, the learning rate of 0.01 is the best, and with the further reduction of the learning rate, the model effect has a disastrous decline. Figure 9 (d) explores the influence of batch size on the classification effect of the model. Related practices have proved that the size of batch is usually 2 nThe effect is best at 128, and further experiments show that as the batch size increases, the model effect improves significantly and reaches the optimum at 128. When the batch size continues to increase, the model cannot learn more subtle features, and the accuracy of the model begins to decline.

[0204] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalently replaced without departing from the purpose and scope of the technical solutions, and they should be covered in the scope of the claims of the present application.

Claims

1. A fine-grained traffic classification method based on an improved residual convolutional network in an SDN environment, characterized by: The method includes the following steps: S1: Classify network traffic categories and define the granularity of network traffic splitting; S2: Traffic classification task in Software-Defined Networking (SDN); S3: Data preparation, data preprocessing, and data visualization; S4: Design a fine-grained traffic classification system based on residual convolutional networks, specifically: If each layer is considered as a nonlinear function, then the entire network can be viewed as a composite nonlinear multivariate function: F ( x )= f n (... f 3( f 2( f 1( x )* w 1+ b 1)* w 2+ b 2)...) in: F ( x ) represents the output of the neural network. f This represents the result function after the activation function has processed the data. x Represents the input to the neural network. w Represents weight, b The output of this chain structure changes exponentially with the number of network layers and the size of the activation function, which is the typical gradient explosion and gradient vanishing problem in deep learning. Deep residual networks are composed of stacked residual modules. The structure of a residual block is as follows: The residual block is divided into two parts: the identity mapping and the residual mapping; let the input of the neural network be... X The expected output is H ( x ), directly mapping the input X If transmitted to the output, F ( x )and X If the channels are the same, they are directly added together and the output result is: H ( x )= F ( x )+ X ,if F ( x )and X If the channels are not the same, then a special one-dimensional convolutional kernel in the convolutional neural network is used to adjust the number of channels to match F(…). x With the same channel, the output result is: H ( x )= F ( x )+W X ,when F ( x When )=0, the output results in the two cases are as follows: H ( x )= X That is, identity mapping; (1) Convolutional layer Let the input shape be n h * n w The shape of the convolution kernel is k h * k w The fill size is p Step size is s The output shape is: The complete feature map is obtained through multiple convolutional kernels, the first... l Layer k Each feature map at location ( i , j ) output It is expressed as follows: in, and They are the first l The first layer k The weight vector and bias term of each convolutional kernel. Therefore, the first l Layer location ( i , j The sensory field centered on ); For each Mini-Batch, a single training process includes m For each training instance, the activation value of each neuron in the hidden layer is transformed as follows: Convolutional layers often require a non-linear activation function at the end to generate a non-linear mapping between the input and output, further enhancing the network's feature learning ability. The output after activation function processing... Represented as: (2) Adaptive average pooling layer With padding set to 0, the size of the pooling kernel is expressed as: In the residual convolutional network, batch_size=n grayscale images of size I=28*28*1 are read from the IDX file. Then, the images are processed through two residual layers, each consisting of a convolutional layer, a batch normalization layer, and an activation function layer. The network results are optimized using batchnorm and ReLU functions. The network is then transformed into n_classes feature maps through a convolutional layer and a normalization layer. Finally, the feature values ​​for each class are output through the adaptive average pooling layer AdaptiveAvgPool2d.

2. The fine-grained traffic classification method based on an improved residual convolutional network in an SDN environment according to claim 1, characterized in that: Specifically, S1 is: Traffic categories are classified according to the granularity of protocol, application, and traffic type; Network traffic can be segmented into granularities including TCP connections, flows, sessions, services, and hosts. Different segmentation granularities result in different traffic units. A flow is defined as all packets that share the same 5-tuple, namely timestamp, source IP, destination IP, protocol, and packet length. A session is a bidirectional flow, including traffic in both directions. Suppose that network traffic consists of a series of consecutive data packets. P = p 1, p 2,..., p i ,... p n ,,in p i Representing the i Each packet is defined as follows: A five-tuple is extracted from the packet header using the traffic splitting tool SplitCap; each packet is defined as... p i =( t i , src i , dst i ,pro i , len i ), i =1,2,...,| p |, among which t i Represents a timestamp. src i Represents the source IP address. dst i Indicates the destination IP address. pro i Indicates the protocol type. len i Indicates the length of the data packet.

3. The fine-grained traffic classification method based on an improved residual convolutional network in an SDN environment according to claim 2, characterized in that: Specifically, S2 is: S21: The switch collects network traffic and sends it to the controller via the OpenFlow protocol; S22: The controller extracts traffic data and then sends the features to the preprocessing module to process the raw dataset; S23: The processing results are sent to the classification module to classify the stream; S24: Send the classification results to the SDN controller; S25: The controller analyzes the traffic composition based on the classification results and makes management and scheduling decisions.

4. The fine-grained traffic classification method based on an improved residual convolutional network in an SDN environment according to claim 3, characterized in that: In step S3, the data preparation is as follows: We obtained 11 classes of raw datasets from public dataset websites, including facebook_audio, facebook_chat, facebook_video, hangout_chat, hangouts_audio, hangouts_video, skype_audio, netflix, vimeo, youtube, and email, and named them CIC_TF.

5. The fine-grained traffic classification method based on an improved residual convolutional network in an SDN environment according to claim 4, characterized in that: In step S3, the data preprocessing is as follows: The process transforms raw PCAP format traffic data into IDX format data recognizable by residual convolutional networks, including traffic segmentation, normalization, visualization, and PNG to IDX conversion. Specifically, this includes: 1) Traffic splitting: The original continuous PCAP traffic data is split into smaller discrete PCAP traffic data at the granularity of session + all layers; empty files and duplicate files are deleted; 2) Normalization: Trim all files to a uniform length of 28*28, 784 bytes. If the file size is greater than 784 bytes, trim it to 784 bytes. If the file size is less than 784 bytes, add 0x00 to the end to make it 784 bytes. After trimming, divide all files into test set and training set in a 1:9 ratio. 3) Image Conversion: This step converts files of the same size processed in step 3) into grayscale images; each byte of the original file represents one pixel, 0x00 represents black, and 0xff represents white. All files are batch converted into 28*28 grayscale images. 4) PNG to IDX: Converts PNG format image data into a data format that the network model can recognize. The IDX filler generator is used to convert image data of the same size into two-dimensional IDX files.

6. The fine-grained traffic classification method based on an improved residual convolutional network in an SDN environment according to claim 5, characterized in that: In S3, the data visualization is as follows: Visualize the partial images generated in step 3), with each grayscale image being 28 * 28 bytes in size.

Images