Network anomaly detection method based on SDN and deep learning framework

Abstract

The invention proposes a network anomaly detection method based on an SDN and a deep learning framework, and relates to the technical field of network detection, and the method comprises the steps: S1, configuring network anomaly detection system data to establish a network anomaly detection model, and receiving a data packet in a network through an SDN controller; s2, performing feature extraction on the data packet to obtain feature data, and preprocessing the feature data; s3, transmitting the preprocessed feature data to a network anomaly detection model for data identification to judge whether the network has abnormal traffic, if so, reducing threats by reconfiguring a recovery strategy, and if not, directly outputting a detection result; according to the method, the network anomaly detection model is established and matched with the SDN controller to perform data identification on the abnormal flow, so that the anomaly detection efficiency can be improved, the false alarm rate is reduced, and the accuracy of data flow anomaly detection is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of network detection, and in particular, to a network anomaly detection method based on SDN and deep learning framework. Background Art

[0002] Network abnormal traffic refers to the data stream that deviates from the normal traffic behavior pattern in the communication network. It is usually caused by reasons such as network attacks, malware, traffic surges, or network device failures. Network traffic anomalies not only lead to a decline in network performance and service quality, but also pose a serious threat to network security. For example, network attackers may use abnormal traffic to conduct denial-of-service attacks, DDoS attacks, intrusion attempts, or data leaks. In addition, abnormal network traffic may cause network outages, data loss, system crashes, and significant damage to critical infrastructure. Detecting and analyzing abnormal traffic to identify abnormal activities has become a hot topic in current network security research. And network abnormal traffic attacks are launched by a single computing node to the server or network with a large amount of abnormal traffic, resulting in a decline in the performance of computer systems or even paralysis.

[0003] Traditional abnormal traffic detection methods extract traffic characteristics from the switchboard flow table and then implement feature detection of abnormal flows. However, when the system receives a large-scale data stream, the cost of maintaining network connection characteristics, state tables, and packet tags will be too high. Traditional abnormal traffic detection methods extract character or numerical features from traffic packets to classify normal or abnormal traffic, and cannot achieve dynamic detection. Especially in the edge computing environment, additional computing resources are still required to complete the detection. In addition, traditional abnormal traffic detection models use early attack data sets, cannot verify the effectiveness of current new attacks, and are difficult to cope with abnormal traffic detection work in the edge computing environment. When the abnormal flow detection model belongs to a shallow classification model, better detection results can be obtained when the data stream feature dimension is small; while when the data stream scale is large and the feature dimension is high, the detection and classification effect is not good, and the intelligence of the abnormal detection model needs to be enhanced to meet the real-time detection requirements in edge computing. At the same time, traditional methods for large-scale real-time detection of network anomalies generally have problems such as low efficiency, high computational complexity, and high false alarm rate. Therefore, traditional network abnormal traffic detection methods face challenges in dealing with network abnormal traffic detection work in edge computing. Summary of the Invention

[0004] The purpose of the present invention is to provide a network anomaly detection method based on SDN and deep learning framework, which can improve the efficiency of anomaly detection and reduce the false alarm rate.

[0005] The technical solution of the present invention is as follows:

[0006] This application provides a network anomaly detection method based on SDN and deep learning framework, which includes the following steps:

[0007] S1. Configure the data of the network anomaly detection system to establish a network anomaly detection model, and receive data packets in the network through the SDN controller;

[0008] S2. Extract features from the data packets to obtain feature data, and preprocess the feature data;

[0009] S3. Transmit the preprocessed feature data to the network anomaly detection model for data identification to determine whether there is abnormal traffic in the network. If so, mitigate the threat by reconfiguring the recovery strategy. If not, directly output the detection result.

[0010] Further, in step S2, the above-mentioned feature data includes source IP, source port, destination IP, destination port, protocol, timestamp, total number of forward packets, minimum packet length, maximum packet length, average packet length, FIN flag, SYN flag, RST flag, PUSH flag, ACK flag, and URG flag.

[0011] Further, in step S2, the process of preprocessing the feature data includes: deleting feature rows, converting text to numerical values, deleting rows where missing values are located, removing label columns, normalizing the feature data, and performing one-hot encoding on the label columns.

[0012] Further, in step S3, the above-mentioned recovery strategy includes: blocking abnormal traffic, redirecting abnormal traffic, and discarding abnormal traffic.

[0013] Further, the above-mentioned network anomaly detection model includes an anomaly scoring network and a reference score generator;

[0014] Among them, the expression of the anomaly scoring network includes:

[0015]

[0016] In the formula, f ij is a feature vector, is a feature representation learner, x i is an input image, x ij is the j-th feature of the i-th image, θ f is a weight matrix, χ(x i ) is the region derived from the image x i , Ω is an intermediate representation space, ξ is an anomaly scoring function, θ s is the weight parameter corresponding to the L elements of the feature vector, L is the number of feature vectors in each image, f ijk is the k-th element of the feature vector, k is the number of normal data, is the weight parameter corresponding to the k-th element of the feature vector, is the bias, ψ is the anomaly score map, and θ is the weight parameter in the neural network corresponding to the anomaly score mapping function;

[0017] The expression of the reference score generator includes:

[0018]

[0019] where μ is the average of all randomly sampled scores, and n i is the i-th anomaly score.

[0020] Furthermore, the training process of the above network anomaly detection model includes:

[0021]

[0022] L(x i , μ ref , σ ref , θ) = (1 - y i ) | dev(x i , θ) | + y i max(0, a - dev(x i , θ))

[0023] where ψ K is the top K anomaly regions of each image, K is the proportion of the region with the largest anomaly score in the image, x i is the input image, θ is the weight parameter in the neural network corresponding to the anomaly score mapping function, M(x i ) is the set of image regions with the largest anomaly score in χ(x i ), χ(x i ) is the region derived from the image x i , x ij is the j-th feature of the i-th image, ψ is the anomaly score mapping, dev(x i , θ) is the loss value, μ ref is the reference score based on the prior anomaly score, σ ref is the standard deviation based on the prior anomaly score, L is the number of feature vectors in each image, y i is the true label of the image, and a is the confidence interval parameter of the standard score.

[0024] Compared with the prior art, the present invention has at least the following advantages or beneficial effects:

[0025] (1) The present invention provides a network anomaly detection method based on SDN and deep learning framework. By establishing a network anomaly detection model and cooperating with the SDN controller to identify abnormal traffic data, the efficiency of anomaly detection can be improved and the false alarm rate can be reduced.

[0026] (2) The present invention uses a model that combines software-defined network and deep learning methods to address the anomaly detection of real-time data streams, enhancing the flexibility and intelligence of the anomaly detection system and significantly improving the accuracy of data stream anomaly detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0028] Figure 1 It is a step diagram of a network anomaly detection method based on SDN and deep learning framework of the present invention;

[0029] Figure 2 It is a schematic structural block diagram of a knowledge definition network architecture;

[0030] Figure 3 It is a real-time anomaly detection architecture diagram of an edge cluster network based on SDN and CNN;

[0031] Figure 4 It is a structure diagram of a network anomaly detection model;

[0032] Figure 5 It is a working principle diagram of an SDN controller;

[0033] Figure 6 It is a schematic diagram of the implementation principle of abnormal traffic detection. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them. Usually, the components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0035] Accordingly, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application, but merely represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts fall within the scope of protection of the present application.

[0036] The following will describe in detail some embodiments of the present application in conjunction with the accompanying drawings. Without conflict, the various embodiments and the various features in the embodiments below can be combined with each other.

[0037] Embodiment 1

[0038] Please refer to Figure 1 , Figure 1 which shows a step diagram of a network anomaly detection method based on SDN and deep learning framework provided by an embodiment of the present application.

[0039] The present application provides a network anomaly detection method based on SDN and deep learning framework, which includes the following steps:

[0040] S1. Configure the data of the network anomaly detection system to establish a network anomaly detection model, and receive data packets in the network through the SDN controller;

[0041] S2. Extract features from the data packets to obtain feature data, and preprocess the feature data;

[0042] S3. Transmit the preprocessed feature data to the network anomaly detection model for data identification to determine whether there is abnormal traffic in the network. If so, mitigate the threat by reconfiguring the recovery strategy. If not, directly output the detection result.

[0043] It should be noted that in step S1, the SDN controller is a software-defined network controller in the control panel of the Knowledge-Defined Network (KDN) architecture. The KDN architecture includes a decision panel, a control panel, and a data panel. By separating the control plane and the data plane, the data plane can be centrally controlled and managed by the SDN controller and divided into three layers.

[0044] Such as Figure 2The figure shows a schematic structural block diagram of a knowledge definition network architecture. The decision panel includes monitoring abnormal data through decision management and feature extraction through machine learning. It mainly provides an application layer, which mainly refers to the upper layer of the SDN controller application, such as other application programs like network management and network security. Developers can use these application programs through a specific SDN application programming interface (API); the control panel provides a control layer, which mainly refers to the control software of the SDN controller. Its function is to control the status and topology information of the entire network, provide control and management on the data plane, and enable the network to quickly adapt to changes; the data panel includes network traffic and network devices, which mainly provides a data layer, which mainly refers to devices such as SDN network ports, switches, routers, and gateways. Its function is to establish communication between network nodes through interconnection at the data level.

[0045] Thus, by collecting current traffic data and passing the data to the SDN controller for feature extraction, the extracted features are passed to the neural network model after preprocessing, and then prediction and recognition results and mitigation strategies are given. Its framework is as Figure 3 shown. By using multiple virtual machines, the sender client sends data to the receiver edge service cluster through the edge cluster network. The CNN server and SDN controller in the edge cluster network predict the availability of data in real time, and then send different types of data according to corresponding strategies, further improving security and availability.

[0046] As a preferred implementation manner, in step S2, the feature data includes source IP, source port, destination IP, destination port, protocol, timestamp, total number of forward packets, minimum packet length, maximum packet length, average packet length, FIN flag, SYN flag, RST flag, PUSH flag, ACK flag, and URG flag.

[0047] As a preferred implementation manner, in step S2, the process of preprocessing the feature data includes: deleting feature rows, converting text to numerical values, deleting rows where missing values are located, removing label columns, normalizing the feature data, and performing one-hot encoding on the label columns.

[0048] As a preferred implementation manner, in step S3, the recovery strategies include: blocking abnormal traffic, redirecting abnormal traffic, and discarding abnormal traffic.

[0049] As a preferred implementation manner, the network anomaly detection model includes an anomaly scoring network and a reference score generator, as Figure 4The following is the structural diagram of the network anomaly detection model; in the training stage, the present application uses an anomaly scoring network to generate anomaly scores for each region of the input image, while the reference score generator generates reference scores based on normal samples, and the anomaly scoring network is optimized by minimizing the loss function; in the testing stage, the anomaly scoring network generates anomaly scores for the test samples, and the test samples exceeding the reference score distribution are identified as abnormal samples; then, an improved ResNet network is used, combining ResNet and ECA modules as the feature representation learner, so as to enhance the multi-scale representation ability and feature extraction ability;

[0050] Among them, the expression of the anomaly scoring network includes:

[0051]

[0052]

[0053] In the formula, f ij is the feature vector, is the feature representation learner, x i is the input image, x ij is the j-th feature of the i-th image, θ f is the weight matrix, χ(x i ) is the region derived from the image x i , Ω is the intermediate representation space, ξ is the anomaly scoring function, θ s is the weight parameter corresponding to the L elements of the feature vector, L is the number of feature vectors in each image, f ijk is the k-th element of the feature vector, k is the number of normal data, is the weight parameter corresponding to the k-th element of the feature vector, is the bias, ψ is the anomaly score mapping, and θ is the weight parameter in the neural network corresponding to the anomaly score mapping function;

[0054] The expression of the reference score generator includes:

[0055]

[0056] In the formula, μ is the average value of all randomly sampled scores, n i is the i-th anomaly score.

[0057] It should be noted that the anomaly scores obtained by the anomaly scoring network need to be optimized. Therefore, the present application uses a prior-driven method to generate reference scores to guide the optimization of the anomaly scores; since the anomaly scores in most datasets conform to the Gaussian distribution, the present application uses Gaussian prior-based reference scores.

[0058] As a preferred embodiment, the training process of the network anomaly detection model includes:

[0059]

[0060] L(x i , μ ref , σ ref , θ) = (1 - y i ) | dev(x i , θ) | + y i max(0, a - dev(x i , θ))

[0061] In the formula, ψ K is the top K anomaly regions of each image, K is the proportion of the region with the largest anomaly score in the image, x i is the input image, θ is the weight parameter in the neural network corresponding to the anomaly score mapping function, M(x i ) is the set of image regions with the largest anomaly score in χ(x i ), χ(x i ) is the region derived from the image x i , x ij is the j-th feature of the i-th image, ψ is the anomaly score mapping, dev(x i , θ) is the loss value, μ ref is the reference score based on the prior anomaly score, σ ref is the standard deviation based on the prior anomaly score, L is the number of feature vectors in each image, y i is the true label of the image, and a is the confidence interval parameter of the standard score.

[0062] Example 2

[0063] In this example, a Ubuntu 16.04 virtual machine is started on a computer, the network service is enabled in multiple VMware virtual machines, and a simple network is set up for testing. Please refer to Figure 5 , which shows the working principle of SDN. In the figure, s1, s2, and s3 represent switches.

[0064] In this example, this application uses the CICIDS2017 dataset to complete the experiment. This dataset is an intrusion detection and defense dataset opened by the Canadian Institute of Cybersecurity in 2017 and is used to train and test the CNN model. The working principle is as follows:

[0065] (1) This application identifies and generates data streams according to the CICIDS2017 data marking method to obtain true and reliable tags and complete system settings. Then, the SDN controller receives data packets in the network. When the controller receives an unknown data packet through the OpenFlow protocol, the data packet contains complete information about the data packet. After receiving the data packet, the controller will parse the data packet layer by layer and extract the data packet information of each layer. In this way, a flow table is created for the data packet based on the extracted information. Then, through the OpenFlow protocol, the packet-out is passed to the switch. The packet-out contains the interface for forwarding the data packet. After that, the data packet submitted by the switch is processed. First, multiple virtual machines and the RYU controller are started. At this time, a data packet is sent, and the switch forwards the data packet to the controller. The time window method is used to collect network traffic data. For common attacks, data is collected once every 2-second time window; for slow attacks, data is collected once every 5-second time window.

[0066] (2) After obtaining the data packet in the controller, corresponding data can be extracted according to the features. The features used in this embodiment include: source IP, source port, destination IP, destination port, protocol, timestamp, total number of forward packets, minimum packet length, maximum packet length, average packet length, FIN flag, SYN flag, RST flag, PUSH flag, ACK flag, and URG flag. According to the hierarchical display, the source IP, destination IP, and the protocol used by the upper layer are extracted at the IP layer. The source port, destination port, fin, syn, rest, push, ack, urg, and other flag information are extracted at the TCP layer. Only the source port and destination port are extracted at the UDP layer. Finally, the packet size, as well as the maximum, minimum, and average sizes of the packet, are calculated.

[0067] (3) The SDN controller transmits the data packet to the network anomaly detection model. After data preprocessing, the network anomaly detection model is used for training and modeling to identify abnormal traffic. If it is normal traffic, the data packet is directly transmitted. If abnormal traffic is identified, three methods, namely blocking, redirecting, and discarding, are used to mitigate the abnormal traffic attack. The flow chart is as Figure 6 shown;

[0068] (4) Abnormal traffic may disrupt the normal operation of the network. The recovery strategy of this application focuses on managing and reconfiguring the detection and recovery mechanisms that are autonomous components in the network. Therefore, in this embodiment, the process of locating abnormal traffic and affected devices is implemented through the system, and these threats are mitigated through reconfiguration strategies. The main operations of implementing the strategy include forwarding, discarding, or modifying data packet fields. Table 1 lists the implemented strategies:

[0069] Table 1 Processing Strategies and Measures Taken

[0070]

[0071] The blocking strategy can cut off the incoming traffic from malicious hosts (port scanning attacks) and block the communication between specific hosts and specific services at the target IP address; the dropping strategy means dropping protocol data units at specific addresses; the redirecting strategy is to distribute the network traffic load generated by abnormal attacks to multiple servers to avoid degrading service performance.

[0072] It can be understood that the structures shown in the figures are only schematic. A network anomaly detection method based on the SDN and deep learning framework may also include more or fewer components than those shown in the figures, or have a different configuration from that shown in the figures. Each component shown in the figures can be implemented using hardware, software, or a combination thereof.

[0073] In the embodiments provided in the present application, it should be understood that the disclosed method can also be implemented in other ways. The above-described embodiments are merely illustrative. For example, the flowcharts or block diagrams in the accompanying drawings show the possible architectures, functions, and operations of the methods and computer program products according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from that marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0074] The above are only the preferred embodiments of the present application and are not used to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0075] It will be apparent to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above, and that the present application can be implemented in other specific forms without departing from the spirit or essential features of the present application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present application. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A network anomaly detection method based on SDN and deep learning framework, characterized in that: The following steps are involved: S1, configure network anomaly detection system data to establish a network anomaly detection model, and receive data packets in the network through the SDN controller; S2, extracting features from the data packet to obtain feature data, and preprocessing the feature data; S3. The preprocessed feature data is transmitted to the network anomaly detection model for data identification to determine whether the network has abnormal traffic. If so, the threat is mitigated by reconfiguring the recovery strategy. If not, the detection result is directly output.

2. A network anomaly detection method based on SDN and deep learning framework as claimed in claim 1, characterized in that: In step S2, the characteristic data includes source IP, source port, target IP, target port, protocol, timestamp, total forward packets, minimum packet length, maximum packet length, average packet length, FIN flag, SYN flag, RST flag, PUSH flag, ACK flag and URG flag.

3. A network anomaly detection method based on SDN and deep learning framework as claimed in claim 1, characterized in that: In step S2, the process of preprocessing the feature data includes: deleting feature rows, converting text into numerical values, deleting rows with missing values, removing label columns, normalizing the feature data, and performing one-hot encoding on the label columns.

4. A network anomaly detection method based on SDN and deep learning framework as claimed in claim 1, characterized in that: In step S3, the recovery strategy includes: blocking abnormal traffic, redirecting abnormal traffic, and discarding abnormal traffic.

5. A network anomaly detection method based on SDN and deep learning framework as claimed in claim 1, characterized in that: The network anomaly detection model includes an anomaly scoring network and a reference score generator; Among them, the expression of the anomaly scoring network includes: x ij ∈χ(x i ),f ij ∈Ω In the formula, f ij is the feature vector, is the feature representation learner, x i is the input image, x ij is the jth feature of the i-th image, θ f is the weight matrix, χ(x i ) is the image x i The derived region, Ω is the intermediate representation space, ξ is the anomaly scoring function, θ s is the weight parameter corresponding to L feature vector elements, L is the number of feature vectors in each image, f ijk is the kth element of the feature vector, k is the number of normal data, is the weight parameter corresponding to the kth element of the feature vector, is the deviation, ψ is the anomaly score mapping, and θ is the weight parameter in the neural network corresponding to the anomaly score mapping function; The expressions for the reference score generator include: Where μ is the average of all random sampling scores, n i is the ith anomaly score.

6. A network anomaly detection method based on SDN and deep learning framework as claimed in claim 5, characterized in that: The training process of the network anomaly detection model includes: L(x i ,m ref ,s ref ,θ)=(1-y i )|dev(x i ,θ)|+y i max(0, a-dev(x i ,i)) In the formula, ψ K are the first K abnormal regions of each image, K is the proportion of the region with the largest abnormal score in the image, x i is the input image, θ is the weight parameter in the neural network corresponding to the anomaly score mapping function, M(x i ) is χ(x i ) is the set of image regions with the largest anomaly score, x(x i ) is the image x i The derived region, x ij is the jth feature of the i-th image, ψ is the anomaly score mapping, dev(x i ,θ) is the loss value, μ ref is the reference score based on the prior anomaly score, σ ref is the standard deviation of the prior anomaly score, L is the number of feature vectors in each image, and y i is the true label of the image, and a is the confidence interval parameter of the standard score.

Images