Intrusion prevention system, operation and maintenance management equipment and traffic anomaly detection method thereof

Abstract

The invention discloses an intrusion prevention system, operation and maintenance management equipment and a traffic anomaly detection method thereof. The traffic anomaly detection method comprises the following steps: acquiring to-be-identified traffic data; the traffic anomaly detection model trained by the method is used to identify the traffic data to be identified, and the abnormal event corresponding to the traffic data to be identified is identified, and the abnormal event comprises a network attack.

Description

Technical Field

[0001] The present invention relates to the field of operation and maintenance management technology, and in particular to an intrusion prevention system, operation and maintenance management equipment and a flow anomaly detection method thereof. Background Art

[0002] With the rapid development of Internet technology, the use of network and traffic is becoming more and more popular. At the same time, there are more and more means of network attacks, and new attack means and technologies emerge in an endless stream, making it impossible for general intrusion prevention systems to effectively identify related network attacks. However, any network attack is transmitted through the network, which will generate traffic data. However, it is time-consuming and labor-intensive to manually check these traffic data, and there are also many unnecessary traffic data, which increases the difficulty of manual investigation.

[0003] The emergence of machine learning and deep learning has brought many innovations to traffic data anomaly detection, which can not only cope with complex and changing network environments, but also greatly improve the timeliness and accuracy of detection. However, in the field of machine learning, such as unsupervised learning methods decision trees and support vector machines, supervised learning cluster analysis and principal component analysis; in the field of deep learning, such as convolutional neural networks, recurrent neural networks, and generative adversarial networks, the accuracy of traffic data anomaly detection is still average, and some context-related traffic anomalies cannot be identified.

[0004] Therefore, how to improve the accuracy of traffic data anomaly detection and effectively predict network attacks is the technical problem to be solved by the present invention. Summary of the invention

[0005] The purpose of the present invention is to provide an intrusion prevention system, operation and maintenance management equipment and a flow anomaly detection method thereof. The flow anomaly detection method constructs a flow anomaly detection model by analyzing and training a large amount of flow data, and then uses the flow anomaly detection model to perform flow anomaly detection, which not only has high accuracy, but also can effectively predict network attacks.

[0006] According to one aspect of the present invention, at least one embodiment provides a method for training a traffic anomaly detection model, comprising: obtaining traffic data, wherein the traffic data is reduced into at least one data point; determining at least one truncation distance of the at least one data point, and obtaining at least one feature of the at least one truncation distance; concatenating the at least one feature to form feature data, and using the feature data to train a tree model to obtain a traffic anomaly detection model.

[0007] According to another aspect of the present invention, at least one embodiment further provides a traffic anomaly detection method, comprising: obtaining traffic data to be identified; using the traffic anomaly detection model trained by the present invention to identify the traffic data to be identified, and identifying abnormal events corresponding to the traffic data to be identified, wherein the abnormal events include network attacks.

[0008] According to another aspect of the present invention, at least one embodiment also provides an operation and maintenance management device, including: a processor, suitable for implementing various instructions; and a memory, suitable for storing multiple instructions, wherein the instructions are suitable for being loaded and executed by the processor: the method for training the traffic anomaly detection model mentioned above in the present invention, and / or the traffic anomaly detection method mentioned above in the present invention.

[0009] According to another aspect of the present invention, at least one embodiment further provides an intrusion prevention system, including: the above-mentioned operation and maintenance management device of the present invention.

[0010] According to another aspect of the present invention, at least one embodiment further provides a computer-readable non-volatile storage medium storing computer program instructions, and when the computer executes the program instructions, it executes: the method for training a traffic anomaly detection model of the present invention, and / or the traffic anomaly detection method of the present invention.

[0011] Through the above-mentioned implementation mode of the present invention, a traffic anomaly detection model is constructed by analyzing and training a large amount of traffic data, and then the traffic anomaly detection model is used to perform traffic anomaly detection, which effectively solves the problem that the traditional density clustering algorithm "is greatly affected by the cutoff distance when performing traffic anomaly detection and does not consider the local characteristics of the data". Not only does it have a high accuracy rate in traffic anomaly detection, it can also effectively predict network attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0013] Figure 1 is a schematic diagram of an intrusion prevention system according to an embodiment of the present invention;

[0014] Figure 2 is a schematic diagram of an operation and maintenance management device according to an embodiment of the present invention;

[0015] Figure 3 is a flow chart of a method for detecting abnormal flow according to an embodiment of the present invention;

[0016] Figure 4 is a flow chart of a method for training a traffic anomaly detection model according to an embodiment of the present invention;

[0017] Figure 5 is an example diagram of a method for training a traffic anomaly detection model according to an embodiment of the present invention;

[0018] Figure 6 It is a rendering of the flow anomaly detection method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] It should be noted that the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or apparatus.

[0021] The emergence of machine learning and deep learning has brought many innovations to traffic data anomaly detection. Not only are there more ways to process huge amounts of traffic data, but the prediction time for smaller amounts of traffic data can be reduced to milliseconds. However, in the application of machine learning and deep learning in traffic data anomaly detection, the model performance and efficiency need to be improved.

[0022] For example, the clustering-based traffic anomaly detection method is actually an outlier detection algorithm. In the actual traffic anomaly detection clustering analysis algorithm, outliers are usually removed as interference data, which often leads to low clustering analysis in traffic anomaly detection. In clustering-based traffic data anomaly detection, clustering is usually performed first and then detection. The detection results of outliers are greatly affected by clustering. The quality of clustering results will directly affect the quality of anomaly detection results, and different numbers of clusters will also lead to large differences in the identified anomalies. The commonly used classic k-means clustering algorithm, after its own algorithm completes clustering, needs to further use distance-based or threshold-based methods to detect traffic data outliers. Different cluster center points or numbers of centers will affect the subsequent anomaly detection. Some density-based clustering algorithms such as DBSCAN and density peak clustering algorithms can perform anomaly detection and clustering simultaneously or separately, greatly reducing the dependence of anomaly detection on clustering results.

[0023] For example, DBSCAN is a typical representative algorithm based on density clustering methods. Its core idea is to calculate the density of each traffic data point, find out the points with higher density, and then gradually merge these high-density points to form clusters. Since it relies on continuously connecting high-density points in the field to discover clusters, it only needs to define the neighborhood size and density threshold, so it has the advantage of identifying clusters of any shape and size in noise. But even so, since DBSCAN uses a global density threshold, it is difficult to distinguish clusters of different densities.

[0024] For example, the core idea of ​​density peak clustering is to assume that each cluster has a point with the highest density as the cluster center. Each cluster center attracts the surrounding points with lower density. These points are clustered into a cluster, and the centers of different clusters are relatively far away and will not interfere with each other. Density peak clustering uses a visualization method to help find different cluster centers. By calculating the density of each point and the distance from each point to its nearest point with higher density, a two-dimensional decision diagram is generated. The points in the upper right corner of the decision diagram represent the centers of different clusters. When applied to traffic anomaly detection, the points in the upper left corner of the decision diagram mean points that are far away from other points with higher density and have very low density. These points are usually abnormal points. In other words, the traffic anomaly detection method based on the density peak clustering algorithm can obtain non-spherical clustering results to cluster different types of traffic data distribution and reduce the impact of clustering results on traffic anomaly detection results, but it uses density-distance decision diagrams to cluster centers, which requires manual selection and brings a greater subjective influence to the clustering results.

[0025] It can be seen that current machine learning and deep learning have more or less some problems when detecting traffic data anomalies, resulting in the current accuracy of traffic anomaly detection being average and unable to identify some context-related traffic anomalies.

[0026] Driven by the above-mentioned technical status, considering that the density peak clustering algorithm can obtain non-spherical clustering results, it can cluster different types of data distributions well; the outlier detection method based on the density peak clustering algorithm can reduce the influence of the clustering results on the anomaly detection results; at the same time, its algorithm complexity is also lower than that of the general k-means algorithm. The present invention fully considers the above-mentioned advantages and improves the existing density peak clustering algorithm, so that at least one embodiment of the present invention provides an intrusion prevention system, such as Figure 1As shown, it includes: an operation and maintenance management device 100 and a computer network 200. There is a communication link between the operation and maintenance management device 100 and the computer network 200, and this link can be established based on Ethernet, Wi-Fi or other network protocols. The operation and maintenance management device 100 operates the computer network 200 through corresponding instructions, so that the network traffic / performance data of the devices in the computer network 200 can be collected and the failure of the devices in the computer network 200 can be checked.

[0027] The operation and maintenance management device 100, such as Figure 2 As shown, it includes: a processor 202, suitable for implementing various instructions; and a memory 204, suitable for storing multiple instructions, the instructions are suitable for being loaded and executed by the processor 202: the method for training the traffic anomaly detection model developed by the present invention, and / or, the traffic anomaly detection method developed by the present invention (this article will be elaborated in detail later, so it will not be repeated here). The traffic anomaly detection model uses the density peak clustering algorithm for model training. The general idea is: by calculating the density and distance of the sample points to generate a decision diagram for judgment, which requires two parameters ρ i and δ i , respectively representing the density of each point, the minimum distance between the density clustering result of each data point and its density center, and obtaining the ρ of the data sample point i and δ i After that, we can make the (ρ i , δ i ) decision diagram. Similarly, the decision diagram can also be used to assist in identifying the cluster center. Once the cluster center is determined, labels are assigned from large to small according to the density of the data sample points, and each data point is assigned to the cluster to which the data point with a larger density value and the closest distance belongs. In this way, all data points can be clustered into their respective clusters in one step without multiple iterations, which effectively improves the algorithm speed.

[0028] The processor 202 may be any suitable processor, for example, implemented in the form of a central processing unit, a microprocessor, an embedded processor, etc., and may adopt an architecture such as X86, ARM, etc.

[0029] Among them, the memory 204 can be various applicable storage devices, such as non-volatile storage devices, including but not limited to magnetic storage devices, semiconductor storage devices, optical storage devices, etc., and can be arranged as a single storage device, a storage device array or a distributed storage device, and the embodiments of the present invention do not limit these. Optionally, corresponding to the memory 204, at least one embodiment of the present invention also provides a computer-readable non-volatile storage medium that stores computer program instructions. When the computer executes the program instructions, it executes: the method for training the flow anomaly detection model developed by the present invention, and / or, the flow anomaly detection method developed by the present invention (this article will be elaborated in detail later, so it will not be repeated here).

[0030] Those skilled in the art will appreciate that the above structure of the operation and maintenance management device 100 is only for illustration and does not limit the structure of the operation and maintenance management device 100. Figure 2 The transmission device is used to receive or send data via a network. In one embodiment, the transmission device is a radio frequency (RF) module, which is used to communicate with the computer network 200 wirelessly.

[0031] The computer network 200 is composed of multiple computer devices and communication links. These computer devices include servers, clients (such as personal computers, smart phones), routers, switches, etc., which are connected together by wired (such as Ethernet cables) or wirelessly, with the purpose of realizing resource sharing and information exchange. The wireless method includes but is not limited to: wide area network, metropolitan area network, local area network or mobile data network. Typically, the mobile data network includes but is not limited to: global mobile communication (GSM) network, code division multiple access (CDMA) network, wideband code division multiple access (WCDMA) network, long-term evolution (LTE) communication network, WIFI network, ZigBee network, network based on Bluetooth technology, etc. Different types of communication networks may be operated by different operators. The type of communication network does not constitute a limitation on the embodiments of the present invention.

[0032] Among them, network traffic will be generated when data is transmitted between computer devices. Generally speaking, traffic anomalies may be a signal of network attacks, so the operation and maintenance management device 100 of the present invention can timely detect such abnormal traffic and take corresponding defense measures by monitoring and analyzing the network traffic of the computer network 200.

[0033] Through the above-mentioned implementation mode of the present invention, the network traffic of the computer network 200 is monitored and analyzed using an operation and maintenance management device. The operation and maintenance management device is equipped with a trained traffic anomaly detection model, which not only has a high accuracy rate in traffic anomaly detection, but can also effectively predict network attacks, and then take corresponding defense measures to solve the problem of inaccurate network status caused by abnormal cascading changes.

[0034] In the above-mentioned hardware and software environment, at least one embodiment of the present invention provides a flow anomaly detection method, which can also be called a flow anomaly detection method based on a density clustering algorithm, and is a network flow anomaly detection technology, which is applicable to the operation and maintenance management device 100 and can be based on density clustering technology: by using multiple truncation distances to calculate the clustering results of the flow data at different intercepts, to achieve rapid flow anomaly detection. Figure 3The flowchart of the traffic anomaly detection method shown in the accompanying drawings should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here. The method may include the following steps:

[0035] Step S301, obtaining traffic data to be identified;

[0036] Step S303, using the traffic anomaly detection model trained by the present invention (which will be described in detail later in this article and will not be described here in detail), the traffic data to be identified is identified, and abnormal events corresponding to the traffic data to be identified are identified, wherein the abnormal events include network attacks. The network attacks include BENIGN, Bot, DDoS, DoS GoldenEye, DoS Hulk, DoSSlowhttptest, DoS slowloris, and PortScan.

[0037] Through the above-mentioned implementation mode of the present invention, the network traffic (such as network transmission traffic) of the computer network is monitored and analyzed based on the trained traffic anomaly detection model, which not only has a high accuracy rate of traffic anomaly detection, but also can effectively predict network attacks.

[0038] Under the above-mentioned software and hardware environment, at least one embodiment of the present invention provides a method for training a traffic anomaly detection model. In view of the shortcomings of machine learning and deep learning applications in the process of traffic data anomaly detection, the traditional density peak clustering technology is improved to create a traffic anomaly detection model based on a density clustering algorithm. The traffic anomaly detection model takes into account the advantages and disadvantages of various clustering algorithms, improves the clustering effect by means of a dynamic intercept, and effectively reduces the influence of the cutoff distance on the traditional density peak clustering algorithm; reduces the dimension of the original data, focuses on several features, strengthens the processing of local features, and enhances the fitting effect of the traffic anomaly detection model for local features. Figure 4 As shown, the method for training a traffic anomaly detection model may include the following steps:

[0039] Step S402, acquiring traffic data, wherein the traffic data is reduced into at least one data point;

[0040] Step S404, determining at least one cutoff distance of at least one data point, and obtaining at least one feature of the at least one cutoff distance;

[0041] Step S406: concatenate at least one feature to form feature data, and use the feature data to train a tree model to obtain a traffic anomaly detection model.

[0042] Through the above-mentioned implementation mode of the present invention, a traffic anomaly detection model is constructed by analyzing and training a large amount of traffic data, and then the traffic anomaly detection model is used to perform traffic anomaly detection, which effectively solves the problem that the traditional density clustering algorithm "is greatly affected by the cutoff distance when performing traffic anomaly detection and does not consider the local characteristics of the data". Not only does it have a high accuracy rate in traffic anomaly detection, it can also effectively predict network attacks.

[0043] In step S402, traffic data is obtained. For example, traffic data with annotated attack types is obtained, wherein the traffic data includes at least one traffic flow; the at least one traffic flow is sorted by feature importance, and the at least one traffic flow after the feature importance sorting is reduced into at least one data point.

[0044] That is to say, the present invention performs feature importance analysis on the original traffic data (referred to as "original data") with labeled attack types, sorts the original data by importance according to the result of the feature importance analysis, and then reduces the dimension of the original data to generate the reduced-dimensional original data, which can be at least one data point.

[0045] In step S404, at least one cutoff distance of at least one data point is determined, and at least one feature of the at least one cutoff distance is obtained. There are M data points in total, numbered 1, 2, ..., m. For example, determining at least one cutoff distance of at least one data point may include: calculating the distance from each data point to the remaining data points to obtain a distance set for each data point, wherein the i-th distance set of the i-th data point is distance[d1, d2, ..., d m ] i , d m Represents the distance between the i-th data point and the m-th data point, d1≤d2≤…≤d m ; Calculate the truncation distance set of each data point through N truncation distances, where the N truncation distances are numbered as dc1, dc2, ..., dc n， The j-th cutoff distance set of the j-th data point is dc[dc1, dc2, ..., dc n ] j , dc n represents the cutoff distance corresponding to the jth data point under the action of the nth t, where t is a set parameter. For example, obtaining at least one feature of at least one cutoff distance may include: n ] jDetermine n cluster centers, calculate the density center of the j-th data point based on the n cluster centers, and then obtain the density center of each data point; select the first n density centers of the density center of each data point, and density cluster each data point based on the first n density centers to obtain a density clustering result; use the density clustering result and the density distance as the features of each predicted data point, thereby obtaining at least one feature, wherein the density distance is the distance between the density clustering result of each data point and its density center.

[0046] For example, Figure 5 , 6 As shown, the present invention: first, the distance between each data point and other data points is calculated for the original data with reduced dimension, and a distance set distance[d1, d2, ..., d m ] i , and sort the distances between the two data points from small to large, and get the sequence d1≤d2≤…≤d m ; Secondly, calculate the cutoff distance dc, set the parameter t, take the (number of distances between two data points × t%) distance as the cutoff distance dc, assign different values ​​to the parameter t, calculate the cutoff distance corresponding to each parameter t, and obtain multiple cutoff distance sets dc [dc1, dc2, ..., dc n ], build a dynamic cutoff distance list; again, determine the cluster center, and for the original data with reduced dimension, calculate the distance of each data point in the radius equal to the cutoff distance dc[dc1, dc2, ..., dc n ] is included, and on this basis, the density value when the point is equal to the cutoff distance dc is calculated, each group of cluster centers is calculated, and the density value results are sorted from large to small; again, the density center of each data point is calculated by determining the cluster center, the first n density centers among the density center points are selected, and the results are gradually merged into n density centers, and the n density centers are the final clustering results; finally, the data points are density clustered to obtain their density clustering results and the distances from their density centers, and the distances from their density centers and the density clustering results are used as the predicted feature data, which are spliced ​​to obtain the feature data corresponding to each cutoff distance dc.

[0047] Step S406, at least one feature is concatenated to form feature data, and a tree model is trained using the feature data to obtain a traffic anomaly detection model. The tree model is XGBOOST (eXtreme Gradient Boosting), which is an efficient gradient boosting algorithm framework. For example, the final feature data obtained by concatenating at least one feature is used to train XGBOOST to obtain a traffic anomaly detection model, and then the trained traffic anomaly detection model is used to complete traffic anomaly detection based on a density clustering algorithm.

[0048] Through the above implementation of the present invention, the clustering results of the traffic data at different intercepts are calculated through multiple truncation distances to achieve rapid traffic anomaly detection. Specifically, the original data is subjected to feature importance analysis and dimensionality reduction operations, and on this basis, multiple truncation distances are used to perform density clustering algorithms for clustering; the traffic data is subjected to a density clustering algorithm to obtain multiple minimum distances and clustering results; the feature data corresponding to each truncation distance dc is obtained, all feature data results are spliced ​​to obtain the final feature data, which is input into the XGBOOST model for training, and finally the traffic anomaly detection based on the density clustering algorithm is completed, which not only has a high accuracy rate of traffic anomaly detection, but also can effectively predict network attacks.

[0049] It should also be noted that although the fast density peak clustering algorithm has better results than distance-based clustering, the accuracy of its clustering results is too dependent on the selection of the cutoff distance. The algorithm uses a global cutoff distance when calculating density, ignoring the local characteristics of data distribution, resulting in unsatisfactory clustering results on some data sets, which in turn affects the accuracy of anomaly detection. For example, in the Flame data set, even if different d_c values ​​are used for density peak clustering, the abnormal outliers cannot be identified.

[0050] The improved distance-based density peak clustering algorithm of the present invention effectively solves the problem that the traditional density clustering algorithm is "greatly affected by the cutoff distance when performing traffic anomaly detection and does not consider the local characteristics of the data". It not only has a high accuracy rate in traffic anomaly detection, but can also effectively predict network attacks.

[0051] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for training a traffic anomaly detection model, characterized in that: include: Acquiring traffic data, wherein the traffic data is reduced in dimension into at least one data point; determining at least one cutoff distance of the at least one data point and obtaining at least one feature of the at least one cutoff distance; The at least one feature is concatenated to form feature data, and the feature data is used to train a tree model to obtain a traffic anomaly detection model.

2. The method according to claim 1, characterized in that Obtaining traffic data includes: Acquire flow data with marked attack types, wherein the flow data includes at least one flow; The at least one flow is sorted by feature importance, and the at least one flow after the feature importance sorting is reduced into at least one data point.

3. The method according to claim 1, characterized in that There are M data points in total, numbered 1, 2, ..., m, and determining at least one cutoff distance of the at least one data point includes: Calculate the distance from each data point to the remaining data points to obtain the distance set of each data point, where the i-th distance set of the i-th data point is distance[d1, d2, ..., d m ] i , d m Represents the distance between the i-th data point and the m-th data point, d1≤d2≤…≤d m ; The truncation distance set of each data point is calculated by N truncation distances, wherein the N truncation distances are numbered as dc1, dc2, ..., dc n， The j-th cutoff distance set of the j-th data point is dc[dc1, dc2, ..., dc n ] j , dc n It represents the cutoff distance corresponding to the jth data point under the action of the nth t, where t is the set parameter.

4. The method according to claim 3, characterized in that The at least one feature of obtaining the at least one cutoff distance comprises: By dc[dc1, dc2, ..., dc n ] j Determine n cluster centers, calculate the density center of the j-th data point based on the n cluster centers, and then obtain the density center of each data point; Selecting the first n density centers of the density center to which each data point belongs, and performing density clustering on each data point based on the first n density centers to obtain a density clustering result; The density clustering result and the density distance are used as features of each predicted data point to obtain at least one feature, wherein the density distance is the distance between the density clustering result of each data point and its density center.

5. The method according to claim 1, characterized in that The tree model is XGBOOST, and using the feature data to train the tree model to obtain a traffic anomaly detection model includes: The feature data is used to train XGBOOST to obtain a traffic anomaly detection model.

6. A method for detecting flow anomaly, characterized in that: include: Obtaining traffic data to be identified; The traffic data to be identified is identified using the traffic anomaly detection model as described in any one of claims 1 to 5, and abnormal events corresponding to the traffic data to be identified are identified, wherein the abnormal events include network attacks.

7. The flow anomaly detection method according to claim 6, characterized in that: The traffic data to be identified is the network transmission traffic of the computer network, and obtaining the traffic data to be identified includes: Get network transmission flow.

8. An operation and maintenance management device, characterized in that: include: A processor, suitable for implementing each instruction; and a memory, suitable for storing multiple instructions, wherein the instructions are suitable for being loaded and executed by the processor: a method for training a traffic anomaly detection model as described in any one of claims 1-5, and / or a traffic anomaly detection method as described in any one of claims 6-7.

9. An intrusion prevention system, characterized in that: include: The operation and maintenance management device as claimed in claim 8.

10. A computer-readable non-volatile storage medium storing computer program instructions, which, when executed by a computer, executes: the method for training a traffic anomaly detection model as described in any one of claims 1-5, and / or the traffic anomaly detection method as described in any one of claims 6-7.

Images