Intrusion detection method and system

Abstract

The present invention discloses an intrusion detection method and system. By collecting network traffic data in groups with labels, the problem of inaccurate detection caused by the traditional method of collecting and analyzing single network traffic data is avoided. The distribution curve of each network traffic data feature is fitted into a behavior distribution feature curve before the type of attack behavior is detected and identified, and the distribution characteristics of the multi-dimensional network traffic data features are reduced to one-dimensional behavior distribution characteristics, which is conducive to better combining the network traffic data features with machine learning and improving the accuracy of the detection results. The behavior distribution feature values ​​of the behavior distribution feature curve are used for machine learning training, and only a basic neural network model is needed to complete the training task, avoiding the excessive reliance of traditional methods on machine learning neural networks. This enables the intrusion detection method provided by the present invention to complete intrusion detection more simply, efficiently and accurately.

Description

Technical Field

[0001] The present invention relates to the technical field of network security intrusion detection, and in particular to an intrusion detection method and system. Background Art

[0002] According to the data used and the detection strategy, intrusion detection can be divided into network-based intrusion detection, host-based intrusion detection system, feature-based intrusion detection, anomaly-based intrusion detection, etc. These methods are based on network traffic data, host behavior data, and fusion network and host data to identify attacks or abnormal behaviors.

[0003] Existing technologies in the field of intrusion detection rely too much on machine learning models. They pursue better detection performance by building increasingly complex machine learning models or by combining machine learning models. In addition, in order to obtain better detection performance on specific data sets, a large number of intrusion detection methods use feature engineering strategies to select feature indicators that can obtain better detection performance. This makes many intrusion detection methods not well applicable and difficult to deploy and use.

[0004] Existing intrusion detection methods still lack research on network behavior characteristics. The research on network behavior characteristics is not in-depth enough and cannot be effectively combined with machine learning models. Many methods propose some feature studies and then only use threshold information for anomaly detection. At the same time, with the rise of machine learning, many studies have even given up in-depth research on network behavior. Summary of the invention

[0005] In view of this, the present invention provides an intrusion detection method and system, which can simply and efficiently complete intrusion detection.

[0006] The specific technical solution adopted in the present invention is as follows:

[0007] An intrusion detection method, comprising:

[0008] Collect network traffic data in groups and obtain network traffic data characteristics;

[0009] Calculate and obtain the distribution characteristic curve of each network traffic data characteristic respectively;

[0010] According to the distribution characteristic curve of each network traffic data characteristic, fitting to obtain a behavior distribution characteristic curve;

[0011] According to the behavior distribution characteristic curve, the type of the corresponding attack behavior is detected and determined.

[0012] Furthermore, the grouping and collecting of network traffic data includes:

[0013] The network traffic data is collected using the source IP and the destination IP as identification information, and a collection lower limit M and a collection upper limit N are set for each group of the network traffic data, wherein, if the data volume of the current behavior is lower than the collection lower limit M, the current behavior is considered to be an invalid behavior, and if the data volume of the current behavior is higher than the collection upper limit N, the first collection data is deleted, and new collection data is added to maintain the data volume equal to the collection upper limit N, wherein M and N are both integers.

[0014] Furthermore, the network traffic data features include: duration of the behavior, number of packets in both the forward and backward directions, number of bytes in both the forward and backward directions, and response time in both the forward and backward directions.

[0015] Furthermore, the distribution characteristic curves of each network traffic data feature are calculated and obtained respectively, including:

[0016] Calculate the proportional distribution information of each of the network traffic data features to obtain the frequency domain information of each of the network traffic data features, and bring the frequency domain information into the Fourier series real value formula to obtain the distribution characteristic curve of each of the network traffic data features, wherein the distribution characteristic curve is expressed by the formula:

[0017] F d (t) = ∑x n sin(p n *2*π*t)

[0018] Among them, x n is the value of the network traffic data feature, p n For x n The proportional probability of , t is the quantity parameter.

[0019] Further, according to the distribution characteristic curve of each network traffic data characteristic, fitting to obtain a behavior distribution characteristic curve includes:

[0020] The distribution characteristic value of the distribution characteristic curve of each network traffic data feature is input into a convolutional autoencoder, and the convolutional autoencoder is fitted to obtain a behavior distribution characteristic curve.

[0021] Furthermore, the detecting and determining the type of the corresponding attack behavior according to the behavior distribution characteristic curve includes:

[0022] The behavior distribution characteristic values ​​of the behavior distribution characteristic curve are used to perform machine learning training to obtain a behavior detection neural network model, and the behavior detection neural network model is used to detect and determine the type of corresponding attack behavior.

[0023] An intrusion detection system, comprising:

[0024] A data collection module is used to collect network traffic data in groups and obtain network traffic data characteristics;

[0025] A distribution characteristic calculation module is used to calculate and obtain the distribution characteristic curve of each network flow data characteristic;

[0026] A distribution feature fitting module, used for fitting a behavior distribution feature curve according to the distribution feature curve of each network traffic data feature;

[0027] The training detection module is used to detect and determine the type of corresponding attack behavior according to the behavior distribution characteristic curve.

[0028] Furthermore, in the data collection module, the network traffic data is collected using the source IP and the destination IP as identification information, and a collection lower limit M and a collection upper limit N are set for each group of the network traffic data, wherein, if the data volume of the current behavior is lower than the collection lower limit M, the current behavior is considered to be an invalid behavior, and if the data volume of the current behavior is higher than the collection upper limit N, the first collected data is deleted, and new collected data is added to maintain the data volume equal to the collection upper limit N, wherein M and N are both integers.

[0029] Furthermore, in the distribution feature calculation module, the proportional distribution information of each of the network traffic data features is calculated to obtain the frequency domain information of each of the network traffic data features, and the frequency domain information is brought into the Fourier series real value formula to obtain the distribution feature curve of each of the network traffic data features, wherein the distribution feature curve is expressed by the formula:

[0030] F d (t) = ∑x n sin(p n *2*π*t) where x n is the value of the network traffic data feature, p n For x n The proportional probability of , t is the quantity parameter.

[0031] Furthermore, in the distribution feature fitting module, the distribution feature value of the distribution feature curve of each network traffic data feature is input into a convolutional autoencoder, and the convolutional autoencoder is fitted to obtain a behavior distribution feature curve.

[0032] Beneficial effects:

[0033] (1) An intrusion detection method collects network traffic data in groups, thereby avoiding the problem of inaccurate detection caused by traditional methods that collect and analyze single network traffic data; the distribution curve of each network traffic data feature is fitted into a behavior distribution feature curve before the type of attack behavior is detected and identified, and the distribution feature of the multi-dimensional network traffic data feature is reduced to a one-dimensional behavior distribution feature, which is conducive to better combining the network traffic data feature with machine learning and improving the accuracy of the detection result.

[0034] (2) Collecting the network traffic data using the source IP and destination IP as identification information can distinguish behaviors well, detect and judge the type of current behavior more effectively and accurately, and avoid collecting data of other behaviors when collecting current behavior data, which affects the accuracy of the detection results. Setting the collection lower limit and collection upper limit of each group of network traffic data can avoid the waste of resources and reduced efficiency caused by the collection of invalid behaviors.

[0035] (3) If the data volume of the current behavior exceeds the collection upper limit, the first collected data is deleted and new collected data is added to maintain the data volume equal to the collection upper limit. This can maintain the uniformity and accuracy of the data collected for the current behavior and avoid the behavior evolution interfering with the detection results due to unlimited collection of large amounts of data.

[0036] (4) Using the most basic network traffic data features as data calculation indicators can improve the applicability of the intrusion detection method of the present invention.

[0037] (5) The behavior distribution characteristic values ​​of the behavior distribution characteristic curve are used for machine learning training, and only a basic neural network model is needed to complete the training task, thus avoiding the excessive reliance of traditional methods on machine learning neural networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a flow chart of an intrusion detection method according to an embodiment of the present invention;

[0039] Figure 2 is a schematic diagram of the embedded relationship between behavior and network traffic data features according to an embodiment of the present invention;

[0040] Figure 3 is a structural block diagram of an intrusion detection system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The present invention provides an intrusion detection method and system. By collecting network traffic data in groups, the problem of inaccurate detection caused by the traditional method of collecting and analyzing single network traffic data is avoided. The distribution curve of each network traffic data feature is fitted into a behavior distribution feature curve before the type of attack behavior is detected and identified, and the distribution characteristics of the multi-dimensional network traffic data features are reduced to one-dimensional behavior distribution characteristics, which is conducive to better combining the network traffic data features with machine learning and improving the accuracy of the detection results. The behavior distribution feature values ​​of the behavior distribution feature curve are used for machine learning training, and only a basic neural network model is needed to complete the training task, avoiding the excessive reliance of traditional methods on machine learning neural networks. This enables the intrusion detection method provided by the present invention to complete intrusion detection more simply, efficiently and accurately.

[0042] In the prior art, based on the scheme of defining attack pattern signatures, signatures are stored in a signature database. Attack detection is performed by analyzing data patterns and matching these stored signatures. Based on the anomaly detection method of benign behavior, any behavior that deviates from normal behavior is considered abnormal. The main disadvantage of this method is that it is difficult to find the boundary between normal and abnormal contours, resulting in a high false alarm rate of intrusion detection. The statistical-based intrusion detection system establishes a distribution model about normal behavior, then detects low-probability events and marks them as potential anomalies. The statistical methods mentioned so far do not detect data flows, but monitor each data packet and regard the data packet as a fingerprint of the data flow. Or based on monitoring Web servers through a statistical framework, a bivariate beta-autoregressive moving average time series model is proposed for the analysis of proportional data that changes over time. Or based on the data fusion anomaly detection method of the blockchain system. By fusing the high-frequency data clustering features in the blockchain, the behavior in the blockchain is defined, and those that do not meet the definition of normal behavior are abnormal.

[0043] Artificial neural network, random forest, K-nearest neighbor, support vector machine, AdaBoost, and naive Bayes are used to classify and verify CSE-CIC-IDS 2018. Fitness function based on feature similarity and fitness function based on classifier accuracy are combined with adaptive migration algorithm for feature extraction. By monitoring each server in the Internet Connection Sharing (ICS) network, a discriminant model for benign packets and attack packets based on packet interval and length of support vector machine is proposed. Multi-class network intrusion detection model based on convolutional neural network. This method uses data transformation algorithm to expand one-dimensional data padding to two dimensions to realize convolution operation. Network anomaly detection method based on autoencoder, which uses autoencoder to capture nonlinear correlation between features. Therefore, convolutional autoencoder is used to reduce the dimension of data features to improve detection accuracy.

[0044] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0045] The present invention distinguishes between benign and abnormal behaviors by identifying behavioral patterns hidden in the data. Existing studies have attempted to extract behavioral patterns from single traffic data. For example, in a single traffic data, whether features such as the number of packets, duration, and number of stream bytes conform to the value pattern of benign behavior. However, with the development of automated tools, attack behaviors can imitate the value pattern of benign behavior and evade detection. However, automated tools cannot completely cover up the differences between attack behaviors and benign behaviors. Therefore, it is necessary to conduct in-depth research on the pattern differences between offensive behaviors and benign behaviors, not just the differences in single traffic data.

[0046] An embodiment of the present invention provides an intrusion detection method. Figure 1 is a flow chart of an intrusion detection method according to an embodiment of the present invention. Figure 1 As shown, the process includes:

[0047] Step 1: Collect network traffic data in groups and obtain network traffic data characteristics;

[0048] In a specific embodiment, the network traffic data is collected in groups, including: collecting the network traffic data with the source IP and the destination IP as identification information, and setting the collection lower limit M and the collection upper limit N of each group of network traffic data, wherein, if the data volume of the current behavior is lower than the collection lower limit M, the current behavior is considered to be invalid behavior, and if the data volume of the current behavior is higher than the collection upper limit N, the first collected data is deleted, and the new collected data is added to maintain the data volume equal to the collection upper limit N, wherein M and N are both integers.

[0049] In a specific embodiment, the network traffic data features include: duration of the behavior, number of packets in both directions, number of bytes in both directions, and response time in both directions.

[0050] In the actual implementation process, behavior is a continuous process, so when collecting data, the behavior should be divided according to the identification. The present invention uses the source IP and destination IP as identification information, and sets the upper and lower limits of the number of behavior determinations to 10 and 25 respectively. Only when more than 10 data are considered valid behaviors, and when more than 25 data are considered to have begun to evolve, the earliest data is deleted, and new data is added to ensure that the number of data is 25. The network traffic features used only include duration, number of packets in the forward and backward directions, number of bytes, and response time, a total of 7 feature indicators.

[0051] Step 2: Calculate and obtain the distribution characteristic curve of each network traffic data characteristic respectively;

[0052] In a specific embodiment, the distribution characteristic curve of each network traffic data feature is calculated respectively, including: calculating the proportional distribution information of each network traffic data feature, obtaining the frequency domain information of each network traffic data feature, and bringing the frequency domain information into the Fourier series real value formula to obtain the distribution characteristic curve of each network traffic data feature, wherein the distribution characteristic curve is expressed by the formula:

[0053] F d (t) = ∑x n sin(p n *2*π*t)

[0054] Among them, x n is the value of the network traffic data feature, p n For x n The proportional probability of , t is the quantity parameter.

[0055] In the actual implementation process, the proportional distribution information of the values ​​of each network traffic data feature in the data is calculated to obtain the frequency domain information about the network traffic data feature; the Fourier series real-valued formula is introduced to calculate the distribution curve function of the network traffic data feature. If the number of data is n, the n values ​​on one period of the distribution curve are calculated as the feature information.

[0056] Step 3: According to the distribution characteristic curve of each network traffic data feature, a behavior distribution characteristic curve is obtained by fitting;

[0057] In a specific embodiment, a behavior distribution characteristic curve is fitted based on the distribution characteristic curve of each network traffic data feature, including: inputting the distribution characteristic value of the distribution characteristic curve of each network traffic data feature into a convolutional autoencoder, and fitting the convolutional autoencoder to obtain the behavior distribution characteristic curve.

[0058] In the actual implementation process, Figure 2 is a schematic diagram of the embedded relationship between behavior and network traffic data features according to an embodiment of the present invention, such as Figure 2 As shown in the figure, when embedding and reducing dimension, the distribution of behavior and network traffic data features is abstracted into a differential manifold structure. According to the characteristics of differential manifold mapping, the Convolutional Autoencoder (CAE) is selected as the embedding fitting algorithm to realize the embedding fitting of multi-feature distribution curves to behavior distribution curves and achieve dimensionality reduction.

[0059] Step 4: Detect and determine the type of corresponding attack behavior based on the behavior distribution characteristic curve.

[0060] In a specific embodiment, the type of the corresponding attack behavior is detected and determined based on the behavior distribution characteristic curve, including: using the behavior distribution characteristic value of the behavior distribution characteristic curve to perform machine learning training to obtain a behavior detection neural network model, and using the behavior detection neural network model to detect and determine the type of the corresponding attack behavior.

[0061] In the actual implementation process, the data after dimensionality reduction, i.e., the behavior distribution feature data, is used as input and handed over to the machine learning model for training and detection. The present invention does not limit the machine learning model. The basic models in the sklearn machine learning library are used in the verification process, including four models of random forest, neural network, K nearest neighbor and guided aggregation algorithm. At the same time, only the default configuration is used, and the neural network model is not adjusted or redesigned. This illustrates the effectiveness of the method proposed in the present invention.

[0062] In the actual implementation process, the behavior type can be determined by setting a detection threshold and an abnormal threshold. If the number of data exceeding the detection threshold in a set of data is greater than the abnormal threshold, the behavior is considered abnormal.

[0063] An embodiment of the present invention further provides an intrusion detection system. Figure 3 FIG. 1 is a structural block diagram of an intrusion detection system according to an embodiment of the present invention. Figure 3 As shown, the intrusion detection system includes:

[0064] A data collection module is used to collect network traffic data in groups and obtain network traffic data characteristics;

[0065] A distribution characteristic calculation module is used to calculate and obtain the distribution characteristic curve of each network flow data characteristic;

[0066] A distribution feature fitting module is used to fit a behavior distribution feature curve according to the distribution feature curve of each network traffic data feature;

[0067] The training detection module is used to detect and determine the type of corresponding attack behavior based on the behavior distribution characteristic curve.

[0068] In a specific embodiment, in a data collection module, network traffic data is collected using source IP and destination IP as identification information, and a collection lower limit M and a collection upper limit N are set for each group of network traffic data. If the data volume of the current behavior is lower than the collection lower limit M, the current behavior is considered to be an invalid behavior. If the data volume of the current behavior is higher than the collection upper limit N, the first collected data is deleted and new collected data is added to maintain the data volume equal to the collection upper limit N, where M and N are both integers.

[0069] In a specific embodiment, in the distribution feature calculation module, the proportional distribution information of each network traffic data feature is calculated to obtain the frequency domain information of each network traffic data feature, and the frequency domain information is brought into the Fourier series real value formula to obtain the distribution feature curve of each network traffic data feature, wherein the distribution feature curve is expressed by the formula:

[0070] F d (t) = ∑x n sin(p n *2*π*t)

[0071] Among them, x n is the value of the network traffic data feature, p n For x n The proportional probability of , t is the quantity parameter.

[0072] In a specific embodiment, in the distribution feature fitting module, the distribution feature value of the distribution feature curve of each network traffic data feature is input into the convolutional autoencoder, and the convolutional autoencoder is fitted to obtain the behavior distribution feature curve.

[0073] In summary, the present invention aims to fully study the distribution feature information of network behavior, and use this feature information in combination with a machine learning model to achieve detection. The present invention uses the simplest and most basic machine learning model and the most basic network traffic indicators. On the one hand, it gets rid of the dilemma of over-reliance on machine learning; on the other hand, in-depth research on the characteristics of network behavior can make the intrusion detection method proposed in the present invention have better applicability. That is, in more network scenarios, using simpler machine learning models can achieve faster, more efficient and more accurate intrusion detection.

[0074] In the above-mentioned embodiments of the present invention, the collection strategy in the data collection stage is the way the present invention identifies network behavior. In the distribution feature calculation stage, it is the core algorithm of the present invention, which is the key technology of how the present invention recognizes behavior and how to analyze behavior characteristics. The 4-stage architecture (i.e., 4 steps, 4 modules) of the present invention includes the main functions of network behavior data collection, network behavior feature extraction, data dimensionality reduction, intrusion detection implementation, etc., which connects the complete process of the present invention.

[0075] The present invention can achieve an accuracy rate of more than 99% on both CICIDS2017 and UNSW-NB15 datasets. At the same time, FPR and FNR are generally lower than 0.01. The performance is superior. The present invention uses the most basic 7 traffic feature indicators, which shows that the present invention can be applied in most network environments with low application difficulty. The present invention uses the most basic machine learning model, with low deployment difficulty, low deployment cost and strong scalability.

[0076] The collection and calculation of behavioral characteristic information of the present invention are different from traditional methods. In the actual implementation process, there is no rigid requirement for the selection of machine learning models. The four machine learning models used in the present invention are exemplary. There may be other machine learning models that can also meet the detection requirements. There may be other alternatives to the solid machine learning model.

[0077] The above specific embodiments only describe the design principle of the present invention. The shapes and names of the components in the description may be different and are not limited. Therefore, those skilled in the art in the field of the present invention may modify or replace the technical solutions recorded in the above embodiments; and these modifications and replacements do not deviate from the creative purpose and technical solutions of the present invention and should all fall within the protection scope of the present invention.

Claims

1. An intrusion detection method, characterized in that: include: Collect network traffic data in groups and obtain network traffic data characteristics; Calculate and obtain the distribution characteristic curve of each network traffic data characteristic respectively; According to the distribution characteristic curve of each network traffic data characteristic, fitting to obtain a behavior distribution characteristic curve; Detecting and determining the type of the corresponding attack behavior according to the behavior distribution characteristic curve; The distribution characteristic curves of each network traffic data feature are calculated separately, including: Calculate the proportional distribution information of each of the network traffic data features to obtain the frequency domain information of each of the network traffic data features, and bring the frequency domain information into the Fourier series real value formula to obtain the distribution characteristic curve of each of the network traffic data features, wherein the distribution characteristic curve is expressed by the formula: F d (t)=∑x n that(p n *2*π*t) Among them, x n is the value of the network traffic data feature, p n For x n The proportional probability of , t is the quantity parameter.

2. The method according to claim 1, characterized in that The grouping and collecting of network traffic data includes: The network traffic data is collected using the source IP and the destination IP as identification information, and a collection lower limit M and a collection upper limit N are set for each group of the network traffic data, wherein, if the data volume of the current behavior is lower than the collection lower limit M, the current behavior is considered to be an invalid behavior, and if the data volume of the current behavior is higher than the collection upper limit N, the first collection data is deleted, and new collection data is added to maintain the data volume equal to the collection upper limit N, wherein M and N are both integers.

3. The method according to claim 2, characterized in that The network traffic data features include: duration of the behavior, number of packets in both the forward and backward directions, number of bytes in both the forward and backward directions, and response time in both the forward and backward directions.

4. The method according to claim 1, characterized in that According to the distribution characteristic curve of each network traffic data characteristic, fitting to obtain a behavior distribution characteristic curve includes: The distribution characteristic value of the distribution characteristic curve of each network traffic data feature is input into a convolutional autoencoder, and the convolutional autoencoder is fitted to obtain a behavior distribution characteristic curve.

5. The method according to claim 1, characterized in that The detecting and determining the type of the corresponding attack behavior according to the behavior distribution characteristic curve includes: The behavior distribution characteristic values ​​of the behavior distribution characteristic curve are used to perform machine learning training to obtain a behavior detection neural network model, and the behavior detection neural network model is used to detect and determine the type of corresponding attack behavior.

6. An intrusion detection system, characterized in that: include: A data collection module is used to collect network traffic data in groups and obtain network traffic data characteristics; A distribution characteristic calculation module is used to calculate and obtain the distribution characteristic curve of each network flow data characteristic; A distribution feature fitting module, used for fitting a behavior distribution feature curve according to the distribution feature curve of each network traffic data feature; A training detection module is used to detect and determine the type of corresponding attack behavior according to the behavior distribution characteristic curve; In the distribution feature calculation module, the proportional distribution information of each of the network traffic data features is calculated to obtain the frequency domain information of each of the network traffic data features, and the frequency domain information is brought into the Fourier series real value formula to obtain the distribution feature curve of each of the network traffic data features, wherein the distribution feature curve is expressed by the formula: F d (t)=∑x n that(p n *2*π*t) Among them, x n is the value of the network traffic data feature, p n For x n The proportional probability of , t is the quantity parameter.

7. The system according to claim 6, characterized in that In the data collection module, the network traffic data is collected using the source IP and the destination IP as identification information, and a collection lower limit M and a collection upper limit N are set for each group of the network traffic data. If the data volume of the current behavior is lower than the collection lower limit M, the current behavior is considered to be an invalid behavior. If the data volume of the current behavior is higher than the collection upper limit N, the first collected data is deleted and new collected data is added to maintain the data volume equal to the collection upper limit N, wherein M and N are both integers.

8. The system according to claim 6, characterized in that In the distribution feature fitting module, the distribution feature value of the distribution feature curve of each network traffic data feature is input into a convolutional autoencoder, and the convolutional autoencoder is fitted to obtain a behavior distribution feature curve.

Images