Network unknown attack detection method, system and device and storage medium

Abstract

The invention belongs to the technical field of network space security, and discloses a network unknown attack detection method, system and device and a storage medium, and the method comprises the steps: obtaining to-be-detected traffic data of a network, and extracting the features of the to-be-detected traffic data; obtaining an attack detection result through an unknown attack detection model according to the characteristics of the to-be-detected traffic data; the method comprises the following steps: acquiring training traffic data, extracting features of the training traffic data, and according to the features of the training traffic data, generating pseudo features of the training traffic data by adopting a preset sample generation model; and based on a generalized zero sample learning training mode, training a preset supervised classifier according to the features of the training traffic data and the pseudo features of the training traffic data to obtain an unknown attack detection model. The problems of small unknown attack samples and unbalanced training samples can be effectively solved, the defect of insufficient unknown attack detection capability is overcome, the detection accuracy is improved, and the network protection level is enhanced.

Description

Technical Field

[0001] The present invention belongs to the field of cyberspace security technology and relates to a network unknown attack detection method, system, equipment and storage medium. Background Art

[0002] The current network threat environment is becoming increasingly complex and dynamic. New attack methods and malware continue to emerge, and network threats have become more hidden and confrontational. At the same time, the types of unknown attacks are increasing, and the tactics and techniques are more novel. If they cannot be effectively detected, they will cause serious consequences such as information leakage, system paralysis and data loss, which will in turn have a serious impact on normal business operations and information assets. However, traditional network security detection methods can only deal with known attack patterns and malicious codes, with a high detection false negative rate and low efficiency, and it is difficult to meet new network security needs. The detection and identification of unknown attacks is a research hotspot and difficulty in the security field today.

[0003] Timely detection of unknown attacks can help defenders understand the attack intent and predict subsequent attacks, guide defense strategies, and improve security defense levels and response capabilities. However, on the one hand, new attacks such as zero-day vulnerability exploits and APTs are highly concealed, difficult to capture, and have little attack sample data; on the other hand, the lack of unknown attack sample data leads to an imbalance in training samples, making it impossible for deep learning-based detection models to accurately identify unknown attacks, and there are deficiencies in detection accuracy and recall rate. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a network unknown attack detection method, system, device and storage medium.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, the present invention provides a method for detecting unknown network attacks, comprising: obtaining network traffic data to be detected and extracting features of the traffic data to be detected; obtaining attack detection results through an unknown attack detection model based on the features of the traffic data to be detected; wherein the unknown attack detection model is trained by the following training method: obtaining training traffic data and extracting features of the training traffic data, and generating pseudo features of the training traffic data using a preset sample generation model based on the features of the training traffic data; based on a generalized zero-sample learning training method, training a preset supervised classifier based on the features of the training traffic data and the pseudo features of the training traffic data to obtain an unknown attack detection model.

[0007] Optionally, obtaining the network's traffic data to be detected includes: adopting a honey spot-based attack data capture method to obtain the network's traffic data to be detected through pre-built IP address-based external network honey spots and / or domain name and path-based internal network honey spots.

[0008] Optionally, the extracting of features of the traffic data to be detected includes: performing data preprocessing on the traffic data to be detected to obtain preprocessed traffic data to be detected, and performing feature extraction on the preprocessed traffic data to be detected according to the selected feature type to obtain features of the traffic data to be detected; the extracting of features of the training traffic data includes: performing data preprocessing on the training traffic data to obtain preprocessed training traffic data, and performing feature extraction on the preprocessed training traffic data according to the selected feature type to obtain features of the training traffic data; wherein the selected feature type is obtained in the following manner: based on the supervised classifier LightGBM model, the feature importance of each feature type is obtained according to the information gain or Gini coefficient of each feature type at each split; and according to a preset feature importance selection threshold, a number of preliminary feature types are selected in descending order of feature importance; the feature correlation coefficient between the several preliminary feature types is calculated using the Pearson correlation coefficient, and the preliminary feature types whose feature correlation coefficient is greater than the feature correlation coefficient threshold are eliminated to obtain the selected feature type.

[0009] Optionally, the preset sample generation model is obtained in the following manner: dividing the features of the training traffic data into known attack type features and unknown attack type features; according to the known attack type features, obtaining the attribute information of each known attack type through a preset attribute extraction network; and according to the known attack type features and the attribute information of each known attack type, training a preset conditional variational autoencoder to obtain a pre-trained conditional variational autoencoder; wherein the attribute extraction network is obtained by removing the last classification layer from a multi-layer perceptron; obtaining a decoder in the pre-trained conditional variational autoencoder to obtain the preset sample generation model.

[0010] Optionally, the method of generating pseudo features of the training traffic data using a preset sample generation model based on the features of the training traffic data includes: sampling a random vector from a prior distribution in the latent space of a pre-trained conditional variational autoencoder; obtaining attribute information of each unknown attack type through a preset attribute extraction network based on the features of the unknown attack types, and combining the random vector with the attribute information of each unknown attack type and the attribute information of each known attack type, respectively, and inputting the combined information into the preset sample generation model to generate known attack type pseudo features and unknown attack type pseudo features; wherein the number of known attack type pseudo features of each known attack type and the number of unknown attack type pseudo features of each unknown attack type are the same.

[0011] Optionally, after obtaining the unknown attack detection model, it also includes: obtaining a preset test index value of the unknown attack detection model; and when the preset test index value of the unknown attack detection model is less than the preset test index value threshold of the unknown attack detection model, retraining the preset supervised classifier by adjusting the training parameters in the training process of the unknown attack detection model until the preset test index value of the unknown attack detection model is not less than the preset test index value threshold of the unknown attack detection model, so as to obtain the final unknown attack detection model.

[0012] According to a second aspect of the present invention, a network unknown attack detection system is provided, comprising: a feature extraction module, used to obtain network traffic data to be detected and extract features of the traffic data to be detected; an attack detection module, used to obtain attack detection results through an unknown attack detection model according to the features of the traffic data to be detected; wherein the unknown attack detection model is trained by the following training method: obtaining training traffic data and extracting features of the training traffic data, and generating pseudo features of the training traffic data using a preset sample generation model based on the features of the training traffic data; based on a generalized zero-sample learning training method, a preset supervised classifier is trained according to the features of the training traffic data and the pseudo features of the training traffic data to obtain an unknown attack detection model.

[0013] Optionally, it also includes a data capture module; the data capture module is used to adopt an attack data capture method based on a honey spot, and obtain the network's traffic data to be detected through pre-built external network honey spots based on IP addresses and / or internal network honey spots based on domain names and paths.

[0014] According to a third aspect of the present invention, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method for detecting unknown network attacks when executing the computer program.

[0015] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned network unknown attack detection method are implemented.

[0016] Compared with the prior art, the present invention has the following effects:

[0017] The network unknown attack detection method of the present invention extracts the features of the flow data to be detected, and then realizes the attack detection of the flow data to be detected through a preset unknown attack detection model, thereby obtaining the attack detection result of the flow data to be detected. Among them, in the training process of the unknown attack detection model, in view of the problem of lack of a large number of unknown attack samples in the real network environment, the pseudo-features of the training flow data are generated by a preset sample generation model, the data volume of the training samples is increased, and the balance of the samples is ensured, thereby effectively overcoming the problem of low model detection accuracy caused by the lack of samples. At the same time, the unknown attack detection model is constructed based on a supervised classifier, the unknown attack detection problem is converted into a supervised classification problem, and the training is carried out based on a generalized zero-sample learning training method, which is more in line with the real scene of network unknown attack detection, thereby ensuring that the unknown attack detection model has a high detection accuracy, realizes accurate detection of unknown attacks on the network, consumes the attack resources of the attacker, reverses the asymmetric situation of attack and defense to a certain extent, discovers unknown attack behaviors in time, and enhances the overall level of network security protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 The figure is a flow chart of a method for detecting unknown network attacks according to an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of a method for detecting unknown network attacks according to an embodiment of the present invention.

[0020] Figure 3 The figure is a structural block diagram of a network unknown attack detection system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0023] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0024] See also Figure 1 In one embodiment of the present invention, a network unknown attack detection method is provided, which can effectively respond to the continuous evolution of new attack methods, timely discover unknown attack behaviors, solve the problems of small unknown attack samples and unbalanced training samples, overcome the defects of traditional detection methods in insufficient unknown attack detection capabilities, improve detection accuracy, and enhance the overall level of network security protection.

[0025] Specifically, the network unknown attack detection method of the present invention comprises the following steps:

[0026] S1: Obtain the traffic data to be detected from the network and extract the features of the traffic data to be detected.

[0027] S2: According to the characteristics of the traffic data to be detected, the attack detection result is obtained through the unknown attack detection model.

[0028] Among them, the unknown attack detection model is trained by the following training method: obtaining training traffic data and extracting the features of the training traffic data, and generating pseudo features of the training traffic data using a preset sample generation model based on the features of the training traffic data; based on the generalized zero-sample learning training method, a preset supervised classifier is trained according to the features of the training traffic data and the pseudo features of the training traffic data to obtain an unknown attack detection model.

[0029] The network unknown attack detection method of the present invention extracts the features of the flow data to be detected, and then realizes the attack detection of the flow data to be detected through a preset unknown attack detection model, thereby obtaining the attack detection result of the flow data to be detected. Among them, in the training process of the unknown attack detection model, in view of the problem of lack of a large number of unknown attack samples in the real network environment, the pseudo-features of the training flow data are generated by a preset sample generation model, the data volume of the training samples is increased, and the balance of the samples is ensured, thereby effectively overcoming the problem of low model detection accuracy caused by the lack of samples. At the same time, the unknown attack detection model is constructed based on a supervised classifier, the unknown attack detection problem is converted into a supervised classification problem, and the training is carried out based on a generalized zero-sample learning training method, which is more in line with the real scene of network unknown attack detection, thereby ensuring that the unknown attack detection model has a high detection accuracy, realizes accurate detection of unknown attacks on the network, consumes the attack resources of the attacker, reverses the asymmetric situation of attack and defense to a certain extent, discovers unknown attack behaviors in time, and enhances the overall level of network security protection.

[0030] In a possible implementation, obtaining the network's traffic data to be detected includes: using a honey spot-based attack data capture method to obtain the network's traffic data to be detected through pre-built IP address-based external network honey spots and / or domain name and path-based internal network honey spots.

[0031] As an active deception tool, honeypots have been maturely applied in industrial control systems, but their attack detection surface is narrow and they are easily identified by attackers. Honey spots make up for the defects of traditional honeypots. They are a group of artificially set network access points deployed around the protected system. They carry the sentinel processes carefully set up by the defenders inside, and their external form is a simulation system of the protected object. Unlike honeypots, honey spots are not actively spread, but are placed on paths that regular users will not visit. Once a honey spot is visited, it can be confirmed that the behavior is an abnormal access. The attack data captured based on honey spots has a large flow, and a large amount of high-value data can be collected. Regarding the construction of honey spots, the present invention adopts two design methods: external network honey spots based on IP addresses and internal network honey spots based on domain names and paths.

[0032] Regarding the construction of an external network honeypot based on IP addresses, the trap is mainly set by setting a large number of IP addresses that are not accessed by normal behavior. Once the address is touched, the user who stepped on the honeypot can be identified as a malicious attacker. The specific construction process includes: 1. Design a virtual network topology based on the network topology and characteristics of the industrial control business system, ensuring that the virtual network topology is logically consistent with the real network topology and preventing the leakage of the core topology. Perform a comprehensive analysis of the real network topology, including network structure, IP address planning, subnet division and routing strategy. (1) Network segment division: The virtual network and the real network use different IP address segments, and the subnet mask is consistent. (2) Network topology structure: The virtual network simulates the real network topology structure, reduces some nodes and connections in the virtual network, or adds some spare nodes or secondary connections to introduce certain differences. Change the topology of some networks, such as replacing the ring structure of some network segments with a star structure. (3) Check whether there is a conflict with the real network, whether there are problems such as IP address overlap and routing loops. 2. According to the characteristics of industrial control business systems, define the service types of honey spots, simulate various service types, such as SCADA protocol, remote access service, etc., and provide various honey spots including device class, web class, middleware class, database class, network device class and network service class to improve the authenticity of honey spots.

[0033] Regarding the construction of intranet honey spots based on domain names and paths, the specific construction process includes: 1. Domain name honey spot construction. After luring attackers into the virtual network, a domain name dictionary is generated based on the domain name blasting dictionary commonly used by attackers, combined with the domain name and subdomain information of the industrial control business field, as a false domain name to attract attackers. 2. Path honey spot construction. (1) Create some false directories and files, and give them simple and common names so that they can be scanned by attackers. Add hyperlinks to false web pages that normal users will not visit to the business web pages. Once the attacker touches these false files or clicks on the false web pages, they will fall into the path honey spot. (2) Find the file paths and web page hyperlinks accessed by normal users through traffic or log analysis, and then based on full path detection and scanning, find the dark paths that normal users will not visit by excluding them, and distribute or deploy high-value honey baits on the dark paths, such as user cookies, accounts, passwords and high-value files, and customize the matching business files according to different business systems to increase the sweetness of the honey bait. Normal users usually do not access this path. In order to search for high-value files, attackers will enter these network nodes and file paths. Once attackers detect and search for these path trap honey spots, the path honey spots can sense and record intranet attack behaviors.

[0034] In addition, in order to deal with the attacker's anti-identification technology and continuously attract attackers to perform attack behaviors in the virtual business system to capture high-value information, a virtual physical business model and a virtual industrial control private protocol can be constructed to ensure that the perception and transmission of data conform to the laws of physics. Among them, a fictitious physical business model is set to characterize the constraints of the physical laws and business processes of the industrial control system, constrain the physical information perception and execution of the primary equipment, and the state changes and operations of the secondary equipment, which not only confuses the attacker's business cognition of the real business system, but also realizes a high-realistic bait environment. A false industrial control private protocol is set in the honey spot to simulate the network stack behavior of the power business system, and the information interaction intensity between the honey spot and the attacker is increased. Whenever the attacker initiates a request, the protocol header of the response data packet is appropriately modified to make it conform to the characteristic protocol of the industrial control business system expected by the fingerprint recognition software (such as Nmap), so as to make an "accurate" response to the attacker, induce the attacker to continue the attack activities, and collect the attacker's valuable attack data. The above two points ensure that the perception and transmission of data conform to the laws of physics and are not easily identified by the attacker's anti-entrapment technology.

[0035] When an attacker launches an attack on the network, he will step on the honey spot. The external network honey spots and the internal network honey spots set up will collect and record the attacker's threat behavior data, such as IP address, attack payload, attack time, and attack target, that is, the network's traffic data to be detected, which can reflect the attacker's intention and attack mode. The collected traffic data to be detected is saved in the data storage device in PCAP format. PCAP (Packet Capture) format is a commonly used network data packet capture format.

[0036] In one possible implementation, see Figure 2 The extracting of features of the flow data to be detected includes: performing data preprocessing on the flow data to be detected to obtain preprocessed flow data to be detected, and performing feature extraction on the preprocessed flow data to be detected according to the selected feature type to obtain the features of the flow data to be detected.

[0037] Among them, the selected feature type is obtained in the following way: based on the supervised classifier LightGBM model, the feature importance of each feature type is obtained according to the information gain or Gini coefficient of each feature type at each split; and according to the preset feature importance selection threshold, a number of preliminary feature types are selected in descending order of feature importance; the Pearson correlation coefficient is used to calculate the feature correlation coefficient between several preliminary feature types, and the preliminary feature types whose feature correlation coefficients are greater than the feature correlation coefficient threshold are eliminated to obtain the selected feature type.

[0038] Specifically, the initially acquired flow data to be detected needs to be preprocessed to facilitate subsequent analysis and processing. In this implementation, flow segmentation, data cleaning, data clipping, and data normalization are performed in sequence to achieve data preprocessing of the flow data to be detected.

[0039] Among them, flow division: each PCAP file contains data packets of different sizes. By using the CICFlowMeter tool, the network flow boundary is divided based on the source address, source port, transport protocol, destination port and destination address quintuple. Data cleaning: delete duplicate data packets, empty data packets and Ethernet-related data in the data link layer (such as MAC address). Data trimming: In order to make the transport layer segment uniform, fill the end of the 8-byte header of the UDP segment with 0, which is equal to the 20-byte header of the TCP segment. Observe the multiple PCAP files divided by the original PCAP file according to the data flow form. If the proportion of data packets with a length less than the set byte length is greater than 75%, and the proportion increases slowly with the increase in the number of bytes, the set byte length is selected as the fixed byte length, and the insufficient length is padded with 0x00, and the excess bytes are discarded.

[0040] Data normalization: The numerical scales of different features vary greatly. For example, the numerical range of one feature is [0.0001, 0.05], while the numerical range of another feature is [0.1e+09, 4e+09]. This will lead to inaccurate feature extraction model. The minimum-maximum method is used to normalize them:

[0041]

[0042] Among them, x is the original value of a feature, x max is the characteristic maximum value, x min is the minimum characteristic value, x norm is the normalized eigenvalue.

[0043] In this implementation, the features of the traffic data to be detected are extracted through feature engineering. Specifically, the supervised classifier LightGBM model is first used to select feature importance, and then the feature correlation coefficient is calculated using the Pearson correlation coefficient, and finally the feature combination with high feature importance and low correlation is retained as the preset feature type.

[0044] Among them, the supervised classifier LightGBM model is used to complete the feature importance selection. Specifically, the decision tree is trained using data. During the training process, the importance of the feature is evaluated according to the information gain or Gini coefficient of the feature at each split. The threshold is selected according to the feature importance. For example, features that account for more than 90% are selected in order from high to low feature importance.

[0045] The supervised classifier LightGBM model is a distributed gradient boosting (GradientBoosting Decision Tree, GBDT) framework based on the decision tree algorithm. The supervised classifier LightGBM model uses a histogram-based algorithm to build a decision tree, adopts a leaf-wise growth strategy, optimizes the cache hit rate, and directly supports category features. Compared with the basic GBDT algorithm and XGBoost (Extreme Gradient Boosting) algorithm, its training speed and efficiency are higher, the memory utilization ratio is lower, and the accuracy is greatly improved.

[0046] The Pearson correlation coefficient is used as the feature correlation coefficient to filter out features with large feature correlation coefficients and retain feature combinations that are highly important and independent of each other.

[0047] Among them, the Pearson correlation coefficient between two variables X and Y is ρ X,Y :

[0048]

[0049] Among them, cov(X,Y) is the covariance of variables X and Y, σ X is the variance of variable X, σ Y is the variance of variable Y, μ X is the mean of variable X, μ Y is the mean of the variable Y.

[0050] The closer the absolute value of the feature correlation coefficient is to 1, the stronger the variable correlation is; the closer it is to 0, the weaker the variable correlation is. The feature correlation coefficient threshold is set to filter out redundant features, and features with feature correlation coefficients greater than the feature correlation coefficient threshold are filtered out to finally obtain the best feature combination, that is, the selected feature type.

[0051] Optionally, extracting the features of the training traffic data includes: performing data preprocessing on the training traffic data to obtain preprocessed training traffic data, and extracting the preprocessed training traffic data according to a selected feature type to obtain the features of the training traffic data.

[0052] Specifically, the preliminary processing of the training flow data, including data preprocessing and feature extraction, is consistent with the process of the flow data to be tested and will not be repeated.

[0053] In a possible implementation, the preset sample generation model is obtained in the following manner: dividing the features of the training traffic data into known attack type features and unknown attack type features; obtaining attribute information of each known attack type through a preset attribute extraction network based on the known attack type features; and training a preset conditional variational autoencoder based on the known attack type features and the attribute information of each known attack type to obtain a pre-trained conditional variational autoencoder; obtaining a decoder in the pre-trained conditional variational autoencoder to obtain the preset sample generation model.

[0054] Specifically, first determine whether the training data is a known attack or an unknown attack based on the characteristics of the extracted training traffic data. If the characteristics of the current training traffic data match the feature library of the border protection detection device, mark the current training traffic data as a known attack type and store the relevant data in the training set; if they do not match, mark it as an unknown attack type and store the relevant data in the test set.

[0055] Using the attack type features and corresponding labels, a multi-layer perceptron (MLP) is used as the classification model M for classification training. After removing the last classification layer from the model M, the attribute extraction network M is obtained. A , input the feature F of the training traffic data, and output the attribute vector A, that is, A = M A (F).

[0056] For each label y, the features of different attack types f i y ,i=1,...,m, get the attribute value After calculating the mean, it is used as the attribute vector corresponding to the label y:

[0057]

[0058] Among them, a y is the attribute vector corresponding to label y, m is the number of samples of attack type corresponding to label y, and f i y is the feature of the i-th attack sample corresponding to label y, is the attribute value of the i-th attack sample corresponding to label y.

[0059] The present invention adopts a generative method to alleviate the overfitting problem existing in the embedded method by means of data enhancement. The generative method converts the zero-sample learning problem into a supervised learning problem by learning the relationship between the attribute space and the real feature distribution, and using the attribute space of the unseen class to generate the features of the unseen class. Specifically, the present invention uses a conditional variational autoencoder (CVAE) to generate pseudo samples to construct the corresponding relationship between the attribute space and the feature space. CVAE introduces conditional information on the basis of VAE (Variational Autoencoder), so that the model can generate samples of a specific type according to given conditional information. Taking the attribute vector A as the conditional information, the encoder E encodes the feature F of the training traffic data and the attribute vector A into the latent space Z, and the decoder D reconstructs the features of the training traffic data corresponding to the attribute vector based on the latent space sampling results and the attribute vector. Formalizing the above expression, the encoder is defined as q(z|f,a), the decoder is defined as q(f|z,a), and the latent space distribution is p(z).

[0060] When training the preset conditional variational autoencoder, the training process is based only on the known attack type features F k and its corresponding attribute space A k , and two loss functions are needed, namely reconstruction loss and KL loss. The reconstruction loss measures the feature F of the input training traffic data and the pseudo feature F of the output training traffic data. The KL loss measures the difference between the potential spatial distribution of the encoder D output and the prior Gaussian distribution. It can be expressed as:

[0061] L=-D KL (q(z|f,a)|p(z))+E q(z|f,a) [logq(f|z,y)]

[0062] Among them, L is the comprehensive loss, D KL is the KL loss.

[0063] In a possible implementation, the method of generating pseudo features of the training traffic data using a preset sample generation model based on the features of the training traffic data includes: sampling a random vector from a prior distribution in the latent space of a pre-trained conditional variational autoencoder; obtaining attribute information of each unknown attack type through a preset attribute extraction network based on the features of the unknown attack type, and combining the random vector with the attribute information of each unknown attack type and the attribute information of each known attack type, respectively, and inputting the combined information into a preset sample generation model to generate known attack type pseudo features and unknown attack type pseudo features, wherein the number of known attack type pseudo features of each known attack type and the number of unknown attack type pseudo features of each unknown attack type are the same.

[0064] Specifically, after the conditional variational autoencoder is trained, the decoder in the trained conditional variational autoencoder is used as the preset sample generation model to generate fake data. A random vector is sampled from the prior distribution in the latent space, and then the random vector is combined with the known attack type feature and the unknown attack type feature A. kUz Input them together into the sample generation model, and the sample generation model generates the random vector according to this random vector and A kUz Generate pseudo-signatures of known attack types and pseudo-signatures of unknown attack types In order to avoid data imbalance, the same amount of pseudo data is generated for each data category.

[0065] By using CVAE based on all types of traffic attribute space, namely A kUz Generate feature representation The classification detection problem of unknown attacks is converted into a supervised classification problem.

[0066] In a possible implementation, based on a generalized zero-shot learning training method, a preset supervised classifier is trained according to the features of the training traffic data and the pseudo features of the training traffic data.

[0067] Most industrial control systems are in normal operation. Data actively captured based on honey spots are considered malicious attack data, i.e., traffic data to be detected. However, it is difficult for traditional machine learning algorithms to detect unknown attacks. Zero-sample learning can alleviate the reliance on unknown attack sample data in industrial control environments. The zero-sample feature is reflected in the fact that attack categories that do not appear in the training data set exist in the test phase data set, i.e., unknown attacks. In real unknown attack capture and detection scenarios, unknown attack samples cannot participate in training. However, in the classification and detection phase, not only real unknown attacks will be faced, but also other types of known attacks. The traditional zero-sample definition only has unseen categories appearing in the test phase, which does not conform to the real scenario. Therefore, a generalized zero-sample learning method is adopted, i.e., the test phase data set contains not only unknown attack samples, but also known attack samples.

[0068] Define known attack categories as Unknown attack category is It is obvious that definition As the training dataset, represents a sample instance of a known attack type, It is the label of the corresponding known attack category. Definition As the test phase dataset, represents the sample instance in the test phase, is the corresponding label. Further, the training phase dataset and the test phase dataset can be refined as follows:

[0069]

[0070] in, X tr is a sample set of known attack types; X tst is the sample set in the test phase, including sample instances of known attack types and sample instances of unknown attack types; X is all sample instances.

[0071] Among them, the preset supervised classifier uses the supervised classifier LightGBM model to predict the unknown attack label. In the real industrial control unknown attack detection scenario, all the traffic data to be detected captured by the honey spot are detection objects, including not only samples of unknown attack categories, but also samples of known attack categories.

[0072] In a possible implementation, after obtaining the unknown attack detection model, it also includes: obtaining a preset test index value of the unknown attack detection model; and when the preset test index value of the unknown attack detection model is less than the preset test index value threshold of the unknown attack detection model, retraining the preset supervised classifier by adjusting the training parameters in the training process of the unknown attack detection model until the preset test index value of the unknown attack detection model is not less than the preset test index value threshold of the unknown attack detection model, thereby obtaining the final unknown attack detection model.

[0073] Specifically, after obtaining the unknown attack detection model, the preset test index values ​​of the unknown attack detection model are obtained, including preset test index values ​​such as F1, accuracy, precision and recall, and then the performance of the unknown attack detection model is evaluated in combination with the preset test index value threshold. When the preset test index value of the unknown attack detection model is less than the preset test index value threshold of the unknown attack detection model, it is considered that the current unknown attack detection model does not meet the expected requirements, and the preset supervised classifier is retrained by adjusting the training parameters in the training process of the unknown attack detection model, such as adjusting the cropping length in the data preprocessing stage, adjusting the model parameters of the supervised classifier, adjusting the feature importance selection threshold, adjusting the feature correlation coefficient threshold, and adjusting the attribute vector dimension, etc., and finally the unknown attack detection model is adjusted to the optimal value according to the preset test index value.

[0074] The network unknown attack detection method of the present invention, the linkage defense system based on rules and feature codes can only detect known attacks and threats, and the traditional machine learning and deep learning models can be used to detect unknown attacks, but the traditional machine learning model cannot extract features from data with unbalanced samples and high feature dimensions to detect unknown attacks, and the deep learning model has a good effect, but relies on a large amount of correctly labeled traffic data, and the current situation of lack of unknown attack samples in the real network environment, by arranging honey spots to capture attack data, and then pre-processing the attack data, feature extraction based on the supervised classifier LightGBM model and the Person correlation coefficient, and then using the multi-layer perceptron to generate attribute vectors, and combining with the pre-built sample generation model based on the conditional variational autoencoder to generate pseudo data, and then training the unknown attack detection model after sample expansion, after completing the training, it can effectively capture and detect unknown attacks, consume the attacker's attack resources, and realize the detection of new unknown network threats, APT and other attacks, which to a certain extent reverses the asymmetric situation of attack and defense.

[0075] The following are device embodiments of the present invention, which can be used to implement the method embodiments of the present invention. For details not disclosed in the device embodiments, please refer to the method embodiments of the present invention.

[0076] See also Figure 3 In another embodiment of the present invention, a network unknown attack detection system is provided, which can be used to implement the above-mentioned network unknown attack detection method. Specifically, the network unknown attack detection system includes a feature extraction module and an attack detection module.

[0077] Among them, the feature extraction module is used to obtain the network's traffic data to be detected and extract the features of the traffic data to be detected; the attack detection module is used to obtain the attack detection result through an unknown attack detection model based on the features of the traffic data to be detected; wherein the unknown attack detection model is trained by the following training method: obtaining training traffic data and extracting the features of the training traffic data, and generating pseudo features of the training traffic data using a preset sample generation model based on the features of the training traffic data; based on the generalized zero-sample learning training method, a preset supervised classifier is trained according to the features of the training traffic data and the pseudo features of the training traffic data to obtain an unknown attack detection model.

[0078] In a possible implementation, the network unknown attack detection system also includes a data capture module; the data capture module is used to adopt an attack data capture method based on a honey point to obtain the network's traffic data to be detected through pre-built external network honey points based on IP addresses and / or internal network honey points based on domain names and paths.

[0079] All relevant contents of each step involved in the embodiment of the aforementioned network unknown attack detection method can be referred to the functional description of the functional module corresponding to the network unknown attack detection system in the embodiment of the present invention, and will not be repeated here.

[0080] The division of modules in the embodiments of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present invention may be integrated into one processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0081] In another embodiment of the present invention, a computer device is provided, the computer device including a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of the network unknown attack detection method.

[0082] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the network unknown attack detection method in the above embodiment.

[0083] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0084] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0085] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0086] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for detecting unknown network attacks, characterized in that: include: Obtaining the traffic data to be detected of the network and extracting the features of the traffic data to be detected; According to the characteristics of the traffic data to be detected, the attack detection results are obtained through the unknown attack detection model; Among them, the unknown attack detection model is trained by the following training method: Acquire training traffic data and extract features of the training traffic data, and generate pseudo features of the training traffic data using a preset sample generation model based on the features of the training traffic data; Based on the generalized zero-shot learning training method, a preset supervised classifier is trained according to the characteristics of the training traffic data and the pseudo-features of the training traffic data to obtain an unknown attack detection model.

2. The method for detecting unknown network attacks according to claim 1, characterized in that: The obtaining of the network traffic data to be detected comprises: The attack data capture method based on honey spots is adopted to obtain the network traffic data to be detected through pre-built external network honey spots based on IP addresses and / or internal network honey spots based on domain names and paths.

3. The method for detecting unknown network attacks according to claim 1, characterized in that: The features of extracting the traffic data to be detected include: Performing data preprocessing on the flow data to be detected to obtain preprocessed flow data to be detected, and performing feature extraction on the preprocessed flow data to be detected according to the selected feature type to obtain features of the flow data to be detected; The features of extracting training flow data include: Performing data preprocessing on the training flow data to obtain preprocessed training flow data, and performing feature extraction on the preprocessed training flow data according to the selected feature type to obtain features of the training flow data; Among them, the selected feature type is obtained by: Based on the supervised classifier LightGBM model, the feature importance of each feature type is obtained according to the information gain or Gini coefficient of each feature type at each split; and according to the preset feature importance selection threshold, several preliminary feature types are selected in descending order of feature importance; The Pearson correlation coefficient is used to calculate the feature correlation coefficients between several preliminary feature types, and the preliminary feature types whose feature correlation coefficients are greater than the feature correlation coefficient threshold are eliminated to obtain the selected feature types.

4. The method for detecting unknown network attacks according to claim 1, characterized in that: The preset sample generation model is obtained in the following way: The features of the training traffic data are divided into known attack type features and unknown attack type features; According to the known attack type characteristics, the attribute information of each known attack type is obtained through a preset attribute extraction network; and according to the known attack type characteristics and the attribute information of each known attack type, a preset conditional variational autoencoder is trained to obtain a pre-trained conditional variational autoencoder; wherein the attribute extraction network is obtained by removing the last classification layer from the multi-layer perceptron; Get the decoder in the pre-trained conditional variational autoencoder to obtain the preset sample generation model.

5. The method for detecting unknown network attacks according to claim 4, characterized in that: The method of generating pseudo features of the training flow data by using a preset sample generation model according to the features of the training flow data includes: Sample a random vector from the prior distribution in the latent space of the pre-trained conditional variational autoencoder; According to the unknown attack type characteristics, the attribute information of each unknown attack type is obtained through a preset attribute extraction network, and the random vector is combined with the attribute information of each unknown attack type and the attribute information of each known attack type and then input into a preset sample generation model to generate known attack type pseudo features and unknown attack type pseudo features; The number of known attack type pseudo-signatures of each known attack type and the number of unknown attack type pseudo-signatures of each unknown attack type are the same.

6. The method for detecting unknown network attacks according to claim 1, characterized in that: After obtaining the unknown attack detection model, the method further includes: Obtain a preset test index value of the unknown attack detection model; and when the preset test index value of the unknown attack detection model is less than the preset test index value threshold of the unknown attack detection model, retrain the preset supervised classifier by adjusting the training parameters in the training process of the unknown attack detection model until the preset test index value of the unknown attack detection model is not less than the preset test index value threshold of the unknown attack detection model, thereby obtaining a final unknown attack detection model.

7. A network unknown attack detection system, characterized in that: include: A feature extraction module is used to obtain the traffic data to be detected in the network and extract the features of the traffic data to be detected; An attack detection module is used to obtain attack detection results through an unknown attack detection model according to the characteristics of the traffic data to be detected; Among them, the unknown attack detection model is trained by the following training method: Acquire training traffic data and extract features of the training traffic data, and generate pseudo features of the training traffic data using a preset sample generation model based on the features of the training traffic data; Based on the generalized zero-shot learning training method, a preset supervised classifier is trained according to the characteristics of the training traffic data and the pseudo-features of the training traffic data to obtain an unknown attack detection model.

8. The network unknown attack detection system according to claim 7, characterized in that: Also included is a data capture module; The data capture module is used to adopt an attack data capture method based on a honey spot, and obtain the network's traffic data to be detected through pre-built external network honey spots based on IP addresses and / or internal network honey spots based on domain names and paths.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the network unknown attack detection method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the network unknown attack detection method according to any one of claims 1 to 6 are implemented.

Images