Method and system for performing network attack detection by causal reasoning through causal converter

Abstract

The invention discloses a network attack detection method and system using causal reasoning by a causal converter, and the method comprises the following steps: inputting a detection set composed of samples containing known and unknown attack types into a classifier based on supervised learning training, and carrying out the classification, thereby obtaining a plurality of known attack types; classifying the known attack types into known attacks and unknown attacks based on a causal converter of semi-supervised learning; and classifying all unknown attacks into a plurality of unknown attack types based on a causal converter of unsupervised learning. The invention aims to guide a classifier to decouple false association through causal reasoning, and realizes intelligent interpretable IOT network attack detection which can identify known network anomalies and can classify unknown anomalies.

Description

Technical Field

[0001] The present invention relates to the field of IOT network security technology, and in particular to a method and system for a causal converter to detect network attacks using causal reasoning. Background Art

[0002] Due to the large number of IoT devices, as well as their open and distributed characteristics, hundreds of millions of interconnected IoT devices are becoming a stage for hackers to show off their skills and introducing potential cybercrime. Compared with traditional computers and other devices, the biggest difference between IoT devices is their small size. In addition, their interconnected nature allows them to connect to multiple devices, so there are more ways to be attacked. In particular, the threat of unknown attacks is growing and difficult to prevent, and is becoming one of the most serious threats to IoT security. In order to cope with the rapid growth in the number and complexity of network viruses, especially the huge challenges brought by unknown network attacks, there is an urgent need for a detection system that can accurately classify known attacks, identify unknown attacks, and perform incremental learning, so as to provide accurate and fully effective detection information to deal with IoT network security threats.

[0003] In the past few years, machine learning has shown great potential in the field of IOT intrusion detection due to its inherent learning and reasoning capabilities. However, intrusion detection not only requires accurate classification of known network attacks, but also requires identification and learning of unknown types such as zero-day attacks, which is in essential conflict with the simple classification function of machine learning and other methods. In particular, for the method of simply classifying network traffic as normal or not for outlier detection, only so-called normal traffic samples can be learned on a large scale in order to identify abnormal behavior. However, this method has no practical deployment significance in the actual IOT network, because most of the so-called samples that are different from normal traffic are likely to be network surges caused by retransmission behavior or group network activities. It would be wrong to judge such surges as network attacks. In addition, although machine learning has a high tolerance for misjudgment, it is extremely costly in the field of IOT security detection. Even a small number of missed reports may cause extremely serious damage to IOT devices due to failure to identify network attacks in a timely manner, especially in the field of intelligent driving, which may lead to vehicle destruction and death.

[0004] Another serious problem faced by machine learning is the poor interpretability of the model and the lack of causality. This is because machine learning and other methods make judgments based on the correlation between features, but this correlation is not a causal relationship. For example, in IOT intrusion detection, machine learning and other methods identify security threats based on the correlation between traffic anomalies and attack types, and find some non-causal features in correlation mining, such as the relationship between various risk factors (such as mass network activities) and traffic anomalies, and use this strong false correlation to make predictions. Therefore, machine learning methods driven by correlation relationships cannot provide real decision-making basis for today's network traffic anomaly diagnosis.

[0005] IoT attack detection is essentially a classification system, and machine learning has the ability to optimize classifier performance by continuously learning and accumulating experience, which is a better solution than traditional detection methods. Therefore, various machine learning methods have been continuously used in attack detection in recent years to improve its detection performance. However, with the development of machine learning, automated and intelligent detection methods have profoundly affected the development of attack detection technology. Existing methods can only solve the problem of attack classification with prior knowledge, without considering the existence of unknown attacks, or only consider whether there is an attack without considering the classification of attacks. However, in actual networks, facing the threats of multiple known attacks and multiple unknown attacks, existing machine learning methods cannot provide meaningful guidance measures for network defense.

[0006] Transformer is a deep learning model that only relies on the self-attention mechanism to calculate input and output, and is completely different from network structures such as recursive neural networks. It has achieved great success in the field of anomaly detection. Some literature believes that this success should be attributed to its powerful unified modeling capabilities in global representation and long-range relations. Faced with the problems of lack of anomaly labels and excessive data volatility in traffic anomaly detection, the prior art proposes an anomaly detection model based on deep transformers, which can quickly perform inference based on a wider range of temporal trend knowledge of traffic samples. In order to protect IoT devices from attacks, the prior art proposes a transformer classifier that can not only effectively process a large amount of data, but also quickly detect whether the network has been invaded. In order to solve the problems of feature redundancy and sample imbalance, the prior art also records the use of position embedding technology to associate the sequential information between features, and then perform feature dimensionality reduction based on variable stacked encoder-decoder, and proposes a robust transformer-based network anomaly detection system (RTIDS). After comparison with deep learning algorithms such as long short-term memory network (LSTM), the effectiveness of RTIDS is verified. Since traffic anomaly detection uses flow-based feature extraction, it can only perform detection after the flow ends. In order to solve this problem, the existing technology proposes a detection method based on the neural attention mechanism based on the time slot function to solve the problem of real-time attack detection, and verifies its performance in terms of accuracy and false alarm rate by comparing with models such as bidirectional LSTM as a baseline. In summary, the transformer ideally solves the problem of low detection rate caused by the lack of abnormal labels and high data volatility faced by current network attack detection, and has ultra-low inference time requirements, and can quickly and accurately identify network traffic anomalies. However, the transformer still only considers whether the network traffic is abnormal, and cannot solve the requirements for accurate classification of known and unknown attacks.

[0007] The detection system based on machine learning is essentially a black box and does not have causal interpretability. First, machine learning cannot gain insight into its own behavior and reasoning through complex algorithms and is prone to generate a large number of false positives, which limits its application in actual networks. Second, due to the lack of causal knowledge, existing machine learning methods can only make predictions based purely on correlation, which limits them to analytical insights and cannot completely solve the management and decision-making problems in network security. Since causal knowledge is crucial to organizational decision-making, if these problems are to be solved, causal knowledge needs to be added to machine learning. In recent years, with the introduction and development of causal reasoning technology, an effective method has been provided to reveal the causal relationship of machine learning methods. For example, the literature (

[31] Zhang, Xingxuan, et al. "Deep stable learning for out-of-distribution generalization." Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2021.) solves the false correlation of deep models by making the features independent through the learning weights of the consistent causal effect training samples, thereby making the machine learning model more focused on distinguishing the true connection between features and labels. The paper (Cui, Peng, and Susan Athey. "Stable learning establishes some common ground between causal inference and machine learning." Nature Machine Intelligence 4.2 (2022): 110-115.) believes that using causal reasoning to guide machine learning can eliminate false correlations and improve the stability of the model in unknown environments. The paper (Jung, Yonghan, JinTian, ​​and Elias Bareinboim. "Estimating identifiable causal effects through double machine learning." Proceedings of the AAAI Conference on ArtificialIntelligence. Vol. 35. No. 13. 2021.) proposes a machine learning method called double debiasing based on causal reasoning, which is highly robust in reducing modeling errors and biases.In summary, the problems faced in machine learning also apply to network attack detection. However, with the in-depth study of causal reasoning technology, causal reasoning-based modeling technology represented by causal machine learning will become a breakthrough in intelligent detection, helping the detection system to infer the causal relationship between network attacks and traffic characteristics from abnormal traffic samples and conduct effective detection, thereby providing a better decision-making basis for network security. Therefore, if causal reasoning is introduced in network attack detection based on machine learning, it is expected to bridge the gap between "detection" and complete "decision-making" in the real network.

[0008] Structural causal modeling, or SCM for short, is a formalized mathematical modeling language that consists of a graph model that represents causal knowledge, causal intervention and counterfactual reasoning that explain causal knowledge, and structural equations that semantically link the two. Each node in SCM corresponds to a variable, and the arrows between variables represent the causal relationship from cause to effect. Figure 1 The schematic diagram of the structural causal model SCM of network attacks in the prior art is shown in FIG. Figure 2 In this example, R represents the confusion factor, A1 and A2 represent the labels of network attacks, and F c1 ~F c3 Indicates the effective feature, F n Represents a noise feature. In the SCM of network attacks, it is an exogenous latent variable, which is composed of factors such as firewalls, antivirus software or network hardware devices. These factors will be different in different network environments or time periods, and may cause traffic anomalies. Represents a known / unknown network attack; is a feature extracted from network traffic data packets, which indicates that the feature has a causal relationship with the network attack, and is a noise feature caused by factors unrelated to the network attack such as the network itself or the deployment environment. Based on Figure 1 , we can make the following assumptions: Assumption 1: If Figure 1 , assuming 0 is normal and 1 is abnormal. According to the prior art, the effective feature F c The relationship between the label A and the network attack is stable and explainable in different environments, so if the effective feature F c =1, then we can infer that label A = 1, otherwise we can know that label A = 0. However, the characteristic value of the noise feature itself has nothing to do with the known attack type, even if we know the noise feature F n =1, nor can we infer that label A = 1. Assumption 2: Assume that the expression of traffic feature values ​​is related to network attacks, confusion factors, and exogenous latent variables. Figure 1 As shown, in <R,A2,F n >In the loop, label A2 and noise feature F n There is no causal relationship between them, but due to the existence of the confusion factor R, the label A2 and the noise feature F nA false association is established. For example, when a network is attacked by a confusing factor DDoS, we find that the traffic anomaly feature contains TCP information anomalies. However, we only know that DDoS attacks contain UDP attacks. According to the fitting of machine learning, it is mistakenly believed that UDP attacks cause TCP information anomalies. In fact, there is no causal relationship between the two. Therefore, the noise feature F n The relationship with label A is unstable and uninterpretable.

[0009] Assuming that f represents the mapping function between the traffic feature set F and the label A, we have:

[0010] A=f(F)=f(F c ,F n ),

[0011] Assume that F1 represents the F1 score for evaluating the overall performance of the model in the mapping function f. Ideally, the detected F1 score is only related to the causal features. However, in the actual deployed network environment, when training with the existing machine learning-based methods, the noise feature F is constantly fitted. n , so F n The existence of will have a certain negative impact on the detection accuracy. So there are:

[0012] F1(f(F c ,F n ))≤F1(f(F c )),

[0013] This leads to Questions 1 to 3:

[0014] Problem 1: The strong false correlation caused by noise features confuses the correspondence between cyber attacks and causal features, reducing the accuracy and stability of the detection system. At the same time, if the false correlation is not eliminated, the detection system will not be explainable.

[0015] exist Figure 1 In the example, it is assumed that it is a known attack, an unknown attack or a zero-day attack. In supervised multi-class classification, a training set containing normal traffic and known attack traffic can be generated using known attack traffic. That is:

[0016] N train ={B,A 2i ,...,A 2k},

[0017] In the above formula, B represents the type set of normal traffic, A 2i ,...,A 2k Indicates the k known attack types in A2. In the multi-class classification of network attack detection, the default detection set N is usually test With Ntrain However, if Figure 1 As shown in the figure, in the actual detection environment, there are not only samples composed of normal traffic and A2, but also an unknown attack set A1 composed of K (K>0) types without prior knowledge. That is, there are:

[0018] N test ={B,A 2i ,...,A 2k ,A 1i ,...,A 1K}≠N train .

[0019] Question 2: In the actual detection goal, not only should the detection samples be classified according to prior knowledge, but also whether the abnormal traffic is an unknown attack type should be detected and classified. However, in the actual detection, supervised learning can only classify the detection samples into known attack types according to prior knowledge, while unsupervised learning can only judge whether the traffic is abnormal and lacks incremental learning ability. The confusion matrix of network attack detection is shown in the following table:

[0020] Label normal abnormal normal TP FN abnormal FP TN

[0021] In the confusion matrix in the table above, the rows represent predicted values ​​and the columns represent true values. TP is the true positive class; FN is the false negative class; FP is the false positive class; TN is the true negative class. In network anomaly detection, precision is usually very important, and its calculation function expression is as follows: Precision = TP / ((TP+FN)).

[0022] Question 3: Although there is usually a high tolerance for precision in machine learning, the tolerance for precision in network security is extremely low. For example, in intelligent driving, if an attack is successful, there will be a risk of car crash and death. Especially for the actual deployed network, its traffic scale is huge and the attacks are frequent, so even a very small false negative class FN will allow a large number of attacks to enter, causing huge losses to the network. Summary of the invention

[0023] Technical problem to be solved by the present invention: In view of the above-mentioned problems in the prior art, a causal converter is provided to detect network attacks using causal reasoning. The present invention aims to guide the classifier to decouple false associations through causal reasoning, thereby achieving intelligent and explainable IOT network attack detection that can both identify known network anomalies and classify unknown anomalies.

[0024] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0025] A causal transformer uses causal reasoning to detect network attacks, comprising the following steps:

[0026] S1, using a classifier based on supervised learning training, inputs the detection set consisting of samples of known and unknown attack types into the classifier for classification, and obtains multiple known attack types;

[0027] S2, the causal transformer based on semi-supervised learning classifies each known attack type into known attacks and unknown attacks;

[0028] S3, the causal transformer based on unsupervised learning classifies all unknown attacks into multiple unknown attack types.

[0029] Optionally, step S1 includes:

[0030] S1.1, learn causal effect weights W for features in a training set of known attack types:

[0031]

[0032] In the above formula, p is the dimension of the feature, E represents the expected value, It means that except for the i-th feature which is set to normal, the values ​​of other features remain unchanged. It means that except for the value of the i-th feature, all other features are set to normal by intervention, and ∑w is the p elements W1~W in the causal effect weight W. p The diagonal matrix diag(W1,W2,…,W p );

[0033] S1.2, causal features are selected from the features in the training set of known attack types based on the causal effect weight W;

[0034] S1.3, select causal features from the features in the training set to train the classifier, so that the classifier establishes a mapping relationship between the input causal features and the known attack types;

[0035] S1.4, input the detection set consisting of samples of known and unknown attack types into the trained classifier for classification, and obtain multiple known attack types.

[0036] Optionally, step S1.2 includes: according to Identify the noise feature F n and the causal feature F c ,in represents the expected value of network anomaly under abnormal eigenvalues, w represents the diagonal matrix of causal effect weight W, E(F c W T ) represents the expected value of network anomaly under causal anomaly characteristics.

[0037] Optionally, the classifier in step S1.3 is an XGBoost classifier, and the output obtained after t iterations of the causal feature sample XGBoost classifier and the XGBoost classifier is The function expression is:

[0038]

[0039] In the above formula, f (k) represents the model of the kth tree of the XGBoost classifier, xc i represents the sample after retaining only the causal features, Wc represents the causal effect weight of the causal features, is the output after the t-1th iteration, f (t) Represents the model of the tth tree of the XGBoost classifier.

[0040] Optionally, when selecting causal features from the features in the training set to train the classifier in step S1.3, the loss function used is And the objective function Obj (t) The function expression is:

[0041]

[0042] In the above formula, p represents the dimension of the sample; It means that the feature of the i-th dimension is intervened to make it normal, with a value of 0; w represents the diagonal matrix of the causal effect weight W; W represents the causal effect weight; It means that the features other than the i-th dimension are intervened to make them normal, and the value is 0; the causal feature F is selected c When training the classifier, the objective function Obj is used (t) The function expression is: m represents the number of samples, Represents the loss contribution of a single sample, y i represents the true label of the i-th sample, is the output after the t-1th iteration, f (t) represents the model of the t-th tree of the XGBoost classifier, xc i represents the sample after retaining only the causal features, Wc represents the causal effect weight of the causal features, Ω(f (t) ) represents the regularization term, which is used to control the model complexity.

[0043] Optionally, step S2 includes:

[0044] S2.1, for sample X, extract the causal effect weight W and the causal feature independence weight G according to the following formula:

[0045]

[0046] In the above formula, p is the dimension of the feature, E represents the expected value, It means that except for the i-th feature which is set to normal, the values ​​of other features remain unchanged. It means that except for the value of the i-th feature, all other features are set to normal by intervention, and ∑w is the p elements W1~W in the causal effect weight W. p The diagonal matrix diag(W1,W2,…,W p ); Indicates that the j-th row sample is set to 0 by intervention, Indicates that all samples except the j-th row sample are set to 0, ∑g is the m elements G1~G in the causal feature independent weight G m The diagonal matrix diag(G1,G2,...,G m ); I represents all samples, i represents the i-th feature, and j represents the j-th row of samples;

[0047] S2.2, the causal effect weight W and the causal feature independence weight G are multiplied by MatMul for the first time and then multiplied by MatMul for the second time with the sample X;

[0048] S2.3, the result obtained by the second matrix multiplication MatMul is subjected to causal association CA and then subjected to the third matrix multiplication MatMul with the sample X. The function expression of the causal association CA is:

[0049]

[0050] In the above formula, CA(Q,K,V) is the output of causal association CA, Q, K, V represent query, key and value, G l,T is the transpose of the causal feature-independent weight G used by layer l to balance the training samples, W l is the causal effect weight W of layer l, X l,T The transpose of the input feature vector or sequence for layer l, p is the dimension of the feature, X l is the sample X of layer l;

[0051] S2.4, the sample X is processed by the prior association PA and Minmax algorithm for sample reconstruction, and the result obtained by the third matrix multiplication MatMul is fused through the addition and normalization module ADD&Norm to obtain forward feedback; the function expression of the prior association PA is:

[0052]

[0053] In the above formula, X l i is the empirical standard deviation of the ith feature in the lth layer, Fi is the eigenvector of the i-th feature, F j is the feature vector of the jth feature, where j≠i;

[0054] S2.5, the forward feedback outputted in step S2.4 and the sample X are fused through the addition and normalization module ADD&Norm to obtain a new forward feedback, and then the new forward feedback is activated through the addition and normalization module ADD&Norm, the linear module and Softmax, and then subtracted from the sample X to obtain a detection result Y, where the detection result Y is a known attack or an unknown attack under a known attack type corresponding to the sample X.

[0055] Optionally, step S3 includes:

[0056] S3.1, aggregate all unknown attacks under known attack types to obtain an unknown attack set;

[0057] S3.2, extract the causal effect weight W and the causal feature independence weight G for all unknown attacks in the unknown attack set to obtain the causal distribution state of the unknown attack:

[0058]

[0059] In the above formula, p is the dimension of the feature, E represents the expected value, It means that except for the i-th feature which is set to normal, the values ​​of other features remain unchanged. It means that except for the value of the i-th feature, all other features are set to normal by intervention, and ∑w is the p elements W1~W in the causal effect weight W. p The diagonal matrix diag(W1,W2,…,W p ); Indicates that the j-th row sample is set to 0 by intervention, Indicates that all samples except the j-th row sample are set to 0, ∑g is the m elements G1~G in the causal feature independent weight G m The diagonal matrix diag(G1,G2,...,G m ); I represents all samples, i represents the i-th feature, and j represents the j-th row of samples;

[0060] S3.3, reconstruct samples of the unknown attack set using the Minmax algorithm. If the error distribution of the unknown attack set before and after reconstruction is less than the preset threshold, the unknown attack set is determined to be indivisible, and the process ends and exits. Otherwise, the unknown attack set is determined to be divisible, and the reconstructed unknown attack set is divided into unknown attack type i and unknown attack type j. For the unknown attack type i and the unknown attack type j, the corresponding unknown attacks are respectively summarized to obtain a new unknown attack set and jump to step S3.2.

[0061] In addition, the present invention also provides a causal converter using causal reasoning to perform network attack detection system, including a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to execute the causal converter using causal reasoning to perform network attack detection method.

[0062] In addition, the present invention also provides a computer-readable storage medium, which stores a computer program or instruction, and the computer program or instruction is programmed or configured to execute the causal converter through a processor to perform a network attack detection method using causal reasoning.

[0063] In addition, the present invention also provides a computer program product, including a computer program or instructions, which are programmed or configured to execute the causal converter through a processor to perform a network attack detection method using causal reasoning.

[0064] Compared with the prior art, the present invention mainly has the following advantages: Aiming at the problem that it is difficult for IOT intrusion detection systems to take into account both network anomaly identification and classification, especially the classification of unknown attacks, which brings huge risks to the use of IOT devices, the present invention guides the classifier to use causal reasoning to detect network attacks through causal reasoning, and can decouple false associations in order to achieve an intelligent and explainable IOT detection that can both identify network anomalies and effectively classify anomalies. It can not only correctly classify known attacks in IOT, but also achieve 100% of network attacks on some data sets without being missed, and can effectively realize the identification and classification of more than 99% of unknown attack types, providing timely and effective guidance for network security defense. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 Schematic diagram of the structural causal model (SCM) of network attacks in the prior art.

[0066] Figure 2 This is an example of a directed acyclic network attack SCM in the prior art.

[0067] Figure 3 This is an example of indirect causation in the prior art.

[0068] Figure 4 Schematic diagram of the basic flow of the method of the embodiment of the present invention.

[0069] Figure 5 Schematic diagram of the process of supervised learning in an embodiment of the present invention.

[0070] Figure 6 Schematic diagram of the structure of the causal converter in an embodiment of the present invention.

[0071] Figure 7 Schematic diagram of the process of step S2 in the embodiment of the present invention.

[0072] Figure 8 Schematic diagram of the process of step S3 in the embodiment of the present invention.

[0073] Fig. 9 It is the detection confusion matrix on the Darknet dataset in an embodiment of the present invention.

[0074] Fig.10 It is the detection confusion matrix on the MalMem dataset in an embodiment of the present invention. DETAILED DESCRIPTION

[0075] Causal inference is to use data from experimental and observational studies to evaluate the causal effects between variables and to explore the causal relationships between multiple variables. When solving the causal relationship between features, intervention operations can be used to estimate the results of an intervention on a certain attack, thereby exploring whether the attack is the cause of the feature abnormality.

[0076] Definition 1 (do operation) do(A=a) means to perform causal intervention on A so that all network attacks A are equal to a specific number a or obey a certain distribution a; PF|do(A=a)) represents the probability that feature F is abnormal after intervention on network attack A[10-11], denoted as P(F|A a ). The do operator indicates that active intervention is being performed rather than passive observation, that is:

[0077] P(F|A a )≠P(F|A=a),

[0078] This concept is impossible to involve in classical statistics. The expected effect of one or more operations on a system is called potential result.

[0079] Assumption 3: Causal inference has a very important assumption of stable unit processing value, which includes two parts: assuming that there may be equal network attacks in the network, if , then no matter how it changes, the potential results of are not affected; the operation accepted by each network attack is unique, and the potential results of the characteristics caused are also unique. That is, either observe the distribution of characteristics when the network is attacked; or observe the distribution of characteristics when the network is normal; it is impossible to observe the distribution of characteristics when and at the same time.

[0080] Definition 2 (Causality): The core of causal effect is to compare the abnormality of each traffic feature when the same network attack is intervened and when it is not intervened, and to believe that the difference between whether a feature is abnormal or not is the effect of intervention relative to no intervention. For example, if, then we can say that the network attack causes the abnormality of traffic feature F. Figure 2 As shown, if the label A is known, the feature F x and F y are independent and identically distributed, then:

[0081] P(F x ,F y |A)=P(F x |A)P(F y |A),

[0082] In the above formula, P(F x ,F y |A) represents feature F x ,F y Regarding the probability distribution of label A, P(F x |A) is feature F x Regarding the probability distribution of label A, P(F y |A) is feature F y Regarding the probability distribution of label A, then we have:

[0083] E(F x ,F y |A)=E(F x |A)E(F y |A),

[0084] In the above formula, E(F x ,F y |A) represents feature F under label A x ,F y The expected value of E(F x |A) represents feature F under label A x The expected value of E(F y |A) represents feature F under label A y Similarly, if the label A is known, E(F x ,F y |A)=E(F x |A)E(F y |A), then we can infer that F x and F y are independent and identically distributed.

[0085] However, in the actual network attack structure, there is no ideal directed acyclic structural causal model SCM, such as Figure 2 , Fx and F y In order to make F x and F y Independent of each other, F can be interrupted by introducing a causal effect weight W x and F y association, making them independent of each other.

[0086] Definition 3: For two non-IID features F x and F y , if there exists a p×1-order weight matrix W such that E(F x ∑WF y |A)=E(F x W|A)E(F y W|A)(where ∑W is a p×p order diagonal matrix, and W i,1 =(∑W) i,i ), then there exists W such that F x With F y Independent of each other.

[0087] In the causal structural model, according to Definition 2, the causal effect between network attack A and a single traffic feature is assumed to be:

[0088] ICE(AF1)=P(F1|A 1 )-P(F1|A 0 ),

[0089] In the above formula, ICE(AF1) represents the causal effect between the network attack A and the feature F1, P(F1|A 1 ) represents the probability distribution of feature F1 under the condition of intervention setting A = 1, P(F1|A 0 ) represents the probability distribution of feature F1 under the condition of intervention setting A=1. However, in network SCM, the dimension of traffic features corresponding to network attacks is p, and p>>1. If you want to quantitatively analyze the causal relationship between the observed feature set F and the network attack A, you need to consider the average causal effect between the network attack A and the feature set F. In order to better analyze the causal effect between network attacks and features in the dataset, we flip the causal direction between network attack A and feature set F, and judge the change of A by intervening in F. So we define the average causal effect size between feature set F and network attack A to judge the causal relationship between network attack A and feature set F, as shown in the following formula:

[0090]

[0091] In the above formula, ACE(ICE(FA)) represents the causal relationship between network attack A and feature set F, ICE(FA) represents the causal effect between network attack A and feature set F, p represents the dimension of feature set F, Representation feature F i The causal effect value between the result label A, In the intervention setting F i =1, the expected value of feature A. In the intervention setting F i =0. We assume that the average result E(A|F) when the feature is abnormal (F=1) is 1 ) and the average result E(A|F) when the network characteristics are assumed to be normal (F = 0) 0 ) is the average causal effect ACE between F and A. A threshold can be set in advance. If it is greater than a certain threshold, it is considered that the network attack caused the occurrence of the feature set. Otherwise, it is considered that there is no causal relationship between the network attack and the feature value set. According to the above formula, the greater the causal effect between A and F, the more the change of A can affect the change of F, and the greater the weight of A's influence on F should be. Conversely, the smaller the causal effect, the smaller the weight. Although the above formula calculates the causal effect between each feature, in the actual causal structure model SCM network, it is often in F. x and F y There is a connection between them, which makes the causal effect weight we obtained cannot truly reflect the causal effect between network attacks and features. However, in the actual solution, Figure 3 As shown, if F is known c Under the premise that x )⊥P(F y |F x ), then ICE AF y )=0, where “⊥” means independence, where P(A|F x ) represents the feature F x The probability distribution of A under the condition, P(F y |F x ) represents the feature F x Under the condition F y The probability distribution of ICE AF y ) represents the result label and feature F y The average causal effect between .

[0092] The prior art proposes For all x, there exists an m×1-dimensional weight G that balances all training samples X so that all features are independent of each other, where Represents feature X iThe empirical probability estimate of the value x.

[0093] The prior art (Luo, Yunan, Jian Peng, and Jianzhu Ma. "When causal inference meets deep learning." Nature Machine Intelligence 2.8 (2020): 426-427.) records the intervention of samples by the do operator By learning The causal feature independence weight G of the sample keeps all features independent, and a variable decorrelation regularizer is proposed to learn the sample weight, as shown in the following formula:

[0094]

[0095] In the above formula, G is the causal feature independent weight, p is the feature dimension of the sample, E is the expected value, which is an m×p-dimensional sample matrix, and the causal feature independent weight G is an m×1-dimensional sample weight matrix. ∑g=diag(G1,G2,...,G m ). I represents all samples; j represents the j-th row sample; It means that except for the sample in the jth row, all other samples are set to 0 by intervention; Indicates that the j-th row sample is intervened and set to 0; T indicates the transposition of the matrix. In the above formula, by focusing on the first moment of the X sample matrix in the training environment, and then reweighting through the causal intervention do operation to keep the features F in X independent, thereby reducing the correlation between features in the training environment and the false correlation between network attacks and features, ensuring the accuracy and uniqueness of the causal effect calculation between A and F.

[0096] Definition 4: (Causal effect weight) Give each feature a corresponding weight according to the size of the causal effect between each feature and the label. We call this weight the causal effect weight. According to Definition 4, theoretically, if the causal feature F c and noise characteristics F n The independence relationship is stable, then:

[0097] E(A|F c ,F n )=E(A|F c )E(A|F n ),

[0098] In the above formula, E(A|F c ,F n ) indicates F c and F n The expected value of A under the feature set composed of c) represents the causal feature F c The expected value of A under the set, E(A|F n ) represents the noise characteristic F n The expected value of A under the set. After the samples are weighted by the causal feature independence weight G, the features remain independent of each other and have no influence on the causal effect of the network attack, so the above formula can be converted into:

[0099] E(GF c ,GF n |A)=E(GF c |A)E(GF n |A),

[0100] In the above formula, E(GF c ,GF n |A) represents independent samples under the result label A (feature GF c ,GF n The expected value of the complete independent sample), E(GF c |A) represents the expected value of the causal independent feature under the condition of the result label A, E(GF n |A) represents the expected value of the noise-independent feature under the result label A, GF c Represents the effective feature (causal feature) F of sample X c The feature after adding the independent weight G of the causal feature, GF n Represents the noise feature F of sample C n The feature after adding the independent weight G of the causal feature, GF c represents the causal feature set, GF n Represents the noise feature set. Since the network attack A is known, A in the above formula is ignored when solving the noise feature, and it can be simplified as follows:

[0101] E(G(F c ,F n ))=E(GF c )E(GF n ),

[0102] In the above formula, E(G(F c ,F n )) represents the expected value of independent samples, E(GF c ) represents the expected value of the causally independent feature,

[0103] E(GF n ) represents the expected value of the noise independent feature. However, in actual sample data, GF c represents the causal feature set, GF nrepresents the noise feature set, and the two are not independent, and we do not know in advance which feature sets are causal or noise. Therefore, in order to filter out the noise features and make the above formula valid in real samples, we add weights to the features according to Definition 4, so that the causal feature set and the noise feature set remain independent:

[0104]

[0105] In the above formula, W is the weight of p×1 dimension, ∑w is diag(W1,W2,…,W p ), T is the transposition function. In order to consistently estimate the causal effect, most equilibrium methods use moments to describe the distribution, and through the do operation, the average difference between intervention and non-intervention can be compared to estimate the causal impact of the feature on the network attack. Therefore, in order to accurately find the causal feature and the noise feature, the causal effect weight W can be learned through the do operation:

[0106]

[0107] In the above formula, p is the dimension of the feature, E is the expected value, G is the independent weight of the causal feature, I is all the features, and i is the i-th feature; It means that except for the i-th feature which is set to normal by intervention, the values ​​of other features remain unchanged; It means that except for the value of the i-th feature, the rest of the features are set to normal. It can be seen that the causal weight can enhance the influence of causal features on labels, decouple the pseudo-correlation between noise features and labels, and improve the stability of the system. For example, the noise feature F that has no causal effect on label A n , its causal weight should be 0, regardless of the noise feature F n No matter how it changes, the impact of the re-weighted judgment on the result label is f(F n )*0=0). Therefore, in this embodiment, each traffic feature F is regarded as a processing variable in turn, and all features are causally re-weighted through causal intervention, so as to improve the stability of the detection system.

[0108] like Figure 4 As shown, the causal converter of this embodiment uses causal reasoning to perform a network attack detection method. This embodiment method includes the following steps:

[0109] S1, using a classifier based on supervised learning training, inputs the detection set consisting of samples of known and unknown attack types into the classifier for classification, and obtains multiple known attack types;

[0110] S2, the causal transformer based on semi-supervised learning classifies each known attack type into known attacks and unknown attacks;

[0111] S3, the causal transformer based on unsupervised learning classifies all unknown attacks into multiple unknown attack types.

[0112] like Figure 4 As shown, the causal transformer of this embodiment uses causal reasoning to perform network attack detection method, which aims to solve the false association between network attacks and features and the classification problem of known / unknown attacks through three modules based on classifiers and causal transformers (known attack classification based on supervised learning, known unknown attack classification based on semi-supervised learning, and unknown attack incremental learning based on unsupervised learning), wherein: supervised learning module: first, N types of known attack samples and normal traffic samples are used as training sets, and causal effect weights are trained based on causal reasoning, and then noise features are removed and causal features are screened out based on the size of the causal effect weights, and finally a classifier is trained based on a supervised method, and finally the detection set composed of known / unknown attack types is input into the classifier for classification. XGBoost semi-supervised learning module: Since in the supervised learning module, all unknown attack types are classified as known attack types based on XGBoost, in order to screen out unknown attack types from known attack types, this module is screened based on the causal transformer of semi-supervised learning. First, based on known attack types, causal effect weights and sample weights are trained through causal reasoning to guide the attention of the transformer to learn the causal distribution characteristics of known attack type samples. Then, based on the prior association, the gap between the causal relationship between normal samples and abnormal samples is widened. Secondly, the difference between the reconstructed output and input is used to determine whether the detected sample is an outlier different from the known attack type. Finally, the known attacks and unknown attacks are classified. Unsupervised learning module: This part is for the classification of unknown attack types determined by the semi-supervised learning module. After screening out the unknown attack types, the unknown attacks are aggregated together, and then the reconstruction error learned by the unsupervised causal transformer is used as the threshold for pairwise classification. Therefore, this module has the ability of incremental learning and can provide classified unknown or zero-day attack type samples for the sample library of known attack types.

[0113] like Figure 5 As shown, step S1 of this embodiment includes:

[0114] S1.1, learn causal effect weights W for features in a training set of known attack types:

[0115]

[0116] In the above formula, p is the dimension of the feature, E represents the expected value, It means that except for the i-th feature which is set to normal, the values ​​of other features remain unchanged. It means that except for the value of the i-th feature, all other features are set to normal by intervention, and ∑w is the p elements W1~W in the causal effect weight W. p The diagonal matrix diag(W1,W2,…,W p );

[0117] S1.2, causal features are selected from the features in the training set of known attack types based on the causal effect weight W;

[0118] S1.3, select causal features from the features in the training set to train the classifier, so that the classifier establishes a mapping relationship between the input causal features and the known attack types;

[0119] S1.4, input the detection set consisting of samples of known and unknown attack types into the trained classifier for classification, and obtain multiple known attack types. Figure 5 In the figure, they are respectively represented as known type 1 to known type n.

[0120] According to the causal effect, if a feature is a noise feature, its causal effect is 0. And because E(0) = 1, after the causal weights of the samples are assigned by the learned causal effect weight W:

[0121]

[0122] If the above equation holds, then E(A|do(F n =1))-E(A|do(F n =0))=0, so it can be considered that F n is the noise feature we need to remove, F c The causal feature we need to select. Therefore, as an optional implementation, step S1.2 of this embodiment includes: Identify the noise feature F n and the causal feature F c ,in

[0123] represents the expected value of network anomaly under abnormal eigenvalues, w represents the diagonal matrix of causal effect weight W,

[0124] E(F c W t ) represents the expected value of network anomaly under causal anomaly characteristics.

[0125] As an optional implementation, the classifier in step S1.3 of this embodiment is an XGBoost classifier, and the output obtained by the XGBoost classifier after t iterations after the causal features are input into the XGBoost classifier is The function expression is:

[0126]

[0127] In the above formula, f (k) represents the model of the kth tree of the XGBoost classifier, xc i represents the sample after retaining only the causal features, Wc represents the causal effect weight of the causal features, is the output after the t-1th iteration, f (t) Represents the model of the tth tree of the XGBoost classifier.

[0128] As an optional implementation, in step S1.3 of this embodiment, when selecting causal features from the features in the training set to train the classifier, the loss function used is And the objective function Obj (t) The function expression is:

[0129]

[0130] In the above formula, p represents the dimension of the sample; It means that the feature of the i-th dimension is intervened to make it normal, with a value of 0; w represents the diagonal matrix of the causal effect weight W; W represents the causal effect weight; It means that the features other than the i-th dimension are intervened to make them normal, and the value is 0; the causal feature F is selected c When training the classifier, the objective function Obj is used (t) The function expression is: m represents the number of samples, Represents the loss contribution of a single sample, y i represents the true label of the i-th sample, is the output after the t-1th iteration, f (t) represents the model of the t-th tree of the XGBoost classifier, xc i represents the sample after retaining only the causal features, Wc represents the causal effect weight of the causal features, Ω(f (t) ) represents the regularization term, which is used to control the complexity of the model. (t) The smallest, then XGBoost has the best tree structure. The more accurate.

[0131] like Figure 6 and Figure 7 As shown, step S2 in this embodiment includes:

[0132] S2.1, for sample X, extract the causal effect weight W and causal feature independence weight G according to the following formula:

[0133]

[0134] In the above formula, p is the dimension of the feature, E represents the expected value, It means that except for the i-th feature which is set to normal, the values ​​of other features remain unchanged. It means that except for the value of the i-th feature, all other features are set to normal by intervention, and ∑w is the p elements W1~W in the causal effect weight W. p The diagonal matrix diag(W1,W2,…,W p ); Indicates that the j-th row sample is set to 0 by intervention, Indicates that all samples except the j-th row sample are set to 0, ∑g is the m elements G1~G in the causal feature independent weight G m The diagonal matrix diag(G1,G2,...,G m ); I represents all samples, i represents the i-th feature, and j represents the j-th row of samples;

[0135] S2.2, the causal effect weight W and the causal feature independence weight G are multiplied by MatMul for the first time and then multiplied by MatMul for the second time with the sample X;

[0136] S2.3, the result obtained by the second matrix multiplication MatMul is subjected to causal association CA and then subjected to the third matrix multiplication MatMul with the sample X. The function expression of the causal association CA is:

[0137]

[0138] In the above formula, CA(Q,K,V) is the output of causal association CA, Q, K, V represent query, key and value, G l,T is the transpose of the causal feature-independent weight G used by layer l to balance the training samples, W l is the causal effect weight W of layer l, X l,T The transpose of the input feature vector or sequence for layer l, p is the dimension of the feature, X l is the sample X of layer l;

[0139] S2.4, the sample X is processed by the prior association PA and Minmax algorithm for sample reconstruction, and the result obtained by the third matrix multiplication MatMul is fused through the addition and normalization module ADD&Norm to obtain forward feedback; the function expression of the prior association PA is:

[0140]

[0141] In the above formula, X l i is the empirical standard deviation of the ith feature in the lth layer, F i is the eigenvector of the i-th feature, F j is the feature vector of the jth feature, where j≠i;

[0142] S2.5, the forward feedback outputted in step S2.4 and the sample X are fused through the addition and normalization module ADD&Norm to obtain a new forward feedback, and then the new forward feedback is activated through the addition and normalization module ADD&Norm, the linear module and Softmax, and then subtracted from the sample X to obtain a detection result Y, where the detection result Y is a known attack or an unknown attack under a known attack type corresponding to the sample X.

[0143] like Figure 6 As shown, the causal transformer in this embodiment is a transformer based on causal attention, which is mainly composed of a causal attention mechanism module and a reconstruction module. The causal attention mechanism module is used to execute steps S2.1 to S2.4, and the reconstruction module is used to execute step S2.5 to obtain the final detection result Y.

[0144] In this embodiment, the network traffic is regarded as a time series, and the traffic feature is a time point. Then, the transformer is used to train and detect whether the network traffic is abnormal. The transformer connects different positions of a sequence through the attention mechanism to calculate a model of the sequence representation. It forces the model to learn to focus on specific parts of the input sequence when decoding, but because of the existence of false correlations, it cannot effectively pay attention to the causal distribution of the sequence. For example, during a shopping festival on a certain website, based on its past history, due to hardware and software limitations, the website would crash when a large number of people flocked to it. However, the hardware and software have been expanded, but they still pay too much attention to the impact of the influx of people on the website and ignore hacker attacks. Figure 6 As shown, in order to solve the problem of false correlation in the self-attention mechanism, this embodiment proposes to add causal weights to the self-attention mechanism to guide the model to learn to focus on the causal feature part with greater causal effect of the input sequence during encoding and decoding, while ignoring the interference of false correlation on attention. First, initialize Q, K, V of the attention mechanism as Where l represents the lth layer of the attention mechanism, W Q ,W K ,W Vis the matrix parameter of the encoder. As shown in the following formula, in this embodiment, the original query vector Q is replaced by the matrix multiplication of the causal weights:

[0145] Q=G l,T W l ,

[0146] In the above formula, G l,T G l The transpose of G l For the lth layer X l The causal feature independence weights G, W l is the sample X of the lth layer l The weight of the causal effect.

[0147] As shown in the following formula, the causal factors learned from the query vector Q are used to obtain the information that the attention mechanism needs to pay attention to, and then the information selection mechanism is summarized to form the causal sequence attention distribution:

[0148]

[0149] In the above formula, Causal_att(Q,K) is the causal sequence attention distribution, X l,T For X l The transpose of , softmax(·) is normalized along the last dimension, and then a discrete distribution based on the causal sequence is formed with p as the dimension.

[0150] Then the causal sequence attention distribution learns the causal relationship from the original sequence, as shown in the following formula:

[0151] Association) adaptively finds the most effective causal association (Causal Association):

[0152]

[0153] In the above formula, CA(Q,K,V) is the output of causal association CA, Q, K, V represent query, key and value, G l,T is the transpose of the causal feature-independent weight G used by layer l to balance the training samples, W l is the causal effect weight W of layer l, X l,T The transpose of the input feature vector or sequence for layer l, p is the dimension of the feature, X l is the sample X of layer l.

[0154] In order to increase the gap between causal association and spurious association, Figure 6 As shown, a prior correlation is used in the attention mechanism, which can be expressed as:

[0155]

[0156] In the above formula, X l i is the empirical standard deviation of the ith feature in the lth layer, F i is the eigenvector of the i-th feature, F j is the feature vector of the jth feature, where j≠i.

[0157] The starting point of the above causal association and prior association is to make the samples with causal association with the labels of the training data more obvious as normal, and the samples with false association with the attack labels more obvious as abnormal. Therefore, the function expression of the combined causal association and prior association is used to optimize the reconstruction error. The loss function used by the causal converter is for:

[0158]

[0159] In the above formula, is the output of the last layer of the reconstruction module, X is the input, λ is the regularization coefficient used to balance the weight between the reconstruction error and the association difference loss, ADis is the symmetric KL distance between the causal association CA and the prior association PA, ||.|| F For Frobenius, ||.|| k is the k-norm. When λ>0, if the model minimizes the loss, it needs to maximize the association difference. Directly maximizing the association difference will cause the scale parameter of the Gaussian kernel to decrease sharply, resulting in meaningless prior distribution. Therefore, in order to better control the process of association learning, the Minimax algorithm is used in this embodiment to optimize the loss function by controlling the back propagation of the causal association CA and the prior association PA.

[0160] When λ<0, the back propagation of the causal association CA is stopped and in the minimization stage, the prior association PA is optimized to make the prior association approximate the causal association CA learned from the original sample. This process will make the prior association adapt to different sample patterns. Its function expression is:

[0161]

[0162] In the above formula, is the total loss function in the minimization phase, ADis detachCA The L1 norm of the association difference calculated after stopping the back-propagation of the causal association CA in the minimization phase. detachCA means stopping the back-propagation of the causal association CA and being in the minimization phase.

[0163] When λ>0, stop optimizing the prior association PA to maximize the difference in causal features between normal samples and abnormal samples. This process is in the maximization stage, which will make the causal association pay more attention to the features with the largest causal effect on the label, making it more difficult to reconstruct features with false associations. Its function expression is:

[0164]

[0165] In the above formula, is the total loss function of the maximization stage, ADis detachPA It is the L1 norm of the association difference calculated after stopping the back-propagation of the prior association PA in the maximization stage, and detachPA means stopping the optimization of the prior association PA.

[0166] First, the causal feature independent weight G of each causal transformer is trained according to the known sample type, and the loss function is optimized according to the calculation formula of the causal feature independent weight G for:

[0167]

[0168] After ensuring the independence of each feature through the causal feature independence weight G, according to the formula E(GF c ,F n )=E(GF c )E(GF n ) trains the causal effect weights, if we assume Then its loss function for:

[0169]

[0170] In order to widen the difference between normal samples (with causal distribution) and abnormal sample distribution (with false correlation distribution) in the causal transformer model, this embodiment optimizes the model by using the Minimax algorithm consisting of the total loss function in the minimization stage and the total loss function in the maximization stage, as shown in the following formula:

[0171]

[0172] In the above formula, is the total loss function under the Minimax strategy, which is used to optimize the causal feature differences between normal samples and abnormal samples. maxdetach(PA) is to stop the back propagation of the prior association PA in the maximization stage, which is used to calculate the association difference. maxdetach(G,W,CA) is to stop the back propagation of the sample weight G, the causal effect weight W and the causal association CA in the maximization stage, which is used to optimize the causal association. λ is a hyperparameter, X is the sample, and CA is the causal association. Under the Minimax algorithm, all detected samples pay more attention to the neighboring areas with stronger causal effect characteristics driven by the causal association difference loss. The error between normal samples that are consistent with the training sample distribution and the reconstructed samples will be reduced, while the gap between abnormal samples that are inconsistent with the training sample distribution and the reconstructed samples will be larger. The samples reconstructed by the above formula are After that, it is normalized by adding the original sample through the point-wise feedforward network, as shown in the following formula:

[0173] out1=BNorm(PA(X)+(ffn(CA(Q,K,V))=>dropout)),

[0174] out2=BNorm(X+(ffn(out1)=>dropout)),

[0175] In the above formula, out1 is the output after the prior association PA and causal association CA, plus the feedforward neural network ffn and the dropout layer, and then normalized by batch normalization BNormBNorm, out2 is the input X plus the output after out1 processing and then the feedforward neural network ffn and the dropout layer, and finally normalized by batch normalization BNorm, BNorm is Batch Normalization, dropout is the pre-set drop threshold, ffn is the feedforward neural network, PA(X) is the result of sample X after the prior association PA. Finally, the sample re-characterized by the causal transformer is reconstructed with the original sample, and the output prediction value pred is:

[0176]

[0177] In the above formula, ADis is the attention distance, ⊙ is the element-wise multiplication, and F i,: is the i-th feature of F, outF2 i,: is the i-th feature of outF2, F represents the feature value of the original sample, outF2 represents the feature value of the reconstructed sample, It means that each feature of the detection sample is judged. If a feature is abnormal, it usually has a higher false association and causal association difference. Therefore, this design can make the reconstruction error and the association error work together to improve the detection performance. The larger the reconstruction error, the larger the value of the predicted value pred, indicating that the sample is more likely to be an abnormal traffic sample.

[0178] like Figure 8 As shown, step S3 in this embodiment includes:

[0179] S3.1, aggregate all unknown attacks under known attack types to obtain an unknown attack set;

[0180] S3.2, extract the causal effect weight W and the causal feature independence weight G for all unknown attacks in the unknown attack set to obtain the causal distribution state of the unknown attack:

[0181]

[0182] In the above formula, p is the dimension of the feature, E represents the expected value, It means that except for the i-th feature which is set to normal, the values ​​of other features remain unchanged. It means that except for the value of the i-th feature, all other features are set to normal by intervention, and ∑w is the p elements W1~W in the causal effect weight W. p The diagonal matrix diag(W1,W2,…,W p ); Indicates that the j-th row sample is set to 0 by intervention, Indicates that all samples except the j-th row sample are set to 0, ∑g is the m elements G1~G in the causal feature independent weight G m The diagonal matrix diag(G1,G2,...,G m ); I represents all samples, i represents the i-th feature, and j represents the j-th row of samples;

[0183] S3.3, reconstruct samples of the unknown attack set using the Minmax algorithm. If the error distribution of the unknown attack set before and after reconstruction is less than the preset threshold, the unknown attack set is determined to be indivisible, and the process ends and exits. Otherwise, the unknown attack set is determined to be divisible, and the reconstructed unknown attack set is divided into unknown attack type i and unknown attack type j. For the unknown attack type i and the unknown attack type j, the corresponding unknown attacks are respectively summarized to obtain a new unknown attack set and jump to step S3.2.

[0184] In this embodiment, the experiment uses two widely used datasets Darknet and MalMem in the field of network attack detection as benchmark datasets, where the Darknet dataset is merged from the two datasets ISCXTor2016 and ISCXVPN2016 to create a darknet dataset covering Tor and VPN traffic. The MalMem dataset simulates the real network environment training and detection of fuzzy malware using malware that is prevalent in today's network systems, including three types of attacks: Ransomware, Spyware, and Trojan. The causal attention mechanism module (causal attention) of the causal transformer (causal transformer) contains 3 parallel layers, and the number of channels of its hidden state is set to 512 and the number of fronts is set to 8. For all datasets, the hyperparameter λ is set to 3, and the ADAM optimizer is applied for optimization, and its initial learning rate is set to. The training cycle of the causal transformer (causal transformer) is set to 10, and the batch size is set to 32. All experiments are completed on the jupyter of SitonHoly Cluster Management Platform (SCMP).

[0185] Based on prior knowledge, the test samples are classified into known attack samples and unknown attack samples. Then, the XGBoost detector is trained based on known attack samples and normal traffic samples to detect known attack samples in real time. Finally, the classifier is used to classify the known / unknown attacks of the test samples into known attack types. Figure 9-10 The following is the detection confusion matrix of the XGBoost detector for each detection set. The columns are the true labels of network traffic, the detection labels of behavioral network traffic, and the numbers on the diagonal represent the number of accurately classified flows. It can be seen that this experiment has achieved a high detection rate for known attacks. The numbers in other positions represent the number of incorrectly classified flows, so it can be seen that unknown attacks are incorrectly classified as normal traffic or known attacks in this part of the experiment.

[0186] Experiment on detecting unknown attacks from known attacks: Although a high classification accuracy rate was achieved for known attacks through the supervised learning module, all unknown attacks were inevitably misclassified as normal traffic or known attacks. In addition, in some data sets, detection samples of known attack types were also classified as normal traffic, which would lead to the emergence of problem 3. In order to solve these problems, the second stage uses a causal transformer to learn the intrinsic causal distribution of normal traffic and each type of known attack sample, so as to detect the misclassified unknown traffic samples. At the same time, in order to verify the performance of the causal transformer, this experiment compared the three indicators of accuracy, precision and F1 score with previous benchmark experiments such as TranAD, LSTM_Univariate, Attention, LSTM_AD, DAGMM, OmniAnomaly, Isolation Forest, SVM, LOF, LSTM and other existing network models. According to Fig. 9 It can be seen that 72 Tors and 449 VPNs are classified as NON-Tor. However, the results obtained after the semi-supervisory module of NON-Tor are shown in Table 1.

[0187] Table 1: Analysis of detection results of the semi-supervised module Darknet.

[0188]

[0189] In Table 1, Ac, Pr, and Fl represent accuracy, precision, and Fl scores, respectively. F1 is the harmonic mean of precision and recall. For these three indicators, the higher the value, the better the performance. As shown in Table 1, the precision is 1, that is, all Tor and VPN samples classified as NON-Tor are judged as abnormal, which is about 50% higher than the best method. Even if it cannot ensure that NON-Tor is classified as normal, it can ensure that all attacks can be detected. In the semi-supervised detector part of Non-VPN, our method can not only ensure that all anomalies are identified, but also ensure that normal traffic will not be misclassified, which is not possible for all other methods. This is because the behavior pattern of normal traffic is larger than that of attack traffic, and after optimization by the causal attention mechanism module (Causal Attention) and the Minimax algorithm, the false correlation is eliminated, and the value of the causal feature re-characterization between normal and abnormal samples is enlarged, making their differences clearer and easier to distinguish. Other methods confuse causal features with noise features. When comparing whether a sample is abnormal, they fit the noise features more. However, the expression value of the noise feature has nothing to do with whether the network is attacked.

[0190] according to Fig.10 It can be seen that a total of 7 unknown attacks in the MalMem dataset were detected as normal, but after the semi-supervised module, as shown in Table 2, its performance indicators are significantly improved compared to the most advanced methods in the past, and both normal and abnormal traffic can be identified. Due to the similarities between known Ransomware, Spyware and unknown Trojans, there is confusion in the classification of the supervised module, but after optimization by the causal attention mechanism module (Causal Attention) and the Minimax algorithm, they can be basically distinguished, achieving a high accuracy.

[0191] Table 2: Analysis of detection results of the semi-supervised module MalMem

[0192]

[0193] Anomaly detection: Compared with the relatively simple detection of known attacks, the detection of unknown attacks is more difficult, and the detection of unknown attacks is also the focus of this paper. Therefore, based on the previous experiments, in order to verify the anomaly detection ability of the causal transformer, this experimental part classifies all attack types as unknown attacks, and then inputs two types of types with normal traffic types into the semi-supervised learning module of the causal transformer for testing. The experimental results are shown in Tables 3 and 4. After training on normal traffic samples, the causal features of normal traffic samples of all data sets are obtained, and the influence of features that are not causally related to normal traffic types is eliminated or reduced, eliminating false associations. Specifically, to detect whether the network is abnormal, only the causal features of normal traffic are compared.

[0194] Table 3: Anomaly detection on the Darknet dataset

[0195] method Ac Pr F1 TranAD 0.9774 0.9646 0.9820 LSTM_Univariate 0.9798 0.9683 0.9839 Attention 0.9809 0.9700 0.9848 LSTM_AD 0.9857 0.9773 0.9885 DAGMM 0.9836 0.9741 0.9869 OmniAnomaly 0.9851 0.9764 0.9881 Causal Transformer 0.9950 0.9919 0.9959

[0196] Table 4: Anomaly detection on the MalMem dataset

[0197] method Ac Pr F1 TranAD 0.9989 0.9989 0.9994 LSTM_Univariate 0.9994 0.9994 0.9997 Attention 0.9994 0.9994 0.9997 LSTM_AD 1.0000 1.0000 1.0000 DAGMM 0.9996 0.9996 0.9998 OmniAnomaly 0.9987 0.9986 0.9993 Causal Transformer 1.0000 1.0000 1.0000

[0198] As can be seen from Tables 3 and 4, the anomaly detection scores of the causal transformer in the three major data sets are the best compared with the latest algorithms.

[0199] Unknown attack classification: As shown in Tables 1 to 4, after the semi-supervised learning module, although the traffic samples that are inconsistent with the known training data are basically distinguished, there are both unknown attacks and known attacks among them, and there may be more than one type of unknown attack. In order to distinguish the two and classify these unknown attacks at the same time, this experiment uses the unsupervised module to achieve this goal, as shown in Tables 5 and 6.

[0200] Table 5: Unknown attack classification results of the unsupervised module Darknet

[0201]

[0202]

[0203] Table 6: Unknown attack classification results of the unsupervised module MalMem

[0204]

[0205] In order to distinguish the unknown outliers detected in the Darknet dataset, this section puts the unknown abnormal traffic samples into an unsupervised module based on a causal transformer for distinction. As shown in Table 5, this method can basically distinguish known attacks Tor and unknown attacks VPN, with an accuracy of more than 99%, and an F1 value of >98%, which is at least 8% higher than the previous methods. Compared with the Darknet dataset, which has only two types of attacks, the MalMem dataset has two types of known attacks and unknown attacks, so as shown in Table 6, the three types of attacks are classified in pairs. In the classification of Ransomware and Spyware, Ransomware and Trojan, its performance indicators are all 100%, which are higher than the best methods. Even in the poor Spyware and Trojan parts, its three major performance indicators are all over 99.9%. Therefore, through the three major learning modules, the known attack types and unknown attack types are basically detected for the Darknet dataset, and the known attack types are accurately classified. It can be seen that the causal transformer of the method of this embodiment keeps the features independent after causal attention CausalAttention, and eliminates the noise features of unknown attacks after optimization with the Minimax algorithm, so that the anomaly detection performance based on unsupervised learning is greatly improved, and it can better distinguish known / unknown attacks, and can effectively classify unknown attacks, and can realize incremental learning of the detection model. After the three major modules of the causal transformer in this embodiment are trained, it can not only ensure that almost all known / unknown attacks will not be misclassified as normal traffic (Darknet and MalMem can achieve 100% correctness), but also achieve more than 99% correct classification of known attack / unknown attack types. Compared with other methods, the causal transformer in this embodiment maintains more than 99% detection performance on all data sets, has strong stability and generalization ability, is adaptable to different data sets and network environments, can provide correct and meaningful guidance for network security defense, and effectively solves the three problems defined in the previous article, including: Problem 1: Due to the strong false correlation caused by noise characteristics, the correspondence between network attacks and causal characteristics is confused, reducing the accuracy and stability of the detection system. Problem 2: In actual detection goals, not only must the detection samples be classified according to prior knowledge, but also whether the abnormal traffic is an unknown attack type and classify it. Problem 3: Although in machine learning, there is usually a high tolerance for precision.However, in network security, the tolerance for precision is extremely low, especially for actually deployed networks, which have huge traffic volumes and are frequently attacked. Therefore, even a very small false negative FN will allow a large number of attacks to enter, causing the network to suffer huge losses.

[0206] In summary, in the actual IOT interconnection scenario, not only known network attack threats are faced, but also many unknown attacks. Existing detection methods can usually only identify and classify known attacks with high accuracy, but little is known about unknown attacks. It is not only difficult to detect unknown attacks, but also difficult to classify unknown attacks. However, if network attacks are not discovered in time and classified to provide defense measures, it will greatly endanger the security of the network and bring huge losses to mankind. At the same time, due to the low tolerance of IOT security, even if the detection accuracy is extremely high, it faces thousands of network attacks, and as long as there are a few attacks, the network may be paralyzed. In order to solve this problem, the method of this embodiment eliminates noise features and the Minmiax algorithm to amplify the causal correlation difference between normal traffic features and abnormal traffic features to distinguish various types of traffic samples, and proposes a detection method based on a causal transformer. The method of this embodiment is divided into three modules: known attack classification based on supervised learning, known / unknown attack classification based on semi-supervised learning, and unknown attack classification based on unsupervised learning. In the supervised learning module, known / unknown attacks are classified into known traffic types through Xgboost. Secondly, in the semi-supervised learning module, the causal feature distribution of known traffic types is learned through causal reasoning, and the causal attention mechanism module (Causal Attention) and the Minimax algorithm are used to optimize and eliminate false associations, and the causal feature distribution differences between different traffic types are enlarged to classify known / unknown traffic types, ensuring that >99.6% of known / unknown attacks on the two data sets will not be misclassified as normal traffic, and even 100% accuracy can be achieved in Darknet and MalMem, which is higher than the most advanced algorithms in the three major performance indicators. Then in the unsupervised learning module, all traffic samples different from known types are collected together and classified through a causal transformer (causal transformer), achieving >99% of the classification of unknown attack types correctly. Experiments have proved that the method of this embodiment can not only correctly classify known attacks, but also ensure that IOT network attacks will not be missed on some data sets, and can effectively realize the identification and classification of unknown attack types, with good generalization performance and stable performance, providing timely and effective guidance for network security defense.

[0207] In addition, this embodiment also provides a causal converter using causal reasoning to detect network attacks. The system includes a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to execute the causal converter using causal reasoning to detect network attacks.

[0208] In addition, this embodiment also provides a computer-readable storage medium, which stores a computer program or instruction, and the computer program or instruction is programmed or configured to execute the causal converter through a processor to perform a network attack detection method using causal reasoning.

[0209] In addition, this embodiment also provides a computer program product, including a computer program or instructions, which are programmed or configured to execute the causal converter through a processor to perform a network attack detection method using causal reasoning.

[0210] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the functions in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0211] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A causal transformer uses causal reasoning to detect network attacks, characterized in that: The steps include: S1, using a classifier based on supervised learning training, inputs the detection set consisting of samples of known and unknown attack types into the classifier for classification, and obtains multiple known attack types; S2, the causal transformer based on semi-supervised learning classifies each known attack type into known attacks and unknown attacks; S3, the causal transformer based on unsupervised learning classifies all unknown attacks into multiple unknown attack types.

2. The causal converter according to claim 1 uses causal reasoning to detect network attacks, characterized in that: Step S1 includes: S1.1, learn causal effect weights W for features in a training set of known attack types: In the above formula, p is the dimension of the feature, E represents the expected value, and G is the independent weight of the causal feature. It means that except for the i-th feature which is set to normal, the values ​​of other features remain unchanged. It means that except for the value of the i-th feature, all other features are set to normal by intervention, I represents all samples, i represents the i-th feature, and ∑w represents the p elements W1~W in the causal effect weight W. p The diagonal matrix diag(W1,W2,…,W p ); S1.2, screening causal features from features in the training set of known attack types based on the causal effect weight W; S1.3, select causal features from the features in the training set to train the classifier, so that the classifier establishes a mapping relationship between the input causal features and the known attack types; S1.4, input the detection set consisting of samples of known and unknown attack types into the trained classifier for classification, and obtain multiple known attack types.

3. The causal converter according to claim 2 uses causal reasoning to detect network attacks, characterized in that: Step S1.2 includes: Identify the noise feature F n and the causal feature F c ,in represents the expected value of network anomaly under abnormal eigenvalue, ∑w represents the diagonal matrix of causal effect weight W, E(F c W T ) represents the expected value of network anomaly under causal anomaly characteristics.

4. The causal converter according to claim 2 uses causal reasoning to detect network attacks, characterized in that: The classifier in step S1.3 is an XGBoost classifier, and the output obtained by the XGBoost classifier after t iterations after the causal features are input into the XGBoost classifier is The function expression is: In the above formula, f () represents the model of the kth tree of the XGBoost classifier, xc i represents the sample after retaining only the causal features, Wc represents the causal effect weight of the causal features, is the output after the t-1th iteration, f () Represents the model of the tth tree of the XGBoost classifier.

5. The causal converter according to claim 3 uses causal reasoning to detect network attacks, characterized in that: The loss function used when selecting causal features from the features in the training set to train the classifier in step S1.3 is And the objective function Obj () The function expression is: In the above formula, p represents the dimension of the sample; It means that the feature of the i-th dimension is intervened to make it normal, and the value is 0; ∑w represents the diagonal matrix of causal effect weight W; W represents the causal effect weight; It means that the features other than the i-th dimension are intervened to make them normal, and the value is 0; the causal feature F is selected c When training a classifier, m represents the number of samples. Represents the loss contribution of a single sample, y i represents the true label of the i-th sample, is the output after the t-1th iteration, f () represents the model of the t-th tree of the XGBoost classifier, xc i represents the sample after retaining only the causal features, Wc represents the causal effect weight of the causal features, Ω(f () ) represents the regularization term, which is used to control the model complexity.

6. The causal converter according to claim 1 uses causal reasoning to detect network attacks, characterized in that: Step S2 includes: S2.1, for sample X, extract the causal effect weight W and causal feature independence weight G according to the following formula: In the above formula, p is the dimension of the feature, E represents the expected value, It means that except for the i-th feature which is set to normal, the values ​​of other features remain unchanged. It means that except for the value of the i-th feature, all other features are set to normal by intervention, and ∑w is the p elements W1~W in the causal effect weight W. p The diagonal matrix diag(W1,W2,…,W p ); Indicates that the j-th row sample is set to 0 by intervention, Indicates that all samples except the j-th row sample are set to 0, ∑g is the m elements G1~G in the causal feature independent weight G m The diagonal matrix diag(G1,G2,...,G m ); I represents all samples, i represents the i-th feature, and j represents the j-th row of samples; S2.2, the causal effect weight W and the causal feature independence weight G are multiplied by MatMul for the first time and then multiplied by MatMul for the second time with the sample X; S2.3, the result obtained by the second matrix multiplication MatMul is subjected to causal association CA and then subjected to the third matrix multiplication MatMul with the sample X. The function expression of the causal association CA is: In the above formula, CA(Q,K,V) is the output of causal association CA, Q, K, V represent query, key and value, G l,T is the transpose of the causal feature-independent weight G used by layer l to balance the training samples, W l is the causal effect weight W of layer l, X l,T The transpose of the input feature vector or sequence for layer l, p is the dimension of the feature, X l is the sample X of layer l; S2.4, the sample X is processed by the prior association PA and Minmax algorithm for sample reconstruction, and the result obtained by the third matrix multiplication MatMul is fused through the addition and normalization module ADD&Norm to obtain forward feedback; the function expression of the prior association PA is: In the above formula, p is the dimension of the feature, X l i is the empirical standard deviation of the ith feature in the lth layer, F i is the eigenvector of the i-th feature, F j is the feature vector of the jth feature, where j≠i; S2.5, the forward feedback outputted in step S2.4 and the sample X are fused through the addition and normalization module ADD&Norm to obtain a new forward feedback, and then the new forward feedback is activated through the addition and normalization module ADD&Norm, the linear module and Softmax, and then subtracted from the sample X to obtain a detection result Y, where the detection result Y is a known attack or an unknown attack under a known attack type corresponding to the sample X.

7. The causal converter according to claim 1 uses causal reasoning to detect network attacks, characterized in that: Step S3 includes: S3.1, aggregate all unknown attacks under known attack types to obtain an unknown attack set; S3.2, extract the causal effect weight W and the causal feature independence weight G for all unknown attacks in the unknown attack set to obtain the causal distribution state of the unknown attack: In the above formula, p is the dimension of the feature, E represents the expected value, and G is the independent weight of the causal feature. It means that except for the i-th feature which is set to normal, the values ​​of other features remain unchanged. It means that except for the value of the i-th feature, all other features are set to normal by intervention, and ∑w is the p elements W1~W in the causal effect weight W. p The diagonal matrix diag(W1,W2,…,W p ); Indicates that the j-th row sample is set to 0 by intervention, Indicates that all samples except the j-th row sample are set to 0, ∑g is the m elements G1~G in the causal feature independent weight G m The diagonal matrix diag(G1,G2,...,G m ); I represents all samples, i represents the i-th feature, and j represents the j-th row of samples; S3.3, reconstruct samples of the unknown attack set using the Minmax algorithm. If the error distribution of the unknown attack set before and after reconstruction is less than the preset threshold, the unknown attack set is determined to be indivisible, and the process ends and exits. Otherwise, the unknown attack set is determined to be divisible, and the reconstructed unknown attack set is divided into unknown attack type i and unknown attack type j. For the unknown attack type i and the unknown attack type j, the corresponding unknown attacks are respectively summarized to obtain a new unknown attack set and jump to step S3.

2.

8. A causal converter uses causal reasoning to perform network attack detection system, including a microprocessor and a memory connected to each other, characterized in that: The microprocessor is programmed or configured to execute the network attack detection method using causal reasoning by the causal converter described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program or instruction stored therein, characterized in that: The computer program or instruction is programmed or configured to execute the network attack detection method using causal reasoning by the causal converter described in any one of claims 1 to 7 through a processor.

10. A computer program product comprising a computer program or instructions, characterized in that The computer program or instruction is programmed or configured to execute the network attack detection method using causal reasoning by the causal converter described in any one of claims 1 to 7 through a processor.

Images