Adaptive intrusion detection method based on CDIVAE and bidirectional time series model

Abstract

This invention discloses an adaptive intrusion detection method based on CDIVAE and a bidirectional temporal model, belonging to the field of network attack detection technology. It uses the GWR algorithm to filter and cluster source domain data, removing a large amount of duplicate data and extracting a clearly distributed and relatively small subset of source domain data. A Gaussian mixture conditional domain-invariant variational autoencoder (GMI) is used to reduce the difference in posterior distribution between the source and target domains. Finally, an intrusion detection is performed using a fusion neural network BRN-BiLSTM, which combines a bidirectional preservative network encoder and a bidirectional long short-term memory network. The BRN first applies bidirectional temporal weighting to the data, and then the BiLSTM identifies and classifies the time-weighted data. The memory unit and gating unit effectively capture data dependencies. This invention achieves better domain-invariant feature extraction and data distribution alignment, and also exhibits better cross-domain intrusion detection performance.

Description

Technical Field

[0001] This invention belongs to the field of network attack detection technology, and particularly relates to a domain-adaptive intrusion detection method based on CDIVAE and bidirectional temporal fusion neural network. Background Technology

[0002] Intrusion Detection System (NIDS) is a crucial line of defense in network security. A stable NIDS capable of timely and accurate detection and response to network attacks is a current research focus, and many researchers have proposed solutions to improve its performance. Kim et al. proposed a DoS attack detection algorithm based on a CNN model, first extracting feature maps from each traffic data point and then detecting them using a CNN model. Zhang et al. proposed an intrusion detection system based on a fusion model of Gaussian mixture model and CNN (SGM-CNN), and used the SMOTE algorithm for data balancing. Zhang et al. also proposed an intrusion detection system based on a fuzzy wavelet neural network, optimizing the neural network parameters using a quantum particle swarm optimization algorithm. All of these studies employ machine learning models to solve the intrusion detection problem and achieve excellent detection results. In addition, existing research has introduced temporal models into intrusion detection. Laghrissi et al. proposed an intrusion detection algorithm based on a Long Short-Term Memory (LSTM) network, focusing on the temporal characteristics of traffic data and using a temporal model for computation, thus making the detection results more accurate. Imrana et al. used a bidirectional LSTM algorithm to treat traffic data as a time series and compute it serially. However, this serial computation method is inefficient when dealing with large-scale traffic data. Transformers, through a multi-head self-attention mechanism, process data in parallel, significantly improving computational efficiency. Farhan et al. introduced Transformers into intrusion detection, using them to learn feature representations of traffic data and then performing detection using a CNN-LSTM model. While these methods all achieve excellent detection results, they are all trained and tested on the same dataset. In real-world applications, network environments are complex and variable, and the distribution of traffic data often changes. Research shows that when training and testing data come from network environments with different distributions, the detection performance of existing NIDS (Network Intrusion Detection and Retrieval System) will be significantly reduced.

[0003] To address the domain offset problem in Intrusion Detection Systems (NIDS), researchers have introduced domain adaptation (DA) techniques into intrusion detection. For example, Zhang et al. proposed a semi-supervised domain adversarial training (SSDAT) framework using domain adversarial learning to solve the domain adaptation problem in smart grid intrusion detection. This framework adjusts the model through adversarial training to cope with new attacks faced by the system. Long et al. proposed a novel deep adaptive network framework that extends deep convolutional networks to domain adaptation scenarios. These methods reduce the difference between the source and target domains, enabling data in the target domain to be recognized by a classifier trained on the source domain, thus achieving equally good performance for the classifier in both domains. However, these methods are resource-intensive in handling large-scale traffic data and are prone to overfitting. Mahdavi et al. proposed an incremental transfer learning-based intrusion detection system framework (ITL-IDS), which uses GWR-based incremental clustering to extract data structure summary subsets from the source and target domain data, thereby reducing system training overhead and avoiding overfitting.

[0004] The methods described above only align the marginal distributions of the source and target domains when performing domain migration on traffic data. In Bayesian theory, the posterior distribution directly affects the detection performance of the model. Therefore, researchers have studied the alignment of the posterior distribution of data. Wu et al. proposed a heterogeneous domain adaptation method based on geometric graph alignment, which represents the relationship between intrusion categories and categories using a graph. They also used the geometric graph alignment method (GGA) and domain adversarial networks (DANN) to reduce the differences in posterior distributions between domains.

[0005] Network Intrusion Detection Systems (NIDS) ensure network security by collecting and analyzing network traffic in real time to detect intrusions. Since networks are mostly in a normal state, and network traffic data is private and secure, publicly available intrusion detection datasets are scarce. Furthermore, publicly available datasets are often collected in laboratory network environments, resulting in significant differences in the characteristic distribution of real-world traffic data. Therefore, NIDS needs strong adaptability, enabling it to accurately detect attack traffic even when the characteristic distribution of traffic data changes. In recent years, Domain Adaptation (DA) has received widespread attention in intrusion detection. DA aims to map source and target domains with different distributions to the same feature space, allowing classification models trained in the source domain to be transferred to the target domain, improving their performance. Currently, feature-based DA methods are widely used in intrusion detection, and some have achieved significant results. However, the collection process generates a large amount of duplicate data in the source domain, making the source domain dataset large and redundant, greatly increasing training costs, and easily leading to overfitting of domain-invariant feature extractors and classifiers. Furthermore, most current methods assume, based on Bayesian decision theory, that the posterior distribution of samples is stable across domains. Therefore, the methods mentioned above only study the alignment of the marginal probability distributions of samples. However, real-world network environments are complex and variable, with diverse attack methods, making it difficult for the posterior distribution to remain stable. In Bayesian decision theory, differences in the posterior distribution directly affect the classification results. In addition, most current intrusion detection models do not fully consider the temporal characteristics of intrusion detection data, resulting in poor feature learning ability and classification performance. Summary of the Invention

[0006] The purpose of this invention is to provide a domain-adaptive intrusion detection method based on GM-CDIVAE and bidirectional temporal fusion neural network, which can better extract domain-invariant features and align data distribution, and also has better cross-domain intrusion detection performance.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] An adaptive intrusion detection method based on CDIVAE and a bidirectional temporal model includes a filtering clustering algorithm based on the GWR algorithm, a domain-invariant feature extraction method based on a Gaussian mixture conditional domain-invariant variational autoencoder, and a fusion neural network model based on a bidirectional preserve-preservation network and a bidirectional long short-term memory network. Specifically, it includes the following steps:

[0009] Step S1: Perform max-min normalization on the data and balance the data using undersampling techniques;

[0010] Step S2: The processed data is filtered and clustered using the GWR algorithm. This algorithm clusters the source domain dataset based on Euclidean distance, while filtering out a large amount of duplicate data and extracting a small subset of source domain data with clear distribution.

[0011] Step S3: Reduce the difference in conditional distributions between the target domain and the source domain by using a Gaussian mixture conditional domain invariant variational autoencoder to ensure alignment of the posterior distributions;

[0012] Step S4: Attack traffic is detected by a fusion neural network of bidirectional hold-alive network encoder and bidirectional long short-term memory network. First, the data is bidirectionally time-weighted by BRN. Bidirectional causal masking and exponential decay weighting are used to avoid model overfitting. At the same time, the computational efficiency of the model is improved by using a serial-parallel hybrid method. Then, the time-weighted data is identified and classified by BiLSTM. It can effectively capture the data dependencies through memory units and gating units.

[0013] Step S5: Verify the adaptive capability of the algorithm through ablation experiments and comparative experiments, and use the F1 score as the evaluation criterion to verify the effectiveness and superiority of the method.

[0014] A further improvement of the technical solution of the present invention is that: in step S2, the source domain data is filtered and clustered by the GWR algorithm to select a smaller subset of data, and the difference in posterior distribution between the source domain and the target domain is reduced by GM-CDIVAE and domain-invariant features are extracted.

[0015] The main steps of the GWR algorithm are as follows:

[0016] S2.1 Randomly select two samples as initial nodes to form set A, and use their sample values ​​as their initial weight vector;

[0017] S2.2 Initialize edge set C as an empty set;

[0018] S2.3, Randomly select a sample ξ;

[0019] S2.4 For each node vi, calculate its distance to the input sample ||ξ-w i ||;

[0020] S2.5 Select the best matching node and the second best matching node

[0021]

[0022] S2.6 If the edge (vs, vt) does not exist, create it: C = C∪{(vs, vt)} s ,v t)}, set the age of edge (vs,vt) to 0; S2.7, calculate the similarity between sample ξ and the best matching node a=exp(-||ξ-w s ||);

[0023] S2.8 If a is less than the similarity threshold aT, and the access intensity is less than the access intensity threshold fT, then add a new node vr between vs and vt, and create a weight vector: w r =0.5(w s +ξ), create edges (vr,vs) and (vr,vt) and delete the old edge (vs,vt);

[0024] S2.9 Otherwise, do not add new nodes, and adjust the positions of the best matching node and its neighbor i:

[0025]

[0026] S2.10. Increase the weight of all edges connected to s: age(vs,vt) = age(vs,vt) + 1;

[0027] S2.11, Update the access strength of the best matching node and its neighbor i:

[0028]

[0029] S2.12 Remove all edges with a weight greater than agemax, and remove all isolated vertices;

[0030] S2.13. If the required number of iterations has not been reached, return to step S2.3; otherwise, end the process.

[0031] A further improvement to the technical solution of this invention lies in the following: In step S3, the Gaussian mixture conditional domain invariant variational autoencoder aligns the class-conditional distribution P(X|Y) of the source domain and the target domain. The specific steps are as follows:

[0032] Step S3.1: Construct a Gaussian Model (GMM) for the source domain data using the EM algorithm; estimate the parameters of the Gaussian distribution corresponding to each class in the reference domain based on the maximum likelihood method, and use the estimated values ​​as the initial values ​​of the GMM. Each component of the GMM converges towards the class conditional probability density function, thus achieving the fitting of the class conditional probability density function. Formulas (11) and (12) are the maximum likelihood estimates of the Gaussian distribution parameters, where... The mean of the sample;

[0033]

[0034] Step S3.2: After obtaining the GMM, each component of the GMM is used as the class conditional distribution of the source domain. CDIVAE uses the components of the GMM as the prior distribution of the prior matching terms. CDIVAE matches the variational distribution to each class conditional distribution of the source domain, so as to align the data distribution after extracting domain-invariant features of the target domain with the class conditional distribution of the source domain. The ELBO of CDIVAE is shown in formula (13).

[0035]

[0036] In the above formula, p GMM (x S |y) represents the source domain data X given the category label y. S The probability density function of q, i.e., the component of class y in the GMM of the source domain; φ (z|x T (,y) represents the target domain sample X with input category y. T The variational density function of the latent variable z, i.e., the probability density function of the domain-invariant eigenvector; p θ (x T |z,y) represents the probability density function of x given a latent variable z of class y;

[0037] Step S3.3, the analytical form of the KL divergence of the d-dimensional Gaussian distribution is shown in formula (14):

[0038]

[0039] The covariance matrix is ​​taken in diagonal form for model calculation, and the reconstruction term is sampled and estimated to obtain the ELBO estimator of CDIVAE:

[0040]

[0041] in and For target domain samples with input category k The expectation and variance vector of the k-th class variational density obtained from the inference network;

[0042] Step S3.4: By combining K encoder-decoder pairs, K variational distributions can be obtained, where K is the number of classes. Based on the additivity of the normal distribution, the probability density p(g(x)) of the domain-invariant feature mapping g(x) is obtained as follows:

[0043]

[0044] Where x is the input target domain sample;

[0045] Step S3.5: Obtain the final domain-invariant eigenmap function through reparameterization.

[0046]

[0047] A further improvement of the technical solution of the present invention is that: in step S4, the bidirectional preserving network BRN based on RetNet is used to extract the global temporal features of traffic data. The BRN adopts a multi-scale preserving mechanism to extract effective information from the long sequence from front to back and from back in a mixed serial and parallel manner. At the same time, it introduces exponential decay weighting to replace softmax in the self-attention mechanism, thereby reducing the computational overhead.

[0048] A further improvement of the technical solution of the present invention is that: the BRN is composed of L identical blocks stacked together, and each block contains two modules: a multi-scale bidirectional preservation MSBR module and a feedforward network FFN module;

[0049] Each MSBR module consists of a bidirectional persistent BR mechanism, which has both serial and parallel forms, allowing the model to be trained in parallel while inference is repeatedly performed. The BR mechanism has a two-layer structure, assuming the first layer is the forward time series and the second layer is the reverse time series. The forward state at time n is defined as... Reverse state is defined as Given an input sequence Where N is the sequence length, d m It is the hidden dimension after encoding, which encodes each component. Mapped into d-dimensional Q, K, V matrices;

[0050]

[0051] The positive and negative structures are mapped using different parameter matrices to obtain different Q, K, and V matrices, and finally the positive output is... and reverse output The final mapping result O is obtained by splicing. n The mapping process is shown in formulas (19)-(21);

[0052]

[0053] Where A is the weight matrix;

[0054] In the serial computation form, the BR mechanism transmits position information through a weight matrix A. To enable it to exhibit long-range attenuation, A is set as a complex diagonal matrix with a modulus of γ and a dimension of d / 2.

[0055]

[0056] in e iθ The real matrix form is as follows:

[0057]

[0058] Substituting formula (22) into formula (21) yields the expression for the forward output:

[0059]

[0060] Where H represents the conjugate transpose. and For rotational position embedding, γ is the position weight. As can be seen from formula (24), the farther away from the current position, the smaller its position weight. Therefore, the exponent of the position weight γ can ensure the long-range decay of the model while recording position information. From formula (24), the parallel representation method of the bidirectional preservation mechanism can be obtained, which is abbreviated as PR:

[0061]

[0062] Where Θ n =e inθ , Indicates conjugate.

[0063] Similarly, the inverse expression for the bidirectional hold mechanism can be obtained.

[0064]

[0065] From (27), we can obtain the parallel representation method NR in reverse order:

[0066]

[0067] The results from the forward and reverse directions are combined to obtain the final result.

[0068] BR(X)=Concat(PR(X),NR(X)) (30)

[0069] A hybrid structure of parallel and serial representations can be used to accelerate training and improve model computation efficiency. The input sequence is divided into blocks, and parallel representation is used for computation within each block, while serial representation is used for computation between blocks. Let B be the length of each block, and its calculation is as follows:

[0070]

[0071]

[0072] BR(X [i] =Concat(PR(X) [i] ),NR(X [i] (33)

[0073] Where [i] represents the i-th block, Q [i] =Q Bi:B(i+1) ,K [i] =K Bi:B(i+1) V [i] =V Bi:B(i+1) .

[0074] The final MSBR is a combination of bidirectional preservation mechanisms in a multi-scale form, with each MSBR using h=d m / d heads, where d is the dimension of Q, K, V in each head, and each head uses a different parameter matrix. A bidirectional retention mechanism assigns a different γ to each head. Additionally, a swing gate is added to increase the nonlinearity of the non-retention layer. The formula is as follows:

[0075]

[0076] Among them W G W O These are all learnable parameters.

[0077] A further improvement of the technical solution of the present invention is that the Bidirectional Long Short-Term Memory Network (BiLSTM) integrates two LSTM layers with opposite directions: one processes the sequence in the forward direction of time, and the other processes the sequence in the opposite direction, so as to obtain more comprehensive contextual information in the sequence data.

[0078] A further improvement of the technical solution of the present invention is that: LSTM consists of three gating units and one memory unit, wherein the calculation formula of the gating unit is shown in formula (43)-formula (45).

[0079] f t =σ(W f x t +U f h t-1 +b f (35)

[0080] i t =σ(W i x t +U i h t-1 +b i (36)

[0081] o t =σ(W o x t +U o h t-1 +b o (37)

[0082] Where f t Represents the Gate of Oblivion, it Represents the input gate, o t Represents the output gate, x t h represents the input data. t This represents the hidden layer, where W, U, and b represent the weights and biases, σ represents the sigmoid activation function, and c represents the current state of the memory cell. t It is obtained by weighting the memory cell state of the previous time step and the current candidate state, and the calculation formula is shown in formula (46) and formula (48).

[0083]

[0084] Finally, the output of the hidden layer of LSTM is shown in Equation (30).

[0085] h t =o t ⊙tanh(c t (40).

[0086] A further improvement of the technical solution of this invention lies in the fact that the sequence of traffic flows contains temporal characteristics; therefore, there is a temporal correlation between network traffic data. Using BRN to weight features can effectively extract the global temporal features of traffic data. Simultaneously, BiLSTM can learn from traffic data from both past and future perspectives, effectively utilizing global temporal features, thereby enabling more accurate detection.

[0087] A further improvement to the technical solution of this invention lies in: the maximization formula Shrink q φ (z|x T ,y) and p GMM (x S |y)KL divergence enables the distribution of the target domain data after extracting domain-invariant features to align with the distribution of the source domain, achieving a cross-domain effect.

[0088] A further improvement to the technical solution of this invention lies in the fact that D in the parallel representation is a combination of causal masking and exponential decay weighting, playing the role of the MASK matrix and Softmax function in the multi-head self-attention mechanism, thus avoiding overfitting of the model; D + and D - By applying causal masking and exponential decay weighting to the data from two different directions, the model can take into account every piece of data in the context during the calculation process.

[0089] The technological advancements achieved by this invention due to the adoption of the above technical solutions are as follows:

[0090] This invention uses the GWR algorithm to filter and cluster source domain data. This algorithm clusters the source domain dataset based on Euclidean distance while filtering out a large amount of duplicate data, extracting a clearly distributed and relatively small subset of source domain data. This solves the problem of overfitting in domain-invariant feature extractors and classifiers caused by the large amount of duplicate data generated during the collection of source domain traffic data.

[0091] This invention applies the GWR-GCN feature extraction method to extract domain-invariant features. By transforming data samples into graph nodes, connecting related samples together to construct a graph structure, and then using graph convolution to mine the correlation information between samples, domain-invariant feature extraction is achieved. Considering the impact of correlation between samples on the extraction of domain-invariant features, the accuracy of feature extraction is improved.

[0092] This invention aligns feature distributions based on GM-CDIVAE. The method first fits the conditional distribution of the source domain by establishing a GMM model, and then reduces the difference between the conditional distributions of the target domain and the source domain through a variational autoencoder, thereby ensuring the alignment of the posterior distribution. This invention considers the issue that the posterior distribution may change across domains, thus performing distribution alignment based on the posterior distribution.

[0093] This invention is based on the BRN-BiLSTM model for intrusion detection. First, the data is bidirectionally time-weighted by BRN, which avoids overfitting through bidirectional causal masking and exponential decay weighting. At the same time, the computational efficiency of the model is improved by using a serial-parallel hybrid approach. Then, the time-weighted data is identified and classified by BiLSTM, which can effectively capture the dependencies of the data through memory units and gating units. Attached Figure Description

[0094] Figure 1 It is the overall flowchart;

[0095] Figure 2 It is a sequential serial structure with a bidirectional hold mechanism;

[0096] Figure 3 It is a forward-order parallel structure with a bidirectional preservation mechanism;

[0097] Figure 4 It is a bidirectional preserving mechanism for the positive-order parallel hybrid structure;

[0098] Figure 5 It is the F1 value of the ablation experiment of the algorithm of this invention. Detailed Implementation

[0099] The present invention will be further described in detail below with reference to embodiments:

[0100] This invention addresses the adaptation problem in Internet intrusion detection systems by proposing a domain-adaptive intrusion detection algorithm based on GM-CDIVAE and a bidirectional temporal fusion neural network. The method uses the GWR algorithm to filter and cluster the source domain dataset, then aligns the source and target domain data using GM-CDIVAE, and finally utilizes a bidirectional temporal fusion neural network for cross-domain network attack detection.

[0101] The intrusion detection algorithm of this invention mainly consists of four parts: the GWR filtering clustering algorithm, the GM-CDIVAE feature distribution alignment network, the bidirectional temporal fusion neural network intrusion detection model, and algorithm evaluation. First, the data is subjected to min-max normalization and undersampling to balance the data. The processed data is then input into the GWR algorithm for filtering clustering. The GWR algorithm clusters the dataset based on Euclidean distance, while filtering out a large amount of duplicate data, extracting a clearly distributed and relatively small subset of source domain data. Second, the GM-CDIVAE distribution alignment algorithm is used to align the domain-invariant features of the source and target domains. First, a Gaussian mixture model (GMM) is established to fit the conditional distribution of the source domain. Then, a variational autoencoder is used to reduce the difference between the conditional distributions of the target and source domains, thus ensuring the alignment of the posterior distributions. Finally, the source domain data with the aligned distributions is fed into a fusion neural network BRN-BiLSTM, which combines a bidirectional preserve-encoder network (BRN) and a bidirectional long short-term memory network (BiLSTM), for training. First, a BRN (Bio-Range Network) is used to apply bidirectional temporal weighting to the data. This avoids overfitting through bidirectional causal masking and exponential decay weighting, while improving computational efficiency through a serial-parallel hybrid approach. Then, a BiLSTM (Bilateral Least Signature Detector) is used to identify and classify the time-weighted data. Its memory and gating units effectively capture data dependencies. Two experimental groups were set up: one with a source domain of NFv2-UNSW-NB15 and a target domain of NFv2-CSE-CIC-IDS2018, and the other with a source domain of NFv2-CSE-CIC-IDS2018 and a target domain of NFv2-UNSW-NB15. Ablation experiments were conducted on both groups. Domain-adaptive cross-domain detection experiments were performed on both groups. Overall performance was compared with other similar algorithms, focusing on accuracy and recall, demonstrating the superiority of the proposed algorithm.

[0102] This invention mainly consists of three parts. First, the source domain data is filtered and clustered using the GWR algorithm to select a smaller subset of data. Then, GM-CDIVAE is used to reduce the difference in posterior distributions between the source and target domains and extract domain-invariant features. Finally, a BRN-BiLSTM model is trained on the source domain based on the extracted domain-invariant features and applied to intrusion detection in the target domain. In the following description, the source domain is denoted as... in Represents the i-th sample and its class label in the source domain dataset, n s The number of samples is represented by ; the target domain is denoted as . in The target domain can be represented by the j-th sample in the dataset and its class label, n. T This indicates the number of available samples, with each sample value represented by x. The overall flowchart is as follows: Figure 1 As shown.

[0103] I. Data Extraction Method Based on GWR Algorithm

[0104] Self-organizing neural networks can dynamically adjust the network structure, enabling them to accurately approximate the input space with a simpler structure. Marsland et al. proposed the Growing When Required (GWR) algorithm, which determines when to add a new node by calculating the node that best matches the current input.

[0105] In the GWR algorithm, two samples are randomly selected from the source domain dataset and used as the initial selection set for iterative expansion. After each iteration, the subset of data from the previous iteration is set as the algorithm's selection set, thus preserving previously learned knowledge. The GWR algorithm also connects the selected samples into a graph structure during iteration, which this invention refers to as the sample graph. Highly similar sample points have higher connectivity and a more concentrated distribution; therefore, the GWR algorithm makes the sample distribution structure clearer. The definitions of symbols and parameters in the algorithm are shown in Table 1.

[0106] Table 1 GWR Algorithm Variable Definition Table

[0107]

[0108] The main steps of the GWR algorithm are as follows:

[0109] S2.1. Randomly select two samples as initial nodes to form set A, and use their sample values ​​as their initial weight vector;

[0110] S2.2. Initialize edge set C as an empty set;

[0111] S2.3. Randomly select a sample ξ;

[0112] S2.4. For each node v i Calculate the distance ||ξ-w| from the input sample. i ||;

[0113] S2.5. Select the best matching node and the second best matching node

[0114]

[0115] S2.6. If edge (v s ,v t If it does not exist, create it: C = C∪{(v s ,v t )}; Will the edge (v s ,v t Set the age of ) to 0.

[0116] S2.7. Calculate the similarity between sample ξ and the best matching node: a = exp(-||ξ - w) s ||);

[0117] S2.8. If a is less than the similarity threshold a T And the access strength is less than the access strength threshold f T Then in v s and v t Add a new node v between r Create a weight vector: w r =0.5(w s +ξ), create an edge (v) r ,v s ) and (v r ,v t ) and delete the old edge (v s ,v t );

[0118] S2.9. Otherwise, do not add new nodes, and adjust the positions of the best matching node and its neighbor i:

[0119]

[0120] S2.10. Increase the weight of all edges connected to s: age(v s ,v t ) = age(v s ,v t +1;

[0121] S2.11. Update the access strength of the best matching node and its neighbor i:

[0122]

[0123] S2.12. Remove ownership greater than age max Remove all isolated points from the edges;

[0124] S2.13. If the required number of iterations has not been reached, return to step (3); otherwise, end.

[0125] II. Distribution Alignment Method Based on Gaussian Mixture Domain Invariant Variational Autoencoder

[0126] 2.1 Gaussian Mixture Model

[0127] Gaussian Mixture Model (GMM) is a probabilistic model used for clustering and density estimation of data. GMM assumes that the data is generated by a mixture of multiple Gaussian distributions, each called a component. Each component has its own corresponding coefficient π. k The expected vector μ k The sum of the covariance matrix ∑ k They determine the shape of the overall distribution. Its probability form is as follows:

[0128]

[0129] Suppose there are N observation data points x1, x2, ..., xn. N The steps for estimating GMM parameters using the EM algorithm are as follows:

[0130] 1. Initialize parameters;

[0131] 2. Step E: Calculate the observed data x j The probability of coming from the k-th sub-model;

[0132]

[0133] 3. M-step: Calculate the model parameters for the new iteration;

[0134]

[0135] 4. Repeat steps 2 and 3 until each parameter value converges;

[0136] 2.2 Variational Autoencoder

[0137] A variational autoencoder (VAE) is a deep generative model used to model probabilistic models containing latent variables. Its structure consists of an encoder and a decoder. The goal of a VAE is to maximize the lower bound of evidence (ELBO).

[0138]

[0139] Where p θ (z) is the probability density function of the prior distribution of the latent variable z, q φ(z|x) is the variational density function of the latent variable z, p θ (x|z) is the probability density function of x given the input latent variable z. Formula (8) consists of an encoder and a decoder: the encoder estimates the variational distribution q through a neural network. φ (z|x), making it approximate the prior distribution p θ (z), this network is called an inference network. The inference network takes x as input and outputs a variational distribution q. φ (z|x); The decoder estimates the probability distribution p through a neural network. θ (x|z), this neural network is called a generator network, and this distribution is called the decoder distribution. The generator network takes the latent variable z as input and outputs the decoder distribution p. θ (x|z); To simplify the model, the prior distribution p is usually used. θ (z) is set to a standard normal distribution, and according to q φ (z|x) Sample L z times l The ELBO estimator is obtained as follows:

[0140]

[0141] Where p θ (x|z l ) is the input hidden variable z l Decoding distribution generated by the generative network and To obtain the target VAE, we can maximize the above formula to obtain the expected value and variance vector of the variational density obtained by the inference network. Finally, we can obtain the deterministic function of the latent variable by reparameterizing. First, we sample a random variable from the standard normal distribution, and then obtain the latent variable z by formula (10).

[0142]

[0143] 2.3 Gaussian Mixture Conditional Domain Invariant Variational Autoencoder

[0144] Methods based on learning domain-invariant feature representations are commonly used domain adaptation methods. Their principle is to map the source and target domains into a common feature space by reducing the distributional differences between them. Existing methods typically assume that the posterior distribution P(Y|X) remains stable when performing cross-domain mapping on intrusion detection data, thus achieving domain alignment through differences in the marginal distributions P(X). However, real-world network environments are complex and volatile, with numerous attack methods, making this assumption difficult to guarantee. According to Bayesian decision rules, as shown in Proposition (I), where K is the number of classes.

[0145]

[0146] The class with the highest posterior probability is the class to which sample X belongs. Therefore, the classification result is directly determined by the posterior probability. According to Bayes' theorem, P(Y) = ... j |X)=P(Y j )P(X|Y j Proposition (I) can be rewritten as:

[0147]

[0148] Therefore, the final classification result is determined by the prior probability P(Y) and the class conditional distribution P(X|Y). The influence of the class prior probability P(Y) can be eliminated by balancing the data. Therefore, this invention focuses on the alignment of the class conditional distributions of the source domain and the target domain.

[0149] This invention proposes a Gaussian mixture conditional domain invariant variational autoencoder (GM-CDIVAE) to align the class-conditional distributions P(X|Y) of the source and target domains. First, a Gaussian mixture model (GMM) is constructed from the source domain data using the EM algorithm. Since the EM algorithm is sensitive to initial values, this invention uses the maximum likelihood method to estimate the parameters of the Gaussian distribution corresponding to each class in the reference domain, and uses these estimates as the initial values ​​of the GMM. This setting allows each component of the GMM to converge towards the class-conditional probability density function, thus achieving fitting of the class-conditional probability density function. Equations (11) and (12) are the maximum likelihood estimates of the Gaussian distribution parameters, where... This is the sample mean.

[0150]

[0151] After obtaining the GMM, each component of the GMM is used as the class-conditional distribution of the source domain, and CDIVAE uses the components of the GMM as the prior distribution of the prior matching terms. With this setting, CDIVAE can match the variational distribution to each class-conditional distribution of the source domain, thereby aligning the data distribution after extracting domain-invariant features from the target domain to the class-conditional distribution of the source domain. The ELBO of CDIVAE is shown in formula (13).

[0152]

[0153] In the above formula, p GMM (x S |y) represents the source domain data X given the category label y. S The probability density function of q, i.e., the component of class y in the GMM of the source domain; φ (z|x T (,y) represents the target domain sample with input category y. X TThe variational density function of the latent variable z, i.e., the probability density function of the domain-invariant eigenvector; p θ (x T |z,y) represents the probability density function of x given a latent variable z of category y. By maximizing formula (13), q can be reduced. φ (z|x T ,y) and p GMM (x S The KL divergence (|y) ensures that the distribution of the target domain data after extracting domain-invariant features aligns with the distribution of the source domain, achieving a cross-domain effect. Simultaneously, the reconstruction term guarantees that the target domain data after extracting domain-invariant features will not lose its original information.

[0154] The analytical form of the KL divergence of the d-dimensional Gaussian distribution is shown in formula (14):

[0155]

[0156] The covariance matrix is ​​taken in diagonal form for model calculation, and the reconstruction term is sampled and estimated to obtain the ELBO estimator of CDIVAE:

[0157]

[0158] in and For target domain samples with input category k The expectation and variance vectors of the k-th class variational density obtained from the inference network.

[0159] By combining K encoder-decoder pairs, we can obtain K variational distributions, where K is the number of classes. Since they all approximate the components of a GMM, the probability density p(g(x)) of the domain-invariant feature map g(x) can be obtained based on the additivity of the normal distribution:

[0160]

[0161] Here, x represents the input target domain sample. Finally, the domain-invariant feature mapping function is obtained through reparameterization.

[0162]

[0163] III. Bidirectional Temporal Fusion Neural Network

[0164] 3.1BRN

[0165] Sun et al. proposed RetNet for long sequence modeling. Its structure is similar to Transformer, enabling low-cost inference, efficient long sequence modeling, and parallel model training. RetNet replaces the multi-head self-attention mechanism in Transformer with a multi-scale preservation module. The preservation mechanism extracts effective information from long sequences through a hybrid serial-parallel approach, and it reduces computational overhead by introducing exponentially decaying weighting instead of sofmtax in the self-attention mechanism.

[0166] Global temporal features are crucial for traffic data detection. However, RetNet can only extract temporal information from data from a single direction. Therefore, this invention proposes a bidirectional hold network (BRN) based on RetNet to extract global temporal features from traffic data.

[0167] BRN consists of L identical stacked blocks, each block containing two modules: a multi-scale bidirectional preserve (MSBR) module and a feedforward network (FFN) module.

[0168] Each MSBR module consists of a bidirectional hold (BR) mechanism, which has both serial and parallel forms, allowing the model to be trained in parallel while inference is repeated. The BR mechanism has a two-layer structure; assuming the first layer is the forward time series and the second layer is the reverse time series, the forward state at time n is defined as... Reverse state is defined as Given an input sequence Where N is the sequence length, d m It is the hidden dimension after encoding. Each component... Mapped into d-dimensional Q, K, V matrices.

[0169]

[0170] The positive and negative structures are mapped using different parameter matrices to obtain different Q, K, and V matrices, and finally the positive output is... and reverse output The final mapping result O is obtained by splicing. n The mapping process is shown in formulas (19)-(21).

[0171]

[0172] Where A is the weight matrix. The above calculation process is the serial calculation form of the BR mechanism, where the ascending serial structure is as follows: Figure 2 As shown, the reverse serial structure is similar.

[0173] In the serial computation form, the BR mechanism transmits position information through a weight matrix A. To enable it to exhibit long-range attenuation, A is set as a complex diagonal matrix with a modulus of γ and a dimension of d / 2.

[0174]

[0175] in e iθ The real matrix form is as follows:

[0176]

[0177] Substituting formula (22) into formula (21) yields the expression for the forward output:

[0178]

[0179] Here, H represents conjugate transpose. and This can be viewed as a rotational position embedding, and γ can be seen as the position weight. From formula (24), it can be seen that the farther away from the current position, the smaller its position weight. Therefore, by introducing an exponent of γ, the long-range decay of the model can be guaranteed while recording position information; this invention calls it the exponential decay factor. From (24), we can obtain the parallel representation method of the bidirectional preservation mechanism, which is abbreviated here as PR:

[0180]

[0181] Where Θ n =e inθ , Parallel structures representing conjugation, bidirectional preservation mechanisms, and orthogonal order are as follows: Figure 3 As shown:

[0182] Similarly, the inverse expression for the bidirectional hold mechanism can be obtained.

[0183]

[0184] From (27), we can obtain the parallel representation method of reverse order, which is abbreviated as NR here:

[0185]

[0186] The results from the forward and reverse directions are combined to obtain the final result.

[0187] BR(X)=Concat(PR(X),NR(X)) (30)

[0188] In parallel representation, D can be viewed as a combination of causal masking and exponentially decaying weighting. It functions similarly to the mask matrix and softmax function in multi-head self-attention mechanisms, thus preventing overfitting. Furthermore, D... + and D - By applying causal masking and exponential decay weighting to the data from two different directions, the model can take into account every piece of data in the context during the calculation process.

[0189] A hybrid structure of parallel and serial representations can be used to accelerate training and improve model computation efficiency. The input sequence is divided into blocks, and parallel representation is used for computation within each block, while serial representation is used for computation between blocks. Let B be the length of each block, and its calculation is as follows:

[0190]

[0191] BR(X [i] =Concat(PR(X) [i] ),NR(X [i] (33)

[0192] Where [i] represents the i-th block, Q [i] =Q Bi:B(i+1) ,K [i] =K Bi:B(i+1) V [i] =V Bi:B(i+1) A bidirectional preserving mechanism for a sequentially parallel hybrid structure is as follows: Figure 4 As shown.

[0193] The final MSBR is a combination of bidirectional preservation mechanisms in a multi-scale form, with each MSBR using h=d m / d heads, where d is the dimension of Q, K, V in each head, and each head uses a different parameter matrix. A bidirectional retention mechanism assigns a different γ to each head. Additionally, a swing gate is added to increase the nonlinearity of the non-retention layer. The formula is as follows:

[0194]

[0195] Among them W G W O These are all learnable parameters.

[0196] 3.2 BiLSTM

[0197] Bidirectional Long Short-Term Memory (BiLSTM) networks integrate two Long Short-Term Memory (LSTM) networks operating in opposite directions, enabling them to consider both past and future information simultaneously, thus providing a more comprehensive understanding of the intrinsic structure of sequence data.

[0198] LSTM is a recurrent neural network structure specifically designed to process sequential data. Through gating units and memory units, it effectively captures long-term dependencies and avoids problems such as vanishing or exploding gradients found in traditional recurrent neural networks (RNNs). LSTM consists of three gating units and one memory unit, with the calculation formulas for the gating units shown in equations (43)-(45).

[0199] f t =σ(W f x t +U f h t-1 +b f (35)

[0200] i t =σ(W i x t +U i h t-1 +b i (36)

[0201] o t =σ(W o x t +U o h t-1 +b o (37)

[0202] Where f t Represents the Gate of Oblivion, i t Represents the input gate, o t Represents the output gate, x t h represents the input data. t Represents the hidden layer. W, U, and b represent the weights and biases. σ represents the Sigmoid activation function. c represents the current state of the memory cell. t It is obtained by weighted sum of the memory cell state at the previous time step and the current candidate state, and the calculation formula is shown in formula (46) and formula (48).

[0203]

[0204] Finally, the output of the hidden layer of LSTM is shown in Equation (30).

[0205] h t =o t ⊙tanh(c t (40)

[0206] BiLSTM further enhances the capabilities of LSTM by introducing two LSTM layers operating in opposite directions: one processes the sequence forward in time, and the other in the opposite direction, to capture more comprehensive contextual information from the sequence data. This bidirectional structure enables BiLSTM to capture richer features and relationships in sequence data, allowing for comprehensive modeling of sequence data that fully considers contextual information and long-term dependencies.

[0207] 3.3 BRN-BiLSTM Classifier

[0208] The sequential order of traffic flows contains temporal characteristics; therefore, there are temporal relationships within the context of network traffic data. Using a BRN (Browser-Related Network) to weight features can effectively extract the global temporal features of traffic data. Simultaneously, BiLSTM can learn from traffic data from both past and future perspectives, effectively utilizing global temporal features for more accurate detection. Therefore, this invention proposes a fusion neural network of BRN-BiLSTM for traffic data detection. BRN-BiLSTM can learn the intrinsic structure of traffic data more deeply, while effectively avoiding model overfitting. It achieves excellent classification results while exhibiting good generalization performance, enabling it to achieve better intrusion detection classification performance.

[0209] IV. The adaptive capability of the algorithm is verified through ablation experiments and comparative experiments. The F1 score is used as the evaluation criterion to verify the effectiveness and superiority of the method.

[0210] 4.1 Experimental Environment and Dataset

[0211] The experimental environment for this invention consisted of a 64-bit Windows 11 operating system, an Intel i5-12500H processor, an NVIDIA RTX 3050 graphics card, and 16GB of RAM. The experimental framework utilized the machine learning library Scikit-learn, and the deep learning libraries Keras and PyTorch.

[0212] To compare the model's adaptability on different datasets, this invention uses the Netflow_v2 versions of the intrusion detection datasets UNSW-NB15 and CSE-CIC-IDS2018 to verify the performance of the proposed algorithm. NFv2-UNSW-NB15 and NFv2-CSE-CIC-IDS2018 share the same 43-dimensional features, converted from their original format to the Netflow standard by Sarhan et al. NFv2-UNSW-NB15 contains 2,390,275 traffic data points, labeled as one benign and nine other attack categories, with benign samples accounting for 96.02% of the total. NFv2-CSE-CIC-IDS2018 contains 18,893,708 traffic data points, labeled as one benign and 14 other attack categories, with benign samples accounting for 88.05% of the total. This invention focuses on binary classification, treating all attack categories as a single Attack class.

[0213] 4.2 Evaluation Indicators

[0214] This invention uses the F1 score (F1-measure) to evaluate the model's performance. The F1 score is calculated using precision and recall, which are verified based on the confusion matrix. The confusion matrix is ​​shown in Table 2. The calculation methods for each indicator are shown in formulas (49)-(51).

[0215] Table 2 Confusion Matrix

[0216]

[0217]

[0218] In this context, TP (True Positive) represents data that is Benign and is predicted to be Benign; TN (True Negative) represents data that is Attack and is predicted to be Attack; FN (False Negative) represents data that is Benign but is predicted to be Attack; and FP (False Positive) represents data that is Attack but is predicted to be Benign.

[0219]

[0220] The invention has been tested, and the intrusion detection model performs very well, consistent with the design expectations.

[0221] To further verify the adaptive capability of the proposed algorithm, the proposed algorithm was compared with other classic algorithms such as Random-Forest, Extra Tree, LSTM NN, Feed Forward NN and DANN in terms of F1 score. The comparison mainly included the detection performance of the algorithm in a single domain, i.e., when the source domain and target domain are from the same dataset, and in a cross-domain scenario, i.e., when the source domain and target domain are from different datasets. The experimental results are shown in Tables 3 and 4.

[0222] Table 3. Detection performance verification of each algorithm in a single domain scenario.

[0223]

[0224] Table 4 Performance verification of each algorithm in cross-domain scenarios

[0225]

[0226] Even within the same dataset, the distribution of traffic data can shift across different time periods. As shown in Table 3, while the performance of various algorithms is roughly similar in this scenario, the algorithm proposed in this invention exhibits the best performance compared to other algorithms, demonstrating its ability to more effectively adapt to distribution shifts caused by time variations. Table 4 shows that the algorithm proposed in this invention demonstrates the best detection performance in both cross-domain scenarios, while the performance of other algorithms deteriorates significantly in cross-domain situations. This highlights the stronger adaptability of the algorithm proposed in this invention when facing datasets from different domains. Furthermore, the DANN model exhibits cross-domain detection performance second only to the algorithm proposed in this invention. This is because DANN aligns the edge distribution of the data through adversarial learning, resulting in superior adaptability compared to other models. However, DANN does not guarantee the domain invariance of the posterior distribution during domain alignment, which makes its adaptability lower than that of the model proposed in this invention.

[0227] To verify the contributions of the GWR algorithm, GM-CDIVAE model, and BRN-BiLSTM model to the algorithm proposed in this invention, the following experiments were conducted under the same experimental environment. For ease of representation, this invention uses A to represent the dataset NFv2-UNSW-NB15 and B to represent the dataset NFv2-CSE-CIC-IDS2018.

[0228] Experiment 1: A standalone BRN-BiLSTM model;

[0229] Experiment 2: GWR algorithm + BRN-BiLSTM model;

[0230] Experiment 3: GM-CDIVAE+BRN-BiLSTM model;

[0231] Experiment 4: The algorithm proposed in this invention is the GWR+GM-CDIVAE+BRN-BiLSTM model;

[0232] The experimental results are shown in Figure 5 ,from Figure 5 The experiments show that the original BRN-BiLSTM model has a relatively low F1 score in cross-domain detection. Introducing the GWR algorithm can reduce the impact of duplicate data on model performance, resulting in a slight improvement in cross-domain detection results. After aligning the source and target domain data using the GM-CDIVAE model, the cross-domain detection performance of the BRN-BiLSTM model is improved to some extent, indicating that the GM-CDIVAE model can reduce inter-domain differences, enabling BRN-BiLSTM to effectively identify data in the target domain, thereby improving the overall adaptive capability of the algorithm. Simultaneously, with the introduction of both the GWR algorithm and GM-CDIVAE, the proposed algorithm achieves the highest F1 scores of 81.88% and 81.71%, respectively.

Claims

1. An adaptive intrusion detection method based on CDIVAE and a bidirectional temporal model, characterized in that: This includes a filtering clustering algorithm based on the GWR algorithm, a domain-invariant feature extraction method based on a Gaussian mixture conditional domain-invariant variational autoencoder, and a fusion neural network model based on a bidirectional preservation network and a bidirectional long short-term memory network, specifically including the following steps: Step S1: Perform max-min normalization on the data and balance the data using undersampling techniques; Step S2: The processed data is filtered and clustered using the GWR algorithm. The GWR algorithm clusters the source domain dataset based on Euclidean distance, while filtering out duplicate data in the source domain data and extracting a small subset of source domain data with clear distribution. Step S3: Reduce the difference in conditional distributions between the target domain and the source domain by using a Gaussian mixture conditional domain invariant variational autoencoder to ensure alignment of the posterior distributions; In step S3, the Gaussian mixture conditional domain invariant variational autoencoder aligns the class-conditional distributions P(X|Y) of the source and target domains. The specific steps are as follows: Step S3.1: Construct a Gaussian Model (GMM) for the source domain data using the EM algorithm; estimate the parameters of the Gaussian distribution corresponding to each class in the reference domain based on the maximum likelihood method, and use the estimated values ​​as the initial values ​​of the GMM. Each component of the GMM converges towards the class conditional probability density function, thus achieving the fitting of the class conditional probability density function. Formulas (11) and (12) are the maximum likelihood estimates of the Gaussian distribution parameters, where... The mean of the sample; Step S3.2: After obtaining the GMM, each component of the GMM is used as the class conditional distribution of the source domain. CDIVAE uses the components of the GMM as the prior distribution of the prior matching terms. CDIVAE matches the variational distribution to each class conditional distribution of the source domain, so as to align the data distribution after extracting domain-invariant features of the target domain with the class conditional distribution of the source domain. The ELBO of CDIVAE is shown in formula (13). In the above formula, p GMM (x S |y) represents the source domain data X given the category label y. S The probability density function of q, i.e., the component of class y in the GMM of the source domain; φ (z|x T (,y) represents the target domain sample X with input category y. T The variational density function of the latent variable z, i.e., the probability density function of the domain-invariant eigenvector; p θ (x T |z,y) represents the probability density function of x given a latent variable z of class y; Step S3.3, the analytical form of the KL divergence of the d-dimensional Gaussian distribution is shown in formula (14): The covariance matrix is ​​taken in diagonal form for model calculation, and the reconstruction term is sampled and estimated to obtain the ELBO estimator of CDIVAE: In the formula and For target domain samples with input category k The expectation and variance vector of the k-th class variational density obtained from the inference network; Step S3.4: K variational distributions are obtained by combining K encoders and decoders, where K is the number of classes. Based on the additivity of the normal distribution, the probability density p(g(x)) of the domain-invariant feature mapping g(x) is obtained as follows: Where x is the input target domain sample; Step S3.5: Obtain the final domain-invariant eigenmap function through reparameterization. Step S4: Attack traffic is detected by a fusion neural network BiLSTM, which combines a bidirectional hold-alive network encoder (BRN) and a bidirectional long short-term memory network (BiLSTM). First, the data is bidirectionally time-weighted by the BRN. Bidirectional causal masking and exponential decay weighting are used to avoid model overfitting. At the same time, the computational efficiency of the model is improved by using a serial-parallel hybrid approach. Then, the time-weighted data is identified and classified by the BiLSTM. The BiLSTM can effectively capture the dependencies of the data through memory units and gating units. Step S5: Verify the adaptive capability of the algorithm through ablation experiments and comparative experiments, and use the F1 score as the evaluation criterion to verify the effectiveness and superiority of the method.

2. The adaptive intrusion detection method based on CDIVAE and a bidirectional temporal model according to claim 1, characterized in that: In step S2, the source domain data is filtered and clustered using the GWR algorithm to select a small subset of data. GM-CDIVAE is then used to reduce the difference in posterior distributions between the source and target domains and to extract domain-invariant features. The main steps of the GWR algorithm are as follows: S2.1 Randomly select two samples as the initial nodes to form set A, and use the values ​​of these two samples as the initial weight vector; S2.2 Initialize edge set C as an empty set; S2.3, Randomly select a sample ξ; S2.4 For each node vi, calculate the distance ||ξ-w from each node vi to the input sample. i ||; S2.5 Select the best matching node and the second best matching node S2.6 If the edge (vs, vt) does not exist, create it: C = C∪{(vs, vt)} s ,v t Set the age of edge (vs, vt) to 0. S2.7 Calculate the similarity between sample ξ and the best matching node: a = exp(-||ξ - w) s ||); S2.8 If a is less than the similarity threshold aT, and the access intensity is less than the access intensity threshold fT, then add a new node vr between vs and vt, and create a weight vector: w r =0.5(w s +ξ), create edges (vr,vs) and (vr,vt) and delete the old edge (vs,vt); S2.9 Otherwise, do not add new nodes, and adjust the positions of the best matching node and its neighbor i: 0 < ε n <ε b <1; S2.

10. Increase the weight of all edges connected to s: age(vs,vt) = age(vs,vt) + 1; S2.11, Update the access strength of the best matching node and its neighbor i: S2.12 Remove all edges with a weight greater than agemax, and remove all isolated vertices; S2.

13. If the required number of iterations has not been reached, return to step S2.3; otherwise, end the process.

3. The adaptive intrusion detection method based on CDIVAE and a bidirectional temporal model according to claim 1, characterized in that: Step S4 uses the RetNet-based Bidirectional Preservation Network (BRN) to extract global temporal features from traffic data. The BRN employs a multi-scale preservation mechanism to extract effective information from long sequences from front to back and back in a mixed serial-parallel manner. At the same time, it introduces exponential decay weighting instead of softmax in the self-attention mechanism to reduce computational overhead.

4. The adaptive intrusion detection method based on CDIVAE and a bidirectional temporal model according to claim 3, characterized in that: The BRN is composed of L identical blocks stacked together, and each block contains two modules: a multi-scale bidirectional preservation MSBR module and a feedforward network FFN module; Each MSBR module consists of a bidirectional persistent BR mechanism, which has both serial and parallel forms, training the model in parallel while repeatedly performing inference. The BR mechanism has a two-layer structure, assuming the first layer is the forward time series and the second layer is the reverse time series. The forward state at time n is defined as... Reverse state is defined as Given an input sequence In this formula, N is the sequence length, and d m It is the hidden dimension after encoding, which encodes each component. Mapped into d-dimensional Q, K, V matrices; In the BR mechanism, the positive and negative structures are mapped using different parameter matrices to obtain different Q, K, and V matrices, and finally the positive output is generated. and reverse output The final mapping result O is obtained by splicing. n The mapping process is shown in formulas (19)-(21); In the formula, A is the weight matrix; In the serial computation form, the BR mechanism transmits position information through a weight matrix A. To enable the BR mechanism to exhibit long-range attenuation, A is set as a complex diagonal matrix with a modulus of γ and a dimension of d / 2. In the formula e iθ The real matrix form is as follows: Substituting formula (22) into formula (21) yields the expression for the forward output: In the formula, H represents the conjugate transpose. and For rotational position embedding, γ is the position weight. As can be seen from formula (24), the farther away from the current position, the smaller the position weight. Therefore, the exponent of the position weight γ ensures the long-range decay of the model while recording position information. From formula (24), the parallel representation method of the bidirectional preservation mechanism is obtained, which is abbreviated as PR: In the formula Θ n =e inθ , Indicates conjugate. Similarly, the inverse expression for the bidirectional hold mechanism is obtained. From (27), we obtain the parallel representation method NR in reverse order: The results from the forward and reverse directions are combined to obtain the final result. BR(X)=Concat(PR(X),NR(X)) (30) A hybrid structure of parallel and serial representations is used to accelerate training and improve model computation efficiency. The input sequence is divided into blocks, and parallel representation is used for computation within each block, while serial representation is used for computation between blocks. Let B be the length of each block, and the computation of a block is as follows: In the formula, [i] represents the i-th block, Q [i] =Q Bi:B(i+1) ,K [i] =K Bi:B(i+1) V [i] =V Bi:B(i+1) , The final MSBR is a combination of bidirectional preservation mechanisms in a multi-scale form, with each MSBR using h=d m / d heads, where d is the dimension of Q, K, V in each head, and each head uses a different parameter matrix. The bidirectional retention mechanism assigns different values ​​to each head, and a swing gate is added to increase the nonlinearity of the non-retention layer, as shown in the following formula: In the formula W G W O These are all learnable parameters.

5. The adaptive intrusion detection method based on CDIVAE and a bidirectional temporal model according to claim 4, characterized in that: Bidirectional Long Short-Term Memory (BiLSTM) integrates two LSTM layers operating in opposite directions: one processes the sequence in the forward direction of time, while the other processes it in the opposite direction, in order to obtain more comprehensive contextual information from the sequence data.

6. The adaptive intrusion detection method based on CDIVAE and a bidirectional temporal model according to claim 5, characterized in that: LSTM consists of three gating units and one memory unit. The calculation formulas for the gating units are shown in formulas (35)-(37). f t =σ(W f x t +U f h t-1 +b f ) (35) i t =σ(W i x t +U i h t-1 +b i ) (36) the t =σ(W o x t +U o h t-1 +b o ) (37) In the formula f t Represents the Gate of Oblivion, i t Represents the input gate, o t Represents the output gate, x t h represents the input data. t This represents the hidden layer, where W, U, and b represent the weights and biases, σ represents the sigmoid activation function, and c represents the current state of the memory cell. t It is obtained by weighting the memory cell state of the previous time step and the current candidate state, and the calculation formula is shown in formula (38) and formula (39). Finally, the output of the hidden layer of the LSTM is shown in Equation (40). h t =o t ⊙tanh(c t ) (40)。 7. The adaptive intrusion detection method based on CDIVAE and a bidirectional temporal model according to claim 5, characterized in that: The order of traffic flows contains temporal characteristics, and there are temporal relationships between network traffic data. By using BRN to perform feature weighting on traffic data, global temporal features of traffic data can be mined. At the same time, BiLSTM learns from traffic data from both past and future directions.

8. The adaptive intrusion detection method based on CDIVAE and a bidirectional temporal model according to claim 1, characterized in that: Maximization formula ELBO=Ε z～qφ [logp θ (x T |z,y)]-KL(q φ (z|x T ,y)||p GMM (x S |y)), shrink q φ (z|x T ,y) and p GMM (x S |y)KL divergence enables the distribution of the target domain data after extracting domain-invariant features to align with the distribution of the source domain, achieving a cross-domain effect.

9. The adaptive intrusion detection method based on CDIVAE and a bidirectional temporal model according to claim 4, characterized in that: In the parallel representation, D is a combination of causal masking and exponentially decaying weighting, serving the functions of the MASK matrix and Softmax function in the multi-head self-attention mechanism, thus preventing overfitting of the model; D + and D - By applying causal masking and exponential decay weighting to the data from two different directions, the model can take into account every piece of data in the context during the calculation process.

Images