A Bayesian incremental learning-based attack detection method and system, and a storage medium

Abstract

The application provides a kind of attack detection method, system and storage medium based on bayesian incremental learning, method includes: step one: data set is collected, and divided into multiple tasks according to year;Step two: using bayesian continuous learning framework, the posterior distribution obtained by learning last task is used as the prior distribution of current task, gradient projection method is used, the gradient of current task is projected to the orthogonal subspace of old task feature space, and projection parameter is obtained;Step three: label bias and noise processing are carried out on task;Step four: minimize new task loss, obtain parameter, and take training strategy according to the result of label bias and noise processing;Step five: find optimal merging solution;Step six: the model parameter after merging is used as the initialization parameter of next task, returns step two, enters next round iteration, until the training of all tasks is completed.The beneficial effects of the application are: it can significantly improve the knowledge retention ability and effectively overcome the problem of catastrophic forgetting.

Description

Technical Field

[0001] This invention relates to the field of cyberspace security technology, and in particular to an attack detection method, system, and storage medium based on Bayesian incremental learning. Background Technology

[0002] With the continuous evolution of cyberattack techniques, cyberspace security faces increasingly serious threats. Cyberattack and defense are a long-term, interdependent process. To achieve their objectives, many attacks are characterized by long durations, complex steps, and high stealth, such as Advanced Persistent Threats (APTs). Traditional static model-based security detection methods are no longer effective in dealing with these threats. In real-world network environments, attackers often persist for months or even years, gradually infiltrating target systems through multiple stages, each employing different attack methods and techniques. This long-term attack process requires security detection models to process data streams with extremely long time spans. During this process, the model must not only be able to identify new attack methods but also retain the memory of early attack characteristics; otherwise, it will be unable to fully reconstruct the attack chain, leading to detection failure.

[0003] In real-world network security systems, we face three interconnected core challenges. First is the challenge of knowledge retention. Due to the long cycles of many complex attacks, models often irreversibly forget previously learned attack characteristics when learning new attack patterns—a phenomenon known in machine learning as "catastrophic forgetting." When a model forgets early attack characteristics, even if the attacker reuses the same techniques in subsequent stages, the system cannot correctly identify them, leading to a severe decline in the ability to analyze the complete attack chain. Second is the problem of data distribution drift. The network environment itself is a dynamically changing ecosystem. With business development, technological upgrades, and evolving user behavior, the statistical characteristics of both normal and attack traffic change significantly. This distribution drift causes a continuous decline in the performance of models trained on historical data, manifesting as a dual problem of label bias and noise bias. Label bias stems from the mismatch between historical labeled data and the current actual distribution, while noise bias arises from the large number of false positives and false negatives during attack detection. The third challenge lies in the complexity of multi-source data correlation analysis. Attack detection often involves multiple data sources, such as network traffic, system logs, and security event alerts, and these data sources have complex spatiotemporal relationships. To fully reconstruct the attack chain, it is necessary to establish a causal relationship model across data sources, linking discrete security events into meaningful attack scenarios. However, most traditional detection methods are limited to single-point detection and lack support for probabilistic reasoning mechanisms across time dimensions, making it impossible to effectively correlate attack steps with large time spans.

[0004] The core principle of Bayesian inference is to treat unknown parameters in the model (such as mean and variance) as random variables and use Bayes' theorem, combined with prior knowledge and observational data, to update the understanding of the uncertainty of these parameters, thereby obtaining their posterior probability distribution. The Bayesian principle stems from the need to quantify uncertainty; it provides a powerful mathematical framework for integrating prior knowledge and observational evidence. For example, based on observed logs, traffic, and alerts, combined with prior knowledge of known attacks, it can infer whether certain network behaviors are attacks. However, for complex models (such as those containing latent variables), the accurate calculation of the posterior distribution often involves cumbersome high-dimensional integrals, resulting in high computational complexity, which limits its widespread application.

[0005] Since the true posterior is difficult to handle, the Laplace approximation is typically used in practical applications, that is, using the multivariate normal distribution to approximate arbitrarily complex posterior distributions. The multivariate normal distribution is a natural extension of the univariate normal distribution to higher-dimensional spaces. Its core principle is to fully describe the probability distribution characteristics of a multidimensional random vector using a mean vector (describing the central position of each variable) and a covariance matrix (characterizing the linear correlation and degree of variation between dimensions). Considering that many traffic features and attack behaviors are not independent but correlated, such as ports and protocols, application types, etc., the structure of the covariance matrix allows the multivariate normal distribution to not only express the variance of each dimension itself but also accurately quantify the covariance relationships between different features. This provides the possibility for probabilistic inference in high-dimensional feature spaces.

[0006] To address the issue of complex attack methods, Basani et al. designed a model based on ensemble graph neural networks (GNNs) and recurrent neural networks (RNNs), using an attention mechanism to discover key features and improve the detection accuracy for complex attacks. Yi et al. employed an incremental learning-based classifier normalization method, primarily used for website fingerprinting, which is unsuitable for attack detection scenarios. Ghadami et al. designed a static model combining parallel CNNs and LSTMs for attack detection and used a hybrid optimization algorithm for feature selection to improve detection efficiency. However, none of these methods effectively solved the aforementioned challenges and problems. Summary of the Invention

[0007] To address the problems in existing technologies, this invention provides an attack detection method based on Bayesian incremental learning, comprising the following steps:

[0008] Step 1, Data Acquisition and Processing: Collect the required dataset and divide it into multiple tasks by year;

[0009] Step 2, First Stage Model Training: Using the Bayesian continuous learning framework, the posterior distribution learned from the previous task is used as the prior distribution for the current task. The gradient projection method is employed to project the gradient of the current task onto an orthogonal subspace of the feature space of the old task, obtaining the projection parameters. ;

[0010] Step 3, Label bias and noise processing: Perform label bias and noise processing on the tasks divided in Step 1;

[0011] Step 4, Second Stage Model Training: From Projection Parameters Begin by continuing to train the model on the current task, minimizing the loss for the new task. , obtain parameters Then, based on the results of label bias and noise processing in step three, corresponding training strategies are adopted.

[0012] Step 5, Bayesian model merging: Projecting the parameters of the old model And the new parameters obtained by the model during training for the current task. By merging the parameters, we obtain the optimal model parameters for all tasks from the beginning to the current task. Then find the optimal merging solution. ;

[0013] Step Six: Merge the model parameters Use these parameters as initialization parameters for the next task, and return to step two to proceed to the next iteration, until training for all tasks is complete.

[0014] As a further improvement of the present invention, step two includes:

[0015] Step 1: Perform offline initial training, i.e., task t=1; for each attack class k, assume that its feature distribution follows a multivariate normal distribution, i.e.:

[0016] ,

[0017] in, Let X represent the probability distribution function of the input data. The labels represent the data, and N represents a multivariate normal distribution function. This represents the mean of data in a multivariate normal distribution. Represents the covariance of the data;

[0018] Using variational distribution Approximate estimation of the true posterior distribution, i.e., variational inference, is as follows:

[0019] ELBO is defined as follows:

[0020] ,

[0021] in This represents the lower bound of optimized evidence for variational inference. It is a reconstruction term, representing the expected log-likelihood of the observed data under the variational distribution. The conditional probability distribution function representing the intermediate variable. It is the KL divergence, a measure of variational distribution versus prior distribution. differences These represent latent variables, namely the mean and variance of the categories. This represents the variational distribution function of the input X. Describe a variational distribution function. Here are the function parameters, z represents the intermediate variable, and the prior distribution is set to... N represents the normal distribution function, 0 represents the mean of the function, and I represents the variance;

[0022] Optimize ELBO through stochastic variational inference:

[0023] ,

[0024] in It's the learning rate; it stops when the change in ELBO is less than a threshold. This represents the mean of the data for Task 1. This represents the covariance of the data from Task 1. Take the first derivative of the Lelbo function;

[0025] Step 2: For the current task t, use the old model parameters Initialize the model, calculate its gradient for the current task, and project the gradient onto an orthogonal subspace of the feature space of the old task to obtain the projection parameters. :

[0026] in Let represent the loss function of the attack detection model in task t;

[0027] After projection, training the model on the new task does not affect the model's performance on the old task; that is, the model's loss function satisfies:

[0028] ,

[0029] in Let represent the cumulative loss function of the attack detection model from task 1 to task t-1.

[0030] As a further improvement of the present invention, step three specifically involves: clustering unlabeled data in the current task to generate pseudo-labels and fitting a temporary distribution. ,in The mean of the data representing the temporary distribution. The covariance of the data representing the temporary distribution, combined with the old categorical distribution. And temporary distributions, forming a unified data distribution estimate that includes all known and potential new categories. ,in N Represents the multivariate normal distribution function. This represents the mean of the data for task i. This represents the covariance of the data for task i. i This represents the i-th task. Let t represent the average value of the current task. The covariance of task t is represented by the label being reassigned using a multivariate Gaussian distribution nearest-nearest-class mean classifier.

[0031] ,

[0032] in This represents the Mahalanobis distance between the data in the current task and category k. The model represents the input from the original input. x The feature vector extracted from it, The inverse matrix of the covariance matrix of class k, also known as the precision matrix. Describes the distribution of category k. T represents Transpose of a matrix; for unlabeled data x i Assigned tags The category k that can provide the maximum similarity value:

[0033] ,

[0034] Label assignment can reduce label bias and noise effects caused by unknown data or incorrectly labeled data.

[0035] As a further improvement to the present invention, after label deviation and noise processing...

[0036] Optimize ELBO through stochastic variational inference:

[0037] ,

[0038] in The learning rate stops when the ELBO change is less than a threshold. After optimization, the posterior distribution of the current task data is updated. .

[0039] As a further improvement of the present invention, in step four...

[0040] The model loss function can be the common cross-entropy loss function or a custom design;

[0041] The specific training strategy is as follows:

[0042] If no new category is found in the results of label bias and noise processing, unconstrained training is adopted. If a new category is found, covariance alignment and early stopping training methods are adopted to prevent the model decision boundary from being biased towards the new category and overfitting on the new category data.

[0043] As a further improvement of the present invention, the training method for covariance alignment and early stopping is specifically as follows:

[0044] By monitoring the ratio of covariance determinant To achieve early stopping:

[0045] ,

[0046] in It's a new category. The determinant of covariance, This represents the sum of the covariances of all data categories that have appeared in all tasks. This represents the covariance of the data in the nth category. This refers to the number of old categories. If the new task contains data from new categories, the second phase of training monitors the ratio of the covariance determinant. ,when Stop training at this time. It is a set threshold, when the ratio of the covariance determinant is... A value close to 0 indicates that the covariance of the new category is aligned with the average covariance of the old category, and the ratio of the covariance determinants is... A value close to 0 is a set threshold, ensuring that the model's decision boundary does not favor new categories, thus avoiding the forgetting of old categories.

[0047] As a further improvement of the present invention, step five also includes the following steps:

[0048] Define the parameter merging formula:

[0049] ,

[0050] in This represents the merged parameters. Represents the combined coefficients of the model parameters;

[0051] Find the optimal merging coefficient This makes the merged parameters Able to minimize cumulative loss Based on Bayesian maximum a posteriori probability estimation, and using the Laplace approximation, the objective function to be optimized is:

[0052] ,

[0053] in It is a precision matrix, which will Substituting into formula (10), Indicates the amount of change in the parameter. , This represents the parameters after model merging, and the parameters are used for the attack detection model. Loss function at time exist Perform a second-order Taylor expansion at this point:

[0054] ,

[0055] in It is the Hessian matrix. This represents the new parameters obtained by the model during training on the current task t. Indicates that the model uses parameters The loss function in task t, due to It is a local minimum, and the first-order term in formula (12) is 0. Therefore, the objective function (11) simplifies to:

[0056] ,

[0057] right Taking the derivative and setting it to 0, we obtain the optimal merging coefficients:

[0058] ,

[0059] Since the Hessian matrix is ​​difficult to calculate and may not be positive definite, the Fisher information matrix is ​​used. As an approximation, the Fishser information matrix is ​​defined as:

[0060] ,

[0061] in Indicates model parameters, The dataset representing task t, This represents the probability density function, given the current model parameters. The probability of seeing data Dt for task t is given below. This indicates that in the probability density function p with respect to parameters The first derivative, which measures the sensitivity of the model parameters to the likelihood of the data, is positive semi-definite and easy to compute; therefore, the optimal solution... The final result is:

[0062] ,

[0063] in It is the sum of all Fisher information matrices from the old tasks, approximately , Represents the Fisher information matrix. Represents the Fisher information matrix for the i-th task. This represents the optimal parameters obtained by training the model on task i.

[0064] As a further improvement of the present invention, in step one, the collected dataset is the Kyoto-2006 dataset, which contains network traffic characteristics and attack labels.

[0065] The present invention also discloses an attack detection system based on Bayesian incremental learning, comprising: a memory, a processor, and a computer program stored in the memory, the computer program being configured to implement the steps of the method described in the present invention when invoked by the processor.

[0066] The present invention also discloses a computer-readable storage medium storing a computer program configured to implement the steps of the method described in the present invention when invoked by a processor.

[0067] The beneficial effects of this invention are as follows: 1. This invention employs a training method combining gradient projection, covariance alignment, early stopping, and adaptive parameter merging. Compared to existing incremental learning attack detection methods, this invention significantly improves knowledge retention and effectively overcomes the catastrophic forgetting problem. 2. This invention uses a Bayesian incremental learning method, taking the posterior distribution learned from the previous task as the prior distribution for the current task. This adapts to the dynamic changes in the network environment and effectively addresses the concept drift problem. 3. This invention addresses label bias and noise, enabling the detection of novel / unknown attack categories. 4. This invention uses Bayesian network causal relationship modeling, which can associate complex attack steps with long time spans and multiple stages, enabling the analysis and reconstruction of attack chains for multi-stage attacks. Attached Figure Description

[0068] Figure 1 This is a schematic diagram of gradient projection of the present invention. Detailed Implementation

[0069] This invention proposes an attack detection method based on Bayesian incremental learning, aiming to construct an intelligent attack detection system with continuous evolution capabilities. Firstly, this invention uses gradient projection to ensure the model's inherent stability when absorbing new knowledge. By constraining parameter updates to an orthogonal subspace of the old task's feature space, the ability to identify old data is minimized when the model is trained with new data, thus achieving near-lossless preservation of existing attack detection capabilities. This mechanism allows the model to minimize the decline in accuracy of old tasks during incremental learning, effectively overcoming the core challenge of catastrophic forgetting in continuous learning. Building upon this, this invention further employs variational inference and covariance alignment techniques, enabling the effective identification of known attack variants while also possessing the open-ended ability to discover unknown or novel attacks. This invention integrates stability preservation, dynamic adaptation, and open discovery capabilities, maintaining persistent and reliable protection performance in a constantly changing network threat environment.

[0070] This invention pertains to attack detection in the field of cyberspace security, specifically involving operating system security, network attack and defense knowledge, and the design and application of neural networks. Based on network attack and defense and operating system knowledge, it utilizes Bayesian principles, gradient projection, and a multivariate normal distribution to approximate arbitrarily complex posterior distributions.

[0071] The technical problem to be solved by this invention:

[0072] This invention proposes an attack detection method based on Bayesian incremental learning. Firstly, it overcomes the "catastrophic forgetting" problem that arises when the model incrementally learns new attack knowledge, ensuring long-term stable identification of historical attack features. Secondly, it enhances the model's adaptive detection capability to continuously changing network data distribution (concept drift), effectively handling label bias and noise interference caused by environmental evolution. Thirdly, it breaks through the limitations of traditional detection methods in identifying single-point attacks, establishing a causal reasoning mechanism capable of analyzing long-term, multi-stage attack steps, enabling the detection and tracing of complex attacks.

[0073] This invention discloses an attack detection method based on Bayesian incremental learning, comprising the following steps:

[0074] Step 1, Data Acquisition and Processing: Collect the required dataset and divide the collected dataset into multiple tasks by year; the collected dataset is the Kyoto-2006 dataset, which spans nearly 10 years from 2006 to 2015 of Kyoto University's campus network, including network traffic characteristics (such as protocol type, traffic volume, connection duration, etc.) and attack tags (such as DDoS, scanning, malware, etc.).

[0075] Step two, the first stage of model training, i.e., the first stage of two-stage model training:

[0076] The network environment is constantly changing over time, and network security data is growing accordingly. Considering that an attack is an event with a specific time of occurrence, dividing tasks according to attack type would disrupt the logic and temporal sequence of the attack event. Therefore, in attack detection scenarios, tasks can only be divided by time. Here, Kyoto-2006 is divided by year, with task t corresponding to data from year t (…). (This refers to data from 2006). Therefore, the data for each task may include both known attack categories and newly added attack categories.

[0077] In the two-stage model training, the first stage uses a Bayesian continuous learning framework, using the posterior distribution learned from the previous task as the prior distribution for the current task, such as... Figure 1 As shown, the gradient projection method is used to project the gradient of the current task onto an orthogonal subspace of the feature space of the old task, thus obtaining the projection parameters. In order to maintain the stability of knowledge about old tasks;

[0078] Step two includes:

[0079] Step 1: Perform offline initial training, i.e., task t=1; for each attack class k, assume that its feature distribution follows a multivariate normal distribution, i.e.:

[0080] ,

[0081] in, Let X represent the probability distribution function of the input data. The labels represent the data, and N represents a multivariate normal distribution function. This represents the mean of data in a multivariate normal distribution. Represents the covariance of the data;

[0082] Due to the true posterior distribution Since it is difficult to calculate directly, a variational distribution function representing the input X is used. This is used to approximate the true posterior distribution, i.e., variational inference. Describe a variational distribution function. Here, z represents the function parameter and is an intermediate variable, similar to the label y. However, since this is not the true posterior distribution, an intermediate variable is used to avoid confusion. z can be understood as a hidden variable that we cannot directly observe, but we believe exists behind the data x and determines the generation of x. The core of variational inference is optimizing the Evidence Lower Bound (ELBO), which is defined as follows:

[0083] ,

[0084] in This represents the lower bound of optimized evidence for variational inference. It is a reconstruction term, representing the expected log-likelihood of the observed data under the variational distribution. It is the KL divergence, a measure of variational distribution versus prior distribution. differences These represent latent variables, namely the mean and variance of the categories. This represents the variational distribution function of the input X. Describe a variational distribution function. Here are the function parameters, z represents the intermediate variable, and the prior distribution is set to... N represents the normal distribution function, 0 represents the mean of the function, and I represents the variance;

[0085] Optimize ELBO through stochastic variational inference:

[0086] ,

[0087] in It's the learning rate; it stops when the change in ELBO is less than a threshold. This represents the mean of the data for Task 1. This represents the covariance of the data from Task 1. Take the first derivative of the Lelbo function.

[0088] Step 2: After initial training is completed on the first task, incremental learning training is performed on subsequent tasks. For the current task t, the old model parameters are used. Initialize the model, calculate its gradient for the current task, and project the gradient onto an orthogonal subspace of the feature space of the old task to obtain the projection parameters. :

[0089] ,

[0090] in Let represent the loss function of the attack detection model in task t;

[0091] After projection, training the model on the new task does not affect the model's performance on the old task; that is, the model's loss function satisfies:

[0092] ,

[0093] in Let represent the cumulative loss function of the attack detection model from task 1 to task t-1.

[0094] Step 3, Label Bias and Noise Processing: Perform label bias and noise processing on the tasks divided in Step 1;

[0095] Step one, which divides tasks by time, inevitably leads to label bias. Therefore, label bias and noise processing are necessary here. Unlabeled data in the current task is clustered to generate pseudo-labels, and a temporary distribution is fitted. (from (mid-sampling), among which The mean of the data representing the temporary distribution. The covariance of the data representing the temporary distribution, combined with the old categorical distribution. And temporary distributions, forming a unified data distribution estimate that includes all known and potential new categories. ,in N Represents the multivariate normal distribution function. This represents the mean of the data for task i. This represents the covariance of the data for task i. i This represents the i-th task. Let t represent the average value of the current task. Let represent the covariance of task t. Reassign labels using a multivariate Gaussian distribution nearest-class-mean classifier:

[0096] ,

[0097] in This represents the Mahalanobis distance between the data in the current task and category k. The model represents the input from the original input. x The feature vector extracted from it, The inverse matrix of the covariance matrix of class k, also known as the precision matrix. Describes the distribution of category k. T represents Transpose of a matrix; for unlabeled data x i Assigned tags The category k that can provide the maximum similarity value:

[0098] ,

[0099] Label assignment can reduce label bias and noise effects caused by unknown data or incorrectly labeled data.

[0100] Optimize ELBO through stochastic variational inference:

[0101] ,

[0102] in The learning rate stops when the ELBO change is less than a threshold. After optimization, the posterior distribution of the current task data is updated. .

[0103] Step four, the second stage of model training, i.e., the second stage of the two-stage model training:

[0104] The second stage, after parametric projection in the first stage, involves the projection parameters... Begin by continuing to train the model on the current task, minimizing the loss for the new task. , obtain parameters The model loss function can be the common cross-entropy loss function, or a custom design can be used. Different training strategies should be adopted based on the results of label bias and noise processing in step three.

[0105] If no new class is found in step three, unconstrained training is performed. If a new class is found in step three, covariance alignment and early stopping are used to prevent the model's decision boundary from being biased towards the new class and overfitting on the new class data. This is achieved by monitoring the ratio of the covariance determinant. To achieve early stopping:

[0106] ,

[0107] in It's a new category. The determinant of covariance, This represents the sum of the covariances of all data categories that have appeared in all tasks. This represents the covariance of the data in the nth category, where n represents the nth category. This refers to the number of old categories. If the new task contains data from new categories, the second phase of training monitors the ratio of the covariance determinant. ,when Stop training at this time. It is a set threshold, such as When the ratio of the covariance determinant A value close to 0 indicates that the covariance of the new category is aligned with the average covariance of the old category, and the ratio of the covariance determinants is... A value close to 0 is a set threshold, ensuring that the model's decision boundary does not favor new categories, thus avoiding the forgetting of old categories.

[0108] Step 5, Bayesian model merging:

[0109] After two stages of model training, the parameter projection of the old model is obtained. And the new parameters obtained by the model during training for the current task. Here, we need to merge the old and new model parameters to obtain the optimal model parameters for all tasks from the beginning to the current task. Then find the optimal merging solution. First, define the parameter merging formula:

[0110] ,

[0111] in This represents the merged parameters. Represents the combined coefficients of the model parameters;

[0112] The goal is to find the optimal merging coefficient. This makes the merged parameters Able to minimize cumulative loss Based on Bayesian maximum a posteriori probability estimation, and using the Laplace approximation, the objective function to be optimized is:

[0113] ,

[0114] in It is the precision matrix (the inverse covariance matrix of the posterior of the old task). Substituting into formula (10), Indicates the amount of change in the parameter. , This represents the parameters after model merging, and the parameters are used for the attack detection model. Loss function at time exist Perform a second-order Taylor expansion at this point:

[0115] ,

[0116] in It is the Hessian matrix. This represents the new parameters obtained by the model during training on the current task t. Indicates that the model uses parameters The loss function in task t, due to It is a local minimum, and the first-order term in formula (12) is 0. Therefore, the objective function formula (11) simplifies to:

[0117] ,

[0118] right Taking the derivative and setting it to 0, we obtain the optimal merging coefficients:

[0119] ,

[0120] Since the Hessian matrix is ​​difficult to calculate and may not be positive definite, the Fisher information matrix is ​​used. As an approximation, the Fishser information matrix is ​​defined as:

[0121] ,

[0122] in Indicates model parameters, The dataset representing task t, This represents the probability density function, given the current model parameters. See the data for task t below. The probability, This indicates that in the probability density function p with respect to parameters The first derivative, which measures the sensitivity of the model parameters to the likelihood of the data, is positive semi-definite and easy to compute; therefore, the optimal solution... The final result is:

[0123] ,

[0124] in It is the sum of all Fisher information matrices from the old tasks, approximately , Represents the Fisher information matrix. Represents the Fisher information matrix for the i-th task. This represents the optimal parameters obtained by training the model on task i;

[0125] Step Six: Merge the model parameters Use these parameters as initialization parameters for the next task, and return to step two to proceed to the next iteration, until training for all tasks is complete.

[0126] Key points of this invention:

[0127] 1. To address the catastrophic forgetting problem in incremental model learning, a method based on gradient projection and Bayesian adaptive parameter merging is designed to maintain a balance between model stability and plasticity.

[0128] 2. To address the issues of label bias and data noise, pseudo-labels are first generated using clustering methods. Then, a multivariate Gaussian distribution nearest-nearest-class mean classifier is used to redistribute the labels, and variational inference is used to approximate the posterior distribution, which can detect novel / unknown attacks.

[0129] 3. To address the challenge of multi-stage attack correlation analysis, a Bayesian network-based attack causal relationship modeling and reasoning method is designed, which uses the posterior distribution of the previous task as the prior distribution of the current task.

[0130] 4. To address the requirements of computational efficiency and model performance, an optimized training strategy based on Fisher information matrix approximation and early stopping is designed to improve efficiency while maintaining model stability.

[0131] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. An attack detection method based on Bayesian incremental learning, characterized in that, Includes the following steps: Step 1, Data Acquisition and Processing: Collect the required dataset and divide it into multiple tasks by year; Step 2, First Stage Model Training: Using the Bayesian continuous learning framework, the posterior distribution learned from the previous task is used as the prior distribution for the current task. The gradient projection method is employed to project the gradient of the current task onto an orthogonal subspace of the feature space of the old task, obtaining the projection parameters. ; Step 3, Label bias and noise processing: Perform label bias and noise processing on the tasks divided in Step 1; Step 4, Second Stage Model Training: From Projection Parameters Begin by continuing to train the model on the current task, minimizing the loss for the new task. , obtain parameters Then, based on the results of label bias and noise processing in step three, corresponding training strategies are adopted. Step 5, Bayesian model merging: Projecting the parameters of the old model And the new parameters obtained by the model during training for the current task. By merging the parameters, we obtain the optimal model parameters for all tasks from the beginning to the current task. Then find the optimal merging solution. ; Step Six: Merge the model parameters Use this as the initialization parameter for the next task, and return to step two to enter the next iteration, until the training of all tasks is completed; Step three specifically involves: clustering unlabeled data in the current task to generate pseudo-labels, and fitting a temporary distribution. ,in The mean of the data representing the temporary distribution. The covariance of the data represents the temporary distribution. And temporary distributions, forming a unified data distribution estimate that includes all known and potential new categories. ,in N Represents the multivariate normal distribution function. This represents the mean of the data for task i. This represents the covariance of the data for task i. i This represents the i-th task. Let t represent the average value of the current task. The covariance of task t is represented by the label being reassigned using a multivariate Gaussian distribution nearest-nearest-class mean classifier. ， in This represents the Mahalanobis distance between the data in the current task and category k. The model represents the input from the original input. x The feature vector extracted from it, The inverse matrix of the covariance matrix of class k, also known as the precision matrix. Describes the distribution of category k. T represents Transpose of a matrix; for unlabeled data x i Assigned tags The category k that can provide the maximum similarity value: ， Label assignment can reduce the impact of label bias and noise caused by unknown data or incorrectly labeled data; In step four, The model loss function can be the common cross-entropy loss function or a custom design; The specific training strategy is as follows: If no new category is found in the results of label bias and noise processing, unconstrained training is adopted. If a new category is found, covariance alignment and early stopping training methods are adopted to prevent the model decision boundary from being biased towards the new category and overfitting on the new category data. The specific training method for covariance alignment and early stopping is as follows: By monitoring the ratio of covariance determinant To achieve early stopping: ， in It's a new category. The determinant of covariance, This represents the sum of the covariances of all data categories that have appeared in all tasks. This represents the covariance of the data in the nth category. This refers to the number of old categories. If the new task contains data from new categories, the second phase of training monitors the ratio of the covariance determinant. ,when Stop training at this time. It is a set threshold, when the ratio of the covariance determinant is... A value close to 0 indicates that the covariance of the new category is aligned with the average covariance of the old category, and the ratio of the covariance determinants is... A value close to 0 is a set threshold, ensuring that the model's decision boundary does not favor new categories, thus avoiding the forgetting of old categories.

2. The attack detection method according to claim 1, characterized in that, Step two includes: Step 1: Perform offline initial training, i.e., task t=1; for each attack class k, assume that its feature distribution follows a multivariate normal distribution, i.e.: ， in, Let X represent the probability distribution function of the input data. The labels represent the data, and N represents a multivariate normal distribution function. This represents the mean of data in a multivariate normal distribution. Represents the covariance of the data; Variational distribution Approximate estimation of the true posterior distribution, i.e., variational inference, is as follows: ELBO is defined as follows: ， in This represents the lower bound of optimized evidence for variational inference. It is a reconstruction term, representing the expected log-likelihood of the observed data under the variational distribution. The conditional probability distribution function representing the intermediate variable. It is the KL divergence, a measure of the variational distribution versus the prior distribution. differences These represent latent variables, namely the mean and variance of the categories. This represents the variational distribution function of the input X. Describe a variational distribution function. Here are the function parameters, z represents the intermediate variable, and the prior distribution is set to... N represents the normal distribution function, 0 represents the mean of the function, and I represents the variance; Optimize ELBO through stochastic variational inference: ， in It's the learning rate; it stops when the change in ELBO is less than a threshold. This represents the mean of the data for Task 1. This represents the covariance of the data from Task 1. This represents taking the first derivative with respect to the Lelbo function; Step 2: For the current task t, use the old model parameters Initialize the model, calculate its gradient for the current task, and project the gradient onto an orthogonal subspace of the feature space of the old task to obtain the projection parameters. : ， in Let represent the loss function of the attack detection model in task t; After projection, training the model on the new task does not affect the model's performance on the old task; that is, the model's loss function satisfies: ， in Let represent the cumulative loss function of the attack detection model from task 1 to task t-1.

3. The attack detection method according to claim 1, characterized in that, After label deviation and noise processing Optimize ELBO through stochastic variational inference: ， in It is the learning rate. It stops when the change in ELBO is less than the threshold. After optimization, the current number of tasks is updated.

4. The attack detection method according to claim 1, characterized in that, Step five also includes the following steps: Define the parameter merging formula: ， in This represents the merged parameters. Represents the combined coefficients of the model parameters; Find the optimal merging coefficient This makes the merged parameters Able to minimize cumulative loss Based on Bayesian maximum a posteriori probability estimation, and using the Laplace approximation, the objective function to be optimized is: ， in It is a precision matrix, which will Substituting into formula (10), Indicates the amount of change in the parameter. , This represents the parameters after model merging, and the parameters are used for the attack detection model. Loss function at time exist Perform a second-order Taylor expansion at this point: ， in It is the Hessian matrix. This represents the new parameters obtained by the model during training on the current task t. Indicates that the model uses parameters Loss function in task t ,because It is a local minimum, and the first-order term in formula (12) is 0. Therefore, the objective function (11) simplifies to: ， right Taking the derivative and setting it to 0, we obtain the optimal merging coefficients: ， Since the Hessian matrix is ​​difficult to calculate and may not be positive definite, the Fisher information matrix is ​​used. As an approximation, the Fishser information matrix is ​​defined as: ， in Indicates model parameters, The dataset representing task t, This represents the probability density function, given the current model parameters. The probability of seeing data Dt for task t is given below. This indicates that in the probability density function p with respect to parameters The first derivative, which measures the sensitivity of the model parameters to the likelihood of the data, is positive semi-definite and easy to compute; therefore, the optimal solution... The final result is: ， in It is the sum of all Fisher information matrices from the old tasks, approximately , Represents the Fisher information matrix for the i-th task. This represents the optimal parameters obtained by training the model on task i.

5. The attack detection method according to claim 1, characterized in that, In step one, the collected dataset is the Kyoto-2006 dataset, which contains network traffic characteristics and attack labels.

6. A system for attack detection based on Bayesian incremental learning, characterized in that, include: A memory, a processor, and a computer program stored on the memory, the computer program being configured to implement the steps of the method of any one of claims 1-5 when invoked by the processor.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program configured to implement the steps of the method according to any one of claims 1-5 when invoked by a processor.

Images